No matter you're a newcomer , Or are you already doing software testing , I believe you must have heard of or been exposed to automated testing . that , What does automated testing mean ?
seeing the name of a thing one thinks of its function , Automated testing is , A practice of transforming human testing behavior into machine testing behavior
, For the most common GUI In terms of automated testing , It is to simulate all kinds of manual operations on the software interface by automated testing tools , And automatically verify whether the results meet the expectations .
Are you a little excited ? This seems to have ushered in an era of automation in which machines replace repetitive manual labor , You can free yourself from simple and repetitive work . But what about the reality ?
The essence of automated testing is to write a piece of code first , Then test another piece of code , Therefore, the implementation of automated test case itself belongs to the development work , It takes a lot of time and effort , And the developed use cases must be updated with the change of the object under test , You also have to pay the cost of maintaining test cases for this .
When you find that the maintenance cost of automated test cases is higher than the test cost saved , Automated testing has lost its value and significance , You also need to make trade-offs on whether to use automated testing .
Why automated testing ?
To give you a better understanding of the value of automated testing , That is why automated testing is needed , Let me first talk to you about the advantages of automated testing ：
*
Automatic testing can replace a large number of manual and mechanical repetitive operations , Test engineers can spend more time on more comprehensive use case design and new function testing ;
*
Automated testing can greatly improve the efficiency of regression testing , Very suitable for agile development process ;
*
Automated testing can make better use of unattended time , To run tests more frequently , It is especially suitable for testing in non working hours , Working mode of failure cases in working time analysis ;
*
Automated testing can efficiently implement some test types that manual testing can not complete or cost a lot , For example, key business 7×24 System stability test and high concurrency scenario stress test ;
*
Automatic testing can also ensure the consistency and repeatability of the operation and verification of each test , Avoid human omission or negligence .
In order to avoid over reliance on automated testing , You also need to understand the disadvantages of automated testing , This will help you get around the problems in the actual work “ pit ”.
*
Automated testing cannot replace manual testing , It can only replace the high frequency of manual testing , Repeated steps of mechanization . Don't expect all tests to be automated , Otherwise, it will not be worth the loss .
*
Automatic testing is far more vulnerable than manual testing , Unable to cope with changes in the system under test , There has always been a joke in the industry “ Development hands shake , Test automation all night ”, This also reflects the fact that the maintenance cost of automated test cases has been high .
The fundamental reason is that automated testing itself does not have any function “ intelligence ”, Just step by step to perform the pre-defined test steps and verify the test results . For obvious errors and unexpected events in the process of execution , Automated testing has no processing power .
*
The workload of automatic test case development is much greater than that of single manual test , Therefore, only when the effective execution times of the developed test cases are greater than or equal to 5 Second time , To recover the cost of automated testing .
*
Manual testing usually finds more defects than automated testing , And automatic testing can only find the defects of regression testing scope .
*
The efficiency of testing largely depends on the design and implementation quality of automated test cases , Unstable automated test case implementations are worse than no automation .
*
The early days of automated testing , Use case development is usually inefficient , A large number of early developed use cases usually mature in the whole automated testing system , After fully mastering the test tools with the test engineer , Need to refactor .
*
Business test experts and automation test experts are usually two groups of people , The former knows business but not automation technology , The latter understands automation technology but not business , Only the two work closely together , In order to carry out automated testing efficiently .
*
Automatic test developers must have certain programming ability , This will be a challenge for traditional manual test engineers .
What kind of project is suitable for automated testing ?
See here , You may be muttering to yourself ,“ Is there any mistake , There are more disadvantages than advantages in automated testing ”. So why are so many enterprise level projects implementing automated testing ? that , What I'm going to talk about next is this , What kind of project is suitable for automated testing ?
first , Stable demand , It doesn't change frequently .
The biggest fear of automated testing is demand instability , The high frequency of requirement change will lead to a sharp rise in the maintenance cost of automated test cases . Use cases that have just been developed and debugged may be due to changes in the interface , Or business process change , Had to redevelop and debug . Therefore, automated testing is more suitable for software projects with relatively stable requirements .
second , Long R & D and maintenance cycle , Regression testing needs to be performed frequently .
1. in my opinion , Software products are more suitable for automated testing than software projects .
first , The life cycle of software products is generally long , Usually, multiple versions will be released one after another , Every release has a lot of regression testing requirements .
meanwhile , There is also plenty of time reserved for automated test development , Can be iterated with the product .
secondly , The execution ratio of automated test cases is higher than 1:5, That is to say, the developed use cases can at least be executed effectively 5 More than one time , The advantages of automated testing can be better reflected .
2. Automated testing for software projects , It depends on the specific situation of the project .
If a short-term one-time project , Even technically speaking, the feasibility of automated testing is very high , But from the input-output ratio （ROI） It is not recommended to implement automation from the perspective of automation , Because the automation use cases that have been painstakingly developed may be executed once or twice , The project is over . I've had more exaggerations , Automated test cases are not yet developed , The project is going online .
therefore , For this kind of short-term one-time project , I think you should choose the manual exploratory test , The first priority is to find defects . For some medium and long-term projects , My advice is ： Automatic test of relatively stable software function , Manually test the functions with great changes or unclear requirements , The ultimate goal is to use 20% To cover it 80% Regression test of .
third , Scenarios that need to run the same test repeatedly on multiple platforms .
There are many such scenes , such as ：
* about GUI test , The same test case needs to be executed on a variety of different browsers ;
* For mobile application testing , The same test case needs to be tested in multiple different environments Android perhaps iOS Version , Or the same test needs to be performed on a large number of different mobile terminals ;
*
For some enterprise software , If there are different customized versions for different customers , Most of the main functions of each customized version are consistent , There may be only slight differences in individual functions , Tests also need to cover all tests for each custom version ;
* ……
These are the best scenarios for automated testing , Because a single test case needs to be executed repeatedly , It can maximize the return on investment of automated testing .
fourth , Some test items cannot be realized by manual test , Or the manual cost is too high .
For all performance and pressure tests , It's hard to do it by hand .
such as , A project requires benchmark performance testing for 10000 concurrent users （Benchmark
test）, Do you really want to find 10000 users to operate the tested software according to your password ? Another example , about 7×24 Hour stability test , Do you want to find a group of users to operate the tested software day and night ?
At this time , You have to rely on automated testing technology , Use the machine to simulate the scene that a large number of users operate the tested software repeatedly . Of course, it is impossible to pass this kind of test GUI Operation to simulate a large number of user behavior , You have to use protocol based automated testing technology , I will describe this in detail in the following performance test section .
fifth , The development of the tested software is relatively standard , It can ensure the testability of the system .
Technically speaking , If we want to achieve stable automated testing , The development process of the software under test must be standardized . such as ,GUI If there are no rules to find , Will cause GUI Automatic control recognition and positioning is unstable , This will affect the efficiency of automated testing .
in addition , The automation of some use cases must require developers to reserve testability interfaces in products , Otherwise, the follow-up automation will be difficult to carry out .
such as , Some users log in , Image captcha required , If the developer does not provide a path to bypass the captcha , So automatic test must rely on optical character recognition （OCR） Technology to pattern recognition of picture captcha , It was designed to prevent the robot from operating , as one can imagine OCR The recognition rate will be very low , It will directly affect the stability of the use case .
sixth , Testers have certain programming ability .
If members of the test team don't have any foundation for development programming , If you want to implement automated testing, there will be greater resistance . This resistance will come from two aspects ：
* Early learning costs are usually large , It is difficult to produce substantial help to the actual project in the short term , At this point, if the management does not have the correct expectations for automated testing , It's likely to stop automated testing ;
*
Test engineers are usually very keen to learn to use automated testing technology , So that the focus of their work will be the wrong shift , Put a lot of energy on the study and practice of automatic test technology , The design of test cases is ignored , This will directly reduce the overall quality of the software .
summary
Automated testing is , A practice of transforming the manual testing of software into machine testing , It can release the test engineer from the repeated mechanical test work , More attention will be paid to the testing of new functions and the design of more comprehensive test cases .
But automated testing is a try “ double-edged sword ”, Although it can liberate the labor force of test engineers to a certain extent , Complete some tests that cannot be done manually , But it doesn't apply to all test scenarios , If the cost of maintaining automated testing is higher than the saved testing cost , Then, in such a project, promoting automated testing will not be worth the loss .
Technology