insights
Tomas’s Post: Manual Testing vs. Test Automation. Pros and Cons – quick overview
In my new blog post, I would like to briefly discuss the main differences between manual testing and automation testing, along with the pros and cons for each approach. I really believe it can be beneficial for both junior testers to provide them with some insights as well as for test managers to help them make the right decisions on their projects. Obviously, we should not forget the client itself, the better educated client, the easier for us testers to deliver quality and results.
As I said in my previous post from the TestingUnited conference, systems are still more and more complex. Testing as a discipline is constantly growing and became more important than ever to support good quality products. More and more people in IT are aware of importance of testing. Despite of this fact, we do not have unlimited resources and unlimited time to test everything. Decrease time to market is one of the expectations in these days. Setting up proper strategy and right balance between manual and automation tests is very important.
Both Manual and automated tests offer several benefits but also disadvantages. I would like to be as objective as possible, I will try not to advocate any of those two options over other. Let’s see how successful I will be at the end.
Before jumping to pros and cons, lets quickly discuss what is manual and automated testing.
Manual Testing
Manual testing is an activity, where testing of an application is done by human. Software tester (but it can also be a business user on the client side) executes tests based on the defines test cases ensuring that the application is working properly and meets defined requirements.
Automated Testing
Automation testing is designed to run automatically using a dedicated tool. Test automation tools are running predefined steps / actions written in the test scripts. Key objective is obviously the same, to check, whether reality is meeting the expected results. Despite the fact it is an automated test, scripts and maintenance have to be done by humans, either tester engineers or developers.
I would like to emphasize that not manual nor automated testing are test types. They are rather an approaches / techniques / way how to execute defined tests. In a very simplified view of this word all test types can be done both manually or automated, even performance tests, if you have enough software testers :). But in real world some tests can be better performed manually and some using automation tool.
Pros and Cons of Manual Testing
Pros of Manual Testing:
- Easier to adapt to changes. Manual testing is well-suited in the case when you are making a lot of unplanned changes to the application and the application needs to be tested post implementing the changes.
- Manual testing helps to identify issues related to the look and feel of the application.
- Manual testing helps to identify usability issues in the application.
- While testing a small change, an automation test would require coding which could be time-consuming. While you could test manually on the fly.
- In many cases need to spend money for test automation tools
- Suitable at the beginning of the project, where functionality is constantly changing and is potentially unstable.
- Easier to prepare test data. Test data can be potentially prepared/modified on the fly during execution.
- Manual testing can be used for exploratory testing
- Less expansive compared to test automation in a short run
Cons of Manual Testing:
- Some tests are difficult or almost not testable manually. E.g. performance tests.
- Less reliable testing results as tests are performed by human. Especially during peaks and under pressure.
- Could be a time-consuming activity, especially when re-executing regression tests multiple times.
- Running manual regression tests may lead to team demotivation and potentially to professional blindness.
Pros and Cons of Test Automation
Pros of automated testing:
- Automation tests are recorded. This allows you to reuse and execute the same kind of testing and repetitive tasks multiple times.
- Once test scripts are configured, test automation requires less time and resources to execute defined scope, compared to manual testing.
- Consistent test runs and thus reliable result
- Fewer error caused by human mistake
Cons of automated testing:
- Initial costs very high. Automation tools can be expensive
- Usually underestimated maintenance costs, it is not one-off activity and requires maintenance of the test scripts on regular basis, otherwise test scripts will not be applicable.
- Pesticide paradox can occur. Scripts will control only what was coded in the script. Running the same scripts many times will stop finding additional defects in the future even they may still exist in the application.
- Too detailed scripts will become unmaintainable. To vague scripts will not serve as expected.
- Tools itself may have their own defects
- Difficult to detect look and feel issue and usability issue
- Different skillset of the test team is required. More technical background is required.
- Basic programming knowledge is required.
Where to typically use manual testing
- Smaller projects – no budget for setting up test automation tools
- Functional testing of a brand-new application. Application in initial stage of development. Content is changing quite often and not stabilized yet.
- Exploratory Testing. This type of testing requires the tester’s knowledge, experience, analytical/logical skills, creativity, and intuition.
- Usability Testing.This is an area in which you need to measure how user-friendly, efficient, or convenient the software or product is for the end users. Testing the look and feel of the application.
- Ad-hoc Testing. In this scenario, there is no specific approach. It is a totally unplanned method of testing where the understanding and insight of the tester is the only important factor
Where to typically use automated testing
- Bigger projects where re-execution of the same test cases is expected in a long run.
- Regression Testing. Best fit for test automation in cases a huge regression set has to be executed frequently.
- Repeated Execution. Testing which requires the repeated execution of a task is best automated. E.g. test data creation.
- Load and Performance Testing. Helps to simulate the work of many concurrent users.
- Unit testing – Testing a specific unit of a software product.
- Data driven testing – executing the same simple scenarios with multiple data sets
Summary
To meet project expectations, control costs and time required for successful completion of a project, you cannot depend solely on manual testing or automated testing approach. You will probably need a right mix of both to get the best results.