The Test Automation Era: Can we kiss manual testing goodbye?
This is an excellent post about the seemingly magic powers of the new industry leader – automated testing. It was originally written by Ilan Malyanker, who has kindly permitted us to post it here.
Automation- the new magic word
The last few years have shown us that there’s an increase in the prestige of testing in general, and in software automation in particular. More and more highly skilled graduates, choose automation testing, as their professional career. Automation tools and simulators have become more reliable, and sophisticated. Test automation’s advantages are too obvious and meaningful to miss out on. Every professional understands the importance of such tests to the organization, and to the elevated quality of the delivered product.
But what is it about automation testing, that makes it so desirable for organizations and to (mainly new) team/group leaders? Well, when performed correctly, automation could be executed in a precise & consistent manner on each execution. It can save time, and ideally use this resource, by constant and ongoing execution of tests. Automation can maximize the use of machines and other setup peripherals available, while covering a wide range of test cases. Automation allows execution of tests in resolution capabilities only possible for a machine, thus avoids human errors.
Is automation really limitless?
Many testing phrases could be used in the automation context. When talking about automation, one might hear expressions like- ‘Functional Tests’, ‘System Tests’, ‘Stress Tests’, ‘Acceptance Tests’, ‘Security Tests’, ‘Sanity Tests’, White/Black Box Tests, etc. All of the above, and much more, could be performed by automated test scenarios, in one way or another. Furthermore, for some testing methodologies, automation could be the ideal solution, like ‘Regression Tests’, or ‘Load Tests’.
But there are test types/styles, which automation cannot be the answer for, by definition. Let’s take ‘Exploratory Testing‘ for instance, which is a testing style that is not predetermined, and without a written, predefined scenario. It is a ‘free-style’ testing type, on which the tester explores the tested application, and performs actions invented on the fly, and while walking through the AUT, he/she finds bugs. It’s important to mention that the exploratory test of today, could be a part of the functional tests of tomorrow, especially if a bug was found during its execution. In this case, the test scenario is documented and added to the regression legacy test flows. More important is the fact that many errors on the AUT are found this way, just by strolling through the framework, exploring it.
These kind of tests (Exploratory), could never be done by automation, since automated tests are predicted, documented flows, whereas exploratory testing is script-less, unexpected and not documented. Since many bugs are found while executing tests in this method, one cannot afford to leave these bugs to be found by actual, real clients after the SW product is released. Manual testers who are familiar with the tested application, and who know its pitfalls, are needed to cover the system by exploring it and revealing its hidden bugs.
What about Usability Testing? Could automated tests be the answer for this testing methodology? Usability testing is a testing style on which the system is checked for conceptual/logical/product-level/UX errors (many times by users out of the QA). On this testing stage, software aspects, like user friendliness, ease of use, and general look & feel of the application, are checked. One might not expect “traditional” bugs to be found on this testing phase, but only “usability bugs”, which require a higher level of system analysis.
Conventional testing are executed with strict correlation to the testing documentation. Success criteria is clearly defined, and tests outcome is measured with Pass/Fail annotations. This is almost opposite to usability testing, on which the tester needs to state the problem, and actually explain (and convince) what he feels is wrong with the system. It’s not a ‘black or white’ situation. This is whyautomation is useless when dealing with usability testing. It is built for dichotomous distinction between success and failure, whereas the outcome of usability, is more descriptive, abstract, and sometimes arguable.
Where lies the real power of automation?
Leaving unit tests and performance tests aside (it’s obvious that coded tests and automation tools, would be vary adequate for those), automated tests are ideally used for regression tests. Elements that are known to be operational and relatively stable, still need to be checked, and this is where automation steps in. All legacy tests, should be included in the regression, as well as new features which were tested and now are integral part of the released version. To all of that, one needs to add the fixed bugs scenarios, which would make the scope of the regression wider over time. This is part of the reason that automated tests best fit this testing type. The released build is mature, and doesn’t need to be changed all the time (like new features and user stories in scrum). The automated tests are developed on a static and stable product, so during test development, debugging would be needed only for the automated tests, and not for the tested application.
Furthermore, covering the regression by reliable automated tests, releases the manual testers to handle other types of tests (Exploratory & Usability for instance :)), increases the range of tests, hence makes the released version better. Manual testers would be free from doing the same monotonous tasks over and over again, and that would surely have a positive impact on their motivation.
Standing still is going backwards
As I tried to demonstrate in this article, there are quite a few good reasons not to neglect manual testing, even if currently, automation is executed in a satisfactory manner. No organization would allow itself to release a product which had not been tested properly. Manual testers are essential for a better quality of the delivery. Apart from exploratory and usability tests, these testers have many issues to cover, and that’s why QA engineers are the ones that are most familiar with the application released.
That said, in this business, everyone must be updated with the advancements in technology. Any professional needs to catch up with the commonly used, and evolving working trends. In spite of the fact that in the foreseeable future, manual testers would still be imperative to any software testing process, they need to realize that scripting and coding in general, are increasingly important for their day to day work. Knowing your way with scripts will undoubtedly increase your efficiency, while minimizing the time spent on repetitive setup and testing tasks. There’s an increasing demand for testers with development capabilities in different levels of coding skills.
Manual testers have been unjustly underrated for years. They (we) were referred to as ‘second best’, as ‘temporary’ & ‘dispensable employees’. No more!! If an answer for the question at the title is needed, then the bottom line is- “No. Manual testing is still a critical part of the overall testing routine.” You’re safe for now. Nevertheless, every professional in general, and especially QA engineers, must constantly aspire to take their work and knowledge to the next level, for their own professional future, and for the benefit of everyone else.
