Test Automation and its necessity in a project
Test automation is a process of running an application or part of application using automation tool or executing script and verifying expected result without manual intervention. Detailed analysis is required if a project needs investing in automation test. Test automation should be seriously considered in large applications however, time and budget should be analysed properly to implement automation in small software.
Difficulties to implement effective Test Automation system
Test automation can be implemented using various automation tools like commercial tool, open source tools and in-house built tools. From selecting suitable automation tool for a project to modifying and updating automation scripts in different phase of project may become challenging for an automation tester. Automation goal or expectation of automation outcome may cause dissatisfaction if all constraints, dependencies are not articulated properly. Implementing an effective framework is a full time task, so any spare time test automation approach would cause inadequate focus in development approach which would result delay in progress. Building and maintaining suitable test automation system in changing project environment requires technical skill and during employee turnover, projects may lose required skills.
Road to effective automation system
Management need to understand the importance as well as challenges of automation system. Dedicated qualified resource, time and budget are needed to deliver a robust and effective test automation framework. Rigorous and effective Quality Assurance practice in place is another important dependency of successful Test Automation. Effective QA practice with well defined Test framework assures requirement traceability and expected behaviour thus ensures to provide better platform for Test Automation system. Some important aspects of test automation are all test cannot be automated, no one tool can support all system components, not all test person can have skill in automation scripting and substantial lead up time required for preparing automation framework and environment.
Benefit: Automated Regression Suit
• Reduced manual regression testing effort
• Minimum additional effort to run tests repeatedly
• A suite of repeatable, robust tests
• Ability to create test data which can be harnessed by manual test team and other members of the project to reduce their efforts to meet test scenario pre-requisites
• Ability to easily expand scope to cover alternative routes through business processes
• A subset of Regression test suit can be built in assisting Shakeout testing
Regression Suit: Few points to consider
• The intended coverage of automation could be described as ‘as much of the regression suite as is practical and possible’. Automation may also be considered for environment shakeout testing, and in rare circumstances for progression testing.
• Automated regression suit should implement all business mandatory test cases. Business mandatory tests are a set of tests designed to touch on aspects that are common to project infrastructure and verify its desirable behaviour.
• Initially all existing priority one and two business cases of the projects should be automated. Automation scripts and associated function can be executed against requested environment
• A common library and shared object repository should be created to provide for a shared and centralised location of automation artefacts and to compose an infrastructure for future efforts
• All test cases should be executed multiple times to ensure they deliver a viable and consistent results
• The automated scripts can bel run after each code drop in the project test environments ensuring regression script validating key functions are not being impacted by the new code drop.
• Automation Candidates: Some test cases will not be suitable for automation; due to the limitations of the test tool, or because automating will have limited value or return on investment. Some factors to consider:
o Test set up requirements & dependencies
o Repeatability & ease of maintenance
o Verifiable results
o Test priority
• The following indicators typically point towards manual regression testing:
o Highly dynamic UI
o Usability testing
o Visual verifications
o Printing verification
o Unpredictable or date dependent outcomes
Solution: Automation Regression Test Scenarios
Functional Test Cases/Steps
The functional steps for the automated test scenarios are important. Initially the Smoke Test and Automation Specification documents can be used if available and certainly relevant test case steps from manual test repository can be used if availble. In the event, if some of this information was not current and did not align with the behaviour observed in the application, SME knowledge and investigation via the application was required to determine the necessary functionality to include. In summary, an automation tester should use a combination of documented sources, plus trial and error activities in the application, consult with the peer, to identify a definitive functional scenario.
Automation Framework Design
An important aspect is to design automation framework. Designing framework is depended on automation scope, purpose, application areas and in general project SDLC. Light framework (scripts may not be reusable in later iterations without modification) can be used to validate functionality in fast paced agile environment. Various types of framework can be adopted. Most projects can be benefited with automation suit implemented on multiple types of frameworks. Herein, mentioning a few framework that are widely used.
Modular Framework: A modular, or functional decomposition framework approach is the coding of repeatable discreet processes or business functions of the project. These functions are coded once only as parameterised functions, or Actions in UFT terminology. Detailed information on all coded actions or functions should be documented in separate spreadsheet. Data dependency here is kept to the minimum and data are created in the scripts. This results in longer test execution time, but this disadvantage is negligible, and is outweighed by the reduced set up effort. Increased emphasis is put on producing a reusable, maintainable suite that can be readily handed over to next automation. Also there will not be a 1 to 1 relationship between automated test scenarios and regression manual test cases.
Data Driven Framework: To accommodate for specific customer data and extend the automation scope, data driven framework can be used Data-driven framework reads client data from an external Excel spreadsheet and feeds the spreadsheet to the script as Input-Data for reading client Information. The scripts are executed to pick up that particular client information and modify Application or other information of the client via data fed from different external spreadsheets or attached UFT built-in datasheet . Test results with various data can be saved in separate Result Spreadsheet by the script.
I have discussed other types of framework in “Automation with Selenium WebDriver” section