QA Test Automation Framework Design Guidelines

A framework is an organization’s way of doing things. With these guidelines, we at Zymr define a set of practices and standards for successfully executing a software project’s test automation.

A test framework consists of a set of processes, standards and interactions between components to design and execute test scripts.

Below are some key parameters that should be considered while building a test automation framework.
Selection of a framework – Different types of frameworks that exist are:

• Data Driven framework – Used when flow of the application remains constant, only the data changes. The data is provided by external medium e.g. – excel sheet, XML etc.
• Keyword driven framework – This framework provides generic keywords that can be used with any type of application. It also provides abstraction from the type of automation tool used and type of application being tested, e.g. – it can test a similar Web and Windows application with the same test case
• Hybrid framework – A hybrid framework is the one which takes advantages from both Data Driven and keyword driven frameworks. These frameworks do not implement generic keywords but implement business logic keywords based on the application being tested.

Support of different application types and versions -A framework should allow re-use of baselines scripts in case different versions/flavors of an applications to be tested. For e.g. An application that is available in iOS, Android, Web, Desktop, Api’s should be testable from a single platform.

Configurability – Configurable items like application, url, versions, paths, ip’s, etc. of a script should be kept in an external file. Once deployed to a system, no manual configuration changes should be required and scripts should automatically configure the required settings.

Maintain Object identification repository -Most common issues faced during automation are object identification changes. Framework should be able to patch such changes easily. This can be achieved by storing all object identification settings at a shared location in the form of external XML file, excel file, database or automation proprietary format.

Execution – Framework might need to cater to below requirements (on a needed bases)

• Execution of individual test cases
• Execution of a combination of tests based on environment, version, regression, build details, etc.
• Re-execution of only failed test cases
• Execution of a test case/test batch based on result of another test case/test batch

Status monitoring – A framework should allow monitoring of the execution status in real time and should be capable of sending alerts in case of failure. This ensures quick turnaround time in event of a failure.

[See Also: QA Automation Testing Services Company]

‍

Reporting – The framework should support html/Excel/Pdf report formats with details about test pass/fail for each test case/suite/test run.

Easy debugging -Debugging takes a lot of time during automation and hence special care needs to be taken for this part. Keyword driven frameworks which use external data source (like a excel spread sheet) to read scripts keywords and process the same are difficult to debug.

Logging – Log generation is important part of execution. It is very important to generate debug information at various points in a test case. This information can help find problem area quickly and reduce the time to make a fix at the same time.

Easy to Use and Run – The framework should be easy to learn and use. Framework should provide a one stop place to run the tests without involving too many changes.

Performance impacts – A framework should also consider the performance impacts of the implementation. A complex framework which increases the load time or execution time of scripts is never desirable. Techniques like caching, compiling all code into single library while execution, no hard sleeps etc. should be used to improve performance whenever possible.

Test Scripts and Test Data – Test scripts and test data should always be separated from each other. Test data input can be in various forms like XML, Json, Excel, text files, database inputs, hash maps etc.
Libraries – A library should contain all reusable components and external connections such as databases, generic functions, application functions etc. Software testers should be exposed only to the implemented libraries and tests should be performed by invoking these libraries.

Coding Standards – Scripting standards should always be maintained across the test automation framework, which will discourage individual coding practices and help in maintaining code uniformity, which makes it easier for software testers and developers to interpret.

Script/Framework Versioning – Versions of framework/scripts should be maintained either in a local repository or versioning tools like Github, which would help in easy monitoring of changes to the software code.

[See Also: The Importance of ServiceNow in Cloud Management]

Modular, re-usable and maintainable test code and data – Test code should always be modular so that it can be re-used. Also modularity makes sure that whenever there are changes in the existing features or new features need to be added, minimum effort is required by the engineers.  E.g. A reusable library can be created, which would help in enhancing application features with minimal effort.
Continuous integration – Framework should get the latest version of the test scripts and execute tests on latest builds of applications periodically. Jenkins like tools should be used to ease the process of continuous integration for deploying latest builds, running latest tests and getting results on a periodic fashion.

• Share test results – Test results should be shared in the form of emails with results attached in the form of an html, pdf, word, excel, etc. to collaborate with other teams.
• Environment independent – Framework should install all required software if run on any new machine other than where it was developed independent of its OS.
• Asserts – Framework should always have properly configured Asserts written in modular functions.

Automation should be considered as a development project and not just record and playback of events. Starting automated testing with a good framework ensures low maintenance.

Everything you need to know about outsourcing technology development

Access a special Introduction Package with everything you want to know about outsourcing your technology development, qa automation testing services company. How should you evaluate a partner? What components of your solution that are suitable to be handed off to a partner? These answers and more below.

‍