Test execution involves the scheduled execution of manual and automated tests to check the functionality, security and performance of software. Test cases must be fully defined, the test environment must be set up correctly and the test data must be up-to-date. Defects and deviations must be precisely documented and forwarded to the development team in order to identify potential problems at an early stage and rectify them before the software goes live or is delivered.
Test Execution
What is test execution?
Challenges in test execution
Test execution brings with it various challenges that can occur during both manual and automated test execution. Overcoming these challenges effectively is crucial for the quality and reliability of the test results.
Inaccuracies in the specification: Unclear or vague wording in the test specification offers too much room for interpretation and leads to varying test results.
Management of manual and automated tests: The parallel management of manual and automated tests requires increased attention to ensure consistency and efficiency.
Different specifications: The differences in the specification of manual and automated test cases complicate their integration.
Designing automated and manual tests: It is a challenge to design automated tests in such a way that they can be performed manually and, conversely, to design manual tests in such a way that they can be automated quickly.
Testing made easy
Test Execution with TestBench
Assistant for manual testing
TestBench offers the option of specifying tests in various ways: Prose text, textual test steps and formalized test design using Keyword Driven and Data Driven Test. This makes TestBench a powerful tool for test specification and test execution. To ensure optimal manual execution, the testers are supported by an assistant: “imbus Test assistant for Online and Remote eXecution” – iTORX for short – which is adapted to the needs of the testers and offers very good support for all types of specification. iTORX creates an overview, enables tests to be executed repeatedly in the same way and also guides through very precise and detailed test specifications. Test execution can be interrupted and resumed. Intermediate results can be saved in TestBench at any time so that they can be continued by other people.
Checking the results of test automation
In addition to supporting manual testing, iTORX can also be used to preview the specified tests and to load the test result logs of a test automation run. It is particularly useful that iTORX can also be used offline by the TestBench system, e.g. in special test environments. This offline use is license-free and can therefore be used on any number of systems by any number of users. Long-running, automated regression tests in particular can generate huge test result logs that can contain several hundred thousand calls to test steps. iTORX has special settings for these large logs that enable high-performance work despite the large volumes.
Flexible integration of automation tools
TestBench supports manual and automated testing – if required also in mixed operation and with different test automation tools. The open interface enables the connection of almost all test automation tools, regardless of whether user interfaces such as Web, SAP or Java Swing are being tested or systems with real-time requirements such as HiL systems need to be controlled. The connection of the open source test framework Robot Framework is particularly well integrated.
Fast documentation of observations and error recording
During the manual test execution or the analysis of the test automation result logs, it is possible to document all observations, whether as comments, screenshots or file attachments. If errors are found, they can be recorded directly on the associated test case or test step.
Management of test protocols and results
The logs and results of manually or automatically executed test cases are transferred to TestBench. Test protocols generated offline can be imported and are then managed in the same way. During automated execution, the execution times are automatically measured and also logged. During manual execution, this time measurement must be activated by the testers and can also be switched off completely.
Integration of CI/CD systems
In addition to the direct execution of automated tests, the tests can also be executed by continuous integration systems such as Jenkins or Atlassian Bamboo. Test instructions can be exported and test results imported via TestBench’s REST API. It is possible to set the associated tests to “running” status so that it is clear which tests are currently being executed. Predefined filters can be used during export so that specific test packages are executed. This allows you to control which tests are to be executed overnight or as regression tests for certain builds or sprints, for example.
All advantages at a glance
Efficient task assignment
Test managers can assign tasks quickly and easily, which avoids long coordination processes during test execution.
User-oriented interface
A convenient test execution view enables testers to familiarize themselves quickly and only see relevant tests and information.
Error recording and precise documentation
Errors can be recorded directly during manual test execution in iTORX. The built-in version management ensures that tests and test results are reliably documented, regardless of the version of a test specification or test run.
Clear instructions
The iTORX test execution assistant displays test instructions step by step, which is particularly helpful for manual testing.
Flexible usage
iTORX is part of TestBench and license-free! It enables both online and offline use, even if there is no connection to the TestBench system.
Seamless integration
Test results are automatically linked to the TestBench test cases and errors are synchronized with external defect management systems.
Flexible automation
Manual tests can be automated at any time, without prior specification. Automation results can be easily imported into the TestBench and reports for manual and automated tests can be created together.
Integration into CI/CD systems
During automatic builds and subsequent test runs, e.g. in Jenkins or Atlassian Bamboo, tests can be automatically retrieved from TestBench, executed and the results imported.
Your questions - our answers
Do you have questions about test execution or are you looking for support in optimizing your test processes? Our experts will be happy to help you. Contact us and we will help you to design your test strategies efficiently and successfully