Test Execution

What is Test Execution?

Test execution involves the scheduled execution of manual and automated tests to check the functionality, security and performance of software. Test cases must be fully defined, the test environment must be set up correctly and the test data must be up-to-date. Defects and deviations must be precisely documented and forwarded to the development team in order to identify potential problems at an early stage and rectify them before the software goes live or is delivered.

Challenges in Test Execution

Test execution brings with it various challenges that can occur during both manual and automated test execution. Overcoming these challenges effectively is crucial for the quality and reliability of the test results.

Inaccuracies in the specification: Unclear or vague wording in the test specification offers too much room for interpretation and leads to varying test results.
Management of manual and automated tests: The parallel management of manual and automated tests requires increased attention to ensure consistency and efficiency.
Different specifications: The differences in the specification of manual and automated test cases complicate their integration.
Designing automated and manual tests: It is a challenge to design automated tests in such a way that they can be performed manually and, conversely, to design manual tests in such a way that they can be automated quickly.

Talk to us now about your test execution challenges

Testing made easy

Test Execution with TestBench

iTORX - Assistant for manual test execution

Assistant for manual testing

TestBench offers the option of specifying tests in various ways: Prose text, textual test steps and formalized test design using Keyword Driven and Data Driven Test. This makes TestBench a powerful tool for test specification and test execution. To ensure optimal manual execution, the testers are supported by an assistant: “imbus Test assistant for Online and Remote eXecution” – iTORX for short – which is adapted to the needs of the testers and offers very good support for all types of specification. iTORX creates an overview, enables tests to be executed repeatedly in the same way and also guides through very precise and detailed test specifications. Test execution can be interrupted and resumed. Intermediate results can be saved in TestBench at any time so that they can be continued by other people.

Blog article about iTORX

Checking the results of test automation

In addition to supporting manual testing, iTORX can also be used to preview the specified tests and to load the test result logs of a test automation run. It is particularly useful that iTORX can also be used offline by the TestBench system, e.g. in special test environments. This offline use is license-free and can therefore be used on any number of systems by any number of users. Long-running, automated regression tests in particular can generate huge test result logs that can contain several hundred thousand calls to test steps. iTORX has special settings for these large logs that enable high-performance work despite the large volumes.

Automated test case, started in TestBench and executed in Robot Framework

Flexible integration of automation tools

TestBench supports manual and automated testing – if required also in mixed operation and with different test automation tools. The open interface enables the connection of almost all test automation tools, regardless of whether user interfaces such as Web, SAP or Java Swing are being tested or systems with real-time requirements such as HiL systems need to be controlled. The connection of the open source test framework Robot Framework is particularly well integrated.

More about Testautomation

Defect detection during test execution in iTORX

Fast documentation of observations and error recording

During the manual test execution or the analysis of the test automation result logs, it is possible to document all observations, whether as comments, screenshots or file attachments. If errors are found, they can be recorded directly on the associated test case or test step.

Management of test protocols and results

The logs and results of manually or automatically executed test cases are transferred to TestBench. Test protocols generated offline can be imported and are then managed in the same way. During automated execution, the execution times are automatically measured and also logged. During manual execution, this time measurement must be activated by the testers and can also be switched off completely.

Integration of CI/CD systems

In addition to the direct execution of automated tests, the tests can also be executed by continuous integration systems such as Jenkins or Atlassian Bamboo. Test instructions can be exported and test results imported via TestBench’s REST API. It is possible to set the associated tests to “running” status so that it is clear which tests are currently being executed. Predefined filters can be used during export so that specific test packages are executed. This allows you to control which tests are to be executed overnight or as regression tests for certain builds or sprints, for example.

All advantages at a glance

Efficient task assignment

Test managers can assign tasks quickly and easily, which avoids long coordination processes during test execution.

User-oriented interface

A convenient test execution view enables testers to familiarize themselves quickly and only see relevant tests and information.

Error recording and precise documentation

Errors can be recorded directly during manual test execution in iTORX. The built-in version management ensures that tests and test results are reliably documented, regardless of the version of a test specification or test run.

Clear instructions

The iTORX test execution assistant displays test instructions step by step, which is particularly helpful for manual testing.

Flexible usage

iTORX is part of TestBench and license-free! It enables both online and offline use, even if there is no connection to the TestBench system.

Seamless integration

Test results are automatically linked to the TestBench test cases and errors are synchronized with external defect management systems.

Flexible automation

Manual tests can be automated at any time, without prior specification. Automation results can be easily imported into the TestBench and reports for manual and automated tests can be created together.

Integration into CI/CD systems

During automatic builds and subsequent test runs, e.g. in Jenkins or Atlassian Bamboo, tests can be automatically retrieved from TestBench, executed and the results imported.