- What is Data Driven Testing?
- Challenges in managing and using test data
- The advantages of Data Driven Testing
- When is Data Driven Testing recommended?
- How does Data Driven Testing work with TestBench?
- How is the test data managed?
- How can tests become even easier to understand?
- Advantages of Data Driven Testing in Manual Testing
Data Driven Testing
Let the data do all the tests
What is Data Driven Testing?
Data Driven Testing (DDT) is a test method in which test cases are executed with different test data. Instead of creating a separate test case for each data combination, DDT uses generic test logic. This runs through identical test steps with different values.
The specific values for inputs and expected results are stored in a data table. Each row in the table corresponds to a test characteristic. This allows tests to be scaled efficiently, test coverage to be increased and redundancies to be avoided. New data combinations are quickly added or updated. Especially in test automation, DDT enables simple and flexible mapping of different scenarios.
Challenges in managing and using test data
Working with test data is associated with various problems, including data management, availability and quality.
Test data management and maintenance
Creating and updating test data is complex. To ensure reliable tests, data must always be up-to-date and representative of real use cases.
Dependencies between data
Some tests require specific combinations of data to generate certain states. If these dependencies are not carefully managed, inconsistent data can lead to delays or instability.
Data quality
Only high-quality test data provides meaningful test results. Incomplete or incorrect data can lead to real errors being overlooked or false error messages being generated.
Specifications for test automation
Test data should be defined independently of the test code so that changes can be made flexibly. Temporary test data that is only required for one test case must also be managed properly.
The advantages of Data Driven Testing
If the behaviour of a system varies with the same processes when using different data, then Data Driven Testing has a number of advantages:
- Higher test coverage – Separating test logic and test data allows many input values to be tested with the same test sequence. This allows different scenarios to be covered, including limit values and special cases.
- Less maintenance – Changes to the test data do not require any adjustments to the test procedure. New data sets can be easily added or updated without changing the test automation.
- Better error analysis – As each test run is logged with different data, error-prone values can be quickly identified. This makes it easier to analyse the cause and rectify errors in a targeted manner.
- Avoidance of redundant tests – A targeted selection of data combinations (e.g. via equivalence classes) avoids unnecessary repetitions. This saves time and reduces unnecessary complexity.
- More efficient test design – Centralised management of test data facilitates its reuse. Instead of defining values multiple times, structured data records can be used specifically for tests.
When is Data Driven Testing recommended?
Data Driven Testing is particularly effective when many data combinations need to be tested or test cases are reused. For data driven applications in particular, DDT reduces manual effort, improves test coverage and increases efficiency.
If test cases are heavily dependent on external test data, DDT enables simple management and scaling. The preparation of test data for inputs or special compilations, such as waybills or insurance forms, simplifies both specification and test execution.
Data Driven Testing is particularly powerful in combination with Keyword Driven Testing.
By default, tests in TestBench contain a data table in which the associated values of the test cases are stored. In the test definition, a parameter table determines which data types are required in the data table.
The data table contains the specific values of the data types that were defined in the parameter table. Each row contains the data for a test case that is used during test execution.
The values from the defined steps are used in the definition of the test case (see Keyword Driven Testing).
When managing the values, groups of values can be created that belong to the same context. For example, the list of valid vehicles and the list of invalid vehicles for e.g. negative tests. The underlying method is called equivalence class decomposition.
Data Driven Testing and Keyword Driven Testing are integrated in TestBench, as also provided for in ISO 29119-5. Details on Keyword Driven Testing can be found here.
This makes the associated values clearer and easier to understand.
The presentation of the data is also clearer and easier to understand in the test definitions and test cases.
If necessary, the inner values of the data type structure can be accessed via their path in the structure, e.g. the vehicle name via ‘VehicleConfiguration.Vehicle.VehicleName’, as shown in the illustration.
Advantages of Data Driven Testing in Manual Testing
Data-driven testing not only increases the efficiency of test automation, but also improves manual tests.
- Faster test execution – Repeated test sequences can be executed more quickly as they do not have to be re-read each time.
- Reduced susceptibility to errors – Instead of typing in values manually, existing data records can simply be run through. In the TestBench, data can also be pasted directly from the clipboard.
- Increased test efficiency – More scenarios can be covered by a targeted variation of test data – without additional manual effort.
Details on test execution can be found here and in the blog post ‘Less effort, more efficiency: the iTORX assistant’.
Your questions - our answers
Do you have questions about Data Driven Testing or are you looking for support in optimizing your testing processes? Our experts will be happy to help you. Contact us and we will help you to design your test strategies efficiently and successfully