The automotive industry is characterized by constant innovation and the continuous pursuit of quality and efficiency. In this highly dynamic environment, companies such as the Marquardt Group are required to optimize their test processes in order to remain competitive. This article describes how Marquardt, in close cooperation with TestBench and imbus AG, has developed a groundbreaking solution for efficient test automation. 

The initial situation

The Marquardt Group is a renowned mechatronics expert. It is active in the automotive sector, among others. The company is characterised by its unique, intelligent solutions based on an overall architecture of mechanics, electronics and software platform. Marquardt manufactures products and systems that are subjected to comprehensive quality testing, whereby special quality assurance solutions are developed for various products.

The products tested are electrical components and systems for vehicles. They are tested using customised hardware-in-the-loop solutions from ibb testing GmbH based on VT standard components from Vector Informatik GmbH. Marquardt uses standardised test systems and individual test environments to connect the DUTs to the test systems.

Marquardt favoured the Keyword Driven Testing approach in conjunction with Data Driven Testing at an early stage. The test sequences were specified in the self-developed test framework based on Excel sheets. However, these were project- and team-specific, which significantly limited reusability – even though many similar or even identical test sequences existed in parallel projects. There was a lack of a centralised test management tool to efficiently track requirements that had already been tested and to enable the reuse of tests.

High maintenance and coordination costs were no longer acceptable

The automotive supply industry is fundamentally faced with the challenge that different customers want different equipment packages for a vehicle. Product variants of vehicle components are therefore not only driven by the evolution of vehicles, but also by the market itself.

Marquardt builds new products or projects on already established solutions, but it is not uncommon for further variants to emerge from these, which are then further developed independently. Variants are versions of a product that generally have a common basis but differ in terms of variant-specific properties or functions.
In this way, several independent sets of test cases are created, which have also been maintained independently of each other despite overlapping content.

 

Although these variants have a common basis, they differ in specific characteristics.

It was therefore known that the products had similar functions and code components in most cases, but that these could not be managed centrally due to minor differences in the product designs. Many tests were carried out manually, which was no longer sustainable with the planned expansion and the expected volume structure. A more efficient quality assurance solution and more comprehensive test automation was required for the future.

To this end, imbus’ test management was expanded in close consultation with Marquardt as a pilot customer to include the management of many product variants.

Our task

In a first step, imbus AG was commissioned by Marquardt with a proof of concept to enable a low-maintenance test specification for several product variants. Comprehensive test cases, which are relevant for several different variants, should only be specified once. This was to be done with the help of a cross-project, centrally controlled, database-based test management solution. In addition, a standardised, reusable and maintainable test automation architecture should be set up in order to exploit the as yet untapped potential of the Keyword Driven Testing approach.

The concept and realisation

imbus AG introduced TestBench as a central test management system in order to specify and manage test cases in a reusable and clearly structured manner. Together, a test automation architecture was defined on the basis of TestBench and with the help of the Test Execution Plugin, which is able to directly control HiL test benches. In addition, a central keyword library was provided for all test steps of the hardware-in-the-loop test benches.

In addition, imbus used the test management system to seamlessly support the entire process from requirements (managed in DOORS) to test design and test automation through to the actual test execution. Particular attention was paid to traceability to ensure that all requirements were validated and verified with error-free tests.

Moreover, imbus developed the TestBench function ‘Variant Management’ together with Marquardt as a pilot customer: A so-called basic test object version (‘basic TOV’) contains the union set of all variant-specific test cases. In conjunction with a suitable assignment of the test cases in the base TOV, the variant-specific test cases can be derived from this at any time.

Furthermore, the test automation architecture was designed in such a way that both the test design and the test automation itself have a modular structure and are therefore as maintainable as possible. All test cases, including the reusable test steps and test case data, are now stored centrally in a globally accessible database. For this purpose, the test design in TestBench is based on centrally provided keywords, which the test designers string together to form a test sequence.

These test sequences then control the automatic test execution – without media breaks and redundancies. The sequence control of the test sequences during automatic test execution takes place on the HiL test systems. These are controlled via a HiL backend implemented by Marquardt itself, which utilises the HiL solution based on Vector CANoe. The test cases are compiled into the scripting language of the HiL and the test scripts are distributed to the selected test benches and controlled. Once the test has been completed, all test results are saved in the HiL backend and stored in the central TestBench repository for analysis and permanent documentation.

The reuse of keywords across test cases and variant management minimises the effects of changes, which leads to very good maintainability of the entire test resources and test automation in particular. It makes sense to define naming conventions for test elements (data types, test steps) as well as a layer architecture for keywords and to make corresponding keywords available centrally across projects. This approach makes it easier to apply specified test cases to different test levels or test systems.

TestBench, was used to centrally manage all test resources (including test cases, test steps and test case data), including variant management. As part of the implementation, the test engineers at all Marquardt locations were trained to use this solution independently and appropriately.

The result

The solution is currently used with the following quantity structure: over 1.5 million test cases, more than 30 product families, up to 150,000 requirements per product, more than 120 test engineers working simultaneously worldwide and around 45 HiL test setups that are controlled directly from the toolchain. In addition, the developed solution was rolled out to all product lines and all international Marquardt locations.

In addition to increasing the efficiency of testing efforts for the entire V&V area by around 10%, the new solution supports the coordination of the globally distributed teams. This is made possible by the central repository, in which all test resources, including links to the requirements, are managed without media disruptions and redundancies.

The Outlook

The DOORS requirements management tool currently in use is to be replaced in the future. Marquardt is currently evaluating possible successors for this. The new requirements management tool is also to be linked to TestBench in the future, for example, in order to continue exchanging requirements and test cases in the background or at the touch of a button.

Marquardt and imbus are currently developing a training course based on the blended learning concept with the aim of enabling new employees to familiarise themselves with the basics of TestBench via self-learning units.

Work with TestBench is to be further automated for users. To this end, the toolchain will be expanded and additional tools integrated to avoid manual imports and exports. This automation is done via the TestBench API, using the new features of TestBench 3.0.