Chapter 9. Evaluate

Table of Contents

9.1. Dashboard
9.1.1. The Overview Reports
9.1.2. The Dashboard Reports
9.2. Report Templates
9.3. Test Run History Report
9.4. Test Runs
9.4.1. The Test Run Report
9.5. Test Case Results
9.5.1. Single Test Case Results
9.6. Test Suite Results
9.6.1. Single Test Suite Results
9.7. Viewing Reasons of Skipped Test Cases
9.8. Issues
9.8.1. Issues by Test Case
9.8.2. Issue Details (Creating a new Issue)
9.8.3. Link Issues
9.9. Bookmarking Results

This chapter describes how Klaros-Testmanagement can be used to gather information about the selected project and evaluate its data.

9.1. Dashboard

The dashboard shows basic statistics about the active project and provides the links to the overview reports. An example of a dashboard is shown in Figure 9.1.

The Dashboard Screen

Figure 9.1. The Dashboard Screen


9.1.1. The Overview Reports

The upper part of the dashboard shows a table with the available overview reports. The following three types of reports are available:

  • Test Environment Overview

    The Test Environment Overview report shows the test runs that have been executed in each test environment containing the execution date and quantity of success, failure and error results. An example of the report is shown in Figure 9.2.

    The Test Environment Overview Report Layout

    Figure 9.2. The Test Environment Overview Report Layout


  • SUT Overview

    The SUT Overview report shows the test runs that have been executed for a system under test containing the execution date and quantity of success, failure and error results. An example of the report is shown in Figure 9.3.

    The SUT Overview Report Layout

    Figure 9.3. The SUT Overview Report Layout


  • Test Suite Overview

    The Test Suite Overview report shows the selected test suite, containing its test cases, description, how many times it was executed and with what results. An example report is shown in Figure 9.4.

    The Test Suite Overview Report Layout

    Figure 9.4. The Test Suite Overview Report Layout


The reports can be generated in three different file formats:

  • PDF

  • HTML

  • CSV

Clicking on a file type icon generates the report will in the selected format.

9.1.2. The Dashboard Reports

The dashboard is a container for multiple reports. Each report has the same structure: It consists of a title bar and an area in which the report is displayed. The title bar of the report contains the reports name, and an edit button ( ). When the edit button is clicked, the parameters of the report can be configured in the resulting dialog. The changes in the configuration can be applied to the report by clicking the Apply button or discarded by clicking the Cancel button. Additionally the name of the report can be configured by clicking on the name in the title bar of the report.

The configured dashboard can be persisted by clicking the Save button in the upper right corner of the dashboard. If the Cancel button is pressed, the changes in the configuration of the dashboard are discarded.

All reports can show data from the currently selected project.

In addition to the above, reports in the Klaros-Testmanagement Enterprise Edition can display data from non-active projects, selected in the report settings.

Klaros-Testmanagement Community Edition comes with three predefined reports:

  • The Project Overview Report

  • The Latest Success Rate Report

  • The Test Activity Report

These reports are described in the following sections.

In addition, Klaros-Testmanagement Enterprise Edition contains three additional reports:

  • The Project Health Report

  • The Test Progress History Report

  • The Test Progress Report

These reports are also described in the following sections.

9.1.2.1. The Project Overview Report

The Project Overview Report shows the main properties of a project listed in a table. The report shows:

  • The number of defined systems under test

  • The number of defined test environments

  • The number of test cases

  • The number of test suites

  • The number of test case results

  • The number of test suite results

  • The average of the number of test case steps

  • The average of the test case execution times

The only configurable property of the report is its display name.

The Project Overview Report

Figure 9.5. The Project Overview Report


9.1.2.2. The Latest Success Rate Report

The Latest Success Rate report shows the ratio of the newest test runs for a selected combination of system under test and test environment.

The configurable properties of the report are:

  • The display name of the report.

  • A boolean value indicating that this report always uses the active project (Klaros-Testmanagement Enterprise Edition only).

  • Alternatively, the fixed project that this report will display.

  • A boolean value indicating that this report always uses the active iteration (if available).

  • Alternatively, the fixed iteration that this report will display (if available).

  • The system under test to display.

  • The test environment to display.

The Latest Success Rate Report

Figure 9.6. The Latest Success Rate Report


9.1.2.3. The Test Activity Report

The Test Activity Report shows the count and the results of the test runs for a selected combination of system under test and test environment in a selected period of time as a histogram.

The configurable properties of the report are:

  • The name of the report.

  • A boolean value indicating that this report always uses the active project (Klaros-Testmanagement Enterprise Edition only).

  • Alternatively, the fixed project that this report will display.

  • A boolean value indicating that this report always uses the active iteration (if available).

  • Alternatively, the fixed iteration that this report will display (if available).

  • The system under test to display.

  • The test environment to display.

  • The time period in days.

The Test Activity Report

Figure 9.7. The Test Activity Report


9.1.2.4. The Project Health Matrix Report

The Project Health Matrix Report shows the project health as weather icons for each defined test environment and system under test in a matrix. The icons are calculated from the execution and success rate in each category which is shown in detail in a tool tip of each icon.

The configurable properties of the report are:

  • The name of the report.

  • A boolean value indicating that this report always uses the active project.

  • A boolean value indicating that this report always uses the active iteration (if available).

  • Alternatively, the fixed iteration that this report will display (if available).

  • Alternatively, the fixed project that this report will display.

  • The execution rates assigned to the different health categories.

  • The success rates assigned to the different health categories.

The Project Health Matrix Report

Figure 9.8. The Project Health Matrix Report


9.1.2.5. The Test Progress Report

The Test Progress Report shows the rate of executed vs. defined tests of a project for a given test environment and a system under test.

The configurable properties of the report are:

  • The name of the report.

  • A boolean value indicating that this report always uses the active project.

  • Alternatively, the fixed project that this report will display.

  • A boolean value indicating that this report always uses the active iteration (if available).

  • Alternatively, the fixed iteration that this report will display (if available).

  • The system under test to display.

  • The test environment to display.

The Test Progress Report

Figure 9.9. The Test Progress Report


9.1.2.6. The Test Progress History Report

The Test Progress History Report shows the rate of executed vs. defined tests of a project for a given test environment, system under test and time period.

The configurable properties of the report are:

  • The name of the report.

  • A boolean value indicating that this report always uses the active project.

  • Alternatively, the fixed project that this report will display.

  • A boolean value indicating that this report always uses the active iteration (if available).

  • Alternatively, the fixed iteration that this report will display (if available).

  • The system under test to display.

  • The test environment to display.

  • The time period of the report.

The Test Progress History Report

Figure 9.10. The Test Progress History Report