Chapter 9. Evaluate

Table of Contents

9.1. The Dashboard
9.1.1. Default Dashboard
9.1.2. Editing a Dashboard Report
9.1.3. Report Types
9.2. Reports
9.2.1. Predefined Reports
9.3. Test Runs
9.3.1. Action
9.3.2. Bulk Actions
9.3.3. Table Operations
9.3.4. The Test Run Report
9.3.5. Details Page
9.3.6. Changes
9.4. Test Case Results
9.4.1. Action
9.4.2. Bulk Actions
9.4.3. Table Operations
9.4.4. Single Test Case Results
9.5. Test Suite Results
9.5.1. Action
9.5.2. Table Operations
9.5.3. Single Test Suite Results
9.6. Issues
9.6.1. Overview Page
9.6.2. Action
9.6.3. Bulk Actions
9.6.4. Table Operations
9.6.5. Details Page
9.6.6. Issue Details (Creating a new Issue)
9.6.7. Link Issue

This chapter shows how data and results can be evaluated, that were gathered during testing. It explains how the dashboard works and how it can be configured, how reports are configured and rendered and how issues can be created in a connected issue management system.

9.1. The Dashboard

A dashboard is an overview page for configurable reports displayed as a chart. Dashboards can be individually configured and shared with or blocked for other users. Each user can configure any number of dashboards. An example of a dashboard is shown in Figure 9.1.

  • New adds a new report to the dashboard.
  • Save saves all changes to the dashboard and its reports.
  • Cancel discards all changes to the dashboard and its reports.
The “Dashboard” Page

Figure 9.1. The Dashboard Page


Users can switch between dashboard at any time. The drop-down list on the upper right page shows all available dashboards (private and shared by other users). Next to this drop down list are several icons:

  •   When selected, all users can use this dashboard.
  •   When selected, this dashboard can only be edited by the owner.
  •   If selected, marks this as the default dashboard.
  •   Creates a private copy of this dashboard.
  •   Deletes this dashboard and all of its reports.

The number of reports that should be shown in a row can be set with the numbered buttons (1 to 4) on the upper right. The currently selected number of reports per row is marked in orange. Reports can be moved to another position by dragging and dropping.

All dashboard reports have the same structure: they consist of a title bar and the area where the chart is displayed. The title bar contains the name of the report and several icons:

  •   Opens up a dialog to edit the report configuration.
  •   Downloads the report as a PNG file.
  •   Expands/collapses the report.
  •   Deletes the report.

9.1.1. Default Dashboard

Upon the first start of Klaros-Test­management, a default dashboard with three predefined reports is created. This dashboard is accessible by all users an is automatically displayed to each new user. Administrators can change the default dashboard by setting any other dashboard as the default dashboard.

[Note] Note

There is always only one default dashboard. If a new default dashboard is set, the previous default dashboard loses this status.

9.1.2. Editing a Dashboard Report

Clicking the icon opens up a dialog that shows all configurable parameters of the report as well as the report type (which cannot be changed). Changes can be applied to the report by clicking the OK button or discarded by clicking the Cancel button.

The “Configure Report” Dialog

Figure 9.2. The Configure Report Dialog


9.1.3. Report Types

All reports initially show data from the currently selected project.

In addition to the data of the active project, data of specific projects can also be displayed in the Klaros-Test­management Enterprise Edition.

Klaros-Test­management Community Edition contains three report types:

  • Project Overview
  • Latest Success Rate
  • Test Activity

The Klaros-Test­management Enterprise Edition also contains four additional report types:

  • System under Test Overview
  • Test Environment Overview
  • Test Progress History
  • Test Progress

These reports are described in detail in the following sections.

9.1.3.1. The Project Overview Report

The Project Overview report shows several metrics of a single project in a chart.

The “Project Overview” Report

Figure 9.3. The Project Overview Report


This report displays the following metrics:

  • The number of executed/not executed test cases.
  • The number of executed/not executed test suites.
  • The number of planned/executed jobs.

The following properties of the Project Overview report diagram are configurable:

  • The name of the report diagram.
  • The selection of whether to use the active project or another one in this report. (Klaros-Test­management Enterprise Edition only).
  • The selection of whether to use the active iteration or another one in this report.
  • If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.2. The Latest Success Rate Report

The Last Success Rate report shows the latest results of executed test cases for the selected combination of system under test (one or more) and test environment (one or more).

The “Latest Success Rate” Report

Figure 9.4. The Latest Success Rate Report


The following properties of the Latest Success Rate report diagram are configurable:

  • One or more systems under test.
  • One or more test environments.
  • The name of the report diagram.
  • The selection of whether to use the active project or another one in this report. (Klaros-Test­management Enterprise Edition only).
  • If the active project is not used, the following objects to be viewed can be selected:
  • The selection of whether to use the active iteration or another one in this report.
  • If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.3. The Test Activity Report

The Test Activity Report shows the test case results for a selected combination of system under test and test environment in a selected period of time as a histogram.

The “Test Activity” Report

Figure 9.5. The Test Activity Report


The following properties of the Test Activity report diagram are configurable:

  • Project
  • Iteration (if available)
  • One or more systems under test.
  • One or more test environments.
  • The name of the report diagram.
  • The selection of whether to use the active project or another one in this report. (Klaros-Test­management Enterprise Edition only).
  • If the active project is not used, the following objects to be viewed can be selected:
  • The selection of whether to use the active iteration or another one in this report.
  • If the active iteration is not used, a dropdown list with the available iterations is displayed here.
  • The time period to be covered in days.

9.1.3.4. The System under Test Overview Report

The System under Test Overview Report shows the success and progress rate of test environments under a single system under test in a radar chart. If fewer than three test environments are configured, a bar chart will be displayed instead.

The “System under Test Overview” Report

Figure 9.6. The System under Test Overview Report


The following properties of the System under Test Overview report diagram are configurable:

  • The Project.
  • Iteration (if available).
  • A system under test.
  • One or more test environments.
  • The name of the report diagram.
  • The selection of whether to use the active project or another one in this report. (Klaros-Test­management Enterprise Edition only).
  • If the active project is not used, the following objects to be viewed can be selected:
  • The selection of whether to use the active iteration or another one in this report.
  • If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.5. The Test Environment Overview Report

The Test Environment Overview Report shows the success and progress rate of systems under test under a single test environment in a radar chart. If fewer than three systems under test are configured, a bar chart will be displayed instead.

The “Test Environment Overview” Report

Figure 9.7. The Test Environment Overview Report


The following properties of the Test Environment Overview report diagram are configurable:

  • Project
  • Iteration (if available)
  • One or more systems under test.
  • A test environment.
  • The name of the report diagram.
  • The selection of whether to use the active project or another one in this report. (Klaros-Test­management Enterprise Edition only).
  • If the active project is not used, the following objects to be viewed can be selected:
  • The selection of whether to use the active iteration or another one in this report.
  • If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.6. The Test Progress Report

The Test Progress report shows the rate of executed versus defined test cases of a project for a given set of test environments and test systems.

The “Test Progress” Report

Figure 9.8. The Test Progress Report


The following properties of the Test Progress report diagram are configurable:

  • Project
  • Iteration (if available).
  • One or more systems under test.
  • One or more test environments.
  • The name of the report diagram.
  • The selection of whether to use the active project or another one in this report. (Klaros-Test­management Enterprise Edition only).
  • If the active project is not used, the following objects to be viewed can be selected:
  • The selection of whether to use the active iteration or another one in this report.
  • If the active iteration is not used, a dropdown list with the available iterations is displayed here.
  • A boolean value indicating that only test cases should be considered that are part of a job.

9.1.3.7. The Test Progress History Report

The Test History report shows the rate of defined test cases versus executed test cases for a project for one or more test environments and one or more systems under test in a given time period.

The “Test Progress History” Report

Figure 9.9. The Test Progress History Report


The following properties of the Test Progress History report diagram are configurable:

  • Project
  • Iteration (if available).
  • One or more systems under test.
  • One or more test environments.
  • The name of the report diagram.
  • The selection of whether to use the active project or another one in this report. (Klaros-Test­management Enterprise Edition only).
  • If the active project is not used, the following objects to be viewed can be selected:
  • The selection of whether to use the active iteration or another one in this report.
  • If the active iteration is not used, a dropdown list with the available iterations is displayed here.
  • The time period in days