Chapter 9. Evaluate

Table of Contents

This chapter shows how data and results can be evaluated, that were gathered during testing. It explains how the dashboard works and how it can be configured, how reports are configured and rendered and how issues can be created in a connected issue management system.

9.1. The Dashboard

9.1.1. Default Dashboard
9.1.2. Editing a Dashboard Report
9.1.3. Report Types

A dashboard is an overview page for configurable reports displayed as a chart. Dashboards can be individually configured and shared with or blocked for other users. Each user can configure any number of dashboards. An example of a dashboard is shown in Figure 9.1.

New adds a new report to the dashboard.
Save saves all changes to the dashboard and its reports.
Cancel discards all changes to the dashboard and its reports.

Figure 9.1. The “Dashboard” Page

Users can switch between dashboard at any time. The drop-down list on the upper right page shows all available dashboards (private and shared by other users). Next to this drop down list are several icons:

When selected, all users can use this dashboard.
When selected, this dashboard can only be edited by the owner.
If selected, marks this as the default dashboard.
Creates a private copy of this dashboard.
Deletes this dashboard and all of its reports.

The number of reports that should be shown in a row can be set with the numbered buttons (1 to 4) on the upper right. The currently selected number of reports per row is marked in orange. Reports can be moved to another position by dragging and dropping.

All dashboard reports have the same structure: they consist of a title bar and the area where the chart is displayed. The title bar contains the name of the report and several icons:

Opens up a dialog to edit the report configuration.
Downloads the report as a PNG file.
Expands/collapses the report.
Deletes the report.

9.1.1. Default Dashboard

Upon the first start of Klaros-Testmanagement, a default dashboard with three predefined reports is created. This dashboard is accessible by all users an is automatically displayed to each new user. Administrators can change the default dashboard by setting any other dashboard as the default dashboard.

	Note
	There is always only one default dashboard. If a new default dashboard is set, the previous default dashboard loses this status.

9.1.2. Editing a Dashboard Report

Clicking the icon opens up a dialog that shows all configurable parameters of the report as well as the report type (which cannot be changed). Changes can be applied to the report by clicking the OK button or discarded by clicking the Cancel button.

Figure 9.2. The “Configure Report” Dialog

9.1.3. Report Types

9.1.3.1. The “Project Overview” Report
9.1.3.2. The “Latest Success Rate” Report
9.1.3.3. The “Test Activity” Report
9.1.3.4. The “System under Test Overview” Report
9.1.3.5. The “Test Environment Overview” Report
9.1.3.6. The “Test Progress” Report
9.1.3.7. The “Test Progress History” Report

All reports initially show data from the currently selected project.

In addition to the data of the active project, data of specific projects can also be displayed in the Klaros-Testmanagement Enterprise Edition.

Klaros-Testmanagement Community Edition contains three report types:

Project Overview
Latest Success Rate
Test Activity

The Klaros-Testmanagement Enterprise Edition also contains four additional report types:

System under Test Overview
Test Environment Overview
Test Progress History
Test Progress

These reports are described in detail in the following sections.

9.1.3.1. The “Project Overview” Report

The Project Overview report shows several metrics of a single project in a chart.

Figure 9.3. The “Project Overview” Report

This report displays the following metrics:

The number of executed/not executed test cases.
The number of executed/not executed test suites.
The number of planned/executed jobs.

The following properties of the “Project Overview” report diagram are configurable:

The name of the report diagram.
The selection of whether to use the active project or another one in this report. (Klaros-Testmanagement Enterprise Edition only).
The selection of whether to use the active iteration or another one in this report.
If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.2. The “Latest Success Rate” Report

The Last Success Rate report shows the latest results of executed test cases for the selected combination of system under test (one or more) and test environment (one or more).

Figure 9.4. The “Latest Success Rate” Report

The following properties of the “Latest Success Rate” report diagram are configurable:

One or more systems under test.
One or more test environments.

The name of the report diagram.
The selection of whether to use the active project or another one in this report. (Klaros-Testmanagement Enterprise Edition only).
If the active project is not used, the following objects to be viewed can be selected:
The selection of whether to use the active iteration or another one in this report.
If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.3. The “Test Activity” Report

The Test Activity Report shows the test case results for a selected combination of system under test and test environment in a selected period of time as a histogram.

Figure 9.5. The “Test Activity” Report

The following properties of the “Test Activity” report diagram are configurable:

Project
Iteration (if available)
One or more systems under test.
One or more test environments.

The name of the report diagram.
The selection of whether to use the active project or another one in this report. (Klaros-Testmanagement Enterprise Edition only).
If the active project is not used, the following objects to be viewed can be selected:
The selection of whether to use the active iteration or another one in this report.
If the active iteration is not used, a dropdown list with the available iterations is displayed here.
The time period to be covered in days.

9.1.3.4. The “System under Test Overview” Report

Only available in Klaros-Testmanagement Enterprise Edition

The System under Test Overview Report shows the success and progress rate of test environments under a single system under test in a radar chart. If fewer than three test environments are configured, a bar chart will be displayed instead.

Figure 9.6. The “System under Test Overview” Report

The following properties of the “System under Test Overview” report diagram are configurable:

The Project.
Iteration (if available).
A system under test.
One or more test environments.

The name of the report diagram.
The selection of whether to use the active project or another one in this report. (Klaros-Testmanagement Enterprise Edition only).
If the active project is not used, the following objects to be viewed can be selected:
The selection of whether to use the active iteration or another one in this report.
If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.5. The “Test Environment Overview” Report

Only available in Klaros-Testmanagement Enterprise Edition

The Test Environment Overview Report shows the success and progress rate of systems under test under a single test environment in a radar chart. If fewer than three systems under test are configured, a bar chart will be displayed instead.

Figure 9.7. The “Test Environment Overview” Report

The following properties of the “Test Environment Overview” report diagram are configurable:

Project
Iteration (if available)
One or more systems under test.
A test environment.

The name of the report diagram.
The selection of whether to use the active project or another one in this report. (Klaros-Testmanagement Enterprise Edition only).
If the active project is not used, the following objects to be viewed can be selected:
The selection of whether to use the active iteration or another one in this report.
If the active iteration is not used, a dropdown list with the available iterations is displayed here.

9.1.3.6. The “Test Progress” Report

Only available in Klaros-Testmanagement Enterprise Edition

The Test Progress report shows the rate of executed versus defined test cases of a project for a given set of test environments and test systems.

Figure 9.8. The “Test Progress” Report

The following properties of the “Test Progress” report diagram are configurable:

Project
Iteration (if available).
One or more systems under test.
One or more test environments.

The name of the report diagram.
The selection of whether to use the active project or another one in this report. (Klaros-Testmanagement Enterprise Edition only).
If the active project is not used, the following objects to be viewed can be selected:
The selection of whether to use the active iteration or another one in this report.
If the active iteration is not used, a dropdown list with the available iterations is displayed here.
A boolean value indicating that only test cases should be considered that are part of a job.

9.1.3.7. The “Test Progress History” Report

Only available in Klaros-Testmanagement Enterprise Edition

The Test History report shows the rate of defined test cases versus executed test cases for a project for one or more test environments and one or more systems under test in a given time period.

Figure 9.9. The “Test Progress History” Report

The following properties of the “Test Progress History” report diagram are configurable:

Project
Iteration (if available).
One or more systems under test.
One or more test environments.

The name of the report diagram.
The selection of whether to use the active project or another one in this report. (Klaros-Testmanagement Enterprise Edition only).
If the active project is not used, the following objects to be viewed can be selected:
The selection of whether to use the active iteration or another one in this report.
If the active iteration is not used, a dropdown list with the available iterations is displayed here.
The time period in days


8.5. Import Test Results		9.2. Reports