| Chapter 7. Evaluate | ||
|---|---|---|
 |  |
|
| Chapter 7. Evaluate | ||
|---|---|---|
| | |
|
Table of Contents
This chapter describes how Klaros-Testmanagement can be used to gather information about the testing project, and evaluate the data about it.
The reports section provides a dashboard, a page for inspecting single test runs and a page for generating a test run history report.
The dashboard shows basic statistics about the active project and provides the links to the overview reports. An example of a dashboard is shown in Figure 7.1 .
The upper part of the dashboard shows a table with the available overview reports. There following three types of reports available:
Test Environment Overview
The Test Environment Overview report shows the test runs that have been executed in each test environment containing the execution date and quantity of success, failure or error result. An example of the report is shown in Figure 7.2 .
SUT Overview
The SUT Overview report shows the test runs that have been executed for a SUT containing the execution date and quantity of success, failure or error result. An example of the report is is shown in Figure 7.3 .
Test Suite Overview
The Test Suite Overview report shows the test plans containing its test cases, description and and how many times it was executed. An example report is shown in Figure 7.4 .
The reports can be generated in five different file formats:
HTML
CSV
RTF
XML
By clicking on an icon of a file type, the report will be generated in the selected format.
The dashboard is a container for multiple reports. Each report has the same
structure:
It consists of a title bar and an area in which the report is displayed.
The title bar of
the report contains the reports' name, and an edit button (
).
When the edit
icon is clicked, the parameters of the report can be configured in the
following dialog.
The changes in the configuration can be applied to the report by
pressing the
button or discarded by pressing the
button. Additionally the name of the report can be
configured by clicking on the name
in the title bar of the report.
The configured dashboard can be persisted by clicking the button in the upper right corner of the dashboard, if the button is pressed, the changes in the configuration of the dashboard are discarded.
All reports are showing the data of the correctly selected project.
The Klaros-Testmanagement Community Edition comes with three predefined reports:
The Project Overview Report
The Latest Success Rate Report
The Test Activity Report
The reports are described in the following sections.
In addition the Klaros-Testmanagement Enterprise Edition contains three additional reports:
The Project Health Report
The Test Progress History Report
The Test Progress Report
The reports are described in the following sections.
The Project Overview Report shows the main properties of a project listed in a table. The Report shows
The number of defined systems under test
The number of defined test environments
The number of test cases
The number of test suites
The number of test case results
The number of test suite results
The average of the count of the test case steps
The average of the test case execution time
The only configurable property of the report is its display name.
The Latest Success Rate shows the ratio of the newest test results that have been run for a selected combination of system under test and test environment.
The configurable properties of the report are:
The display name of the report.
A boolean value indicating that this report always uses the active project (Klaros-Testmanagement Enterprise Edition only).
Alternativly, the fixed project that this report will display.
The system under test.
The test environment.
The Test Activity shows the count and the results of the test runs for a selected combination of system under test and test environment in a selected period of time as a histogram.
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project (Klaros-Testmanagement Enterprise Edition only).
Alternativly, the fixed project that this report will display.
The system under test.
The test environment.
The time period in days.
The Project Health Matrix Report shows the project health as weather icons for each defined test environment and system under test in a matrix. The icons are calculated from the execution and success rate in each category.
![[Important]](images/important.png) | Klaros-Testmanagement Enterprise Edition Feature |
|---|---|
|
This feature is only available in the Klaros-Testmanagement Enterprise Edition. |
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project.
Alternativly, the fixed project that this report will display.
The execution rates assigned to the different health categories.
The success rates assigned to the different health categories.
The Test Progress Report shows the rate of executed vs. defined tests of a project for a given test environment and a system under test.
| Klaros-Testmanagement Enterprise Edition Feature |
|---|---|
|
This feature is only available in the Klaros-Testmanagement Enterprise Edition. |
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project
Alternativly, the fixed project that this report will display.
The system under test.
The test environment.
The Test Progress History Report shows the rate of executed vs defined tests of a project for a given test environment, system under test and time period.
| Klaros-Testmanagement Enterprise Edition Feature |
|---|---|
|
This feature is only available in the Klaros-Testmanagement Enterprise Edition. |
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project.
Alternativly, the fixed project that this report will display.
The system under test.
The test environment.
The time period of the report
| Klaros-Testmanagement Enterprise Edition Feature |
|---|---|
|
This feature is only available in the Klaros-Testmanagement Enterprise Edition. |
The Report Templates screen allows to generate the reports to PDF. The
icon starts the generation and opens (if necessary) a new window to enter the needed
parameters.
How to create
user defined reports is explained in the section
Section 8.2, “
Report Templates
”
.
If the user defined report is parameterized, all needed parameters must be entered to start the generation of the report.
![[Note]](images/note.png) | Note |
|---|---|
|
The button gets visible after entering all needed parameters. |
The
Single Test Run Report
screen allows to generate a report
containing the
result of the
test run selected by clicking on
the
Action icon.
(
Figure 7.13
).
The Single Test Run Report page lists all test runs that have been made in the active project. To get a better overview about the test runs the sorting and searching functionality of the Filter / Sort panel can be used.
It is possible to search for reports of single test runs and sort the results of the search with the filtering and sorting options. The filtering and sorting options become visible by opening the Filter / Sort panel. The Filter / Sort panel contains two tables. The left table contains the filtering parameters, the right table contains the sorting parameters.
Below the two tables you find two buttons:
|
|
The button applies the filter and sorting criteria specified in the two tables above and presents the filtered and sorted result in the table below. |
|
|
The button clears the tables with the filter and sorting criteria and displays all reports of single test runs. |
Each line in the filter table represents a criterion that restricts the result set that is displayed in the table below, i.e. the set of all Reports of Single Test Runs is filtered for the conjunction of all criteria listed in the filtering table.
By
clicking the
button, a new and empty row for a filter criterion is appended to the table. The
table with the
filtering criteria has
four columns:
The Field column denotes the field in the table that the criterion is filtering for. In the Filter / Sort panel of the Single Test Run Reports page are four fields that can be filtered: ID, Test Environment, System Under Test and Timestamp.
The Type column denotes the operator that is used for the criterion. In the Single Test Run Reports page the following operators are available:
|
Equals |
Only the items that exactly match the value of the criterion are shown. |
|
Not equals |
The items that do not equal the value of the criterion are shown in the result table. |
|
Greater than or equals |
Items that are greater than or equal the value of the criterion are shown. |
|
Less than or equals |
Items that are less than or equal the value of the criterion are shown. |
|
Greater than |
Items that are greater than the value of the criterion are shown. |
|
Less than |
Items that are less than the value of the criterion are shown. |
|
Like |
Items that have a part that matches value of the criterion are shown. |
The Value column defines the value for the filtering criterion.
The
Action
column contains a button (
)
for deleting the criterion in the current line.
The table that contains the list of test results can be sorted. The criteria for the sorting are specified in the right table in the Filter / Sort panel. It is possible to specify more than one sorting option. If there are more than one sorting options the uppermost sorting criterion has the highest priority, lowest row in the sorting criterion table has the lowest priority.
When the
button is clicked, a new empty row for a sorting criterion is appended at the
bottom of the
sorting options table. Each row in the sorting options table has in
the
Action
column a
button (
).
By pressing this button the row with the sorting criterion
can be removed.
The sorting options table contains three columns:
The Order By column denotes the field in the results table that is sorted for. The fields that can be sorted for in the Single Test Run Reports screen are ID, Test Environment and System under Test.
The Type column defines the direction of the sorting sequence, i.e. descending or ascending.
The Action column contains a button for deleting the sorting criterion.
The report is generated in PDF format and its layout is shown in Figure 7.14 : The report details: the date of creation of test run and its result, specifying the number of test cases executed with success, failure and error. Additionaly a pie chart is shown summarizing the result.
The Test Run History screen allows to generate a report containing a summary with two result history graphics and test run details of each project, with the run ID, date of test execution , total of test cases and number of success, failure and error results ( Figure 7.15 ).
Klaros-Testmanagement can export these test run report in five different types:
HTML
CSV
RTF
XML
The type can be selected in field Choose File Type . The start date and end date can be selected using the calendar component in Figure 7.15 .
The test run history can be generated by clicking the button.
The report is generated and its layout is shown in Figure 7.16 . The report details the date of creation of each test run and its results, specifying the number of test cases executed with success, failure and error. Two line charts visualize these results.
| | |
|
6.4.
Creating an issue
|  | 7.2.
Test Results
|
