| Chapter 9. Evaluate | ||
|---|---|---|
 |  |
|
| Chapter 9. Evaluate | ||
|---|---|---|
| | |
|
Table of Contents
This chapter describes how Klaros-Testmanagement can be used to gather information about the selected project and evaluate the data about it.
The reports section provides a dashboard, a page for creating user-defined reports (see Chapter 6, Define, only available in Klaros-Testmanagement Enterprise Edition), a page for inspecting single test runs and a page for generating a test run history report.
The dashboard shows basic statistics about the active project and provides the links to the overview reports. An example of a dashboard is shown in Figure 9.1 .
The upper part of the dashboard shows a table with the available overview reports. The following three types of reports are available:
Test Environment Overview
The Test Environment Overview report shows the test runs that have been executed in each test environment containing the execution date and quantity of success, failure or error result. An example of the report is shown in Figure 9.2 .
SUT Overview
The SUT Overview report shows the test runs that have been executed for a SUT containing the execution date and quantity of success, failure or error result. An example of the report is is shown in Figure 9.3 .
Test Suite Overview
The Test Suite Overview report shows the test suite, containing its test cases, description and how many times it was executed. An example report is shown in Figure 9.4 .
The reports can be generated in five different file formats:
HTML
CSV
RTF
XML
By clicking on an icon of a file type, the report will be generated in the selected format.
The dashboard is a container for multiple reports. Each report has the same
structure:
It consists of a title bar and an area in which the report is displayed.
The title bar of
the report contains the reports name, and an edit button (
).
When the edit
icon is clicked, the parameters of the report can be configured in the
resulting dialog.
The changes in the configuration can be applied to the report by
clicking the
button or discarded by clicking the
button. Additionally the name of the report can be
configured by clicking on the name
in the title bar of the report.
The configured dashboard can be persisted by clicking the button in the upper right corner of the dashboard, if the button is pressed, the changes in the configuration of the dashboard are discarded.
All reports show data from the correctly selected project.
Klaros-Testmanagement Community Edition comes with three predefined reports:
The Project Overview Report
The Latest Success Rate Report
The Test Activity Report
The reports are described in the following sections.
In addition, Klaros-Testmanagement Enterprise Edition contains three additional reports:
The Project Health Report
The Test Progress History Report
The Test Progress Report
The reports are described in the following sections.
The Project Overview Report shows the main properties of a project listed in a table. The Report shows
The number of defined systems under test
The number of defined test environments
The number of test cases
The number of test suites
The number of test case results
The number of test suite results
The average of the count of the test case steps
The average of the test case execution time
The only configurable property of the report is its display name.
The Latest Success Rate shows the ratio of the newest test results that have been run for a selected combination of system under test and test environment.
The configurable properties of the report are:
The display name of the report.
A boolean value indicating that this report always uses the active project (Klaros-Testmanagement Enterprise Edition only).
Alternatively, the fixed project that this report will display.
The system under test.
The test environment.
The Test Activity shows the count and the results of the test runs for a selected combination of system under test and test environment in a selected period of time as a histogram.
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project (Klaros-Testmanagement Enterprise Edition only).
Alternativly, the fixed project that this report will display.
The system under test.
The test environment.
The time period in days.
The Project Health Matrix Report shows the project health as weather icons for each defined test environment and system under test in a matrix. The icons are calculated from the execution and success rate in each category.
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project.
Alternativly, the fixed project that this report will display.
The execution rates assigned to the different health categories.
The success rates assigned to the different health categories.
The Test Progress Report shows the rate of executed vs. defined tests of a project for a given test environment and a system under test.
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project
Alternativly, the fixed project that this report will display.
The system under test.
The test environment.
The Test Progress History Report shows the rate of executed vs defined tests of a project for a given test environment, system under test and time period.
The configurable properties of the report are:
The name of the report.
A boolean value indicating that this report always uses the active project.
Alternativly, the fixed project that this report will display.
The system under test.
The test environment.
The time period of the report
The Report Templates screen allows to generate PDFs of user-defined reports. The
icon starts the generation and , if necessary, opens a new window to enter the needed
parameters.
The process of creating user-defined reports is explained in the section
Section 10.2, “
Report Templates
”
.
If the user-defined report is parameterized, all required parameters must be entered to start the generation of the report.
![[Note]](images/note.png) | Note |
|---|---|
|
The button becomes visible after entering all required parameters. |
The
Single Test Run Report
screen allows the user to generate a report
containing the
result of the
test run selected by clicking on
the
Action icon.
(
Figure 9.13
).
The Single Test Run Report page lists all test runs that have been made in the active project. To get a better overview of the test runs, the sorting and searching functionality of the Filter / Sort panel can be used.
It is possible to search for single test runs and sort the results of the search with the filtering and sorting options. The filtering and sorting options become visible by opening the Filter / Sort panel. The Filter / Sort panel contains two tables. The table on the left contains the filtering parameters, while the table on the right contains the sorting parameters.
Below the two tables are two buttons:
|
|
The button applies the filter and sorting criteria specified in the two tables above and presents the filtered and sorted result in the table below. |
|
|
The button clears the tables with the filter and sorting criteria and displays all projects. |
Each line in the filter table represents a criterion that restricts the result set that is displayed in the table below, i.e. the set of all single test runs is filtered for the conjunction of all criteria listed in the filtering table.
By
clicking the
button, a new and empty row for a filter criterion is appended to the table. The
table with the
filtering criteria has
four columns:
The Field column denotes the field in the table that the criterion is filtering for. The fields which can be filtered in the Filter / Sort panel of the single test run reports page are ID, Test Environment, System Under Test and Timestamp , as well as any user-defined properties. The process of maintaining the user-defined properties is described in section Section 6.1.1.8, “Editing the user-defined properties of a project” . User-defined properties are only available in Klaros-Testmanagement Enterprise Edition.
The Type column denotes the operator that is used for the criterion. The following operators are available on the single test run reports page:
|
Equals |
Only the items that exactly match the value of the criterion are shown. |
|
Not equals |
The items that do not equal the value of the criterion are shown in the result table. |
|
Greater than or equals |
Items that are greater than or equal the value of the criterion are shown. |
|
Less than or equals |
Items that are less than or equal the value of the criterion are shown. |
|
Greater than |
Items that are greater than the value of the criterion are shown. |
|
Less than |
Items that are less than the value of the criterion are shown. |
|
Like |
Items that have a part that matches value of the criterion are shown. |
The Value column defines the value for the filtering criterion.
The
Action
column contains a button (
)
for deleting the criterion in the current line.
The table that contains the list of single test runs can be sorted. The criteria for the sorting are specified in the right table in the Filter / Sort panel. It is possible to specify more than one sorting option. If there is more than one sorting option, the uppermost row in the sorting criterion table has the highest priority and the lowest row has the lowest priority.
When the
;
button is clicked, a new empty row for a sorting criterion is appended at the
bottom
of the
sorting options table. Each row in the sorting options table has in
the
Action
column a
button (
).
By pressing this button the row with the sorting criterion
can be removed.
The sorting options table contains three columns:
The Order By column denotes the field in the results table that the sorting applies to. The fields that can be sorted for on the single test run reports page are ID, Test Environment, System Under Test and Timestamp , as well as any user-defined properties. The process of maintaining the user-defined properties is described in section Section 6.1.1.8, “Editing the user-defined properties of a project” . User-defined properties are only available in Klaros-Testmanagement Enterprise Edition.
The Type column defines the direction of the sorting sequence, i.e. descending or ascending.
The Action column contains a button for deleting the sorting criterion.
The report is generated in PDF format and its layout is shown in Figure 9.14 . The report details the date of creation of test run and its result, specifying the number of test cases executed with success, failure and error. Additionaly a pie chart is shown summarizing the result.
The Test Run History screen allows to generate a report containing a summary with two result history graphics and test run details of each project, with the run ID, date of test execution , total of test cases and number of success, failure and error results ( Figure 9.15 ).
Klaros-Testmanagement can export these test run report in five different types:
HTML
CSV
RTF
XML
The type can be selected in field Choose File Type . The start date and end date can be selected using the calendar component in Figure 9.15 .
The test run history can be generated by clicking the button.
The report's layout is shown in Figure 9.16 . The report details the date of execution of each test run and its results, specifying the number of test cases executed with success, failure and error. Two line charts visualize these results.
| | |
|
8.4.
Creating an issue
|  | 9.2.
Test Results
|
