Table of Contents
Klaros-Testmanagement is a modern web application which helps you to organize and manage your test process. It documents all test relevant data by storing it in a database and gives an overview about the test progress of a project or piece of software at any time. A single item of test relevant data will be called Artifact throughout this document.
The main artifact is called Project. It contains all the other artifacts that are needed to define, execute and evaluate tests. For more details on defining projects see Section 6.1.
The central artifact of a test project is the Test Case which represents a single test. A test case can be executed in an automated or manual fashion. An automated test case refers to a piece of software code that gets executed and in return delivers a test result, usually in form of a test result file. A manual test case contains beside other entries one or more Test Case Steps which hold detailed written instructions and test conditions for the testers executing the test case.
A test case step consists of a
Precondition
(the state of the System under Test before the test starts), an
Action
(explains what the user has to do/test/try out). After the action the outcome should be
the
Expected Result, and a
Postcondition
(the state of the system under test after the action).
Klaros-Testmanagement guides the user through the test case step by step.
During the test execution every step will be assigned a
Test Case Step Result
which contains the verdict of the test case step.
Passed
, Failed
,
Error
or Skipped
are the possible verdicts which are available to the user.
If the precondition, postcondition or expected result does not
match the description, the User should choose Failure
.
If an unexpected error in the system prevents the user from
carrying out the step (for example, a button is missing) they
should choose Error
.
If one of the test case step result verdicts is marked as Error
or
Failure
, the whole test case result is also marked with
that particular verdict.
Only if every test case step is marked as passed, the encapsulating
test case result is marked as passed.
Automated test cases usually have no test case steps because they are executed with an automated test program (e.g. JUnit). These test case results can be imported and stored with the other test related data into the database.
Requirements are another type of artifact in Klaros-Testmanagement which represent a condition or capability of the system under test. Each test case can relate to one or more requirements and vice-versa. More information regarding requirements is shown in ( Section 2.8).
Test Segments are grouped, predefined and encapsulated Test Case Steps that can be created separately and reused in Test Cases .
When defining test cases, specific test case steps can be exactly the same in multiple test cases. To make sure these test case step sequences are identical in each test case, it is possible to encapsulate test case steps into test segments. These are inserted into a test case like normal test case steps and can be reused in different test cases within the same project.
They have the same attributes like normal test case steps mentioned above. The only difference is, that they can only be used as is and not be further edited when referenced in test cases.
Multiple test cases can be combined into another artifact that is called Test Suite. When executing a test suite, all test cases that are part of this test suite are executed in a row by the tester.
The same test cases can be part of multiple test suites and changes to a test case will automatically be reflected in all test suites referencing it.
Every time a tester executes test cases or test suites, a Test Run containing the corresponding Test Case Results is created. The test run consists of all test case results created during the execution of a test case or test suite and stores additional information about the circumstances of the test run shown in the next section.
The simplest case is the execution of a single test case. This generates a test run with a single result (the test case result). If a test suite is executed, there will usually be more than one test result (one result for each test case). The test suite result lists all the test results of the executed test suite.
Test runs can be paused and continued at a later time.
This means that it is possible that not all results that are stored in the
database already have a known verdict, in this case the verdict is listed as Unknown
.
Whenever a test case or test suite is executed, it needs to be executed in a defined Test Environment, and one executes it against a defined System under Test.
The result of a test is influenced by the version of the system being tested (system under test / SUT), since issues are detected in a particular software version, but may then be fixed in a later version.
Test environments represent external conditions which may have impact on the test result, like e.g. the operating system the system under test in running in. To document this properly, every test run is linked to a particular system under test and test environment.
An Iteration groups systems under test, test environments and requirements within a project for a selectable time frame. Every iteration represents a selected phase of the project and its artifacts including all jobs and test results executed during this phase. Grouping the artifacts like this makes it easier to identify different test cycles in a project. This allows for better integration of the test process especially with agile software development practices.
To confirm if the system under test meets its quality standards it is needed to document which of the system requirements are covered by tests ( Coverage) and how many of them have been executed with a passed result ( Compliance).
To enable this traceability, it is needed to link requirements to one or more test cases and vice versa.
From this set of information Klaros-Testmanagement is able to automatically determine coverage and compliance information for the whole project or selected artifacts once the first test results are available.
Many users may execute tests in Klaros-Testmanagement simultaneously. To
orchestrate this process it is recommended to plan the testing activities of the users.
The planned execution of a test case or a test suite by a user is
defined in the form of a Job
(which is also an artifact). Jobs usually reference a test case or test suite. The
intended test environment and
system under test may be defined beforehand by the creator of a job.
Jobs also contain a start date (when the testing should
begin), a due date (a deadline for the execution of the test),
and may be assigned a priority (how urgent the test execution is).
Available priorities are:
Trivial
,
Minor
,
Major
,
Critical
and Blocker
.
Jobs may also contain status information like
Resolved
,
Reopened
,
Closed
,
In Progress
or
New
to document their progress.
After a job has been created it can be assigned to a user. This user then can execute the job and set its results. A job may be repeated and executed multiple times, so one job can refer to more than one test run.
In addition to test execution other job types exists allowing to schedule reviews of test cases or requirements and specify arbitrary tasks in a textual form.
Jobs can be nested and individually assigned to users. Therefore it is possible to track progress of larger, distributed test activities.
Klaros-Testmanagement defines four roles available to users:
Administrator,
Manager,
Tester
and Guest.
Only a user with the Administrator
role is able to create and assign other users.
Guests
are only able to view artifacts and reports,
they are not able to change anything in the system.
Testers
have more permissions than those in the role of
guest. They are not only able to view artifacts but in
addition they are allowed to execute jobs, test cases and test suites
and edit test results.
Managers
inherit all rights of a tester and also possess additional rights to
create and update artifacts like test cases, requirements and more.
Administrators
have all available rights in the system.
In addition to the rights a manager has they are able to create and delete projects
and users as well as configuring system-wide parameters.
Each users has a defined default role which is assigned by an administrator when
creating a user account.
The Manager
, Tester
and Guest
roles can also be assigned to users on a per-project basis
providing a project based security scheme.
Detailed information about the role permissions can be found in Appendix A).
Klaros-Testmanagement can be integrated with issue tracking systems like Atlassian Jira, Bugzilla and many more (see Section 12.5). If an error of the system under test is detected in one of the test case steps, the user can easily create an entry in the issue tracking system from within Klaros-Testmanagement which is automatically linked and tracked inside both systems.