Retest failed automated tests

Michael Dietrich-Spitzley, geändert vor 8 Jahren.

Retest failed automated tests

Youngling Beiträge: 2 Beitrittsdatum: 07.07.16 Neueste Beiträge
Hi,

we plan the following process:
  • automated testresults are added using the REST-Intervace
  • failed tests shall be retested manually
Problems:
  • We found no (efficient) way to find all failed automated tests within a job or a testsuite over all SUTs
  • When a testcase is marked as "automated", the teststeps are deactivated
How are you working with update of automated testresults?

Kind regards,
Michael
thumbnail
Torsten Stolpmann, geändert vor 8 Jahren.

RE: Retest failed automated tests

Jedi Council Member Beiträge: 759 Beitrittsdatum: 12.02.09 Neueste Beiträge
Hi Michael,

you should find the failed test case results in the results tab of the job or test suite in question. If there are too many objects to check for you this way, you may also have a look at the iteration you are currently working in.

If you are choosing to rerun your automated tests manually, there are basically two options available for you:

1. Switch the test case to manual mode

Switch the test case to Manual and execute the test using the manual test runner. When done, switch back to automated mode.
This is the simplest method to resolve this. Manual test results will show up like the previous automated test results in the result history.

2. Use a different revision of your test case

If a previous, manual revision of your test case exists (typically just before you have automated your test case), you may execute this revision instead. You may also choose to create a new, manual revision of your test case, especially if no such previous revision exists. Please note that each revision keeps a different set of test results, so you may have to take care of which revision you are looking at during evaluation.

The second method may leave a larger footprint in your system (a new revision is now possibly available), so I would recommend solution 1, especially when the manual retest is only a temporary solution.

All methods may require to add manual test steps to the test case. This indeed is only possible, when execution mode is set to manual. This page has been disabled intentionally in automated mode to not confuse the typical users of automated tests.

Kind regards,

Torsten
Michael Dietrich-Spitzley, geändert vor 8 Jahren.

RE: Retest failed automated tests (Antwort)

Youngling Beiträge: 2 Beitrittsdatum: 07.07.16 Neueste Beiträge
Hi Torsten,

thanks for quick reply.

I also prefer the first variant, but it is connected to a lot of manual administration effort. Is there a switch or something like this available to configure the behavior?

Of course, automated tests should not appear during a testrun. But for retesting purpose it should be possible to see the teststeps and add/update results for the current testrun.

Kind regards,
Michael