Re-use test run results in new test run

Jacob Brodal
Jacob Brodal Member Posts: 11
edited March 13 in Peer-to-Peer Support Forum

Hi,

Given a case of two variants of a product with similar characteristics, we run 100% of all test cases for variant A, but based on equivalency, we only need to run 80% of all test cases for variant B.

For the 20% of test cases we did not run for variant B, we would like to re-use results from variant A test runs.

Additional considerations:

  • Variant B test runs needs to be exported for external submissions
  • The variants may evolve in different directions

What is the best way to re-use test runs in a new test run? 

------------------------------
Jacob Brodal
Systems Engineer
3Shape
------------------------------

Comments

  • Alessandro Valli
    Alessandro Valli Member, Data Exchange, Jama Connect Interchange™ (JCI) Posts: 785
    edited September 2023

    Hi Jacob,

    the question is : do you need separate test runs at all? I mean: do you need a test run run item for variant A and a test run item for variant B? If not, you can use the same test run item for both variants. You have to simply give them the related attributes (via tags, e.g.). In case having plenty of variants consider defining the no attribute case as valid for all variants (better to maintain).
    In case of automated tests it should not be an issue to generated new test runs; manually performed tests are key here.

    ------------------------------
    Alessandro
    Systems Engineer
    SICK AG
    ------------------------------
    -------------------------------------------
    Original Message:
    Sent: 09-04-2023 21:38
    From: Jacob Brodal
    Subject: Re-use test run results in new test run

    Hi,

    Given a case of two variants of a product with similar characteristics, we run 100% of all test cases for variant A, but based on equivalency, we only need to run 80% of all test cases for variant B.

    For the 20% of test cases we did not run for variant B, we would like to re-use results from variant A test runs.

    Additional considerations:

    • Variant B test runs needs to be exported for external submissions
    • The variants may evolve in different directions

    What is the best way to re-use test runs in a new test run? 

    ------------------------------
    Jacob Brodal
    Systems Engineer
    3Shape
    ------------------------------
    Alessandro
    Systems Engineer
    SICK AG
  • [Deleted User]
    [Deleted User] Posts: 103
    edited September 2023

    Hi Jacob,

    For re-using test case results, I recommend creating a separate test plan for the test cases whose results are to be reused.  Then the overall results for a given variant can be calculated by combining the test results from the appropriate set of test plans.

    ------------------------------
    Geoffrey Clemm
    Jama Software
    ------------------------------
    -------------------------------------------
    Original Message:
    Sent: 09-04-2023 21:38
    From: Jacob Brodal
    Subject: Re-use test run results in new test run

    Hi,

    Given a case of two variants of a product with similar characteristics, we run 100% of all test cases for variant A, but based on equivalency, we only need to run 80% of all test cases for variant B.

    For the 20% of test cases we did not run for variant B, we would like to re-use results from variant A test runs.

    Additional considerations:

    • Variant B test runs needs to be exported for external submissions
    • The variants may evolve in different directions

    What is the best way to re-use test runs in a new test run? 

    ------------------------------
    Jacob Brodal
    Systems Engineer
    3Shape
    ------------------------------

Categories