Support

 View Only
  • 1.  system validation - same tests for multiple systems

    Posted 12-17-2021 05:37
    Edited by Jonathan Pagaard 12-17-2021 05:37

    I am currently planning validation test runs. Here I am curious about what would be the best way to validate the same use cases for e.g. 3 different systems (both a HW/SW integrated product system) with 3 different users. I am considering to:

    - Make a test plan "Validation test 1" and include all validation protocols and then run the protocols with User 1 for system 1. Then duplicate the test plan and call it "Validation test 2" and continue with User 2 for system 2.
    - If a use case is not passed, then re-run both test plans with the not passed protocol as a new test cycle in both test runs. 

    Any other way to handle it, or is it the right way to go?



    ------------------------------
    Jonathan Pagaard
    Systems Engineer
    ------------------------------


  • 2.  RE: system validation - same tests for multiple systems

    Posted 29 days ago
    Hi Jonathan,

    Thanks for posting this! For variant testing involving multiple systems and/or users, our recommendation would be to first add the duplication at the Test Case level (under Verifications). From there, you can create each of the multiple testing plans with test groups accordingly.

    I hope that helps! 


    ------------------------------
    Carly Rossi // she/her
    Community Manager // Jama Software
    Portland, OR
    ------------------------------



  • 3.  RE: system validation - same tests for multiple systems

    Posted 28 days ago
    Hi all,

    in my understanding, these use cases are quite generic (used in 3 different systems). I would locate the original use case items in a kind of catalog project ready to be reused in the single projects (3 separate ones for Jonathan). The 3 projects can have a different context, thus the uses cases can ba adapted/changed. Having separated projects means using the option to separate test plans.

    I discourage my users to use the approach suggested by Carly. Here you generate a lot of duplicate items. The solution is difficult to maintain (change to a use case definition should be performed to several items). IMHO the discrimination should occur at execution level, not at definition. I suggest to discriminate test runs associated to the same test case by using a custom field.

    Best,


    ------------------------------
    Alessandro
    Systems Engineer
    SICK AG
    ------------------------------



  • 4.  RE: system validation - same tests for multiple systems

    Posted 28 days ago
    Appreciate the insight, Alessandro! If I'm understanding your recommendations correctly, having the discrimination occur at the Test Run level could lead to an inaccurate test result. This is because of the way Test Case Status is recorded for a Test Case Item in Jama. Remember, the latest result would be recorded on a Test Case OR prioritization would kick in if a Test Case is part of multiple active Test Plans. By splitting the Test variants out further upstream (at the Test Case level), you would avoid accidental/inaccurate reporting.

    ------------------------------
    Carly Rossi // she/her
    Community Manager // Jama Software
    Portland, OR
    ------------------------------



  • 5.  RE: system validation - same tests for multiple systems

    Posted 17 days ago
    Happy New Year Carly,
    we do not use the Test Case Status.
    The way the test case status is calculated is in our use case only confusing.
    • we normally have the same test case on more than one test plan
    • the latest test runs is not necessarily the desired one.
    At the end of the day I have written a VTL report based on our conventions.
    BTW, it would be great to have the possibility to determine the way the test case staus is calculated ...

    Best,
    Alessandro

    ------------------------------
    Alessandro
    Systems Engineer
    SICK AG
    ------------------------------



  • 6.  RE: system validation - same tests for multiple systems

    Posted 13 days ago
    Happy New Year, Alessandro! Thank you for sharing that, and for your feedback -- while we do have expanding the flexibility of test case status calculations on our radar, it isn't an option at this time.

    ------------------------------
    Carly Rossi // she/her
    Community Manager // Jama Software
    Portland, OR
    ------------------------------