Support

Expand all | Collapse all

Clarify how Test Case to Test Run works

  • 1.  Clarify how Test Case to Test Run works

    Posted 05-24-2018 18:40
    Hi,
    I've used Jama for Req management a lot in previous Med devices roles, but haven't used the test functionality so much in my past.  Now in my current company we're looking at potentially fully using it I need to understand the finer details of test case-test run management.  Currently using the 30day trial.

    I have noticed the following:
    If I have executed a test run on v1 of the test case, but have updated the test case to v2 after executing the test run there does not seem to be an indication that the latest version of the test case has not been executed.  ie if I add in a new step or change a value, the results of the latest test run (executed on V1 test case) still shows in the test case.
       a) how do I see/determine that a test case in a list of testcases has not been executed to the latest version?
       b) again baseline of a test case at a point in time also will show the latest test run, even if it's not related to the baselined test steps in the test case.
       c) I understand that there's no specific relationship like link between test case and test run, but surely if the test case changes (perhaps because of a requirement change and not necessarily because it failed in this test case) We need to be able to do a regression analysis as to which test cases should be executed?
       d) if I'm looking to show a historical view of the test executed against a baselined set of test cases how do I filter the more recent information out?

    What i'm looking for is a way to show the Requirement - protocol - report traceability for each Release.  which in my head seems like it should be requirement - test case - test run.

    Given the above, what is the best way to manage changes to test cases?, do i need to lock test cases, then duplicate/sync whole sets of test cases instead of baselineing?






    ------------------------------
    Jeffrey
    ------------------------------


  • 2.  RE: Clarify how Test Case to Test Run works

    Posted 05-30-2018 07:40
    Hi there Jeffrey,

    These are some really great questions!  Since you asked them so neatly, I'll actually copy paste them and answer them one at a time below.

    • how do I see/determine that a test case in a list of testcases has not been executed to the latest version?
      • At this point, if you navigate to the Test Runs tab within those different Test plans, you should see a column on the right hand side that says "Status" and this will show you if an item within that specific Test Cycle has been run within that Test Cycle or not.  The most up to date version should ideally be the most recent cycle. If you want to learn more about Test Cycles, this is a good place to read up about them.
    • again baseline of a test case at a point in time also will show the latest test run, even if it's not related to the baselined test steps in the test case.
      • When you refer to the "latest run" are you referring to different Test Cycles and saying that the content of items is not updated across Test Cycles?  I'm sorry to say that I'm a bit confused here and some steps to follow to recreate or screenshots would be helpful.
    • I understand that there's no specific relationship like link between test case and test run, but surely if the test case changes (perhaps because of a requirement change and not necessarily because it failed in this test case) We need to be able to do a regression analysis as to which test cases should be executed?
      • At this point, I believe what you're looking for is actually the Test Cycle option listed above, which would let you view specific failures and successes throughout the different iterations of the Test Run.  Would you be able to clarify if that's what you're looking for specifically?
    • if I'm looking to show a historical view of the test executed against a baselined set of test cases how do I filter the more recent information out?
      • Again, this would really be that Test Cycle option that we've looked into a fair amount.  That would basically let you run different iterations and then create exports that show the different iterations.  We do also offer some Reports (click on "Reports" in the upper right hand corner of your page while in the Test Center) that cover "Baseline compared to" options that may be exactly what you're looking for.

    I hope that all of this helps, Jeffrey and I'll keep an eye out for any clarification that you're able to provide!

    ------------------------------
    Nick S
    Jama Software
    ------------------------------



  • 3.  RE: Clarify how Test Case to Test Run works

    Posted 07-29-2018 18:23
    Hi Nick,

    Thanks for the reply, I was side tracked since the reply, but have come back to the topic,
    with regards to the below question and answer:

    I understand that there's no specific relationship like link between test case and test run, but surely if the test case changes (perhaps because of a requirement change and not necessarily because it failed in this test case) We need to be able to do a regression analysis as to which test cases should be executed?
    • At this point, I believe what you're looking for is actually the Test Cycle option listed above, which would let you view specific failures and successes throughout the different iterations of the Test Run.  Would you be able to clarify if that's what you're looking for specifically?

    a) No i'm not talking about test cycle here, here's a scenario to consider.
    1. Write a test case for a requirement
    2. create a test plan and do first test run - it passes
    3. update test case because of a requirement change
    4. Note that when I look in the test case the last test run shows that the test case passed  - nothing would clearly tell me that I haven't tested the updated test case or that the test run was executed on an older version, unless I went through and specifically looked at all the details in the test run.
    What i've noticed is that the version of test case doesn't seem to be captured or there isn't a clear indication that the test run results shown is "suspect"
    b) Additionally if I go to Create new cycle and select only new/not run items, the test case that has been updated does not appear in the list.
    If I want to execute just the test cases that have changed, how do I do this?

    Hope this can be clarified or point me in the right direction as to how this should work.

    ------------------------------
    Jeffrey
    ------------------------------



  • 4.  RE: Clarify how Test Case to Test Run works

    Posted 05-11-2020 13:01
    I am facing this similar issue. We would like to basically prevent users from executing test run if the requirement has changed. I cannot see an easy way to identify this

    ------------------------------
    Jason Barton
    VMS
    Palo Alto CA
    6504246216
    ------------------------------



  • 5.  RE: Clarify how Test Case to Test Run works

    Posted 05-26-2020 11:14
    Jason:

    Yes, there doesn't seem to be any way to automate this within the Test Center. The only way I could see this working is if someone is actively managing the Test Center cycles and cases by perhaps monitoring the Requirements for changes and then applying the correlating business workflow to the Test Center; starting a new Test Cycle, deleting old test cycles, changing the Test Case titles to be incremental and reflect versioning, etc.

    Best,

    ------------------------------
    Chloe Elliott
    Jama Software
    Portland OR
    ------------------------------