Support

 View Only
  • 1.  Test Runs - "Pass with Errors" concept.

    Posted 05-25-2020 06:18

    Regarding Test Run, I don't understand what is 'Pass with Errors' useful to. And I am not comfortable with the idea of ​​declaring that I have passed a test while it revealed errors. I suspect that we will want to prevent our users from checking this option.

    Where could I find some use case examples for this option?
    How could I make this option unusable/inaccessible to users?

    Regards


    ------------------------------
    Frederic GARFAGNI
    Fresenius Kabi
    Development Methods & Tools
    GRENOBLE - France
    ------------------------------


  • 2.  RE: Test Runs - "Pass with Errors" concept.

    Posted 05-25-2020 06:59
    Hi Frederic,
    we mostly use 'Pass with Errors' as 'Fail accepted'. That are Fails, but with User override (accept it anyway ... so to say).
    Regards,

    ------------------------------
    Alessandro
    Systems Engineer
    SICK AG
    ------------------------------



  • 3.  RE: Test Runs - "Pass with Errors" concept.

    Posted 05-26-2020 07:57
    Alessandro:

    You always contribute such great ideas and workflows! I granted our new "Collaborator" badge to your community profile!


    Keep it up!



    ------------------------------
    Chloe Elliott
    Jama Software
    Portland OR
    ------------------------------



  • 4.  RE: Test Runs - "Pass with Errors" concept.

    Posted 05-26-2020 08:31
    Somehow related to this,

    what I am missing is a state (both Test Case Status, Test Run Status ) like "inconclusive" in addition to "blocked".
    • Blocked should be "not finished, but blocked"
    • Inconclusive should be "finished, but neither passed, nor failed"

    Best,

    ------------------------------
    Alessandro
    Systems Engineer
    SICK AG
    ------------------------------



  • 5.  RE: Test Runs - "Pass with Errors" concept.

    Posted 06-17-2020 05:53
    Ideally, we should be able to provide our own pick list of criteria for test run statuses.

    RAD

    ------------------------------
    Rory Duncan
    NASA
    Kennedy Space Center FL
    3218612345
    ------------------------------



  • 6.  RE: Test Runs - "Pass with Errors" concept.

    Posted 06-17-2020 10:50
    Rory and Alessandro:

    The Test Center is an area of focus right now. Here is a list of Idea cards from other Community members in the new Ideation module so far: Test Center Idea Cards

    Take a look, vote on any ideas you like and make new ones!

    ------------------------------
    Chloe Elliott
    Jama Software
    Portland OR
    ------------------------------



  • 7.  RE: Test Runs - "Pass with Errors" concept.

    Posted 11-30-2023 12:49

    We haven't released the ability to create custom statuses, however,  in 9.5 we released some options which may provide needed flexibility for you:

    Organization Administrators may configure how the system will behave when users skip test steps and don't assign them a status:

    • Assign "Passed" to all skipped steps

    • Do not assign status to any skipped steps

    • Allow tester to select status for skipped step (i.e. ignore, pass, fail)

    Instructions:

    1. Navigate to Admin > Organization Admin > Item types
    2. Select the Test Run item type in the left panel
    3. Select the Status field in the right panel and click the pencil icon to edit
    4. From the popup window, select the desired default behavior for completing steps out of order 
    5. Click save

    Organization Administrators may configure "Not applicable"/"N/A" as an additional status option. Once configured, testers will see the option to mark some or all steps in a run as 'Not applicable'. Any steps marked as 'Not applicable' will be excluded from the overall pass/fail result of the run.

    Instructions:

    1. Navigate to Admin > Organization Admin > Item types
    2. Select the Test Run item type in the left panel
    3. Select the Status field in the right panel and click the pencil icon to edit
    4. From the popup window and under the status options, select the box for "Not Applicable (N/A)" status
    5. Click save

    The product team would like to learn more about what other statuses would you add (if that capability were available) and how would you use them to improve efficiency/outcomes. Feel free to add any insights the product team will use to understand your use case here:

    https://community.jamasoftware.com/product-ideas-suggest/ideation-home/viewidea?IdeationKey=fd8701ec-d410-497b-bee7-858fc86923bf#ItemCommentPanel



    ------------------------------
    Ashley Ernst
    CA
    ------------------------------