Support

Expand all | Collapse all

Jama Software Automatically Passes Skipped Test Steps

  • 1.  Jama Software Automatically Passes Skipped Test Steps

    Posted 04-26-2017 22:12
    Edited by Dana Medhaug 06-27-2017 15:14
    For example, when executing a test case with 8 test steps. Test steps 1, 2, 3 = Pass. Skip test steps 4 and 5 and pass step test 6. Jama Software will automatically mark test steps 4 and 5 as 'Pass'. I expect Jama Software to notify the tester that test steps 4 and 5 were skipped, and NOT allow test step 6 = pass.

    ------------------------------
    Ken G
    ASR
    ------------------------------


  • 2.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 04-30-2017 06:21
    Edited by Dana Medhaug 06-27-2017 15:14
    We have found the same issue. We don't always run the tests in the same sequence (equipment availability) and JAMA Software erroneously marks skipped tests as pass. JAMA Software has refused to fix the problem.

    ------------------------------
    Randy - Delta Controls
    ------------------------------



  • 3.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 05-05-2017 16:08
    Edited by Dana Medhaug 06-27-2017 15:15
    Hi Ken,
    It is by design that Jama Software passes all steps in this circumstance. For better or worse (in yours and Randy's case, worse), Jama Software is designed to pass the whole test if the final step is passed. The steps function assumes that steps are completed in numerical order. But Randy is right; at this point we don't have any plans to change this.

    ------------------------------
    Kristina King
    Jama Software
    ------------------------------



  • 4.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 05-08-2017 01:35
    Edited by Dana Medhaug 06-27-2017 15:19
    Hi Kristina,

    I see the Arguments behind this Feature. But for our tester, it was perceived more or less as an issue. It is not always the case that the steps were written, that step 3 passed meant step 1 and 2 should be passed.
    For us, it would be better, when the tester could choose step 3 passed, Jama Software should not do anything with the result of step 1 and 2.

    Thanks

    ------------------------------
    Lien Bäcker
    Gira, Giersiepen GmbH & Co. KG
    ------------------------------



  • 5.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 06-22-2017 18:30
    Edited by Dana Medhaug 06-27-2017 15:25
    I can imagine that this functionality to auto pass is useful when reviewing requirements, but for testing, this is risky. I can't imagine a tester reviewed this functionality when it was implemented and said 'yes let's auto skip steps by default to save time.
    It takes away from the detailed approach of testing and not missing items.

    There are a few ways things function in Jama Software that doesn't make sense for testing. I understand people can approach testing differently, but in testing there are a few obvious things that are mandatory for testing (not skipping steps, providing more than just Passed or Failed for a test step - i.e descoped, or N/A)


    ------------------------------
    Diana
    ------------------------------



  • 6.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 06-23-2017 08:39
    Edited by Dana Medhaug 06-27-2017 15:30
    Hi Diana,

    This feature was specifically designed to function this way, and like all features, there was real-world testing before it shipped. But this was many years ago, and the average Jama Software customer might have changed since then. I'm curious-is your company in the same position as Ken's, where Step order does not necessarily mean Execution order?

    ------------------------------
    Kristina King
    Jama Software
    ------------------------------



  • 7.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 06-25-2017 16:31
    Hi Kristina,
    Step order does mean Execution order generally, but there is a scenario where for example I may need to check that multiple fields have a certain property. In this case, the test steps would have:
    Test Step
    1             Verify that x applies for the following fields
    2              Field 1
    3              Field 2
    4              Field 3
    5              ..etc

    In this case I might verify Field 3 first because I am waiting for something for Field 2. I'd rather split out each field so they each get proper attention than bunching it all into one test case. This is a simple example, but you can see one case where this is applicable.
    Something to consider in a future release.

    Regards,
    Diana

    ------------------------------
    Diana
    ------------------------------



  • 8.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 06-25-2017 23:28
    Edited by Dana Medhaug 06-27-2017 15:31
    I would suggest Jama Software to provide an option to either toggle the skip feature to auto pass the skipped steps or not. Default = not.

    ------------------------------
    Thanks,
    Ken G
    ------------------------------



  • 9.  RE: Jama Software Automatically Passes Skipped Test Steps

    Posted 02-21-2020 01:40
    Hi,
    any update?
    We find this behavior very problematic and I expect it to be the same for most users that are not in the Software industry.

    Feature request> Test Run: option to disable Steps auto-fill

    Steps in Test Runs were obviously develop for Software evaluation. Also by the name, it's clear that a sequence of test "steps" is assumed: when defining a step as PASSED or FAILED there is a feature that auto-fills previous steps as PASSED when undefined (NOT_RUN). This is great for clicks saving provided that steps are in sequence.
    For Hardware test, I don't see the need to check steps of a sequence but to check the same Test Case under different external conditions (temperature, loads, supply voltages…). So a use Steps for my different condition but I'm then very annoyed by the auto-fill feature since the conditions were not necessarily defined in the order they are finally tested.
    My proposal is then to add an option in the Test Case configuration to enabling/disabling the step auto fill. It would also be great to be able to customize the Step field and related table headers.

    ------------------------------
    Christian Binard
    ON semiconductor
    Belgium
    ------------------------------