Test Management FAQ

By Kristina posted 04-18-2015 00:47

  
What do all of these terms mean?
When testing, there are four required levels of hierarchy; these are from the top-level down.
Test Plan: a high level grouping of Test Cases
Test Group: a breakdown/organization of Test Cases inside of a Plan
Test Cycle: could represent a build, release, or unit of testing
Test Run: an individual execution of a Test Case inside of a Cycle

Some business rules that apply here:
Test Plans and Test Groups can have only one instance of a Test Case.
Test Cycles must be created in order to execute Runs.
Test Run results can be recorded only once per Case per Cycle.

What permissions are required to create Test Plans and execute Test Runs?
Either a creator or Test Runner license with Create/Edit permissions at the Project Level is needed to create Plans. Note that prior to releases Hosted 2014/06/21 and On-premises 2014.2, creating Plans required the "Manage Project" role. Executing Runs requires a Creator or Collaborator license with Read permissions to the Project. 

What is a Test Run in relation to a Test Case?
A Test Run is an item derived from a Test Case. This means the core information, Name and Description, are copied from the Case into the Run. However, no relationships are transferred, nor is a relationship created between the Case and Run. Please see this article for more details on this relationship.

Is it possible to have custom Test Case fields show in Test Runs?
Test Runs are derived from Test Cases, however, they do not inherit all core fields or custom fields. To see how to workaround this, check out this article.

What is a Test Cycle?
When you create a Test Cycle, you are choosing to execute tests based on related cases in Test Groups. The Cycle is like a snapshot of the Test Cases included in that Group or Groups. This means that the Test Runs included in the Cycle are essentially copies of the Cases as they existed when the Cycle was created.

Is there a way to add individual Test Cases to a Test Cycle?
It is not possible to add individual Test Cases to a Cycle; they must first be added to a Test Group. (Although there is nothing that will prevent you from creating groups of one.) Note that Cases can exist in just one Group per Test Plan.

What happens to a Test Run if I delete the Test Case from a Test Group?
The Run will remain in your Test Cycle unless you do one of two things:
-You can delete the individual test run from the cycle. Select the ID of the run in the Test Runs view. You'll be taken to the single-item view of the test run, and from here, you can select Action > Delete.
-If the Run has not yet been executed, you can edit the Cycle and remove the entire Test Group from the Cycle.

How does the Test Case Status work?
The Test Case Status is a unique field calculated with the results of the Test Runs derived from that Case. It is not something a user can update because it considers a number of things, including the "priority" of individual test RUN statuses (what takes precedence when calculating the CASE status), whether Test Plans are archived, whether a setting in Admin "Include Unexecuted Test Runs in Status Calculations" is  checked, etc. To learn more about how this status is calculated, check out this post.

Can I add additional values to the Test Case/Test Run Status pick list?
As noted previously, the Status is a unique calculated field. You won't be able to add another value in this Status Pick List because it is tied directly to the functionality of the Test Center. As a workaround we recommend adding a custom field or Pick List to the Item Type. Just note that if you add a custom field to the Test Case Item Type it will not be inherited by the Test Run.

What do you recommend when you need to run the same Test Case many times within one Test Plan, for say, developing software for multiple browsers?
It depends on the end results you need for reporting. The easiest thing to do is to structure individual Test Cycles for each browser. The catch with this method is that because the Test Runs are tied to the same Test Case, the Case Status will not reflect individual browser results. Given this, it is common for software companies developing on multiple platforms or browsers to create a Case for each configuration. Reuse can come in very handy here! For more on the topic, check out the thread regarding variations on a test case.

I'm trying to add Test Cases to a Test Group, but Jama is telling me they "already exist in this Test Plan." I don't see them—where might they be hidden?
If you have enough Test Groups in your Plan, Jama will start hiding the visible cases to ensure Groups remain visible. If you run into this message, check your Groups to see if there is a More... link visible.



What does "Make Test Runs Current" do?
If a Test Case has been edited since a Test Cycle was created, a pencil icon will appear next to the Test Run. You can then opt to "Make Test Runs Current," which will update the Name and Description fields in the selected Runs to match the Case. Note that the result of the test will be reset to Not Run.

Do you have an integration with Selenium?
Jama does not offer an integration with Selenium. However, we do have a REST API that allows you to update the status of Test Runs with relative ease.

What is the easiest way to import Test Cases into Jama?
The easiest way to import Test Cases is via Excel spreadsheets, as importing via API does not allow the creation of Test Steps. To see a walk-through of this process, check out this article.

What happens when you archive a Test Plan?
When you have completed a Test Plan but need to maintain it for your records, you can archive it. This means that Test Case statuses will no longer be affected by the result of the Test Runs in the Plan.

Is it possible to review Test Plans in the Review Center?
Because Test Plans are not items that live in the hierarchy, they cannot be sent to the Test Center. However, you can create a filter that pulls your particular cases from the Plan and then create a review based on that filter.

What information from a Test Case can be sent to a Review?
By default, the Name and Description fields of Cases can be visible within the Review Center. Find out how to add them.

Where can I learn more about Test Management?
An expert Jama consultant, Dennis Hodge, answered customer questions during an "Ask Jama" session, and that can be viewed via this link. Additionally, the User Guide will walk you through creating a Test Plan.

#TestManagement
2 comments
489 views

Comments

07-28-2015 18:57

Thanks, Swoo. Test Management is at its surface very straightforward, but there are plenty of gotchas ;)

07-28-2015 16:32

Thanks for sharing. This is an interesting article that goes over some of the finer points regarding test management that I don't know previously.