Knowledge Base

Connecting Jama to your Automated Testing Tool

By Iman posted 01-25-2017 12:48

  

Why Connect Jama to your Automated Testing Tool?

Jama is used for managing requirements and the manual test cases that validate and verify those requirements.  Jama provides traceability between the various layers of requirements down to manual test cases so teams can ensure test coverage and show the results of those manual tests.  However, many teams also validate/verify their requirements through automated testing via a separate Automation Test Tool (ATT).  If you want to demonstrate traceability from your requirements to both manual AND automated testing results in Jama, you can leverage Jama’s API to bring automated test results into Jama. We’ve provided two sample approaches to connecting Jama to your ATT below, but be sure to review the considerations before you get started section prior to moving forward with your implementation.

Jama Sample Approaches

If we’ve learned one thing from working with our customers, it’s that all of them are unique and that there is no one size fits all solution for any implementation. However, we also know that customers are looking for guidance based on our experience on how to best leverage Jama’s capabilities.  Connecting Jama to your ATT is no different, so we’ve created two high-level sample approaches that should help jump start your implementation.  These approaches don’t iron out all of the necessary details you’ll need to implement your unique solution, but provide direction on two different methods of connecting your ATT to Jama as well as sample code for implementing the method of your choice.

Sample Approach #1 – Jama Initiated Tests

Jama’s manual test center offers functionality for your team to manually create Test Cases and Test Plans for executing those tests.  It also offers some out of the box reporting via Test Plan summary and Test Plan detail reports.  Teams familiar with using Jama’s test center to execute manual test cases can use a similar approach to initiate automated tests by communicating directly with Jama.

When to Use this Approach

  • Already using Jama for manual test execution
  • Want automated test results to fit in seamlessly with manual test results
  • Want to leverage Jama’s built in test center functionality for test planning and reporting
  • Want to automate running tests in Jama
  • No additional setup in Jama is desired

Functional Flow

Leveraging this approach allows Jama test cases to be pointed at automated test scripts in your ATT. Once these test cases are manually assigned to a test plan and put in a test cycle to be run, the test run is assigned to an “Automated Test User”. A script can find these test runs using Jama’s API and can notify your ATT to execute the tests and retrieve the results. The results are parsed and updated through the API and appear in Jama just as if they were executed manually within Jama’s test center.

flowchart

  • Jama Admin is available for  creating the necessary automated test user
  • Test leads should fully understand managing and executing manual tests in Jama prior to implementing this approach
  • Developers writing API scripts understand Jama and Jama data structure including test center functionality in detail
  • Automated test scripts are written, managed and executed in automated testing tool
  • Amount of test results being brought into Jama should not exceed performance recommendations (teams can archive or delete test plans and/or projects at specific intervals to maintain the appropriate amount of data in Jama)

Technical Implementation Details

Use the scripts at GitHub.com/JamaSoftware/Automated-Testing as a starting point for this approach. You’ll need some information from your Jama configuration to configure the scripts:

  • The API ID of the automated test user.  When test runs are assigned to this user the script will run whatever command(s) appear in the “action” column of the test run.
  • The API ID of test runs in your Jama instance.
  • Credentials for an account to access and update the test run results.  This can be the automated test user or any user with appropriate permissions.

Technical Flow

TechnicalFlow.png

Sample Approach #2 – Automated Testing Tool Initiated Tests

The Test Trigger script checks Jama for any test runs that are assigned to the automated test user.

  1. The Test Trigger script locks the test run and initiates the test execution in the ATT.
  2. The ATT places the results in a specified location.
  3. The Results Trigger script find and parses those results.
  4. The Results Trigger script updates the test run and unlocks it.

For teams not using Jama’s Test Center for manual testing or who don’t want to interact with Jama in order to initiate the automated test scripts we recommend not leveraging the Jama test center at all.

When to use This Approach

  • Not using Jama’s Test Center for manual testing
  • Want to initiate automated tests to be run from within the ATT

Functional Flow

In this approach your Jama Admin will create a new item type in Jama that represents your automated test results.  This item type should be included in your relationship rule and contain a name, description and status field at a minimum.  Your ATT will be used to trigger all automatic testing and the Jama API will be used to bring those results into your project AND to relate those results to the upstream requirements they are testing.

FuncationalFlow2.png

Assumptions:

  • Unique Jama ID (document key) of the satisfied requirement is available to test that’s satisfying it.
  • Jama Admin is available for  creating the necessary item type
  • Jama business and test leads should fully understand managing data in Jama.
  • Developers writing API scripts understand Jama and Jama data structure in detail
  • Automated test scripts are written, managed and executed in automated testing tool
  • Amount of test results being brought into Jama should not exceed performance recommendations (teams can manually archive or delete projects at specific intervals to maintain the appropriate amount of data in Jama)

Technical Implementation Details

Use the script at GitHub.com/JamaSoftware/Automated-Testing as a starting point for this approach. You’ll need some information from your Jama configuration to configure the script:

  • The API ID of the set item type.  This is a system-defined item type.
  • The API ID of the test results item type.  This is the new item type created for this approach.
  • The API ID of the project that will hold the test results.
  • The unique name of the field containing the Pass/Fail picklist in the test results item type.
  • The API IDs of the ‘pass’ and ‘fail’ options in the above picklist.

Technical Flow

TechnicalFlow2.png
  1. The ATT executes the Jama Proxy script when it’s done running a test.
  2. The Jama Proxy script receives:
    1. The document key of the satisfied requirement
    2. Any information that should be included in the test result item’s description
    3. The results of the test
  3. The Jama Proxy script adds the item to Jama.
  4. A relationship is established between the new test result item and the satisfied requirement

Considerations before you get started

Jama is designed to manage and execute manual test cases/results.  Automated testing allows a larger number of tests to be executed much more frequently, and therefore generates quantities of results beyond the intended functionality of Jama.  So, you don’t want to simply replicate your ATT in Jama.  Your ATT is designed to manage all aspects of your automated testing (scripts, historical results, reporting, etc) and the majority of this management should remain within your ATT.  Prior to jumping in and starting to write code to connect your ATT to Jama, you should identify your overall approach and ensure it aligns with best practices of using Jama.  Several steps you should take are:

Considerations1.png

  • Include all of the necessary team members in developing your approach as bringing automated test results into Jama can benefit and impact many different areas of your organization, including:
    • QA and Jama Process Leads to define desired goals and objectives
    • QA and Jama Technical Leads to design and implement the solution
    • Developers who will be writing the API scripts to connect Jama to your ATT
  • Review the Recommendations for Jama Performance in the Enterprise to inform your overall approach and strategy – Hint: your ATT is likely to generate far more results than are recommended to be in Jama over the long term so consider:
    • Bringing in automated test results only at key milestones/intervals during a project rather than each time an automated script is run
    • Deleting or archiving automated test results/projects from Jama on consistent intervals to avoid exceeding recommendations for optimal Jama performance
    • Storing automated test results in separate Jama project(s)
  • Develop and refine your overall implementation approach – Hint: your approach shouldn’t be replacing major pieces of your ATT, but should be tied to providing traceability to the requirements in Jama at key intervals of your project
  • Leverage one of Jama’s two high-level approaches above to jumpstart your implementation

Interested in having someone guide you through this process?

Jama’s Professional Services team can assist you with practical guidance on both the desired cross-team processes and the supporting technical workflow.  Contact your Customer Success manager for more information.



#REST #tutorial

3 comments
1988 views

Comments

08-23-2017 13:10

Hello @Felix, Once you create an automated user, you can retrieve the user's API ID via your swagger documentation (baseURL/api-docs). The {GET /users} endpoint will allow you to enter a user's username, and retrieve the user and their API ID.

08-23-2017 04:23

hello, where can i find the API ID of automation user?

03-08-2017 06:50

Hello,  The images seem to be missing from this blog post. It would really help understanding if they were added. Many thanks, Ruth