Jama Performance In Enterprise Recommendations for System Administrators

Preston Mitchell
edited January 25 in

Information Moved to the Knowledge Base in January 2024

Jama Connect® Performance In Enterprise Recommendations for System Administrators

Introduction 

Jama Connect is a platform that is successfully used at scale for many different use scenarios. By design, Jama Connect can be configured and used in many ways. The fact that the platform is so configurable makes it difficult to provide “hard” guardrails and performance recommendations. Instead, the purpose of this document is to: 

  1. Define “typical” usage profile of Jama Connect
  2. Provide a matrix where performance may be affected during specific usage scenarios 
  3. Provide recommendations to mitigate performance risks 

Assumptions

This article assumes your Jama Connect instance is in our Cloud environment or, if Self-Hosted, the application is installed correctly utilizing supported software and hardwareWe require that the application server be on a dedicated system with no other applications running, with a separate dedicated system for databases. It is acceptable to share the database server with databases for other applications. 

Typical Usage Profile

A typical data usage profile is where users should not see any performance or scalability issues in typical day-to-day use of Jama Connect. There is always a small risk of performance impacts in any environment, though. Jama engineering teams extensively test Jama for performance, and you can read more about our performance testing approach here: Jama Connect Performance Whitepaper

When describing the typical usage profile, it is important to understand that Jama Connect differentiates “active” vs “inactive” items. Inactive items may be old versions of requirements or entire archived projects. These inactive items are not included in search index and do not materially impact performance of the system. All the numbers below reference “active” aka current version of items. So while a particular project may have 50,000 active requirements, there might be 5x inactive items for the project due to old versions of those items.  

Category Count (active) Notes
Total items in database 10 million No hard system limit but upper limit recommendation of 10 million active items, with much higher # of inactive items in database.
Average concurrent user sessions over course of single day 2500 Jama supports many more total licensed users in systems. 2500 concurrent users means 2500 users performing application functions within a 5-minute window.
Average items per project tree 50,000 No hard-upper limit but above typical usage, users may encounter more performance issues moving items or doing project-wide actions like whole project baseline.
Average items per container (e.g. set/folder) 250 No hard upper limit but recommend upper limit of 1000. Set and folder containers are meant to help organize data.
Average relationships per item 5 Average # of relationships per item across all items. No hard-upper limit in system.
Average Test Plans per project 10 No hard-upper limit, many more Test Plans may exist in archived state.
Average Test Cases per Test Plan 2,500 No hard-upper limit, recommend upper limit below 5,000.
Average Cycles per Test Plan 5 No hard-upper limit, Recommend upper limit below 10 cycles per plan as each cycle creates a new Test Run item for each test case.
Average attachments/images per project 5,000 Typically these are image files stored on application server and will impact HD space needed for self-hosted customers. No hard-upper limit for Jama Cloud customers. 

Performance Impact Matrix 

Below is a summary of what functions may lead to performance issues in certain data profile situations.  

  • Green: expect no performance issues 
  • Yellow: may encounter rare performance issues depending on usage 
  • Orangemore likely to see performance issues as # of concurrent users or volume of active items used exceeds the “typical usage” profile  

The more complex and linked your data structure, the more related items you will pull into memory, and the more processor-intensive it will be to manage.  

If you believe your organization usage of Jama will likely fall above “typical usage” profile and frequently utilize these functions, consider installing multiple Jama instances to split the data and load.  

 

Typical Profile 
If you are operating Jama Connect within our description of “typical usage” profile, the application will be very performant and reliable. Some functions, such as reusing very large # of items, will create a high load on the system. Jama Connect has built-in threshold warnings for users in these scenarios. For example, if a user plans to reuse 1000 items, they will get a pop-up warning that the batch reuse may affect performance. It is not a hard limit. Jama will still complete the task but it’s a guardrail to encourage users to do complex functions in smaller batches.  

Concurrent Users Exceeds Typical Usage 
If you are operating Jama Connect within our description of a “typical” profile but you have a higher # of concurrent users (greater than 2500 performing multiple actions within 5-minute window. 

Jama may still perform well in many usage scenarios, but you may encounter more performance issues or slowness as a higher volume of users are performing high-load actions on the system concurrently. For example, a higher number of users may concurrently create complex filter queries across entire system or export large documents with thousands of items which may stress the system.  

Volume of Items Exceeds Typical Usage 
If you are operating Jama Connect beyond the volume of items defined in the “typical usage” profile, you are more likely to see performance issues with specific functions. This is especially true if multiple users are performing high-load actions concurrently. Please carefully review the performance recommendations section. If you believe your organization usage of Jama will likely fall above “typical usage” profile and frequently utilize functions that may lead to performance issues, consider installing multiple Jama instances to split the data and load.  


Performance Risks and Recommendations By Function

Periodic Performance Issues: If you find that your users are reporting that Jama periodically seems to ‘slow down’ and the issues do not appear to relate to any specific area. 

Troubleshooting: Monitor the CPU/Memory utilization to see resources are being consumed by large operations. Talk to your power users and see if any of them are creating Reviews, Baselines or generating reports on large numbers of items. 

Recommendations: Break large operations up into smaller operations. Run large operations at the end of the day when less users will be on the system. Increase system resources to reduce impact of periodic large operations. 

Function-Specific Performance Issues 

# 

Function 

Performance Issues 

Troubleshooting 

Recommendations 

1 

Viewing Items 

  • List or Reading View takes a long time to load items
  • Check with your administrator to see if any known system outages or performance problems are occurring

  • Check the total number of items being returned
    • Look at the number of pages of data available (e.g. page 1 of ?) 
  • Check to see if large images are being displayed 
  • Typically, users should not see performance issues viewing items
    • If they do, it is usually indicative of larger system issues on server

  • If you are viewing a large flat list of items, consider breaking data into multiple containers (250-1000 items per container) and try to limit searches/browsing to these containers

  • Hide fields that have large amounts of data or images (e.g. Description Fields) 

2 

Comment on item 

 

  • Opening the comments widget takes a long time to show comments

  • Adding a new comment takes a long time. 
  • Check with your administrator to see if any known system outages or performance problems are occurring

  • Look to see if any of the comments contain large images 
  • Typically, users should not see performance issues viewing items
    • If they do, it is usually indicative of larger system issues on server
       
  • Ensure users not to add extremely large images to the comment stream
    • We have seen comments with dozens of 100mb images and this causes slowness in loading comment views

3 

Create / Edit Item 

  • Saving an item takes a long time or never completes
  • Check with your administrator to see if any known system outages or performance problems are occurring 
  • Typically, users should not see performance issues creating or editing single items
    • If they do it is usually indicative of larger system issues on server 

4 

View traceability 

  • Loading the initial Trace View and navigating upstream/downstream takes a long time
  • Exporting Trace View takes a long time
  • Jama Connect is designed to store a complex network of trace relationships typical in System Engineering and Requirements Management
    • If your usage exceeds the "typical usage" profile, retrieving a broad and deep hierarchy from Jama Connect can generate long-running transactions, which will reduce availability for other users

  • The Trace View is designed to show thousands of items and their relationships
    • It loads items incrementally as the user navigates, allowing relationship data exploration
    • Due to the live and dynamic nature of the view, the volume of data the browser and Jama Connect must concurrently track may be very high
  • To view traceability for large scale data, it is recommended users create a filter of only the relevant data they want to explore, then use that filter to activate Trace View

  • Users can search and filter data in the List View before opening the Trace View
    • Further filtering is available from the Trace View as well. 

5 

Search for item(s) 

  • Searching for items may affect performance if a high # of concurrent users are routinely querying across the entire data base (e.g. using "All Projects" filter or search)  
  • Check with your administrator to see if any known system outages or performance problems are occurring
  • By default, all searches in Jama are project specific (users must proactively change to search across all projects)
    • Encourage users to perform targeted searches within specific projects

  • Monitor server load during high usage spikes

6 

Batch edit/delete  items 

  • Batch editing dozens or hundreds of items is a modest load activity
    • Batch editing thousands of items will have a performance cost as it causes a high load of concurrent application requests
    • If you need to delete hundreds (or thousands) of items, it's much faster to delete the containers containing those items rather than the items themselves (e.g. from a batch select in List View
  • Determine if users are routinely batch editing or deleting items
    • If yes – is there an alternative workflow that would be more performant? 
  • Use batch edit function in reasonable batches (hundreds or less) or during off hours for a larger load

  • Delete items at container level in tree or use a filter in the list view to only view containers you wish to delete

7 

Move item(s) in tree 

  • Moving an item in the tree causes Jama to renumber the tree location and heading # (e.g. 3.1.4) of all items in the project

  • We've observed that moving items and renumbering the placement in explorer tree is fast enough for projects with thousands of items, but slower for projects with tens of thousands
    • This is exacerbated if the project has thousands of items in a flat list under single containers (e.g. a set or folder)

  • A flat list of thousands of items is ok for archived projects but may cause slowness if you expect users will routinely move the items in that large flat list

  • The Project Explorer tree setting 'Max items displayed in explorer tree"  setting represents the max items the tree will display to a user in a single container (one level of hierarchy)
    • It does not indicate the total items in the project tree or the total items that will display in the Reading/List View

  • Because most customers organize items in multiple sets, folders and sub-folders, customer rarely hit this max, or generally prefer to organize the data for easier consumption 
  • Check to see if there are large, flat lists in the Jama project

  • Understand why users are batch moving items routinely and see if there is an alternative workflow. 
  • Utilize a logical structure of sets and folders and avoid very long, flat lists in Explorer Tree

  • It is more performant to move folders of items instead of individual items
    • This is especially true when moving large numbers of items as Jama needs to update the tree location and heading # for each item

  • The Project Explorer tree setting 'Max items displayed in explorer tree" should remain below 500
    • This number indicates the max number displayed in a flat list under a single set/folder container  

8 

Participate in a review 

  • The Review Center was designed for reviewing a typical-sized document
    • Reviews with thousands of items are not recommended
      • If you create larger reviews, they will perform slowly and generate high load, not to mention some unhappy reviewers!  
  • Check the number of items included in the review
    • A typical review would contain less than 250 requirements items

  • This is not a hard limit and Jama Admins can monitor which reviews exceed the typical threshold in the Admin > System Health Report 
  • We recommend reviews with less than 250 items, not only for performance reasons, but also as a business best practice
    • Limiting the volume of items allows a more agile process by facilitating more frequent, iterative reviews

9 

Reuse items 

  • Reusing dozens or hundreds of items is a modest-load activity
    • Reusing thousands, tens, or hundreds of thousands of items generates a lot of load
    • During the reuse operation, items involved will be locked and others will be unable to edit until reuse is complete 
  • Jama Connect has a guardrail warning if a user plans to reuse a large # of items
    • This is not a hard limit, just a warning that it may take some time to complete
  • Consider running large reuse operations during off-hours or low load times
    • Avoid syncing thousands of items reused items at a time

10 

Create Test Cycles & Runs 

  • Jama Connect was designed to create and manage manual tests, assuming hundreds of items in a cycle
    • Test plans that exceed the typical usage profile may cause Jama Software to perform more slowly

  • Jama Connect can be linked to an automated test system
    • Be thoughtful about how much test data you are inserting automatically through the REST API
      • For example, adding automated nightly test results to 100,000 test cases, you will quickly exceed recommended system volume of the typical usage profile 
  • Jama Admins can monitor which Test Plans exceed the typical threshold in the Admin > System Health Report 
  • Consider breaking your large test plans into multiple, smaller plans

  • Automated test integrations - do not populate Jama with thousands of nightly test results
    • Instead, consider only integrating results from final build / regression runs 

11 

Create baselines 

  • In a project with tens of thousands of items, creating a baseline on the entire project will generate a high load on the system and take a long time to complete
  • Jama Admins can monitor which projects exceed the typical usage threshold in the Admin > System Health Report  
  • If your project size is larger than typical Jama usage, consider creating baselines on the specific sub-set or filter of items rather than the entire project
    • If an entire project baseline is necessary, consider creating it during off hours or times of low use 

12 

Export data to documents 

  • Reports and Word/Excel exports with hundreds of items is a modest load activity
    • Thousands of items generate very high load and high memory consumption
      • They also can take a very long time to complete, which destabilizes the system if they are run concurrently with other high load activities 
  • Jama Admins can monitor which export & report run times are exceeding the typical threshold in the Admin > System Health Report 

 

 

  • Consider splitting large reports into smaller documents if they must be generated frequently and consider generating very large reports during off hours  

13 

API scripts / integrations 

  • Integrations can run quickly and generate a lot of load by constantly polling Jama Connect’s API
  • Monitor how many integrations and API scripts are hitting the system concurrently 
  • Leverage Jama Connect's REST API throttling features to ensure that your integration doesn't introduce unnecessary load on your servers

14 

Dashboards 

  • Loading the project and project dashboard takes a long time 
  • Check to see if any complex filters that pull data from multiple projects are being utilized 
  • Adding widget that return results of complex filters will add considerable load to the system every time the project dashboard is accessed
    • When possible, use direct links to the filters


Preston
Jama Software