Jama Performance In Enterprise Recommendations for System Administrators

By Preston Mitchell posted 05-02-2019 13:45

Like

Introduction

Jama Connect is a platform that is successfully used at scale for many different use scenarios. By design, Jama Connect can be configured and used in many ways. The fact that the platform is so configurable makes it difficult to provide “hard” guardrails and performance recommendations. Instead, the purpose of this document is to:

Define “typical” usage profile of Jama Connect
Provide a matrix where performance may be affected during specific usage scenarios
Provide recommendations to mitigate performance risks

Assumptions

This article assumes your Jama Connect instance is in our Cloud environment or, if Self-Hosted, the application is installed correctly utilizing supported software and hardware. We require that the application server be on a dedicated system with no other applications running, with a separate dedicated system for databases. It is acceptable to share the database server with databases for other applications.

Typical Usage Profile

A typical data usage profile is where users should not see any performance or scalability issues in typical day-to-day use of Jama Connect. There is always a small risk of performance impacts in any environment, though. Jama engineering teams extensively test Jama for performance, and you can read more about our performance testing approach here: Jama Connect Performance Whitepaper

When describing the typical usage profile, it is important to understand that Jama Connect differentiates “active” vs “inactive” items. Inactive items may be old versions of requirements or entire archived projects. These inactive items are not included in search index and do not materially impact performance of the system. All the numbers below reference “active” aka current version of items. So while a particular project may have 50,000 active requirements, there might be 5x inactive items for the project due to old versions of those items.

Category	Count (active)	Notes
Total items in database	3 million	No hard system limit but upper limit recommendation of 5 million active items, with much higher # of inactive items in database.
Average concurrent user sessions over course of single day	250	Jama supports many more total licensed users in systems. 250 concurrent users means 250 users performing application functions within a 5-minute window.
Average items per project tree	50,000	No hard-upper limit but above typical usage, users may encounter more performance issues moving items or doing project-wide actions like whole project baseline.
Average items per container (e.g. set/folder)	250	No hard upper limit but recommend upper limit of 1000. Set and folder containers are meant to help organize data.
Average relationships per item	5	Average # of relationships per item across all items. No hard-upper limit in system.
Average Test Plans per project	10	No hard-upper limit, many more Test Plans may exist in archived state.
Average Test Cases per Test Plan	2,500	No hard-upper limit, recommend upper limit below 5,000.
Average Cycles per Test Plan	5	No hard-upper limit, Recommend upper limit below 10 cycles per plan as each cycle creates a new Test Run item for each test case.
Average attachments/images per project	5,000	Typically these are image files stored on application server and will impact HD space needed for self-hosted customers. No hard-upper limit for Jama Cloud customers.

Performance Impact Matrix

Below is a summary of what functions may lead to performance issues in certain data profile situations.

Green: expect no performance issues
Yellow: may encounter rare performance issues depending on usage
Orange: more likely to see performance issues as # of concurrent users or volume of active items used exceeds the “typical usage” profile

The more complex and linked your data structure, the more related items you will pull into memory, and the more processor-intensive it will be to manage.

If you believe your organization usage of Jama will likely fall above “typical usage” profile and frequently utilize these functions, consider installing multiple Jama instances to split the data and load.

Typical Profile
If you are operating Jama Connect within our description of “typical usage” profile, the application will be very performant and reliable. Some functions, such as reusing very large # of items, will create a high load on the system. Jama Connect has built-in threshold warnings for users in these scenarios. For example, if a user plans to reuse 1000 items, they will get a pop-up warning that the batch reuse may affect performance. It is not a hard limit. Jama will still complete the task but it’s a guardrail to encourage users to do complex functions in smaller batches.

Concurrent Users Exceeds Typical Usage
If you are operating Jama Connect within our description of a “typical” profile but you have a higher # of concurrent users (greater than 250 performing multiple actions within 5-minute window. See our performance testing methodology for more explanation on the 5-minute window concept Jama Connect Performance Whitepaper)

Jama may still perform well in many usage scenarios, but you may encounter more performance issues or slowness as a higher volume of users are performing high-load actions on the system concurrently. For example, a higher number of users may concurrently create complex filter queries across entire system or export large documents with thousands of items which may stress the system.

Volume of Items Exceeds Typical Usage
If you are operating Jama Connect beyond the volume of items defined in the “typical usage” profile, you are more likely to see performance issues with specific functions. This is especially true if multiple users are performing high-load actions concurrently. Please carefully review the performance recommendations section. If you believe your organization usage of Jama will likely fall above “typical usage” profile and frequently utilize functions that may lead to performance issues, consider installing multiple Jama instances to split the data and load.

Performance Risks and Recommendations By Function

Periodic Performance Issues: If you find that your users are reporting that Jama periodically seems to ‘slow down’ and the issues do not appear to relate to any specific area.

Troubleshooting: Monitor the CPU/Memory utilization to see resources are being consumed by large operations. Talk to your power users and see if any of them are creating Reviews, Baselines or generating reports on large numbers of items.

Recommendations: Break large operations up into smaller operations. Run large operations at the end of the day when less users will be on the system. Increase system resources to reduce impact of periodic large operations.

Function-Specific Performance Issues

#	Function	Performance Issues	Troubleshooting	Recommendations
1	Viewing Items	List or Reading View takes a long time to load items	Check with your administrator to see if any known system outages or performance problems are occurring Check the total number of items being returned Look at the number of pages of data available (e.g. page 1 of ?) Check to see if large images are being displayed	Typically, users should not see performance issues viewing items If they do, it is usually indicative of larger system issues on server If you are viewing a large flat list of items, consider breaking data into multiple containers (250-1000 items per container) and try to limit searches/browsing to these containers Hide fields that have large amounts of data or images (e.g. Description Fields)
2	Comment on item	Opening the comments widget takes a long time to show comments Adding a new comment takes a long time.	Check with your administrator to see if any known system outages or performance problems are occurring Look to see if any of the comments contain large images	Typically, users should not see performance issues viewing items If they do, it is usually indicative of larger system issues on server Ensure users not to add extremely large images to the comment stream We have seen comments with dozens of 100mb images and this causes slowness in loading comment views
3	Create / Edit Item	Saving an item takes a long time or never completes	Check with your administrator to see if any known system outages or performance problems are occurring	Typically, users should not see performance issues creating or editing single items If they do it is usually indicative of larger system issues on server
4	View traceability	Loading the initial Trace View and navigating upstream/downstream takes a long time Exporting Trace View takes a long time	Jama Connect is designed to store a complex network of trace relationships typical in System Engineering and Requirements Management If your usage exceeds the "typical usage" profile, retrieving a broad and deep hierarchy from Jama Connect can generate long-running transactions, which will reduce availability for other users The Trace View is designed to show thousands of items and their relationships It loads items incrementally as the user navigates, allowing relationship data exploration Due to the live and dynamic nature of the view, the volume of data the browser and Jama Connect must concurrently track may be very high	To view traceability for large scale data, it is recommended users create a filter of only the relevant data they want to explore, then use that filter to activate Trace View Users can search and filter data in the List View before opening the Trace View Further filtering is available from the Trace View as well.
5	Search for item(s)	Searching for items may affect performance if a high # of concurrent users are routinely querying across the entire data base (e.g. using "All Projects" filter or search)	Check with your administrator to see if any known system outages or performance problems are occurring	By default, all searches in Jama are project specific (users must proactively change to search across all projects) Encourage users to perform targeted searches within specific projects Monitor server load during high usage spikes
6	Batch edit/delete items	Batch editing dozens or hundreds of items is a modest load activity Batch editing thousands of items will have a performance cost as it causes a high load of concurrent application requests If you need to delete hundreds (or thousands) of items, it's much faster to delete the containers containing those items rather than the items themselves (e.g. from a batch select in List View	Determine if users are routinely batch editing or deleting items If yes – is there an alternative workflow that would be more performant?	Use batch edit function in reasonable batches (hundreds or less) or during off hours for a larger load Delete items at container level in tree or use a filter in the list view to only view containers you wish to delete
7	Move item(s) in tree	Moving an item in the tree causes Jama to renumber the tree location and heading # (e.g. 3.1.4) of all items in the project We've observed that moving items and renumbering the placement in explorer tree is fast enough for projects with thousands of items, but slower for projects with tens of thousands This is exacerbated if the project has thousands of items in a flat list under single containers (e.g. a set or folder) A flat list of thousands of items is ok for archived projects but may cause slowness if you expect users will routinely move the items in that large flat list The Project Explorer tree setting 'Max items displayed in explorer tree" setting represents the max items the tree will display to a user in a single container (one level of hierarchy) It does not indicate the total items in the project tree or the total items that will display in the Reading/List View Because most customers organize items in multiple sets, folders and sub-folders, customer rarely hit this max, or generally prefer to organize the data for easier consumption	Check to see if there are large, flat lists in the Jama project Understand why users are batch moving items routinely and see if there is an alternative workflow.	Utilize a logical structure of sets and folders and avoid very long, flat lists in Explorer Tree It is more performant to move folders of items instead of individual items This is especially true when moving large numbers of items as Jama needs to update the tree location and heading # for each item The Project Explorer tree setting 'Max items displayed in explorer tree" should remain below 500 This number indicates the max number displayed in a flat list under a single set/folder container
8	Participate in a review	The Review Center was designed for reviewing a typical-sized document Reviews with thousands of items are not recommended If you create larger reviews, they will perform slowly and generate high load, not to mention some unhappy reviewers!	Check the number of items included in the review A typical review would contain less than 250 requirements items This is not a hard limit and Jama Admins can monitor which reviews exceed the typical threshold in the Admin > System Health Report	We recommend reviews with less than 250 items, not only for performance reasons, but also as a business best practice Limiting the volume of items allows a more agile process by facilitating more frequent, iterative reviews
9	Reuse items	Reusing dozens or hundreds of items is a modest-load activity Reusing thousands, tens, or hundreds of thousands of items generates a lot of load During the reuse operation, items involved will be locked and others will be unable to edit until reuse is complete	Jama Connect has a guardrail warning if a user plans to reuse a large # of items This is not a hard limit, just a warning that it may take some time to complete	Consider running large reuse operations during off-hours or low load times Avoid syncing thousands of items reused items at a time
10	Create Test Cycles & Runs	Jama Connect was designed to create and manage manual tests, assuming hundreds of items in a cycle Test plans that exceed the typical usage profile may cause Jama Software to perform more slowly Jama Connect can be linked to an automated test system Be thoughtful about how much test data you are inserting automatically through the REST API For example, adding automated nightly test results to 100,000 test cases, you will quickly exceed recommended system volume of the typical usage profile	Jama Admins can monitor which Test Plans exceed the typical threshold in the Admin > System Health Report	Consider breaking your large test plans into multiple, smaller plans Automated test integrations - do not populate Jama with thousands of nightly test results Instead, consider only integrating results from final build / regression runs
11	Create baselines	In a project with tens of thousands of items, creating a baseline on the entire project will generate a high load on the system and take a long time to complete	Jama Admins can monitor which projects exceed the typical usage threshold in the Admin > System Health Report	If your project size is larger than typical Jama usage, consider creating baselines on the specific sub-set or filter of items rather than the entire project If an entire project baseline is necessary, consider creating it during off hours or times of low use
12	Export data to documents	Reports and Word/Excel exports with hundreds of items is a modest load activity Thousands of items generate very high load and high memory consumption They also can take a very long time to complete, which destabilizes the system if they are run concurrently with other high load activities	Jama Admins can monitor which export & report run times are exceeding the typical threshold in the Admin > System Health Report	Consider splitting large reports into smaller documents if they must be generated frequently and consider generating very large reports during off hours
13	API scripts / integrations	Integrations can run quickly and generate a lot of load by constantly polling Jama Connect’s API	Monitor how many integrations and API scripts are hitting the system concurrently	Leverage Jama Connect's REST API throttling features to ensure that your integration doesn't introduce unnecessary load on your servers
14	Dashboards	Loading the project and project dashboard takes a long time	Check to see if any complex filters that pull data from multiple projects are being utilized	Adding widget that return results of complex filters will add considerable load to the system every time the project dashboard is accessed When possible, use direct links to the filters

1 comment

1135 views

Permalink

Comments

Devang Sanghrajka

08-02-2022 08:21

Dear Jama Team,

I understand that this page may be valid... however, I have seen a few performance related updates in the past few releases. Will this page be updated accordingly? or will it updated to the latest performance numbers as needed?

Thank you!

Blogs