Performance and Load Testing

Why Performance Test?

Performance testing can be used to answer critical questions about the application, such as:

  • Will pages load fast enough to satisfy end users?
  • Can the application store all of the data that will be collected?
  • Will performance be consistent over time?
  • Which areas of my application should I improve?

Answering these types of questions reduces risk, informs current business decisions and assists future planning. Performance testing is needed to determine if the application will perform within specification given a particular situation. It also validates if the resources provided to the application are sufficient for the given volume and data load. The "right" amount of performance testing can only be determined by the project sponsor and stakeholders after considering the importance or visibility of the application, potential revenue loss due to poor performance, and other risk factors.Appian designers should ensure that end user interfaces perform within the specification before they promote the feature to higher environments. If interfaces do not perform adequately during unit testing, they will not improve during performance testing. There are several ways that interface performance can be captured and various troubleshooting techniques. Please see Interface Performance and Debugging for more details.

Process Performance Baseline

When designing process models, designers need to pay attention to the response times between user tasks to ensure appropriate user experience. Similarly to interfaces, processes need to perform appropriately during unit tests, before they can be accepted for performance testing. Performance of individual activities is recorded in logs, and designers can utilize the Appian Health Check to find out if there were any expressions or smart services that could impact end users’ experience. Health Check also provides an HTTP Response Time chart that provides an aggregate view of performance within the environment. These tools should be utilized after every round of functional testing to monitor emerging performance risks. Absence of Health Check should not be a blocker to analyze performance test results. You can still analyze Appian Logs in their tool of choice (Splunk, SolarWinds, Grafana, New Relic, Excel).  Additionally, designers can use network monitoring tools built into their browser, to record the exact performance of requests between user clicks during their unit testing.

Performance Testing

Performance testing should be done once the application is in a stable build with the majority of the functionality built out. The app does not have to be complete to get started, but needs to pass the QA/QC cycle, so that performance results are not skewed by application errors. Additionally, performance testing needs to be executed for major releases of the application. Over time, the test will grow to include additional applications and functionality as your platform use evolves.

There are two load factors to consider during performance testing:

  • Data Load - how much data is in the system during performance testing? This is a frequently overlooked factor but applications need to be tested with a pre-loaded set of data in order to ascertain how it will perform in a realistic situation. Performance testing is often executed with blank databases, which will not be the case 3, 6, 12 months or even at application go-live. Results from such testing will be inaccurate.  Please read the Database Volume Testing play for tips on how to load data into your environment.
  • Load Volume - how many users are in the system creating transactions? When designing your test, you need to find out how many business processes are executed in a given timeframe by how many users. You also need to make sure that all other significant activity in the system is tested, to set up a realistic transaction volume scenario. Tests need to be designed to emulate real user activity, with appropriate pauses between interactions.

More details on methodology, complete with test design templates and helpful tips, can be found under Performance Testing Methodology.

Performance testing should have its own project plan and it should be considered its own agile project. A backlog of performance improvement fixes will start to build with each nightly performance test run. The business should prioritize the improvements based on impact and severity. Promote performance fixes on a nightly basis and rerun the performance test to determine the impact of each change. 

Tools

You can execute performance tests using standard web application performance testing tools. These include JMeter and LoadRunner. Performance test with the framework your team has the most experience with.