Scaling Appian Infra without downtime

Question

Hi Team, 
 During one of the recent external assessment of our system reliability ,one of the questions came up was as follows 
 --It is the peak time of your business 
 --Suddenly you notice that the engine is choking due to load 
 In this scenario ,is there any way to scale up on the fly without causing any downtime to your users.

naveenp878 · Accepted Answer

Hi George,

There are many issues can decrease the site performance. Predominantly the heap memory issues for the Appian production or platform support teams in any company. The below points may benefit for better handling the performance issues.

1. You can use the plugin application 'log reader' (community.appian.com/.../log-reader) to monitor the heap memory, system memory and engine performance etc., This application also facilitates to inform prior to the application users or developers on specific threshold has been crossed so that we can take some pro-active measures to minimise the memory and other engine performance issues. This is very useful application for monitoring purposes.
2. Firstly try to identify stranded transaction instances, which are hanging and either kill or monitor them at the earliest.
3. Observe the expression rules executing over the period of time. for example expression rules executing more than 5s for the certain volume. If you are using 17.x versions, under admin console performance section we can find these expressions and can fix them accordingly.
4. Observe any infinite loop is running in the process instances and you may monitor them.
5. If you are using cloud and having VPN connections for consuming any services, then find out the connection are good and intact. This issue is basically comes due to expiration of threads in the Appian memory.
6. You may monitor the users activity/volume for the past hour in admin console under user activity section.
7. refer the system logs and observe any exceptions recorded.
8. Post martom analysis, use another plugin application 'Appian health check' (community.appian.com/.../appian-health-check) which reads the entire logs in the system for the past 90 days and gives you various metrics to analyse what could the anomalies and trouble some design objects.

Hope these points helps you to better monitoring especially the production environments.

Thanks

jims419 · Answer

There are many different "layers" in the Appian architecture, and every app has its own performance/usage profile. The engine layer would be hard to scale without downtime, but I would be very surprised if this was subject to short term spikes. The app sever layer may be at more risk for this, but it should be feasible to add nodes into the app server cluster seamlessly. Database is another layer that can see performance bottlenecks, but that's usually in the hands of the DBAs.