Hi All,
There is a scheduler process that runs hourly and fetches data from a third-party system, then syncs it into an Appian table. This process is handling around 1,400 records.
For each record, data needs to be queried from an integration. Currently, the parent process batches 50 records at a time and calls a sub-process to process each record.
Inside the sub-process, the integration call is made, and the data is constructed and synced into the Appian table. In total, 29 batches are running.
Since this runs hourly, it is taking around 12 minutes to complete in production for all 29 batches. However, the sub-process is being highlighted as one of the top processes consuming engine time, and it is also consuming a noticeable amount of CPU.
I’m not sure if this is missing any design guidance. Can someone help how to resolve this health check highlight?
Note- Each integration call takes around 600 ms.
Health check Highlight is below.
Discussion posts and replies are publicly visible
Many environments have similar legacy schedulers - full revamps cost extra effort. My Suggestion, Try running sub-processes asynchronously for engine balancing and reduce frequency if possible (e.g., 2-hourly). If it still flags, open a support ticket to suppress this health check point as a known design issue; Appian can whitelist it so it won't flag future runs. Health checks guide best practices, but exceptions exist for valid legacy cases.
Thank you for the suggestions!
The reason we kept it synchronous is to regulate number of API's being made at the same time. If we make it async, then many concurrent calls may happen and also many concurrent instances might get created in system. So I went with synchronous (Delete completed instances).
Yeah, I am also going to suggest making the scheduler 2 hour once to client.
Earlier this MNI was present as a single expression rule inside which API is being called in foreach. The rule was being called from script task. Which was triggering long running work in backend. After which this is changed as MNI with 250 batch which has created memory spike & CPU utilization. Now reduced batch size to 50. We need to monitor the CPU for a while to see whether the memory became stable.
I have also made sure this job doesn't overlap with other Scheduler's timing in Production to make sure system resource is not getting overloaded with multiple items at the same time