a!Start process-more than 1000 instances

Question

Hi All, 
 can use MNI configuration for start process smart service. As we have an advantage of process will start different or same engine based on the load and memory consumption will not be much . 
 
 Bcoz, we have design where we need start more than 1000 process instances ? 
 
 will it allow iterate more than 1000 atleast incase of start process smart service -since memory issue will handled by loading into different engine?

Madhusudan Sharma · Answer

Hi Aravind , Appian doesn't allow more than 1000 nodes instances, even if we are using Start Process Smart Service with the MNI configurations. Please find the below screenshots as a PoC. 
 Parent process and configs:

Started process for debugging:

However, if you want to invoke a sub-process more than 1000 times, I would suggest creating another parent process and calling your original parent process with similar MNI configs. 
 Hope it answers your question.

davel001150 · Answer

As Chris said, the question you need to ask yourself is not "How can we make over 1000 processes?" The question you need to ask yourself is rather, "How can I do what I want without creating over 1000 processes?" Why? Too many processes are bad. That's why Appian limits to 1000. That's also why, if I remember rightly, Appian won't allow you to run an a!startProcessLink with an a!forEach to let you spawn a million of them.

Madhusudan Sharma · Answer

Another solution which I can think of is, using a loop instead of MNI. You can create a loop and increment the count based on the number of instances you want to create. To avoid the 1000 node instances error, select the " Delete previously completed/cancelled instances " in the "Other" tab of the nodes (Script Task and Start Process Smart Service). Refer to the below screenshots.

Deleting the instances will free up the memory thus, won't affect even if you create 1000+ instances. However, I advise you to revisit your design and try to process in batches, if possible.

davel001150 · Answer

If you really insist on it, what you need to do is start a certain number of processes (20, 1000) and have each of those take a batch of your data, and each load their own 1000 processes with startProcess. You can probably have all the spawners run asynchronously, and each of them will have 1000 subprocesses. That can get you up to a million. I've done that to try to quickly create 10 million folders on a Docker container before. That said, I was doing it on a container where I wasn't worried about breaking anything. If it got destroyed, I could just spin up a new one. Let me tell you, it got destroyed. The million processes didn't end before I ran out of system resources and my server died. I was doing them sequentially; 1000 simultaneously making 1000 each one at a time. Never more than 1000 running at a time, and my server still died.

Danny Verb · Answer

Appian just released a feature to execute more than 1000 instances docs.appian.com/.../Appian_Release_Notes.html