Hi all,
I have a process that I want to run, for the amounts of data/records on an array, BUT, I want to control the number of instances it runs each time.
For example, I have 20k records but I want the process to execute 30 each time.
Discussion posts and replies are publicly visible
mrios0001 said:For example, I have 20k records but I want the process to execute 30 each time.
There are multiple strategies, one way is to query batchsize 30 in a!queryRecordType() and kickoff Start Process Smart Service for each of the 30 instances and a way to check if the processes are complete and kickoff next batch (One of the ways to check if the kickoff subprocesses are completed using send/receive message nodes).
Kicking off lot of processes this way is not recommended/not a best practice. There are some cases it is fine, like initial application setup/data migration etc.
Please refer to similar use case
You could create a super-process that passes the next 30 from the list into a sub-process, which is synchronous and arranged in a loop. The sub-process it's called runs a subprocess MNI, once for each item in the 30. Configure it so that all instances start at the same time, but the node doesn't move on until all instances are complete. In the super-process, if there are more, collect the next 30, and pass as the parameter in the subprocess again. You need to nest the subprocesses 2 levels deep so you don't hit the 1000 limit (Instead you have a theoretical 1,000,000 limit).
That should get you continuously processing no more than 30 at a time until they're all done, (Though you may run into difficulty with the 1000 MNI limit should the numbers significantly change. You could do 50 in parallel per sub-loop and not find many problems 'til you hit 50,000)
I highly suggest some investigation to see if you can achieve the same result in an expression using a!forEach, no process modeler at all. If it is possible, it's also possibly several orders of magnitude faster than this approach.
If you are still looking for a potential solution you could look into the Transaction Manager located in the App Market. This App also gives you the ability to adjust/tune batch sizes/cycle time to obtain better performance and to load balance batches of through your application servers. Another benefit is that you can throttle the batches so that they don't overload your servers.
community.appian.com/.../transaction-manager