a!Start process-more than 1000 instances

Certified Lead Developer

Hi All,

can use MNI configuration for start process smart service. As we have an advantage of process will start different or same engine  based on the load and memory consumption will not be much .

Bcoz, we have design where we need start more than 1000 process instances ? 

will it allow iterate more than 1000 atleast incase of start process smart service -since memory issue will handled by loading into different engine?

  Discussion posts and replies are publicly visible

Parents
  • 0
    Certified Lead Developer

    If you really insist on it, what you need to do is start a certain number of processes (20, 1000) and have each of those take a batch of your data, and each load their own 1000 processes with startProcess.  You can probably have all the spawners run asynchronously, and each of them will have 1000 subprocesses.  That can get you up to a million.  I've done that to try to quickly create 10 million folders on a Docker container before.  That said, I was doing it on a container where I wasn't worried about breaking anything.  If it got destroyed, I could just spin up a new one.  Let me tell you, it got destroyed.  The million processes didn't end before I ran out of system resources and my server died.  I was doing them sequentially; 1000 simultaneously making 1000 each one at a time.  Never more than 1000 running at a time, and my server still died.

Reply
  • 0
    Certified Lead Developer

    If you really insist on it, what you need to do is start a certain number of processes (20, 1000) and have each of those take a batch of your data, and each load their own 1000 processes with startProcess.  You can probably have all the spawners run asynchronously, and each of them will have 1000 subprocesses.  That can get you up to a million.  I've done that to try to quickly create 10 million folders on a Docker container before.  That said, I was doing it on a container where I wasn't worried about breaking anything.  If it got destroyed, I could just spin up a new one.  Let me tell you, it got destroyed.  The million processes didn't end before I ran out of system resources and my server died.  I was doing them sequentially; 1000 simultaneously making 1000 each one at a time.  Never more than 1000 running at a time, and my server still died.

Children
No Data