Preventing Scheduled Process Models from kicking off multiple instances after system downtime?

Certified Senior Developer
Does anyone have any recommended approaches to preventing a process model that runs on a recurring schedule from kicking off multiple instances after the system is down when it was scheduled to run? We have some processes that run nightly, but can create race-conditions if multiple instances try to run simultaneously.

Thinking about maybe putting in an "Advanced Options" Expression to force it to actually be the scheduled start time (or at least close), and just bypassing running until the next scheduled time that Appian is actually up on the server. Would that do the trick?

OriginalPostID-254265

  Discussion posts and replies are publicly visible

  • 0
    Certified Lead Developer
    In the past I've run process analytics to see if any active process instances exist for a given process model. I can then terminate the process if I see other active instances or continue on if no active instances exist.
  • 0
    Certified Senior Developer
    We've tried creating database locks for similar processes, but if they are started at roughly the same time, there are still race conditions between a lock being checked and the lock being set. (i.e. both processes may get through the gateway, and then both set the lock).
  • 0
    Certified Lead Developer
    If you want to go with a database locking schema instead of using process analytics, you can check out this post forum.appian.com/.../e-215481
  • 0
    Certified Lead Developer
    For scheduled downtime, before shutdown, we flip a constant to no. Each one of those models first evaluate the constant and if no go to end. Then change constant after they complete.
    For nonscheduled downtime, obviously this doesn't work.
  • 0
    Certified Senior Developer
    Christine - The constant method is out. We used this in a couple places, and ended up with major issues in production with our collaboration engine completely out of control (it grew to 42 GB as the constant amassed 55,000 versions). We had to delete the constant and re-import it as a temporary fix to the memory leak, but we're moving away from using the "Update Constant" smart service and using this pattern.
  • 0
    Certified Senior Developer
    I see you're talking about manually flipping a constant, rather than doing it automatically for a lock on the process.

    This complicates shut-down/start-up procedures and is an easy step to forget to perform. Hopefully we can come up with a decent programmatic solution.
  • 0
    Certified Lead Developer
    It seems to me like your 'advanced options' idea might be the best shot for now - i.e. install a gateway shortly after the start node to determine whether the current time is the same as the scheduled starting time (or maybe within a certain margin), and otherwise just exit.

    Also if the DB lock thing is still something you'd consider, I was able to devise a SQL 'trick' that will read a DB value while simultaneously incrementing it, tightly enough that 2 processes executing simultaneously wouldn't get a race condition. The same might work in your case.
  • 0
    Certified Lead Developer
    Keep in mind you could set a Boolean in the Db and execute a queryrule instead of using a constant.
  • 0
    Certified Senior Developer
    The trouble with using the DB is if two or more instances start simultaneously, there is still a race condition (since querying and writing are two separate transactions). But that might be a good use case for the locking mechanism / @Version annotation detailed in the above link.

    mschmitt - what was your SQL 'trick'? We do have the Execute Stored Procedure plugin installed, so doing an SP might be one way to accomplish a single transaction read+write.
  • 0
    Certified Lead Developer
    In my case I'm just using the Query DB node, using a command sequence I was able to find which basically locks the record for update. It's been a while, but I remember testing this by intentionally launching parallel instances simultaneously and it still worked as needed - ymmv. I'll transcribe the basic "setup screen" sql statements (with line numbers) here:

    1 -- START TRANSACTION
    2 -- SELECT COLUMN_TO_UPDATE FROM TABLE_NAME WHERE ID = ac!selectedId FOR UPDATE
    3 -- UPDATE TABLE_NAME SET COLUMN_TO_UPDATE = COLUMN_TO_UPDATE + 1 WHERE ID = ac!selectedId
    4 -- COMMIT

    This allows us to grab the current value of COLUMN_TO_UPDATE and pass that value back into the process via the data outputs, while also incrementing it by one. I assume similar could be done for other data types / operations, such as flipping a boolean flag, following this example.