RE: Process instances and checkpoints

Peter Lewis — Wed, 05 Aug 2020 12:54:12 GMT

In this case a transaction refers to any time we update data, so it most cases it's even more granular than execution of a node. For instance, if I start a node, the node now becomes active (so its status has changed), so that would be committed as a transaction. However, if a node is reassigned to someone else, that would also update information and would require a transaction.

The best way to think about it is if you are able to see the information in the process monitoring view, then that information has already been committed!

RE: Process instances and checkpoints

scristalli — Wed, 05 Aug 2020 07:07:01 GMT

Thank you Peter Lewis for the very detailed answer.

May I ask what consitutes a transaction inside a process instance? Does the execution of every node count as a separate transaction, or only certain nodes are considered for "commit"? Thank you.

RE: Process instances and checkpoints

Peter Lewis — Tue, 04 Aug 2020 22:15:01 GMT

This is not entirely true - process instances live in memory until they are archived. You can set an archival policy to archive your process instance data a certain number of days after the process is completed or canceled.

I think it's useful to discuss a bit more about how Appian engines work. In Appian, we have a bunch of different kinds of engines that store data related to running Appian. For example, we have engines related to documents (content engine), users (personalization engine), processes (process exec and analytics engines), etc. The main engine you're talking about above is the process execution engine, which stores information about each in-flight process.

There are also two main parts to the engine: the in-memory database that stores your data and a .kdb file that is saved to disk. The .kdb file consists of a snapshot of the database and a transaction file. It's important to note that any transaction that occurs in Appian will write to both the in memory database and the transaction log in the .kdb file. This way, even if there is a sudden shutdown, we still have all the transactions saved to disk (so we can replay all the transactions to get the database to its proper state).

However, as you can imagine the transaction log can get to be quite large. If we actually did need to replay all the transactions from the beginning, it could take quite a while. So, this is where we use checkpointing. The purpose of checkpointing is to copy the current state of the in-memory database to the .kdb as the database snapshot. Now, if a shutdown occurs, we only have to replay the transactions since the last snapshot, which should allow us to restart Appian much more quickly.

All this to say, it shouldn't be necessary to perform a checkpoint as a step in an Appian process because the engine data is always stored both in memory and on disk. In fact, Appian will automatically checkpoint if the expected time to replay all the transactions since the last successful checkpoint exceeds a given threshold.