You are currently reviewing an older revision of this page.
Illustrate and explain the process of a single engine checkpointing, and how the associated output appears in the <APPIAN_HOME>/logs/service_manager.log.
<APPIAN_HOME>/logs/service_manager.log
The Appian Engines use a process called "checkpointing" to persist data stored in the in-memory engines to disk. This is triggered either manually or automatically.
Note: The remainder of the document will use the following notation:
<APPIAN_HOME>/services/data/temporary
<APPIAN_HOME>/services/conf/service_manager.conf
<APPIAN_HOME>/services/data/archived
<APPIAN_HOME>/server/process/exec/00/gw1
When a checkpoint is first requested or triggered, it is placed into a queue within the Service Manager component. Checkpoints occur serially, in the order that they are placed into the queue. In the logs, this looks like:
YYYY-MM-DD HH:MM:SS.mmm [KomodoEventBus-XXXX] INFO com.appian.komodo.engine.status.CheckpointingStateProvider - Checkpoint <checkpoint id> requested for: <engine id>
Service Manager will also log output indicating the type of checkpoint that was requested, e.g. automatic or user triggered.
Automatic:
YYYY-MM-DD HH:MM:SS.mmm [CheckpointManager-<engine id>-X] INFO com.appian.komodo.engine.checkpoint.CheckpointManager - HandlingCheckpointRequestedEventAutoCheckpointRequestedEvent{createdInstant=YYYY-MM-DDTHH:MM:SS.mmmZ,engineId=<engine id>>, requestId=<checkpoint id>,engineShuttingDown=false, checkpointContext=null}YYYY-MM-DD HH:MM:SS.mmm [CheckpointManager-<engine id>-X] INFO com.appian.komodo.engine.checkpoint.CheckpointManager - QueueingCheckpointRequestedEventAutoCheckpointRequestedEvent{createdInstant=YYYY-MM-DDTHH:MM:SS.mmmZ,engineId=<engine id>, requestId=<checkpoint id>,engineShuttingDown=false,checkpointContext=CheckpointContext{checkpointId=<checkpoint id>,minTxId=XXXX, kdbFileIndex=XXX,retentionMetadata=serviceManager.transaction.<engine id>-X@XXXX,initiatingHost=<host>, executionHost=<host>}}
User Requested (using the checkpoint script):
checkpoint
YYYY-MM-DD HH:MM:SS.mmm [CheckpointManager-<engine id-X] INFO com.appian.komodo.engine.checkpoint.CheckpointManager - HandlingCheckpointRequestedEventUserCheckpointRequestedEvent{createdInstant=YYYY-MM-DDTHH:MM:SS.mmmZ,engineId=<engine id>, requestId=<checkpoint id>,engineShuttingDown=false, checkpointContext=null}YYYY-MM-DD HH:MM:SS.mmm [CheckpointManager-<engine id-X] INFO com.appian.komodo.engine.checkpoint.CheckpointManager - QueueingCheckpointRequestedEventUserCheckpointRequestedEvent{createdInstant=YYYY-MM-DDTHH:MM:SS.mmmZ,engineId=<engine id>, requestId=<checkpoint id>,engineShuttingDown=false,checkpointContext=CheckpointContext{checkpointId=<checkpoint id>,minTxId=XXXX, kdbFileIndex=X,retentionMetadata=serviceManager.transaction.<engine id>-X@X,initiatingHost=<host>, executionHost=<host>}}
As can be seen, in both of the above outputs, the CheckpointRequestedEvent is handled by putting it on the checkpointing queue, to be processed.
CheckpointRequestedEvent
Service Manager will dequeue a checkpoint request in order to process it. This is associated with:
YYYY-MM-DD HH:MM:SS.mmm [KomodoEventBus-XXXX] INFO com.appian.komodo.engine.status.CheckpointingStateProvider -Checkpoint <checkpoint id> dequeued for: <engineid>YYYY-MM-DD HH:MM:SS.mmm [KomodoEventBus-XXXX] INFO com.appian.komodo.engine.checkpoint.CheckpointWatcher - Waiting for checkpoint <checkpoint id> for <engine id>YYYY-MM-DD HH:MM:SS.mmm [KomodoEventBus-XXXX] INFO com.appian.komodo.engine.status.CheckpointingStateProvider - Checkpoint <checkpoint id> started for <engine id>
After this output, Service Manager will be actively checkpointing the specific engine that the request was associated with. While an engine is being checkpointed, a temporary *.writing kdb is created in the temporary folder associated with the engine undergoing a checkpoint. This is usually located at <APPIAN_HOME>/services/data/temporary/<engine id>.
<APPIAN_HOME>/services/data/temporary/<engine id>
Service Manager will complete the checkpoint operation and perform some operations on the file system:
<APPIAN_HOME>/server/**/gw1
These appear in the logs as:
YYYY-MM-DD HH:MM:SS.mmm [checkpoint-watcher-<engine id>-XXX-X] INFO com.appian.komodo.engine.checkpoint.LocalCheckpointWatcherTask - Notified of Kimage created for checkpoint <checkpoint id> file<path to temporary>/temporary/<engine id>/<engineid>-<checkpoint id>.written at YYYY-MM-DD HH:MM:SS.mmmZYYYY-MM-DD HH:MM:SS.mmm[checkpoint-watcher-<engine id>-XXX-X] INFO com.appian.komodo.engine.checkpoint.SingleGatewayCheckpointCopier - Moving KCheckpoint to next live: <path to temporary>/temporary/<engine id>/<engine id>-<checkpoint id>.written to/<APPIAN_HOME>/server/<path to engine gateway>/gw1/<kdb name>.kdbYYYY-MM-DD HH:MM:SS.mmm [checkpoint-watcher-<engineid>-XXX-X] INFO com.appian.komodo.engine.checkpoint.LocalCheckpointWatcherTask - Finishedcheckpoint <checkpoint id> for <engine id>YYYY-MM-DD HH:MM:SS.mmm [KomodoEventBus-XXXX] INFO com.appian.komodo.engine.status.CheckpointingStateProvider - Checkpoint<checkpoint id> completed for <engine id>YYYY-MM-DD HH:MM:SS.mmm [CheckpointManager-<engine id>-X] INFO com.appian.komodo.engine.checkpoint.CheckpointManager - Checkpoint <checkpoint id> execution complete
Once the checkpoint is complete Service Manager will perform cleanup of the prior "live" kdb, and will also instruct the Kafka component to mark any transactions that were persisted to disk in the checkpoint as ready for cleanup. In the logs, this is associated with:
YYYY-MM-DD HH:MM:SS.mmm[checkpointArchiver-XXX-X]INFO com.appian.komodo.engine.checkpoint.SingleGatewayCheckpointCopier -Archiving kdb image from <APPIAN_HOME>/server/<path to engine gateway>/gw1/*.kdb to <path to archived>/archived/<engineid>/*.kdb...YYYY-MM-DD HH:MM:SS.mmm [KomodoEventBus-XXXX]INFO com.appian.komodo.log.KafkaLogCleaner - Updating committed offset for serviceManager.transaction.<engine id>-X from XXXX to XXXXYYYY-MM-DD HH:MM:SS.mmm [KomodoEventBus-XXXX] INFO com.appian.komodo.log.KafkaLogCleaner - Marking all transaction log entries for <engine id> that are an earlier offset than XXXX for deletion.
This article applies to Appian 17.3 and later. The output was captured on Appian 19.4 and may be slightly different on older / newer versions of Appian.
Last Reviewed: January 2020