KB-2307 Engine disconnect "Unable to append transaction *** for <engine> to the transaction log"

Symptoms

Engines disconnect with the following exception inside the service_manager.log:

YYYY-MM-DD HH:MM:SS,mmm [TransactionLogHandler-<engine>-99-1] ERROR com.appian.komodo.translog.TransactionLogHandler - Unable to append transaction ######### for <engine> to the transaction log [EngineRequest{updateFlag=true, functionName=performEngineWork, params=null, credentials=null, localeId=, timeZoneId=, timestampInteger=-#########, timestampDouble=-#########, randomInteger=#########, isForMigration=false, transactionId=0, interfaceName=PROCESS, shard=null, engineTopologyName=null, apiVersion=9, correlationId=[-34, -106, 76, -8, -64, 5, 64, 101, -120, -27, -120, -66, 84, -39, -18, 9]}]
com.appian.komodo.translog.TransactionSequenceException: ######### is not the next transaction id in the sequence after #########
at com.appian.komodo.log.KafkaTransactionLogService.append(KafkaTransactionLogService.java:187) ~[core-1.236.29.jar:1.236.29]
at com.appian.komodo.translog.TransactionLogHandler.persistRequestToLog(TransactionLogHandler.java:201) ~[core-1.236.29.jar:1.236.29]
at com.appian.komodo.translog.TransactionLogHandler.lambda$channelRead$0(TransactionLogHandler.java:180) ~[core-1.236.29.jar:1.236.29]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_252]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.46.Final.jar:4.1.46.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
YYYY-MM-DD HH:MM:SS,mmm [TransactionLogHandler-execution01-99-1] ERROR com.appian.komodo.translog.TransactionLogHandler - Writing failed inflight request to .l file
YYYY-MM-DD HH:MM:SS,mmm [KomodoEventBus-58103] ERROR com.appian.komodo.engine.connection.KProcessManager - Terminal engine fault received for <engine>. Killing K process. [event received: TransactionAppendFailedEvent{createdInstant=YYYY-MM-DDTHH:MM:SS.mmmZ, engineId=execution01, transactionId=#########}]
YYYY-MM-DD HH:MM:SS,mmm [KomodoEventBus-58149] ERROR com.appian.komodo.engine.EngineSupervisor - Received terminal engine fault: TransactionAppendFailedEvent{createdInstant=2020-07-17T16:42:59.611Z, engineId=<engine>, transactionId=#########}

The engine is able to recover in a few minutes:

YYYY-MM-DD HH:MM:SS,mmm [KomodoEventBus-58212] INFO com.appian.komodo.engine.status.EngineStatusMonitor - <engine> is now EngineStateAndMode{engineState=RUNNING, mode=PRIMARY}

Cause

Another process is accessing the log files which the K engine is trying to write to. On Windows, another process having the file open will completely prevent K from writing to the file. This can force the engine into a bad state and cause the above error. 

Action

Disable any other processes from accessing the files which the K engines write to. In particular, customers using Windows have seen this issue when running some sort of file scanner (often an antivirus or security tool) which scans Appian files. For more information, please refer to the relevant documentation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: May 2024

Related
Recommended