The Kafka transaction logs in the <APPIAN_HOME>/services/data/kafka-logs directory are using large amounts of disk space or continue growing indefinitely without clearing.
<APPIAN_HOME>/services/data/kafka-logs
The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.
Service Manager tells Kafka to cleanup old logs after it has performed 3 successful checkpoints. Every time Service Manager is restarted, that count will reset. Even if checkpointing is occurring every day, if Service Manager is also restarted every day, then the logs will never get cleaned up because it won't reach the 3 successful checkpoint threshold.
Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing and the directory continues to grow, then manually run the checkpoint script 3 times to force Service Manager to cleanup the old logs.
Refer to the document on Configuring Application Checkpointing for more information.
This article applies to Appian 17.3 and later.
Last Reviewed: February 2021