https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-spaceen-USTelligent Community 12https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-spaceWed, 10 Feb 2021 04:01:00 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Parmida Borhanihttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsCurrent Revision posted to Appian Knowledge Base by Parmida Borhani on 2/10/2021 4:01:00 AM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space or continue growing indefinitely without clearing.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <p><span>Service Manager tells Kafka to cleanup old logs after it has performed 3 successful checkpoints. Every time Service Manager is restarted, that count will reset. Even if checkpointing is occurring every day, if Service Manager is also restarted every day, then the logs will never get cleaned up because it won&#39;t reach the 3 successful checkpoint threshold.</span></p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing and the directory continues to grow, then manually run the <a href="https://docs.appian.com/suite/help/latest/Checkpoint_Script.html" rel="noopener noreferrer" target="_blank">checkpoint script</a> 3 times to force Service Manager to cleanup the old logs.</p> <p>Refer to the document on <a href="https://docs.appian.com/suite/help/latest/Configuring_Application_Checkpointing.html" rel="noopener noreferrer" target="_blank">Configuring Application Checkpointing</a> for more information.&nbsp;</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed:&nbsp;February 2021</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: kafka, logging, engines, Architecture, disk usage, infrastructure</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/9Sun, 07 Feb 2021 22:41:03 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Parmida Borhanihttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 9 posted to Appian Knowledge Base by Parmida Borhani on 2/7/2021 10:41:03 PM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space or continue growing indefinitely without clearing.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <p><span>Service Manager tells Kafka to cleanup old logs after it has performed 3 successful checkpoints. Every time Service Manager is restarted, that count will reset. Even if checkpointing is occurring every day, if Service Manager is also restarted every day, then the logs will never get cleaned up because it won&#39;t reach the 3 successful checkpoint threshold.</span></p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing and the directory continues to grow, then manually run the <a href="https://docs.appian.com/suite/help/latest/Checkpoint_Script.html" rel="noopener noreferrer" target="_blank">checkpoint script</a> 3 times to force Service Manager to cleanup the old logs.</p> <p>Refer to the document on <a href="https://docs.appian.com/suite/help/latest/Configuring_Application_Checkpointing.html" rel="noopener noreferrer" target="_blank">Configuring Application Checkpointing</a> for more information.&nbsp;</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed: October 2019</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: kafka, logging, engines, Architecture, disk usage, infrastructure</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/8Mon, 14 Oct 2019 04:54:40 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Parmida Borhanihttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 8 posted to Appian Knowledge Base by Parmida Borhani on 10/14/2019 4:54:40 AM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space or continue growing indefinitely without clearing.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <p><span>Service Manager tells Kafka to cleanup old logs after it has performed 3 successful checkpoints. Every time Service Manager is restarted, that count will reset. Even if checkpointing is occurring every day, if Service Manager is also restarted every day, then the logs will never get cleaned up because it won&#39;t reach the 3 successful checkpoint threshold.</span></p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing and the directory continues to grow, then manually run the <a href="https://docs.appian.com/suite/help/latest/Checkpoint_Script.html" rel="noopener noreferrer" target="_blank">checkpoint script</a> 3 times to force Service Manager to cleanup the old logs.</p> <p>Refer to the document on <a href="https://docs.appian.com/suite/help/latest/Configuring_Application_Checkpointing.html" rel="noopener noreferrer" target="_blank">Configuring Application Checkpointing</a> for more information.&nbsp;</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed: October 2019</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: logging, engines, Architecture, disk usage, infrastructure</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/7Mon, 23 Jul 2018 12:58:43 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Jordan Horwathttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 7 posted to Appian Knowledge Base by Jordan Horwat on 7/23/2018 12:58:43 PM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing, contact Appian Technical Support.</p> <p>Refer to the document on <a href="https://docs.appian.com/suite/help/latest/Configuring_Application_Checkpointing.html" target="_blank">Configuring Application Checkpointing</a> for more information.&nbsp;</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed: January 2018</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: logging, engines, Architecture, disk usage, infrastructure</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/6Wed, 17 Jan 2018 02:10:58 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Parmida Borhanihttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 6 posted to Appian Knowledge Base by Parmida Borhani on 1/17/2018 2:10:58 AM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing, contact Appian Technical Support.</p> <p>Refer to the document on <a href="https://docs.appian.com/suite/help/latest/Configuring_Application_Checkpointing.html" target="_blank">Configuring Application Checkpointing</a> for more information.&nbsp;</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed: January 2018</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: logging, engines, Architecture, disk usage</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/1Tue, 16 Jan 2018 22:47:31 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Tristan Newmannhttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 1 posted to Appian Knowledge Base by Tristan Newmann on 1/16/2018 10:47:31 PM<br /> <h2>Symptoms</h2> <p>The Kafa transaction logs in the &lt;APPIAN_HOME&gt;/services/data/kafka-logs directory are using large amounts of disk space.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs store transactions that have yet to be persisted to the on disk KDB&#39;s through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for significant amount of time.</p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has come down. Ensure that configurations are not in place delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing, contact Appian Support.</p> <h2>Affected Versions</h2> <ul> <li>This article applies to all versions of Appian running version 17.3 or later</li> </ul><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: engines, architecture, disk usage</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/5Tue, 16 Jan 2018 21:10:16 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Parmida Borhanihttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 5 posted to Appian Knowledge Base by Parmida Borhani on 1/16/2018 9:10:16 PM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing, contact Appian Technical Support.</p> <p>Refer to the document on <a href="https://docs.appian.com/suite/help/latest/Configuring_Application_Checkpointing.html" target="_blank">Configuring Application Checkpointing</a> for more information.&nbsp;</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed: January 2018</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: logging, engines, architecture, disk usage</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/4Tue, 16 Jan 2018 21:09:58 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Parmida Borhanihttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 4 posted to Appian Knowledge Base by Parmida Borhani on 1/16/2018 9:09:58 PM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing, contact Appian Technical Support.</p> <p>Refer to the document on <a href="https://docs.appian.com/suite/help/latest/Configuring_Application_Checkpointing.html" target="_blank">Configuring Application Checkpointing</a> for more information.&nbsp;</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed: January 2018</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: engines, architecture, disk usage</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/3Tue, 16 Jan 2018 19:48:34 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Parmida Borhanihttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 3 posted to Appian Knowledge Base by Parmida Borhani on 1/16/2018 7:48:34 PM<br /> <h2>Symptoms</h2> <p>The Kafka transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs are responsible for storing transactions that have yet to be persisted to the KDBs on disk through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for a significant amount of time.</p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has decreased. Ensure that configurations are not in place that delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing, contact Appian Technical Support.</p> <h2>Affected Versions</h2> <p>This article applies to Appian 17.3&nbsp;and later.</p> <p>Last Reviewed: January 2018</p><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: engines, architecture, disk usage</div> https://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space/revision/2Tue, 16 Jan 2018 17:47:58 GMTd3a83456-d57b-489c-a84c-4e8267bb592a:02613d8d-12a2-469d-ad6e-3111dba39369Tristan Newmannhttps://community.appian.com/support/w/kb/780/kb-1455-kafka-logs-using-high-amounts-of-disk-space#commentsRevision 2 posted to Appian Knowledge Base by Tristan Newmann on 1/16/2018 5:47:58 PM<br /> <h2>Symptoms</h2> <p>The Kafa transaction logs in the <code>&lt;APPIAN_HOME&gt;/services/data/kafka-logs</code> directory are using large amounts of disk space.&nbsp;</p> <h2>Cause</h2> <p>The transaction logs store transactions that have yet to be persisted to the on disk KDB&#39;s through checkpointing. These logs can grow large and use higher amounts of disk space if the system has not checkpointed for significant amount of time.</p> <h2>Action</h2> <p>Initiate a checkpoint and verify that the disk usage in that directory has come down. Ensure that configurations are not in place delay or leave long periods between checkpointing. If the disk usage does not come down after checkpointing, contact Appian Support.</p> <h2>Affected Versions</h2> <ul> <li>This article applies to all versions of Appian running version 17.3 or later</li> </ul><div style="clear:both;"></div> <div style="font-size: 90%;">Tags: engines, architecture, disk usage</div>