KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Ryan Good — Tue, 30 Jul 2024 14:19:20 GMT

Current Revision posted to Appian Knowledge Base by Ryan Good on 7/30/2024 2:19:20 PM

Symptoms

After an unexpected shutdown of the entire cluster which hosts Appian high availabilty servers, Kafka fails to start up on a particular node with the following type of error seen in the <APPIAN_HOME>/logs/service-manager/kafka/server.log :

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

DISCLAIMER:

This KB article is currently under review given its age and the relevant updates in the product since its last review. At this time, it is advised to not following the steps for resolution.

If you believe that your environment is currently experiencing the below behavior, please open a case with Appian Support for further review.

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: high availability, kafka, engines, zookeeper, infrastructure

KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Ryan Good — Tue, 30 Jul 2024 13:31:45 GMT

Revision 15 posted to Appian Knowledge Base by Ryan Good on 7/30/2024 1:31:45 PM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

DISCLAIMER:

This KB article is currently under review given its age and the relevant updates in the product since its conception. At this time, it is advised to not following the steps for resolution.

If you believe that your environment is currently experiencing the below behavior, please open a case with Appian Support for further review.

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: high availability, kafka, engines, zookeeper, infrastructure

KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Ryan Good — Tue, 30 Jul 2024 13:30:50 GMT

Revision 14 posted to Appian Knowledge Base by Ryan Good on 7/30/2024 1:30:50 PM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

DISCLAIMER:

This KB article is currently under review given its age and the relevant updates in the product since its conception. At this time, it is advised to not following the steps for resolution.

If you believe that your environment is currently experiencing the below behavior, please open a case with Appian Support for further review.

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: high availability, kafka, engines, zookeeper, infrastructure

KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Ryan Good — Tue, 30 Jul 2024 13:30:02 GMT

Revision 13 posted to Appian Knowledge Base by Ryan Good on 7/30/2024 1:30:02 PM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

DISCLAIMER:

This KB article is currently under review given its age and the relevant updates in the product since its conception. At this time, it is advised to not following the steps for resolution.

If you believe that your environment is currently experiencing the below behavior, please open a case with Appian Support for further review.

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: high availability, kafka, engines, zookeeper, infrastructure

KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Ryan Good — Tue, 30 Jul 2024 13:26:08 GMT

Revision 12 posted to Appian Knowledge Base by Ryan Good on 7/30/2024 1:26:08 PM

DISCLAIMER:

This KB article is currently under review given its age and the relevant updates in the product since its conception. At this time, it is advised to not following the steps for resolution.

If you believe that your environment is currently experiencing the below behavior, please open a case with Appian Support for further review.

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: high availability, kafka, engines, zookeeper, infrastructure

KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Ryan Good — Tue, 30 Jul 2024 13:24:28 GMT

Revision 11 posted to Appian Knowledge Base by Ryan Good on 7/30/2024 1:24:28 PM

DISCLAIMER:

This KB article is currently under review given its age and the relevant updates in the product since its conception. At this time, it is advised to not following the steps for resolution.

If you believe that your environment is currently experiencing the below behavior, please open a case with Appian Support for further review.

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: high availability, kafka, engines, zookeeper, infrastructure

KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Jordan Horwat — Mon, 23 Jul 2018 10:23:55 GMT

Revision 10 posted to Appian Knowledge Base by Jordan Horwat on 7/23/2018 10:23:55 AM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: high availability, kafka, engines, zookeeper, infrastructure

KB-1589 Kafka fails to start after an unexpected shutdown in a high availability environment

Parmida Borhani — Wed, 30 May 2018 05:18:29 GMT

Revision 9 posted to Appian Knowledge Base by Parmida Borhani on 5/30/2018 5:18:29 AM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: kafka, high availibility, engines, zookeeper

DRAFT KB-XXXX Kafka fails to start after an unexpected shutdown in a high availability environment

Parmida Borhani — Thu, 24 May 2018 01:06:25 GMT

Revision 8 posted to Appian Knowledge Base by Parmida Borhani on 5/24/2018 1:06:25 AM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: kafka, high availibility, engines, zookeeper

Kafka fails to start after an unexpected shutdown in a high availability environment

Parmida Borhani — Thu, 24 May 2018 01:05:58 GMT

Revision 7 posted to Appian Knowledge Base by Parmida Borhani on 5/24/2018 1:05:58 AM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: kafka, high availibility, engines, zookeeper

Kafka fails to start after an unexpected shutdown in a High Availability environment

Tejas Kargutkar — Wed, 23 May 2018 12:58:24 GMT

Revision 6 posted to Appian Knowledge Base by Tejas Kargutkar on 5/23/2018 12:58:24 PM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on all servers which host Zookeeper.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: kafka, high availibility, engines, zookeeper

Kafka fails to start after an unexpected shutdown in a High Availability environment

Parmida Borhani — Tue, 08 May 2018 04:06:43 GMT

Revision 5 posted to Appian Knowledge Base by Parmida Borhani on 5/8/2018 4:06:43 AM

Symptoms

After an unexpected shutdown of Appian high availabilty servers, Kafka fails to start up on a particular node with the following type of error seen in the <APPIAN_HOME>\logs\service-manager\kafka\server.log :

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on the server where Kafka is unable to start.
Start Appian.

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: May 2018

Tags: kafka, high availibility, engines, zookeeper

Kafka fails to start after an unexpected shutdown in a High Availability environment

Tejas Kargutkar — Thu, 19 Apr 2018 15:07:41 GMT

Revision 4 posted to Appian Knowledge Base by Tejas Kargutkar on 4/19/2018 3:07:41 PM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components (Service manager, Enginer, Kakfa, Zookeeper, Jboss, Search-server) are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on the server where Kafka is unable to start.
Start Appian

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: April 2018

Tags: kafka, high availibility, engines, zookeeper

Kafka fails to start after an unexpected shutdown in a High Availability environment

Tejas Kargutkar — Thu, 19 Apr 2018 15:07:29 GMT

Revision 3 posted to Appian Knowledge Base by Tejas Kargutkar on 4/19/2018 3:07:29 PM

Symptoms

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components (Service manager, Enginer, Kakfa, Zookeeper, Jboss, Search-server) are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on the server where Kafka is unable to start.
Start Appian

Affected Versions

This article applies to Appian 17.3 and later.

Last Reviewed: April 2018

Tags: kafka, high availibility, engines, zookeeper

Kafka fails to start after an unexpected shutdown in a High Availability environment

Tejas Kargutkar — Mon, 26 Mar 2018 22:00:03 GMT

Revision 1 posted to Appian Knowledge Base by Tejas Kargutkar on 3/26/2018 10:00:03 PM

Symptoms

After an unexpected shutdown of Appian high availabilty servers, Kakfa fails to start up on a particular node with the following type of error seen in the kafka log:

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.

Delete the <APPIAN_HOME>/services/data/zookeeper folder on the server where Kafka is unable to start.

Start Appian the normal way you do it.

Tags: kafka, high availibility, engines, zookeeper

Kafka fails to start after an unexpected shutdown in a High Availability environment

Tejas Kargutkar — Mon, 26 Mar 2018 18:01:48 GMT

Revision 2 posted to Appian Knowledge Base by Tejas Kargutkar on 3/26/2018 6:01:48 PM

Symptoms

After an unexpected shutdown of Appian high availabilty servers, Kakfa fails to start up on a particular node with the following type of error seen in the kafka log:

Exiting because log truncation is not allowed for partition serviceManager.transaction.process-design-0, 
Current leader 1's latest offset 6656337 is less than replica 2's latest offset 6656340 (kafka.server.ReplicaFetcherThread)

Cause

This can happen due to the mismatch in the transaction ids logged across the Kafka nodes.

Action

Make sure all Appian components are shut down.
Delete the <APPIAN_HOME>/services/data/zookeeper folder on the server where Kafka is unable to start.
Start Appian the normal way you do it.

Tags: kafka, high availibility, engines, zookeeper