KB-2314 Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

Ryan Good — Tue, 09 Jul 2024 20:38:55 GMT

Current Revision posted to Appian Knowledge Base by Ryan Good on 7/9/2024 8:38:55 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

Upon starting services, the console output shows the following:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.u.ServiceUtils@48] - Exiting JVM with code 3

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Tomcat:

Inside the tomcat-stdout.log, found inside <APPIAN_HOME>/logs/, the one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm SEVERE [main] org.apache.catalina.startup.HostConfig.deployDescriptor Error deploying deployment descript
or [<APPIAN_HOME>/tomcat/apache-tomcat/conf/Catalina/localhost/suite.xml]
 java.io.FileNotFoundException: ../../../conf/tomcatResources.xml (Permission denied)

YYYY-MM-DD HH:MM:SS,mmm [localhost-startStop-1] ERROR com.appiancorp.common.initialize.MigrateNavigationConfig - Could not merge navigation-config stored in content with the one provided by default. java.io.FileNotFoundException: <APPIAN_HOME>/_admin/accdocs*/2/*.xml (Permission denied)

Cause

Necessary component files are owned by non-Appian user accounts. Appian should have its own user and root must not be used as it may lead to unexpected behavior. For more information, please see the following documentation.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <APPIAN_HOME> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <APPIAN_HOME> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: July 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Wed, 05 Jun 2024 21:23:54 GMT

Revision 14 posted to Appian Knowledge Base by pauline.delacruz on 6/5/2024 9:23:54 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

Upon starting services, the console output shows the following:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.u.ServiceUtils@48] - Exiting JVM with code 3

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Tomcat:

Inside the tomcat-stdout.log, found inside <APPIAN_HOME>/logs/, the one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm SEVERE [main] org.apache.catalina.startup.HostConfig.deployDescriptor Error deploying deployment descript
or [<APPIAN_HOME>/tomcat/apache-tomcat/conf/Catalina/localhost/suite.xml]
 java.io.FileNotFoundException: ../../../conf/tomcatResources.xml (Permission denied)

YYYY-MM-DD HH:MM:SS,mmm [localhost-startStop-1] ERROR com.appiancorp.common.initialize.MigrateNavigationConfig - Could not merge navigation-config stored in content with the one provided by default. java.io.FileNotFoundException: <APPIAN_HOME>/_admin/accdocs*/2/*.xml (Permission denied)

Cause

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <APPIAN_HOME> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <APPIAN_HOME> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: June 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Fri, 05 Apr 2024 15:02:51 GMT

Revision 13 posted to Appian Knowledge Base by pauline.delacruz on 4/5/2024 3:02:51 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

Upon starting services, the console output shows the following:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.u.ServiceUtils@48] - Exiting JVM with code 3

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Tomcat:

Upon starting the app server, one or more of the following errors are displayed:
YYYY-MM-DD HH:MM:SS SEVERE [main] org.apache.catalina.startup.HostConfig.deployDescriptor Error deploying deployment descript
or [<APPIAN_HOME>/tomcat/apache-tomcat/conf/Catalina/localhost/suite.xml]
java.io.FileNotFoundException: ../../../conf/tomcatResources.xml (Permission denied)

2024-04-05 14:20:35 INFO [main] org.apache.coyote.AbstractProtocol.start Startingq ProtocolHandler ["http-nio-8080"]
2024-04-05 14:20:35 SEVERE [main] org.apache.catalina.util.LifecycleBase.handleSubClassException Failed to start component [Connector["http-nio-8080"]]
org.apache.catalina.LifecycleException: Protocol handler start failed

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <APPIAN_HOME> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <APPIAN_HOME> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: April 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Wed, 03 Apr 2024 21:34:31 GMT

Revision 12 posted to Appian Knowledge Base by pauline.delacruz on 4/3/2024 9:34:31 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

Upon starting services, the console output shows the following:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.u.ServiceUtils@48] - Exiting JVM with code 3

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Tomcat:

Upon starting the app server, the following is:

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 22:10:11 GMT

Revision 11 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 10:10:11 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

Upon starting services, the console output shows the following:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.u.ServiceUtils@48] - Exiting JVM with code 3

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Tomcat:

Upon starting services, the console output shows the following:

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 21:45:27 GMT

Revision 10 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 9:45:27 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

The console output shows the following and hangs on startup:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.u.ServiceUtils@48] - Exiting JVM with code 3

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 21:44:45 GMT

Revision 9 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 9:44:45 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

The console output shows the following and hangs on startup:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.u.ServiceUtils@48] - Exiting JVM with code 3

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 21:41:51 GMT

Revision 8 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 9:41:51 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

The console output shows the following and hangs on startup:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, one or more of following error(s) are shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME> .
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, please open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 21:39:51 GMT

Revision 7 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 9:39:51 PM

Symptoms

Users may experience one or more of the following symptoms when starting services.

Zookeeper:

The console output shows the following and hangs on startup:

Starting QuorumPeerMain...
process QuorumPeerMain running
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available
waiting for Zookeeper to be available

Inside the zookeeper.log, found in <APPIAN_HOME>/logs/service-manager/zookeeper, the following error(s) is shown:

YYYY-MM-DD HH:MM:SS,mmm [myid:] - ERROR [PurgeTask:o.a.z.s.DatadirCleanupManager$PurgeTask@143] - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

YYYY-MM-DD HH:MM:SS,mmm [myid:0] - ERROR [main:o.a.z.s.ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory <APPIAN_HOME>/services/data/zookeeper/version-2

Kafka:

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME>
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 21:26:24 GMT

Revision 6 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 9:26:24 PM

Symptoms

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, found inside <APPIAN_HOME>/logs/service-manager/kafka, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users inside <APPIAN_HOME>
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership of the affected files to be under the Appian user.
Attempt startup.
If the issue persists, open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 21:20:30 GMT

Revision 5 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 9:20:30 PM

Symptoms

Upon starting services, the console output shows the following:

Starting Kafka...
waiting for process Kafka
process Kafka running
local Kafka broker is not registered
<<< Kafka broker initialization failed for localhost:9092. Check the broker logs for more information >>>
Deleting Kafka index files in case initialization failed due to a corrupt index file.
Unable to start service Kafka
com.appian.komodo.translog.KafkaNotRunningException: Kafka process died

Inside the server.log, the one or more of following error(s) are shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while loading log dir /usr/local/appian/appian224/appian/services/data/kafka-logs (kafka.server.LogDirFailureChannel) java.nio.file.AccessDeniedException: /usr/local/appian/appian224/appian/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users in Appian_home
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. For High Availability (HA) Sites: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership to be under the Appian user
Attempt startup
If the issue persists, open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 04 Mar 2024 19:27:09 GMT

Revision 4 posted to Appian Knowledge Base by pauline.delacruz on 3/4/2024 7:27:09 PM

Symptoms

Upon starting services, the console output shows the following:

Inside the server.log, the following error is shown:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR Error while deleting the clean shutdown file in dir <APPIAN_HOME>/services/data/kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: <APPIAN_HOME>/services/data/kafka-logs/serviceManager.transaction.<engine>-#/####################.log

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Check to see if files are owned by different users in Appian_home
1. sudo find <Appian_home> -printf '%u:%g\n' | sort -t: -u
2. HA: sudo find -L <Appian_home> -printf '%u:%g\n' | sort -t: -u
Change ownership to be under the Appian user
Attempt startup
If the issue persists, open a support ticket for further investigation.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: March 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 26 Feb 2024 21:46:02 GMT

Revision 3 posted to Appian Knowledge Base by pauline.delacruz on 2/26/2024 9:46:02 PM

Symptoms

Upon starting services, the console output shows the following:

Cause

Necessary component files are owned by non-Appian user accounts.

Action

To check the file permissions inside the environment, run the following:

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: February 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 26 Feb 2024 18:11:29 GMT

Revision 2 posted to Appian Knowledge Base by pauline.delacruz on 2/26/2024 6:11:29 PM

Symptoms

This section is required for this template. All symptoms of the issue, including relevant screenshots and log lines, should be included here. The symptoms should be as descriptive as possible.

Cause

This section is required for this template. When writing a cause for a particular article, ensure the following is taken into consideration:

The cause is concise and does not disclose more information than absolutely necessary.
The cause does not share any Appian source code. Source code for plugins/shared components are fine.
If this is an issue outside of the Appian product and the third-party vendor included documentation referencing the bug, provide their documentation for reference.

For any enhancements/defects, use one of the following as applicable:

This issue has been reported to the Appian Product Team. The reference number for this issue is AN-XXXXX.
This product use case has been logged to the Appian Product Team for consideration to add this functionality in the product. Kindly note it is not Appian Support’s policy to disclose how or when a product use case will be implemented.
This issue has been hotfixed via AN-XXXXX in Appian X.X.
This issue has been hotfixed via AN-XXXXX in Appian X.X to X.X.
This use case has been incorporated in Appian X.X.
This use case has been incorporated in Appian X.X to X.X.

When logging a product defect, ensure you use the known issues tag as well as the open issues or fixed issues tag, depending if the issue is fixed or not.

If no cause is known and this is not something Engineering can reproduce/needs to know about, then say “The root cause of this issue is currently unknown.” If you use that as your cause, then a valid action must be present.

Action

This section is required for this template and contains the steps to resolve the issue. If an issue in an article is solved by a hotfix and/or major version of Appian, use the following language to indicate that:

Apply the latest hotfix to your Appian installation.
Upgrade to the latest version of Appian.
Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

If there is a product issue with Engineering that is open and a workaround for that issue exists, include it in the Action section until the fix is available. After the fix is made available, use that as a workaround instead.

Otherwise, the content in this section should wholly solve a customer’s issue without producing any undesirable side effects. If there is at least one valid action that doesn’t cause undesirable side effects, all other actions with negative side effects should go in the Workaround section.

If there is no action or workaround that resolves the issue, use the following language:

There is no action or workaround currently available for this issue. If you are facing this issue, please open a support case with Appian.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: February 2024

Tags: Tomcat, kafka, zookeeper, permissions, infrastructure, service manager

[DRAFT SP-9623] Kafka/Zookeeper/Tomcat related data ownership being non-Appian user account prevents services from starting up

pauline.delacruz — Mon, 26 Feb 2024 18:04:59 GMT

Revision 1 posted to Appian Knowledge Base by pauline.delacruz on 2/26/2024 6:04:59 PM

Symptoms

This section is required for this template. All symptoms of the issue, including relevant screenshots and log lines, should be included here. The symptoms should be as descriptive as possible.

Cause

This section is required for this template. When writing a cause for a particular article, ensure the following is taken into consideration:

The cause is concise and does not disclose more information than absolutely necessary.
The cause does not share any Appian source code. Source code for plugins/shared components are fine.
If this is an issue outside of the Appian product and the third-party vendor included documentation referencing the bug, provide their documentation for reference.

For any enhancements/defects, use one of the following as applicable:

This issue has been reported to the Appian Product Team. The reference number for this issue is AN-XXXXX.
This product use case has been logged to the Appian Product Team for consideration to add this functionality in the product. Kindly note it is not Appian Support’s policy to disclose how or when a product use case will be implemented.
This issue has been hotfixed via AN-XXXXX in Appian X.X.
This issue has been hotfixed via AN-XXXXX in Appian X.X to X.X.
This use case has been incorporated in Appian X.X.
This use case has been incorporated in Appian X.X to X.X.

When logging a product defect, ensure you use the known issues tag as well as the open issues or fixed issues tag, depending if the issue is fixed or not.

Action

Apply the latest hotfix to your Appian installation.
Upgrade to the latest version of Appian.
Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

If there is no action or workaround that resolves the issue, use the following language:

There is no action or workaround currently available for this issue. If you are facing this issue, please open a support case with Appian.

Workaround

This section is optional and should only be used as an alternative action if there is a massive technical debt associated with performing the action in the above section for customers (such as upgrading). It can also be used if there is a solution to the issue in the article but is more inconvenient to implement or it causes other undesirable behavior and wouldn’t warrant being an action.

Affected Versions

This article applies to all self-managed versions of Appian.

Last Reviewed: Month YYYY