Unable to start Data server

RE: Unable to start Data server

jarunb — Wed, 13 Nov 2024 14:07:22 GMT

Hii I am getting below error

C:\appian243\appian\data-server\bin>start.bat

0 [main] INFO com.appian.data.server.Watchdog - Data Server is starting using the [ALL_COMPONENTS_MODE] startup mode.

17 [main] INFO com.appian.data.TopologyFactory - Could not find C:\appian243\appian\data-server\bin\..\conf\appian-topology.xml. The default topology configuration will be used.

75 [main] INFO com.appian.data.server.config.CustomPropertiesConfiguration - Configuration file custom.properties does not exist. Default settings will be used.

90 [main] ERROR com.appian.data.server.Watchdog - Shutting down with exit code 1...

java.lang.RuntimeException: No data server security properties file found. File: C:\appian243\appian\data-server\bin\..\conf\data-server-sec.properties
at com.appian.data.SecurityConfig.<init>(SecurityConfig.java:22)

at com.appian.data.server.DataServerConfig.<init>(DataServerConfig.java:287)

at com.appian.data.server.DataServerConfig.fromSysProps(DataServerConfig.java:305)

at com.appian.data.server.Watchdog.main(Watchdog.java:1467)

RE: Unable to start Data server

prakashbabu87 — Fri, 25 Oct 2024 07:32:40 GMT

Hi Abdullah thank you for your reply before we followed the steps this the error we are getting as you can see in the log below i underlined those msgs data server started successfully

{"level":"INFO","time":"2024-10-25T06:57:41.932Z","logger":"watchdog","caller":"LogRotation.java:58","thread":"log-rotator","msg":"LogRotation is enabled. To disable, set property "wd.log.rotation.interval.seconds" in file "C:\appian24.1\appian\data-server\bin\..\conf\custom.properties" to a negative number"}
{"level":"INFO","time":"2024-10-25T06:57:43.016Z","logger":"watchdog","caller":"Watchdog.java:1349","thread":"main","msg":"Data Server started successfully."}
{"level":"INFO","time":"2024-10-25T06:59:58.716Z","logger":"watchdog","caller":"DataClientImpl.java:500","thread":"branch-cleaner","msg":"data-server failed to write: APNX-3-0100-004"}
{"level":"ERROR","time":"2024-10-25T06:59:58.729Z","logger":"watchdog","caller":"BranchCleanup.java:59","thread":"branch-cleaner","msg":"Error executing branch cleanup."}
com.appian.data.client.AdsException: APNX-3-0100-004: No-op historical store gateway will not process write requests
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.appian.data.client.AdsExceptionBuilder.build(AdsExceptionBuilder.java:133)
at com.appian.data.client.DataClientImpl.buildWriteError(DataClientImpl.java:1110)
at com.appian.data.client.DataClientImpl.writeWithRetry(DataClientImpl.java:551)
at com.appian.data.client.DataClientImpl.write00(DataClientImpl.java:498)
at com.appian.data.client.DataClientImpl.lambda$write0$6(DataClientImpl.java:479)
at com.appian.data.client.DataClientImpl.runWithAdditionalGroupsIfNeeded(DataClientImpl.java:266)
at com.appian.data.client.DataClientImpl.write0(DataClientImpl.java:478)
at com.appian.data.client.DataClientImpl.write(DataClientImpl.java:458)
at com.appian.data.client.DataClientImpl.write(DataClientImpl.java:446)
at com.appian.data.server.BranchCleanup.onInterval(BranchCleanup.java:53)
at com.appian.data.server.IntervalTask.safeRun(IntervalTask.java:77)
at com.appian.data.server.IntervalTask.run(IntervalTask.java:63)
at java.lang.Thread.run(Thread.java:750)

RE: Unable to start Data server

Abdullah A — Fri, 25 Oct 2024 07:16:28 GMT

sorry there missing steps I did not write:

start.bat -p <your password> -s kafka,zookeeper (start just 2 services )
execute script: :<APPIAN_HOME>/services/kafka-<version>/bin/kafka-topics.bat --bootstrap-server <hostname>:9092 --delete --topic ads_transaction_effe
stop all components
delete files in folder: <Appian home>\data-server\data\hs
start all components

if this did not solve your problem, I think you need to open support ticket with Appian

RE: Unable to start Data server

prakashbabu87 — Thu, 24 Oct 2024 13:22:12 GMT

ill start up in no-op mode"}
{"level":"INFO","time":"2024-10-24T13:00:57.950Z","logger":"watchdog","caller":"TransactionLogRecovery.java:80","thread":"main","msg":"checking for transaction effects Kafka topic existence"}
{"level":"INFO","time":"2024-10-24T13:01:47.979Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:02:38.019Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:03:28.034Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:04:18.054Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:05:08.071Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:05:58.097Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:06:48.112Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"ERROR","time":"2024-10-24T13:06:58.116Z","logger":"watchdog","caller":"Watchdog.java:1483","thread":"main","msg":"Shutting down with exit code 1..."}
java.lang.IllegalStateException: Historical store directory (C:\appian24.1\appian\data-server\bin\..\data\hs) and/or snapshot root directory (C:\appian24.1\appian\data-server\bin\..\data\ss) are not empty, however, kafka topic doesn't exist. Is the kafka log lost or corrupted?
	at com.appian.data.server.TransactionLogRecovery.checkTopicExistsWithRetry(TransactionLogRecovery.java:106)
	at com.appian.data.server.TransactionLogRecovery.verify(TransactionLogRecovery.java:63)
	at com.appian.data.server.Watchdog.initializeData(Watchdog.java:471)
	at com.appian.data.server.Watchdog.startup(Watchdog.java:354)
	at com.appian.data.server.Watchdog.main(Watchdog.java:1478)
{"level":"ERROR","time":"2024-10-24T13:06:58.721Z","logger":"watchdog","caller":"Watchdog.java:1483","thread":"main","msg":"Shutting down with exit code 1..."}
com.appian.data.server.UnrecoverableError: Lost WebSocket connection with localhost:5400. The target JVM process exited abnormally. Check for earlier errors (e.g. JVM startup errors).
	at com.appian.data.server.HeartbeatListener$SyncHeartbeatListenerBuilder.create(HeartbeatListener.java:165)
	at com.appian.data.server.JvmStarter.startSync(JvmStarter.java:41)
	at com.appian.data.server.Watchdog.startWatchdogDaemon(Watchdog.java:1348)
	at com.appian.data.server.Watchdog.main(Watchdog.java:1472)
Caused by: java.lang.Exception: WebSocket closed.
	at com.appian.data.server.HeartbeatListener$HeartbeatListenerFuture.lambda$null$1(HeartbeatListener.java:215)
	at io.vertx.core.http.impl.WebSocketImplBase.handleClosed(WebSocketImplBase.java:620)
	at io.vertx.core.http.impl.WebSocketImpl.handleClosed(WebSocketImpl.java:47)
	at io.vertx.core.http.impl.Http1xClientConnection.handleClosed(Http1xClientConnection.java:812)
	at io.vertx.core.net.impl.VertxHandler.lambda$channelInactive$3(VertxHandler.java:153)
	at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366)
	at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43)
	at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:229)
	at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:221)
	at io.vertx.core.net.impl.VertxHandler.channelInactive(VertxHandler.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:303)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:411)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:376)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
	at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
	at io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:750)

RE: Unable to start Data server

prakashbabu87 — Thu, 24 Oct 2024 13:19:21 GMT

Hi Abdullah after following the steps you mentioned i got the following errors

ill start up in no-op mode"}
{"level":"INFO","time":"2024-10-24T13:00:57.950Z","logger":"watchdog","caller":"TransactionLogRecovery.java:80","thread":"main","msg":"checking for transaction effects Kafka topic existence"}
{"level":"INFO","time":"2024-10-24T13:01:47.979Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:02:38.019Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:03:28.034Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:04:18.054Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:05:08.071Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:05:58.097Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"INFO","time":"2024-10-24T13:06:48.112Z","logger":"watchdog","caller":"TransactionLogRecovery.java:97","thread":"main","msg":"waiting for transaction effects Kafka topic to be available"}
{"level":"ERROR","time":"2024-10-24T13:06:58.116Z","logger":"watchdog","caller":"Watchdog.java:1483","thread":"main","msg":"Shutting down with exit code 1..."}
java.lang.IllegalStateException: Historical store directory (C:\appian24.1\appian\data-server\bin\..\data\hs) and/or snapshot root directory (C:\appian24.1\appian\data-server\bin\..\data\ss) are not empty, however, kafka topic doesn't exist. Is the kafka log lost or corrupted?
at com.appian.data.server.TransactionLogRecovery.checkTopicExistsWithRetry(TransactionLogRecovery.java:106)
at com.appian.data.server.TransactionLogRecovery.verify(TransactionLogRecovery.java:63)
at com.appian.data.server.Watchdog.initializeData(Watchdog.java:471)
at com.appian.data.server.Watchdog.startup(Watchdog.java:354)
at com.appian.data.server.Watchdog.main(Watchdog.java:1478)
{"level":"ERROR","time":"2024-10-24T13:06:58.721Z","logger":"watchdog","caller":"Watchdog.java:1483","thread":"main","msg":"Shutting down with exit code 1..."}
com.appian.data.server.UnrecoverableError: Lost WebSocket connection with localhost:5400. The target JVM process exited abnormally. Check for earlier errors (e.g. JVM startup errors).
at com.appian.data.server.HeartbeatListener$SyncHeartbeatListenerBuilder.create(HeartbeatListener.java:165)
at com.appian.data.server.JvmStarter.startSync(JvmStarter.java:41)
at com.appian.data.server.Watchdog.startWatchdogDaemon(Watchdog.java:1348)
at com.appian.data.server.Watchdog.main(Watchdog.java:1472)
Caused by: java.lang.Exception: WebSocket closed.
at com.appian.data.server.HeartbeatListener$HeartbeatListenerFuture.lambda$null$1(HeartbeatListener.java:215)
at io.vertx.core.http.impl.WebSocketImplBase.handleClosed(WebSocketImplBase.java:620)
at io.vertx.core.http.impl.WebSocketImpl.handleClosed(WebSocketImpl.java:47)
at io.vertx.core.http.impl.Http1xClientConnection.handleClosed(Http1xClientConnection.java:812)
at io.vertx.core.net.impl.VertxHandler.lambda$channelInactive$3(VertxHandler.java:153)
at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366)
at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43)
at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:229)
at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:221)
at io.vertx.core.net.impl.VertxHandler.channelInactive(VertxHandler.java:153)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:303)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:411)
at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:376)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
at io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:750)

RE: Unable to start Data server

prakashbabu87 — Wed, 23 Oct 2024 08:26:11 GMT

Thank you. I will execute those and get back to you

RE: Unable to start Data server

Abdullah A — Wed, 23 Oct 2024 08:23:37 GMT

sorry ..it is mistake

RE: Unable to start Data server

prakashbabu87 — Wed, 23 Oct 2024 08:16:35 GMT

Hi Abdullah thank you for you reply. I have a small doubt in the 2nd point you said execute 2 scripts but i see the same script name mentioned twice can you send the 2 scripts that i need to execute again.

RE: Unable to start Data server

Abdullah A — Wed, 23 Oct 2024 07:46:32 GMT

hi fixed by follow below step:

1-start.bat -p <your password> -s kafka,zookeeper (start just 2 services )
2- execute script: :<APPIAN_HOME>/services/kafka-<version>/bin/kafka-topics.bat --bootstrap-server <hostname>:9092 --delete --topic ads_transaction_effe
stop all components
start all components

RE: Unable to start Data server

prakashbabu87 — Wed, 23 Oct 2024 07:06:47 GMT

Hi, getting the same error and health.bat is showing false for healthy status any resolutions