Data-Server is not coming up

Hello everyone,

We are facing one strange issue while upgrading from 18.4 to 19.4 Appian version. We are not able to start data-server and we have copied .K4 license as well properly in dataserver.

Below is the error log coming up while running start.sh

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See www.slf4j.org/codes.html for further details.
{"level":"INFO","time":"2019-12-05T07:15:34.712Z","logger":"watchdog","caller":"Watchdog.java:1090","thread":"main","msg":"Starting Data Server in the background"}
{"level":"INFO","time":"2019-12-05T07:15:34.715Z","logger":"watchdog","caller":"JvmStarter.java:41","thread":"main","msg":"Starting JVM process: Config{class=com.appian.data.server.Watchdog, hostAndPort=localhost:5400, args=[-f]}"}
{"level":"ERROR","time":"2019-12-05T07:15:37.422Z","logger":"watchdog","caller":"Watchdog.java:1185","thread":"main","msg":"Shutting down with exit code 1..."}
com.appian.data.server.UnrecoverableError: Lost WebSocket connection with localhost:5400. The target JVM process exited abnormally. Check for earlier errors (e.g. JVM startup errors).
at com.appian.data.server.HeartbeatListener$SyncHeartbeatListenerBuilder.create(HeartbeatListener.java:165)
at com.appian.data.server.JvmStarter.startSync(JvmStarter.java:37)
at com.appian.data.server.Watchdog.startWatchdogDaemon(Watchdog.java:1091)
at com.appian.data.server.Watchdog.main(Watchdog.java:1174)
Caused by: io.netty.handler.codec.http.websocketx.WebSocketHandshakeException: Connection closed while handshake in process
at io.vertx.core.http.impl.Http1xClientConnection$HandshakeInboundHandler.channelInactive(Http1xClientConnection.java:759)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelInactive(CombinedChannelDuplexHandler.java:420)
at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:393)
at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:358)
at io.netty.handler.codec.http.HttpClientCodec$Decoder.channelInactive(HttpClientCodec.java:285)
at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:223)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1417)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)
at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:913)
at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

Below is the error in watchdog.log

{"level":"ERROR","time":"2019-12-04T21:09:19.348Z","logger":"watchdog","caller":"Watchdog.java:1185","thread":"main","msg":"Shutting down with exit code 1..."}
java.lang.IllegalStateException: Historical store directory (/appian/data/AppianHomeRAP/current/data-server/data/hs) and/or snapshot root directory (/appian/data/AppianHomeRAP/current/data-server/data/ss) are not empty, however, kafka topic doesn't exist. Is the kafka log lost or corrupted?
at com.google.common.base.Preconditions.checkState(Preconditions.java:199)
at com.appian.data.server.TransactionLogRecovery.verify(TransactionLogRecovery.java:50)
at com.appian.data.server.Watchdog.initializeData(Watchdog.java:296)
at com.appian.data.server.Watchdog.startup(Watchdog.java:241)
at com.appian.data.server.Watchdog.main(Watchdog.java:1180)

Can some one help us on this.

Thanks-

Ruchi Agarwal