Data-Server times out absolutely every time I try to start it, yet is still running

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 18:27:04 GMT

Fixed it!

In the appian.topology.xml file i changed data server host to "localhost", then changed the FQDN in custom.properties to "localhost" as well, now data-server starts.

I still don't have appian because of jboss errors, but we'll deal with that later.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 16:58:27 GMT

this is in the wd-5400 log

2018-05-07 14:18:08,187 [main] INFO com.appian.data.server.JvmStarter - Starting JVM: C:\Program Files\Java\jdk1.8.0_171\jre\bin\java.exe -server -Duser.timezone=GMT -Xms64m -Xmx128m -Dds.home=C:\appian181\appian\data-server\bin\.. -cp C:\appian181\appian\data-server\bin\..\lib\appian-data-server-classpath-0.72.6.jar com.appian.data.server.Watchdog -f
2018-05-07 14:18:09,641 [main] INFO com.appian.data.server.replication.TopicManager - Connecting to Zookeeper: localhost:2181
2018-05-07 14:18:40,750 [main] ERROR com.appian.data.server.Watchdog - Shutting down with exit code 1...

So when I use "localhost" it JUST shuts down with no explanation why it would want to do that. Exit code 1, whatever that is.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 16:54:59 GMT

Found them.

If I use "localhost" I get this error:

0 [main] INFO com.appian.data.TopologyFactory - Loading topology C:\appian181\appian\data-server\bin\..\conf\appian-topology.xml
301 [main] INFO com.appian.data.server.config.CustomPropertiesConfiguration - Configuration file custom.properties does not exist. Default settings will be used.
314 [main] INFO com.appian.data.server.Watchdog - Initializing log4j (processName=wd-5400, logsDir=C:\appian181\appian\data-server\bin\..\..\logs\data-server) from C:\appian181\appian\data-server\bin\..\conf\log4j.properties
2018-05-07 14:45:28,514 wd-5400 [main] INFO com.appian.data.server.replication.TopicManager - Connecting to Zookeeper: localhost:2181
2018-05-07 14:45:59,639 wd-5400 [main] ERROR com.appian.data.server.Watchdog - Shutting down with exit code 1...

If I use the other FQDN I get this error:

0 [main] INFO com.appian.data.TopologyFactory - Loading topology C:\appian181\appian\data-server\bin\..\conf\appian-topology.xml
293 [main] INFO com.appian.data.server.config.CustomPropertiesConfiguration - Configuration file custom.properties does not exist. Default settings will be used.
306 [main] INFO com.appian.data.server.Watchdog - Initializing log4j (processName=wd-5400, logsDir=C:\appian181\appian\data-server\bin\..\..\logs\data-server) from C:\appian181\appian\data-server\bin\..\conf\log4j.properties
2018-05-07 15:03:48,850 wd-5400 [main] INFO com.appian.data.server.replication.TopicManager - Connecting to Zookeeper: localhost:2181
2018-05-07 15:03:49,185 wd-5400 [main] INFO com.appian.data.server.replication.TopicManager - Connecting to Zookeeper: localhost:2181
2018-05-07 15:03:56,011 wd-5400 [main] INFO com.appian.data.server.Watchdog - Data Server is not configured to run on 'DESKTOP-063AN1L' according to the topology. It is configured to run on 'data-server'. DataServer is shutting down.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

Benhail — Tue, 08 May 2018 16:40:54 GMT

So is the server running? But you are still worried about your license?

RE: Data-Server times out absolutely every time I try to start it, yet is still running

Justin Watts — Tue, 08 May 2018 16:40:05 GMT

Under the main logs folder (docs.appian.com/.../Logging.html

So something like ../appian/logs/data-server

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 16:31:32 GMT

Where are the data-server logs? I've looked through the entire data-server folder and every subfolder and found no logs.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 15:05:04 GMT

Status.bat in data-server folder does nothing. Just newline.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

Justin Watts — Tue, 08 May 2018 15:00:36 GMT

What do the data-server logs say? Maybe there is something useful in there

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 14:59:51 GMT

And I do have k4.lic at C:\appian181\repo\appian\data-server\engine\bin\q\w64

This is Windows 10, 64 bit OS.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 14:58:03 GMT

I renamed the FQDN from "localhost" back to what it was before, and nearly all the services came up PRIMARY mode. I shut down one straggler, started the one straggler up again, used status.bat to confirm all are up and running in PRIMARY mode, same error.

I'm not being refused connection to 5400 now, I'm all the way back to having it timeout because it can't find heartbeat.

Interesting thing, it times out, but it says it's running. Stop.bat on the data-server works fine. A few seconds and it's down like it never had an error.

When I run netstat with all the services and the data server up, it still shows absolutely nothing about port 5400.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

Justin Watts — Tue, 08 May 2018 14:25:08 GMT

If you can run /services/bin/_status.sh and get results, I don't expect that your K license is an issue. The FQDN of the machine is what the license looks at, not configuration in other Appian files. Your services should be in PRIMARY mode when they are up (assuming you are not running a cluster)

One other thing to check - do you have a Q license? I would expect a different error if you didn't but might as well double-check to remove it as an issue. Refer to:
docs.appian.com/.../Requesting_and_Installing_a_license.html

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 14:09:45 GMT

I think I know where the issue lies, further back. Whenever I get all the services up using services\bin\start.bat, they're all in REPLICA mode. It happened so much that today is the first time I ever saw PRIMARY mode to let me know that was actually an error. The listener isn't listening on port 5400. Now I'm getting STOPPING state REPLICA mode due to leadership change on a third of the services and port 5400 is refusing connection.

I changed my FQDN from "DESKTOP-063AN1L" (closest thing to FQDN I could find on this machine) to "localhost" in another file to try to correct another issue. Would that be invalidating my license?

RE: Data-Server times out absolutely every time I try to start it, yet is still running

Benhail — Tue, 08 May 2018 14:00:03 GMT

There is no harm in trying David! But just make sure, where all you have changed the 5400 port number!

As far as my memory goes, I had a different issue and I tried to change the port number but that didn't help and I had to resort to other solutions!

Keep us updated, we will try to come up with a solution.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 13:22:22 GMT

All the ports are the same, all of the uncommented code in each of the the appian.topology.xml files is identical, even the one in repo.

Would you recommend manually entering another available port, 5500 for instance, to see what that does?

RE: Data-Server times out absolutely every time I try to start it, yet is still running

davel001150 — Tue, 08 May 2018 13:13:11 GMT

Thank you for the advice. I ran your request after shutting everything down, and there's nothing using that port. I can try reinitializing everything and let you know what's using the port (if anything) when I get all the services back up. I'm going to make sure all the topology files match per Justin Watts' suggestion first.

RE: Data-Server times out absolutely every time I try to start it, yet is still running

Benhail — Tue, 08 May 2018 12:21:40 GMT

com.appian.data.server.UnrecoverableError: Established WebSocket connection but failed to receive heartbeat from data-server:5400.

Mostly the Port Issue.

Follow this link to see which process is using that port 5400:
stackoverflow.com/.../how-can-you-find-out-which-process-is-listening-on-a-port-on-windows

For more details:
stackoverflow.com/.../47202911

RE: Data-Server times out absolutely every time I try to start it, yet is still running

Justin Watts — Tue, 08 May 2018 12:16:14 GMT

The timeout makes me think its a port issue. Did you validate your appian-topology.xml files all match up?

I expect an entry like this in both the /data-server/conf and suite.ear/conf topology locations. I'm pretty sure by default Appian doesn't include these entries: