We've had an error occur a couple times now on different days with different

We've had an error occur a couple times now on different days with different plugins. We have a scheduled task setup to restart Jboss every morning we received the following error:
11:33:03,879 INFO [stdout] (ServerService Thread Pool -- 92) 2015-12-21 11:33:03,879 [ServerService Thread Pool -- 92] ERROR com.atlassian.plugin.manager.PluginEnabler - Unable to start the following plugins due to timeout while waiting for plugin to enable: psshared.htmldisplay

Any Ideas on what could cause this?

Thank you in advance.

OriginalPostID-185556

OriginalPostID-185556

  Discussion posts and replies are publicly visible

  • Another instance of the error:
    11:33:03,048 INFO [stdout] (ServerService Thread Pool -- 56) 2015-12-15 11:33:03,048 [ServerService Thread Pool -- 56] ERROR com.atlassian.plugin.manager.PluginEnabler - Unable to start the following plugins due to timeout while waiting for plugin to enable: com.ptna.plugins.uvcallsub
  • Jared, there are various reasons this can happen, but the general idea is that if the application server is taking too long to start, some plugin deployments may time-out and get skipped. One thing that might prolong application server start up and cause this to happen is if you are trying to load a large number of plugins. Frequently, plugins that were skipped will be loaded after the application server has started as a hot-deployment, but it is best to check the server log or console output to be sure. If plugin is not loaded automatically after the application server has started (assuming you have enabled hot-deployment of plugins), you can hot-deploy the plugin manually.
  • Currently we're on Appian 7.5 and have been on this version since October 2014. This has not happened up until now. We have not changed the number of plugins since April 2015 where we added one. What could be causing the application server to start up slower a year later? Is there a setting we can change to up the timeout to avoid this happening?
  • It could be that the computer itself is running a bit slower, as many computers do slow down over time. It could also be that updates to the plugins have caused them to increase in size or complexity and consequently take longer to load, even though the total number of plugins has remained the same.

    How is memory usage on the system? How long ago did you start implementing daily restarts of the application server using a scheduled task? Are you always seeing this error now, or are there days when all of the plugins get loaded successfully?
  • The scheduled tasked has been used since February. Prior to that we ran the task on 6.6.1 for a couple years. When we upgraded to 7.5 we didn't realize it wasn't working until February since the name of the service changed and fixed the scheduled task. there were no issues with the restart until last week.

    This first time this occured was 12/15/2015. All the plugins successfully installed everyday prior to 12/15 and also installed successfully from 12/16/2015-12/20/2015. We have not upgraded any of the custom plugins recently.
  • Thanks for the info. It sounds like there are some days where it is taking a little bit longer to start up than usual. Do you have any other user-initiated processes or scheduled tasks running at that time? How much of the total system memory is being used during server start-up? Which environment(s) is this occurring in?

    It looks like you've encountered problems in the past with the application server taking a long time to start up ( forum.appian.com/.../e-148090 ), though that sounds like it was in a different environment. Nevertheless, if the application server typically takes a long time to start up anyway, it wouldn't necessarily take a dramatic change in the system to cause plugins to start hitting the timeout threshold.
  • This is happening in our production environment. We cannot afford to have to do a hot deploy every morning. Is there some setting we can adjust to extend the timeout of plugins to avoid this from happening?

    We are currently working on getting a report on the memory and cpu usage of the servers to compare the days it had the issue to other days with the issue.
  • Jared, I would encourage you to look into why the application server is taking a long time to start up in order to head off any potential problems (and you are, of course, welcome to reach out to us for help with this). That said, as a short-term fix, you can increase the timeout by doing the following:

    Go to <JBOSS_HOME>/bin, then

    (For Windows)
    Add the following line to the bottom of standalone.custom.bat:
    set "CUSTOM_JAVA_OPTS=%CUSTOM_JAVA_OPTS% -Datlassian.plugins.enable.wait=300"

    (For Linux)
    Add the following line to the bottom of standalone.custom.sh:
    CUSTOM_JAVA_OPTS="$CUSTOM_JAVA_OPTS -Datlassian.plugins.enable.wait=300"
  • In addition, is there a particular reason that you are restarting your production environment each morning?
  • Penn Treaty has been with Appian since I believe Appian 5.x or 6.x and there was a reason why the daily restart was needed, but that was before I joined the company. Now we continue to do it as a precaution.

    We found that the issue is not with our application server but the environment where the server is might have had a changed and we are working with our operations team in troubleshooting what changed in the environment where the server is.

    Thank you for the setting information.