KB-XXXX System emails/process emails are no longer being sent with a "Connection refused" error

Symptoms

• System emails for account creation are not sent.
• System emails for resetting a password are not sent.
• Process models with Send Email nodes are paused.
• The application server log will contain the following errors:
  

  ERROR com.appiancorp.ra.workpoller.WorkItemListener - The following exception occurred while attempting to complete work item [(WorkId: 14, WorkType: [AbstractMailRequest: from=null, to=<Email>, subject=<Subject>], Response: null, ServerName: NotificationsEmail)] - Exception:
  javax.resource.spi.work.WorkCompletedException: com.appiancorp.process.engine.EmailBodyException: Email body failed to render
  …
  Caused by: javax.ejb.EJBException: com.appiancorp.process.engine.EmailBodyException: Email body failed to render
  …
  Caused by: javax.net.ConnectException: Connection Refused

Cause

In order for the application server to send an email, it exposes a HTTP endpoint that it itself invokes in order to generate the email body that will be used in the sent email. When the app server is unable to invoke the endpoint (due to a configuration or networking issue), it will be unable to generate email bodies, and thus unable to send emails.

The underlying cause of the issue can vary, but is usually associated with web server or application server configurations, or networking configurations.

Troubleshooting

• Use the telnet utility to attempt to access the value declared custom.properties for the MAIL_SERVER_AND_PORT configuration to verify network connectivity
• Use the netstat utility to ensure that the port is being listened to by the expected OS process associated with the application server

Action

• Confirm that <APPIAN_HOME>/conf/custom.properties has the correct and expected configurations for MAIL_SERVER_AND_PORT and MAIL_SCHEME.
• Confirm that the standalone.xml and Context.xml configurations have declared the appropriate http/https listener in order to accept http connections.