Is there a way to redirect Appian to another login_failure.jsp ...

Which web server are you using?

Jboss

Redirection is handled more at the web server level, like with IIS or Apache. Which one of those are you using?

Additionally, a redirect like this would require a custom authentication setup, using override files for Spring security. It could even require a custom Java class. Do you currently have a custom authentication strategy? Either way, this would be a pretty in-depth customization to set up through Spring and a custom class. I'd recommend taking a look at the documentation:
forum.appian.com/.../Authentication.html

Also, I could create an enhancement request to make this sort of thing easier in the future, perhaps through OOTB configurations. Let me know if you want me to submit one to our developers.