SAML Authentication issue

Hello,

I have been trying to establish SSO for Appian. I have uploaded .pem file and IdP metadata as well, also I have filled out all rest the information required in SAML form of Admin portal. But I encountered below issue when tried to test the configuration. 

Authentication was unsuccessful.

Reason: The user you attempted to sign in as does not authenticate through SAML.

Close this window and return to Appian Administration Console.

I see this is tomcat logs : 

ERROR com.appiancorp.security.auth.saml.SamlTestServlet - Error occurred during SAML authentication test: null
com.appiancorp.security.auth.saml.exception.NonSamlUserException


FYI, IdP is Azure Ad. Interesting part is Azure sign in logs mentions that the connection was successful even though we see above issue response from Appian