CI/CD: Automated test a!startRuleTestsApplications via Web Api call returns incorrect test result

Hi everyone. I have created a web api call and run this function a!startRuleTestsApplications as part of my CI/CD (using Jenkins) practices. However, i noticed that Appian is not returning me the test results correctly.

Example, in my API response, it says Test A in expression rule A fails with this message:

"Failure/Error: Error at input: paymentEvaluation" where paymentEvaluation is of Any Type data type

But when i ran manually by going into the rule A in Appian and click 'Run All Tests', it shows me success. Im assuming the error happens via API call is because in my Test A input expression, i am calling another rule to return me the data as my input. 

 

Other than hardcoding the expected values in the Test Case expression, does anyone know why this is happening and what can I do to work around it?

  Discussion posts and replies are publicly visible

  • Rabiah Khairy (sitirabiatula)  Can you share what you are using as the input expression for 'paymentEvaluation'?

    One theory: All tests will be run with the user context that was used to authenticate the WebAPI call. If that user is different than the user you logged into the expression rule designer and hit "Run All Tests" with, that might be why you are seeing different behavior. It's possible your input expression is trying to query for data but that particular user account doesn't have access to query that data store.

    Separately, if your input expression IS calling a query, I'd recommend against that. This will make your test cases fragile and dependent on the data that might change. There is some guidance around this (and other topics) in the Rule Testing with Appian page in our documentation.

  • Hi Kevin. Thanks for your reply and advice. Ive attached a different example. This rule queries the database for result. This test case is essentially to ensure the value return is of an expected type.  All our CDTs reside in a different package as recommended by Appian while this rule resides in the published application

    However, when i run the test rule via startRuleTestsApplications, the result showed that this test case failed as follows: 

    - Test case: Test 001 - CAPDEV Project Info with Child Entities ( id: 38913 )

           Failure/Error: Assertion expression returned false