How to Create Expression Rule Test Cases

Expression rule test cases enable unit testing with minimal overhead by reducing the time required to test while increasing code quality. Because expression rules have a wide variety of uses, creating meaningful test cases will speed up unit, regression, and exploratory testing. Use the following guidelines to create test cases for your expression rules that will result in greater code coverage for your application. The first section provides guidelines for creating useful test cases, and the following sections explain types of expression rule test cases in greater detail with Appian examples.

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

• Create test cases from the start of application development to aid regression testing. 
• If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
• When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
• When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
• Use test!output to assert the output equals what you would expect.
• Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

If your expression rule has a limited number of outcomes, use the below examples to guide your test case creation. Use test!output to assert that the output is equal to the value you expect. Each outcome should have a test case and corresponding assertion.

• Nested if()s: Create 1 test case per possible condition
• Choose(): Create 1 test case per key
• Mathematical Comparisons: Create 1 test case per possible comparison 
  • For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

If your expression rule has an unlimited number of outcomes, use the below examples to guide your test case creation. Use test!output where it makes sense to ensure the output is equal to the expected value.

• Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
  • For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
    • A>B (output is positive)
    • A<B (output is negative)
    • A=B (output is 0)

• Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

• Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case. 
  • For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
    • A work day (true)
    • A weekend day (false)
    • A holiday (false)

• • Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
  • If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
• Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
  • If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
  • If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error handling test cases should be created based on the functions and inputs of the rule. All rules should have null handling test cases (unless already covered by a parent rule), while other error handling scenarios should be determined at the Designer’s discretion. All error handling cases should have the test output match an asserted output.

Error Handling Test Case Examples

• Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

• Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
  • For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

• Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
  • For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

• Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

• Queries: 
  • For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
  • For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
    • On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity. 
    • If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
    • If the query does not ignore null filter values, create null-handling test cases for the rule. 
    • For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
  • Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Note: Expression rules containing queries should only include the a!queryEntity() function. Any expressions to manipulate the data or output different values based on the query results should be done in separate expression rules, using the results of the query as a rule input. This allows flexibility to assert that the manipulation or conditional output operates correctly without relying on transaction data that can change across Appian environments.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

These are test cases a user could have set up for the rule provided in Figure N. The first box contains Coverage Tests. They provide code coverage by testing with expected inputs and outputs. The second box contains Error Handling Tests. These tests are staged with inputs that would typically result in a denominator of 0. Finally, the third box contains Null Handling Tests. This is another specific type of error handling tests. They are used to verify that our rule can properly handle null inputs. Note that more than one was required for this rule.

Regression Testing with Test Cases

Test cases make it easier and faster to regression test. Before making a change to an existing expression rule, run all existing test cases within the rule and resolve any that have failed. After making the change, add coverage and error handling test cases to test the change. Finally, run all test cases to ensure the update didn’t break any existing cases.

Before a deployment, run the Automated Rules Testing Application (ART) in the target environment (non-production) to ensure all test cases work as expected before the deployment. If any rules do not work, resolve the issues in the lower environment prior to the deployment. After the deployment, re-run ART in the target environment to ensure all new and old test cases work as expected.

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.