How to Create Expression Rule Test Cases

Appian Max Team — Tue, 23 Apr 2024 13:23:03 GMT

Current Revision posted to Guide by Appian Max Team on 4/23/2024 1:23:03 PM

Expression rule test cases enable unit testing with minimal overhead by reducing the time required to test while increasing code quality. Because expression rules have a wide variety of uses, creating meaningful test cases will speed up unit, regression, and exploratory testing. Use the following guidelines to create test cases for your expression rules that will result in greater code coverage for your application. The first section provides guidelines for creating useful test cases, and the following sections explain types of expression rule test cases in greater detail with Appian examples.

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

If your expression rule has a limited number of outcomes, use the below examples to guide your test case creation. Use test!output to assert that the output is equal to the value you expect. Each outcome should have a test case and corresponding assertion.

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

If your expression rule has an unlimited number of outcomes, use the below examples to guide your test case creation. Use test!output where it makes sense to ensure the output is equal to the expected value.

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error handling test cases should be created based on the functions and inputs of the rule. All rules should have null handling test cases (unless already covered by a parent rule), while other error handling scenarios should be determined at the Designer’s discretion. All error handling cases should have the test output match an asserted output.

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Note: Expression rules containing queries should only include the a!queryEntity() function. Any expressions to manipulate the data or output different values based on the query results should be done in separate expression rules, using the results of the query as a rule input. This allows flexibility to assert that the manipulation or conditional output operates correctly without relying on transaction data that can change across Appian environments.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

These are test cases a user could have set up for the rule provided in Figure N. The first box contains Coverage Tests. They provide code coverage by testing with expected inputs and outputs. The second box contains Error Handling Tests. These tests are staged with inputs that would typically result in a denominator of 0. Finally, the third box contains Null Handling Tests. This is another specific type of error handling tests. They are used to verify that our rule can properly handle null inputs. Note that more than one was required for this rule.

Regression Testing with Test Cases

Test cases make it easier and faster to regression test. Before making a change to an existing expression rule, run all existing test cases within the rule and resolve any that have failed. After making the change, add coverage and error handling test cases to test the change. Finally, run all test cases to ensure the update didn’t break any existing cases.

Before a deployment, run the Automated Rules Testing Application (ART) in the target environment (non-production) to ensure all test cases work as expected before the deployment. If any rules do not work, resolve the issues in the lower environment prior to the deployment. After the deployment, re-run ART in the target environment to ensure all new and old test cases work as expected.

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

Tags: testing, Architecture

How to Create Expression Rule Test Cases

joel.larin — Thu, 11 Jan 2024 15:13:04 GMT

Revision 13 posted to Guide by joel.larin on 1/11/2024 3:13:04 PM

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

Tags: testing, Architecture

How to Create Expression Rule Test Cases

joel.larin — Thu, 11 Jan 2024 15:08:52 GMT

Revision 12 posted to Guide by joel.larin on 1/11/2024 3:08:52 PM

youtu.be/ATqelb2CJj4

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

Tags: testing, Architecture

How to Create Expression Rule Test Cases

joel.larin — Thu, 11 Jan 2024 14:57:30 GMT

Revision 11 posted to Guide by joel.larin on 1/11/2024 2:57:30 PM

youtu.be/vMuNwRFdty4

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

Tags: testing, Architecture

How to Create Expression Rule Test Cases

joel.larin — Tue, 31 Oct 2023 16:07:22 GMT

Revision 10 posted to Guide by joel.larin on 10/31/2023 4:07:22 PM

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

Tags: testing, Architecture

How to Create Expression Rule Test Cases

joel.larin — Tue, 31 Oct 2023 15:06:27 GMT

Revision 9 posted to Guide by joel.larin on 10/31/2023 3:06:27 PM

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

How to Create Expression Rule Test Cases

joel.larin — Tue, 31 Oct 2023 15:06:01 GMT

Revision 8 posted to Guide by joel.larin on 10/31/2023 3:06:01 PM

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

How to Create Expression Rule Test Cases

joel.larin — Tue, 31 Oct 2023 15:05:24 GMT

Revision 7 posted to Guide by joel.larin on 10/31/2023 3:05:24 PM

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

How to Create Expression Rule Test Cases

joel.larin — Tue, 31 Oct 2023 15:04:21 GMT

Revision 6 posted to Guide by joel.larin on 10/31/2023 3:04:21 PM

Introduction

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples*

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples*

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

*Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

How to Create Expression Rule Test Cases

joel.larin — Tue, 31 Oct 2023 14:59:36 GMT

Revision 5 posted to Guide by joel.larin on 10/31/2023 2:59:36 PM

Introduction

Test Case Creation Guidelines

The following guidelines apply to expression rules which output a value:

Create test cases from the start of application development to aid regression testing.
If an expression rule calls another expression rule, do not duplicate test cases already created in the child within the parent rule. The parent rule should contain test cases that test any logic within the parent rule itself and should assume the child has sufficient test cases.
When possible, break down large expressions to individual rules so that it is easier to ensure test coverage with expression rule test cases
When expression rule test cases are evaluated, the expression is evaluated first, followed by the assertion within the test case. Avoid the use of now() in test case assertions, as this can trigger false negatives due to the dateTime type evaluating the values down to the millisecond. Instead, check whether the field is not blank.
Use test!output to assert the output equals what you would expect.
Use Exploratory Testing to identify edge cases for which to create test cases.

Types of Test Cases

When creating test cases, consider the two types of test cases: coverage tests and error handling tests. Every expression rule should always have at least one of each kind of test case.

Test Type	Description	Implementation
Coverage	A coverage test ensures the rule inputs result in the intended output. In other words, an area of the rule has been sufficiently “covered” via a test case.	Set inputs to expected values.
Error Handling	An error handling test ensures the expression rule has proper error handling.	Set inputs to values that would typically break the rule.

Creating Coverage Tests

Use the diagram below to determine which test cases are required and how to set assertions.

Coverage Test Case Examples*

Limited Number of Outcomes

Nested if()s: Create 1 test case per possible condition
Choose(): Create 1 test case per key
Mathematical Comparisons: Create 1 test case per possible comparison
- For example, for a greater than comparison, you must have 3 cases covering when the input is less than, equal to, or greater than the comparative value

Unlimited Number of Outcomes

Calculations: Based on the calculation being run, identify inputs that will cover groupings of possible outcomes. The asserted output should be calculated outside of Appian and then asserted within the case.
- For example, if you have a rule that is executing the following calculation: A-B, with integers A and B, test cases should cover the following scenarios as they all would reveal something different about the calculation:
  - A>B (output is positive)
  - A<B (output is negative)
  - A=B (output is 0)

Integrations: Expression Rule Test Cases can be used to test integration connectivity prior to a deployment. Create an expression rule that calls the integration and a test case that passes in data to populate the request. Set the assertion to “expression evaluates to true”. The expression should check that the “success” output of the integration = true. Add these expression rules to an app that is deployed prior to your business application.

Date Times: Create test cases which cover all groupings of expected outcomes, including all known edge cases. Assert that the output is correct for each test case.
- For example, if a rule checks whether a date is a work day (calisworkday(datetime)), create test cases with the following inputs to test the possible outcomes. Use the value in parenthesis to assert that test!output is the correct value:
  - A work day (true)
  - A weekend day (false)
  - A holiday (false)

- Consider test cases that go “stale” (rules where the output depends on whether inputs are in the past, future, etc). Avoid using hard coded dates and instead, use the appropriate Appian date and time functions (now(), today(), etc.) to ensure that test inputs will last for the lifetime of the application. As mentioned previously, do not use now() in test case assertions.
- If the rule contains calculations on dates in particular timezones, use the gmt() and local() functions to enforce a timezone and assert the correct output for that timezone.
Type Constructors: Test cases are only necessary to test any logic within the rule that is outside of the constructor itself. “Assertion expression evaluates to true” is typically useful for these types of test cases.
- If the rule takes a raw set of data and only casts items that meet specific criteria, a test case can be created in which you validate that all items in the output meet that criteria.
- If the rule casts individual fields from a rule input to a new data type, create a test case that asserts that each field equals what is expected by using test!output.

Creating Error Handling Tests

Error Handling Test Case Examples*

Null Handling: Set one, some, or all rule inputs to null to ensure the rule executes properly and outputs an expected value. In many cases, more than one null handling test case may be necessary.

Invalid Mathematical Expressions: In rules that execute a calculation, provide inputs that produce an invalid calculation, such as a denominator of zero. Ensure the rule still executes properly and outputs the expected value.
- For example, if your rule is calculating the square root of the difference between A and B, create a case in which B > A.

Functions that error given invalid inputs: If a rule uses functions that produce an expression error if the input is invalid (e.g. user/group functions, not(), etc), create a test case for each invalid input to verify that expression errors are avoided.
- For example, if your rule has a username input, cases should be created in which a random non-username string is passed in. The rule should be coded to handle this case properly.

Date Times: Create test cases with datetimes outside of what the rule expects to ensure that the rule handles them appropriately.

Queries:
- For queries in which data is consistent across environments, regardless of user interaction with the system (i.e. reference data), create test cases with data that will always be returned and assert that the test output matches the desired value.
- For queries on transactional data, expression rule test cases should not be used to validate that a particular data set is returned because data can vary based on the environment and user interaction. However, test cases can still be used in the following ways:
  - On all expression rules that have queries, you can create a test case which asserts that the “Test completes without errors” to validate database connectivity.
  - If the query ignores null filter values, include a test case which passes in values for all possible filters and asserts that the test completes without errors to ensure that all field names and filter configurations are valid.
  - If the query does not ignore null filter values, create null-handling test cases for the rule.
  - For all filters that are created, test cases can be used to validate that the returned results match the filter value that was passed in. For example, if “true” is passed into an “isActive” filter in the query, a test case can validate that all results returned have isActive=true. For test cases like these, include appropriate null handling for when no data matches the filter criteria.
- Use test cases to assert that the output is of the same data type as the data entity being queried. This can be done using the “Assertion expression evaluates to true” assertion option, where the expression would check that the type of test!output matches the expected type using the typeof() function.

Example Expression Rule

An example expression rule that has coverage tests and error handling tests can be found below.

Figure N : Sample Expression Rule

Figure M : Sample Test Cases

Regression Testing with Test Cases

*Note: These examples are not all-encompassing. Best judgement should be used to determine what cases are required.

How to Create Expression Rule Test Cases

matt.cosenza — Tue, 31 Oct 2023 13:25:50 GMT

Revision 4 posted to Guide by matt.cosenza on 10/31/2023 1:25:50 PM

sdfs

How to Create Expression Rule Test Cases

matt.cosenza — Tue, 31 Oct 2023 13:22:49 GMT

Revision 3 posted to Guide by matt.cosenza on 10/31/2023 1:22:49 PM

Automated testing reduces the amount of manual testing necessary in an Appian application and reduces risk at each stage of the application development lifecycle. Automated testing can be used throughout the Appian application development lifecycle to:

Test new rules created within an Appian application such as conditional fields, complex business logic, or validations
Complete functional testing to ensure all requirements are verified and validated
Regression test the application to ensure that existing functionality is not affected when:
- Appian application gets promoted to different environments
- New functionality is created
- Concurrent development teams are established
Fast forward to a particular step in a workflow to perform manual testing from there

Automated testing should be implemented early in a project to prevent buildup of functionality for which test scripts must be written. The longer a team waits to set up automated testing, the more burdensome it can become to do so. Conversely, if test scripts are written for functionality that is highly likely to change, it can be time-consuming to update them to keep up with changing requirements. So, a project team must work together to achieve a balance between these alternatives by determining when and for what purposes to design automated testing scripts.

It is not recommended to have 100% test coverage of an application through Automated Testing due to the below reasons:

It is time consuming to plan/implement test scripts and run the tests to cover the entire application.
We currently do not have a method to gauge completeness of test coverage for an application.
Some areas of the application may change frequently over time and integrations may depend on external systems.

These factors can make it difficult to get a good return on the time investment put into building test scripts to cover all functionality in an application.Given that 100% of the application cannot reliably be validated through automated testing, the core team (developers, PO, QAs) must collaborate in order to identify highest priority items to be addressed. Teams should consider factors such as level of effort to create new automated tests and level of risk in determining where to focus test coverage. Based on this collaboration, the automated testing suite can be incrementally updated to ensure highest value areas are targeted first.

Automated testing is not the same as performance testing. To find more information on performance testing best practices and tools such as JMeter and Loadrunner, see Performance and Load Testing.

Automated Testing Tools for Appian Applications

The following tools can be used to automate testing for Appian applications. The Tools In Appian section describes automated testing features that come OOTB with Appian, which should be used wherever possible. The External Tools section includes the available tools which were not built for Appian, and the Tools Built for Appian section describes tools built for Appian from external tools to facilitate automated testing in Appian.

Expression Rule Test Cases: Test cases can be configured for expression rules to test outputs with any combination of inputs. These function as automated tests built into the expression rule object. See the documentation for more information and instructions on when and how to use expression rule test cases. By importing the Automated Rule Testing application to an environment, administrators can run all test cases regularly and leverage reporting on failures and successes. This will help identify bugs in expression rules faster, speed up regression testing after rules have changed, and make smoke testing deployments even faster.

These should generally only be used when necessary. Otherwise, it is recommended to use FitNesse For Appian or Appian’s Selenium API with Cucumber (described in the Tools Built for Appian section below) because these are preconfigured to work for Appian applications.

Selenium: Selenium is an automated testing tool that is able to control a web browser. Selenium has two different products: WebDriver and IDE. The WebDriver is a collection of language specific bindings that drive a web browser while the IDE is an add-on to Firefox that is able to record and playback interactions with the web browser.

For more information, see the Selenium Website

FitNesse: FitNesse is automated testing software which uses a wiki language and allows for a low barrier to entry and ease of use. FitNesse test scripts and test suites are created in the wiki language that map out to functions written in a development language that wrap a Selenium Webdriver to perform actions on an Appian application.

For more information, see the FitNesse Website

Cucumber: Cucumber is an automated test tool utilizing Behavior Driven Development (BDD) that allows testers to specify test scenarios in a human readable form. Cucumber can be used to create test scripts in plain language with Gherkin syntax. One benefit of Cucumber is that the Product Owner or business users are able to write test cases without a technical background. These test scripts map to functions in a development language (such as Java or Ruby) that wrap a Selenium Webdriver to perform the specified actions on an Appian application.

For more information, see the Cucumber Website

Appian has created the following tools to reduce the amount of work required to automate testing of Appian applications.

Appian Selenium API: The Appian Selenium API allows you to run Appian UI tests directly from a JUnit Test Case, create your own Gherkin steps and step definitions with Cucumber, and run tests from any java based test tool of your choice without having to create the necessary xpath for selenium to interact with Appian. When using Cucumber, it is recommended to use this API to customize step definitions to match the business users’ phrasing.

Download the Appian Selenium API

FitNesse for Appian: Appian has written WebDriver fixtures to leverage Selenium and FitNesse to create FitNesse for Appian. This tool is easier to set up than Fitnesse and provides a faster way to build automated tests for Appian applications. This is complemented by the FitFam application, which allows users to create FitNesse test scripts for complex interfaces solely by interacting with the interface.

Download FitNesse for Appian
For more information, see the FitNesse for Appian Documentation
For implementation guidance, see the FitNesse for Appian Best Practices Play

Cucumber for Appian: Cucumber for Appian is a client-side tool that provides a sample set of step definitions similar to those provided by FitNesse for Appian. Note that these step definitions are not application specific. Features and scenarios written with these sample step definitions will not embrace the Behavior Driven Development (BDD) aspects of Cucumber. Cucumber for Appian is a good starting point for teams not familiar with Cucumber to get familiar with Cucumber before writing their own cucumber step definitions in Java. Additionally, the sample step definitions will allow testers to write a fairly complete set of tests without needing to know Java.

Teams experienced in writing automated tests as code might find the sample step definitions provided by Cucumber for Appian limiting. For example, gherkin syntax does not allow for saving variables between steps or looping. We recommend leveraging the Appian Selenium API to write your own step definitions for testers that want to trade the simplicity of Cucumber for Appian with the power provided by code. Custom Step Definitions in tandem with the Appian Selenium API will allow you to write future proof tests that will work between upgrades and embrace BDD.

Download Cucumber for Appian

For more information, see the Cucumber for Appian Playbook Play

How to Create Expression Rule Test Cases

matt.cosenza — Tue, 31 Oct 2023 13:18:22 GMT

Revision 2 posted to Guide by matt.cosenza on 10/31/2023 1:18:22 PM

Test new rules created within an Appian application such as conditional fields, complex business logic, or validations
Complete functional testing to ensure all requirements are verified and validated
Regression test the application to ensure that existing functionality is not affected when:
- Appian application gets promoted to different environments
- New functionality is created
- Concurrent development teams are established
Fast forward to a particular step in a workflow to perform manual testing from there

It is not recommended to have 100% test coverage of an application through Automated Testing due to the below reasons:

It is time consuming to plan/implement test scripts and run the tests to cover the entire application.
We currently do not have a method to gauge completeness of test coverage for an application.
Some areas of the application may change frequently over time and integrations may depend on external systems.

Automated testing is not the same as performance testing. To find more information on performance testing best practices and tools such as JMeter and Loadrunner, see Performance and Load Testing.

Automated Testing Tools for Appian Applications

For more information, see the Selenium Website

For more information, see the FitNesse Website

For more information, see the Cucumber Website

Appian has created the following tools to reduce the amount of work required to automate testing of Appian applications.

Download the Appian Selenium API

Download FitNesse for Appian
For more information, see the FitNesse for Appian Documentation
For implementation guidance, see the FitNesse for Appian Best Practices Play

Download Cucumber for Appian

For more information, see the Cucumber for Appian Playbook Play

How to Create Expression Rule Test Cases

joel.larin — Wed, 13 Sep 2023 14:07:57 GMT

Revision 1 posted to Guide by joel.larin on 9/13/2023 2:07:57 PM

dfs