Mine Appian Applications

This guide provides guidance and recommendations for designing Appian applications that can provide meaningful business insight when mined using Appian Process Mining. 

 This guide covers the following topics:

• Process Mining, Start to Finish for Appian Applications
• Event Data Capture for Your Appian Applications
• Key Attributes for Better Process Mining Insights

New in Appian 23.2. Appian Record events allow you to seamlessly track who takes action on your records and when, with minimal configuration on your part. This feature greatly reduce the time and complexity needed to create business events. It is highly recommended to use the Record events feature over other custom built alternatives.

Process Mining, Start to Finish for Appian Applications

Before you start using process mining to extract meaningful insights from your Appian applications, a brief background on the discipline of process mining is recommended. 

The steps necessary to mine an Appian application are depicted below.

Step 1: Design your application with events in mind

When planning your events, remember that events can be “anything” that happens in a process. You should consider capturing enough events that you can provide a realistic and faithful picture of the processes' timelines and how they are executed. 

• Not every node in a process model should correspond to an event. 
• Alternatively, only recording when the processes start and end will not give you enough information to understand how the processes are executed.

When developing an Appian application, it is common practice to capture:

1. Business data related to what the application is about e.g. invoices, on-boarding, contracts. The business entities are referred to as Cases in process mining.  This data will be transformed into Case Logs by Mining Prep.
2. Timestamps, events and/or milestones describing when and how the processes are executed. This data will be transformed into Events Logs by Mining Prep.

The level of granularity for the events is application specific and based on your business requirements. At a minimum, we recommend capturing one event per user task or wizard of related tasks. Additional events could be captured for system-to-system integrations or for tasks performed in third-party systems; especially when these are asynchronous or variable in duration.

Use one or more of the following common patterns to capture your events:

1. For well “structured” workflows that sequence a series of distinct tasks
   • Persist events in the process model after each task or set of related user tasks.
2. For record centric, “case management”-style applications using record actions
   • Persist events after completion of the short-lived action processes

Refer to the section “Event Data Capture in your Appian Applications” for details on patterns used to model and persist events.

Certain minimum data requirements must be met in order to create an event log and mine a business process. Refer to the Appian Process Mining documentation for more information: Data Sets and Event Logs.

Step 2: Import application data into Mining Prep

Import application data (events and business entity data) into Mining Prep using one of the following methods:

1. Use the Mine Appian feature to connect Mining Prep to Appian Records
   • This requires to create record types for the event table and other related tables
2. Connect Mining Prep to the database over JDBC
3. Manually import CSV files

Step 3: Transform application data in Case/Events Logs and load into Process Mining

Using Mining Prep, aggregate and transform your application data to create a “flat” representation of processes timelines. This may include:

• Joining to “reference data” tables to resolve identifiers into human friendly descriptive names e.g. statusId 3 == “Completed”
• Joining to auxiliary tables to extract additional attributes that help describe the events or the cases

When you're finished with transformations, load the cleaned data to process mining for modeling and analysis.

For example:

Step 4: Analyze the discovered models and KPIs to extract insights

After event data is loaded into Process Mining, the discovered model visualization displays the existing process. Use the case and event data to slice and dice the activities and discover meaningful insights.

Additional information about Steps 2 to 4 is available in the Appian documentation page Process Mining, Start to Finish.

Event Data Capture For Your Appian Applications

This section describes 3 design patterns for capturing event data as part of your Appian applications:

1. The out of the box Appian Record events that is fast and easy to configure. This is the recommended approach for application events
2. The Audit History pattern is based on audit data that is manually created and saved in the database.
3. The Process Task Data pattern is based on process task data that is automatically generated by Appian when tasks are completed.

These patterns can be used independently or together within the same Appian application in order to capture event data. The table below summarizes their main characteristics.

Record Event	Audit History Pattern	Process Task Data Pattern
Events are configured for the main business record types in the application	Audit data can represent any tasks, business events or business milestones as designed in the application	Audit data is limited to user input tasks. All events generated are tied to users completing tasks
Events are created automatically by Appian when you use the Write Records smart service	Audit data is created manually by populating a CDT and persisting the values in a database table	Audit data is created automatically by Appian when user input tasks are executed. A process model retrieves it using a Process Task Report and persists it in a database table
Events generated are populated by Appian based on the event type that you provide	Event attributes can be generated using expressions rules	The Case ID attribute and other event attributes must be designed as process variables in all process models containing user input tasks to mine
Additional events can be created using the Record event guided user experience in the Record Type designer	Additional events require development to be generated and persisted in the database	Additional events from new applications or new tasks can be captured using this pattern without additional code

Real life implementations of these patterns include:

• Foreign keys to reference data such as related entities, statuses, types, etc.
• Additional attributes describing the events

Use these patterns as starting points for designing tailored solutions that work for your domain model and your requirements.

Record Events

The Record events feature was released in Appian 23.2. 

 Record events allow you to seamlessly track who takes action on your records and when, with minimal configuration on your part. You’re able to generate and capture data about your processes within your standard Appian workflows. 

Audit History Pattern

The Audit History Pattern relies on audit data manually created and saved into a database to create Process Mining events.

The Audit History application (available in the Appian AppMarket) showcases an example of this pattern. The audit data is created by a script task and inserted into a dedicated database table.

The CDT named “AH_AuditHistory” represents an audit history entry/row. It contains the necessary data points to generate the key attributes needed for process mining such as the case identifier, timestamps and activity. 

To Mine an application that uses the Audit History pattern, map the data stored by the CDT “AH_AuditHistory” to the Process Mining event log attributes. 

Process Mining Log Attribute	Audit History Data
Case ID	Use the combination of the fields “recordId” and “recordType” or a unique business identifier if available.
Activity	Use the field “action”
Start Date	Use the field “startTimestamp”
End Date	Use the field “endTimestamp”

When using this pattern, you should: Tailor the audit history CDT and table to your application: Use a business identifier if available Add extra attributes that are important for the events and that can change overtime e.g. the name of the person that performs the action, a status, an invoice amount. (See Process Mining Data Set)  Implement one CDT and table per application or main business entity to capture the corresponding events

Process Task Data Pattern

The Process Task Data pattern relies on the task metrics generated by the Appian platform to create Process Mining events. To use this pattern, save the task metrics and additional contextual data into the database; to be later imported into Mining Prep.

In order to extract this process task data, use:

    1. A task report that gets the task data for a set of process models or for the entire platform

    2. A Custom Data Type (CDT) that maps to the data from the task report

    3. A process model - running on a daily timer - that executes the task report, stores the outputs as a list of CDTs and persists this data into a database

To Mine an application that uses process task data, map the data stored in the process task table to the Process Mining event log attributes:

Process Mining Log Attribute	Process Task Data
Case ID	Use the attribute CASE_ID Do not use the process model ID. Use a business attribute that is unique to each case/business entity. This value is configured as pv!processVariableName in the report.
Activity	Use the attribute TASK_NAME. The static task name must follow the recommendations related to the event activity name as mentioned in this document.
Start Date	Use the attribute START_TIME
End Date	Use the attribute END_TIME
Event attributes	Create one event attribute for “Owner” “Assignees” “Status” “Lag duration” “Work duration”

Key Attributes for Better Process Mining Insights

Generating meaningful insights using Process Mining is tied to the quality of the events and attributes captured in the event log. Contextual information about the events and the cases is also critical for the analysis.

This section lists some of the key attributes that you should pay close attention to when capturing data and generating event logs.

Event End Dates

Although Appian Process Mining can be used with events that only have start dates, having both start dates and end dates provide a deeper level of granularity to the analysis.

Start dates and end dates allow Appian Process Mining to calculate the duration of the events as well as the lag time between subsequent events.

Event Activity Names

One of the required attributes for Appian Process Mining, the activity name, is used to group events across cases to create the discovered model and calculate different statistics.

When capturing data in Appian to later create the event log for Process Mining, you must define your events in such a way that their names are clear and precise but not case specific.

Recording this attribute correctly is critical to decrease the complexity of data transformation that may be required to create the event logs. 

Below are the recommendations for recording this attribute correctly.

The purchase request 123-abc had been approved by John on 1/1/1970 DON’T  Not generic. It contains case specific information that would prevent Process Mining from aggregating similar events from other cases.  Requires complex transformation to be used in Process Mining

Approved DON’T  Too succinct Potentially not unique in the context of a case/business process. For example, within an Order To Cash process, invoices, orders and discounts could be “Approved”. Not enough context around the action. What was approved? Who did it?

Purchase Request Approved DO Simple and human readable Generic without any case specific data Contain the entity (purchase request) and the action performed (approved) Allows for easily capturing the alternative path as “Purchase Request Rejected” Well suited for analyzing high level aggregated activities within a business process without including the context of the actor

Purchase Request Approved By Finance DO Simple and human readable Generic without any case specific data Contain the entity (purchase request), the action performed (approved) and the actor (Finance) Allows for easily capturing the alternative path as “Purchase Request Rejected by Finance” Well suited for analyzing low level activities within a business process and differentiating the approval tasks per actors