Working with Advanced CDT Structures

Many-to-Many Relationships

A many-to-many relationship occurs when one or more rows in a table are associated with one or more rows in another table. This is a conceptual design pattern and can be achieved in the database by adding a join table representing the many to many relationship between the two main tables as two individual one to many relationships between each main table and the join table

Two one-to-many relationships established between the two main tables are not the same as one many-to-many relationship. A many-to-many relationship cannot be created using the "@OneToMany" annotation, but keep in mind that Appian supports an annotation to create a many-to-many relationship: "@ManyToMany".

To further clarify: two unidirectional one-to-many relationships will create two mapping tables for the relationship, whereas a unidirectional many-to-many relationship will only create one mapping table. Also, with two unidirectional one-to-many relationships, both entities know about each other, whereas on a truly unidirectional many-to-many relationship, only one entity knows about the other.

Definitions

Unidirectional = Only one of the entities knows about the other.

Bidirectional = Both entities know about each other.

Owner = Entity that stores the id of the other.

Inverse = The entity referred to by the owner.

Full list of relationships

  • One-to-One > - Unidirectional owner, Only one of the entities knows about the other
  • One-to-One <>> - Bidirectional owner*
  • One-to-One <<> - Bidirectional inverse*
  • One-to-Many > - Unidirectional owner (requires separate mapping table)
  • One-to-Many <<> - Bidirectional inverse (other end is Many-to-One <>>)
  • Many-to-One > - Unidirectional owner
  • Many-to-One <>> - Bidirectional owner (other end is One-to-Many <<>)
  • Many-to-Many > - Unidirectional owner (requires separate mapping table)
  • Many-to-Many <>> - Bidirectional owner (requires separate mapping table) Unsupported by Appian.
  • Many-to-Many <<> - Bidirectional inverse (requires a separate mapping table) Unsupported by Appian.

* One-to-One bidirectional tables are both technically owners, but JPA requires that one is defined as an inverse.

Click here for further reading on these relationships and how they work: http://meri-stuff.blogspot.com/2012/03/jpa-tutorial.html

Example many-to-many relationship in Appian

  • A student can be enrolled in multiple classes
  • A class can enroll multiple students

The join table in this example maps one student id to one class id.

We can create a CDT in Appian using an XSD and JPA annotation which models the many-to-many relationship using the JPA annotation @ManyToMany.

Benefits

  • Avoids multiple Write to Data Store nodes to update the required data.  
  • Uses one query rule to pull back an object and its dependencies into one CDT, versus three query rules to get back the same data into three different CDTs (student, enrollment, relevant classes)

Steps

  1. Create an XSD for the CDT to represent the CLASS table
  2. Create an XSD for the STUDENT table, but add CLASSES as its own column with many-to-many JPA notation. Please note the following:
      • “cascade” is set to NONE. This is to prevent writebacks to CLASS in the cases where there is nothing filled in CLASS_ENROLLMENT_BY_STUDENT->CLASSES.
    • NB: If desired, you can create the ENROLLMENT table with an id column before creating this CDT and the XSD will map to the existing table - having an id on this table (and in the CDT) will be useful if searches such as “Get students for class id” will be done; however, the CDT created in Step 4 also covers this scenario.
    • XSD example - CLASS_ENROLLMENT_BY_STUDENT - index=true.zip
  3. Create an XSD for the CDT to represent the simple STUDENT table (without CLASS detail)
  4. Create an XSD for the CLASS table but adding STUDENT as its own column to reflect the many to many relationship. It will be similar to the “CLASS_ENROLLMENT_BY_STUDENT” CDT but an inverse.  
  5. Import XSD in the following order
    • CLASSES
    • CLASS_ENROLLMENT_BY_STUDENT
    • STUDENT
    • STUDENT_ENROLLMENT_FOR_CLASS
  6. Add CDTs to the Appian Business Data datastore and click Publish - allow Appian to create the tables. When the datastore publishes, note that it creates only 3 tables - called “STUDENT”, “CLASS”, and the join table “ENROLLMENT”

Utilizing @SecondaryTable Annotation

@SecondaryTable joins one or more tables together, appears flat but actually has a parent/child “hidden” structure, appears like a one-to-one, and is used rarely.

The @SecondaryTable annotation is used to specify the secondary tables for the annotated entity class. It only applies to a very specific and limited use case: a single entity that is spread across multiple tables. The tables share the same key and are in a one-to-one relationship, and the annotation is meant exclusively to join two tables where the secondary table has a column that matches the primary table's primary key column.

For example, in this diagram, the primary key in class - ID - matches to the column CLASS_ID in CLASS_DETAIL:

The purpose is to be able to split tables with a large number of columns into several tables, and then be able to create a CDT that seamlessly joins all the tables to mimic one big table - much like a view. Specifying one or more secondary tables indicates that the data for the entity is stored across multiple tables.

Please note that @SecondaryTable annotation is not meant to join lookup or reference tables that map against specific non-primary key columns. For mapping lookup or reference data, a regular nested CDT is required - see Nested CDT Structures section for further information. 

For further reading on this topic, click here: http://en.wikibooks.org/wiki/Java_Persistence/Tables#Multiple_tables_with_foreign_keys

http://en.wikibooks.org/wiki/Java_Persistence/Tables#Multiple_tables 

XSD example: CLASS_WITH_DETAIL.zip

Secondary Tables

When working with a large number of columns in a table, you may run into the issue of size limits. For example, MySql imposes a 65535-byte limit on a single table. To avoid this issue, consider using the @SecondaryTable annotation in your XSD file, to allow you to split the data into more than one table, while still being able to use the same CDT structure. With this annotation, you can split your data across smaller tables and have them seamlessly join off of a primary key, and the final result will behave as one table.

Within the <xsd:annotation> structure, add the following annotation:

@SecondaryTable(name="NameOfSecondaryTable", pkJoinColumns=@PrimaryKeyJoinColumn(name=<COLUMN_NAME_OF_PRIMARY_KEY_IN_SECONDARY_TABLE>,referencedColumNName="COLUMN_NAME_OF_FOREIGN_KEY_IN_PRIMARY_TABLE")

In the CDT, define the fields of the secondary table using the following annotation:

@Column(table="NameOfSecondaryTable>", name="COLUMN_NAME_IN_SECONDARY_TABLE")

 

When querying, writing to, or deleting from this datastore entity, you can proceed as if the CDT was written without this annotation. Appian will write the data to the appropriate primary and secondary tables, linking the two by the specified primary and foreign keys. 

Nested CDT Structures

Nested CDTs can be used for the retrieval and storage of individual attributes or nested structures, reducing query calls in some cases when nested. For example, bringing back associated lookup data when returning a CDT - which can be a one-to-one or one-to-many relationship.

For example - if we add a column to STUDENT to denote the major the student has chosen - the CDT can be nested to bring back the relevant MAJOR_REF at the same time as the STUDENT.

XSD Example: MAJOR.zip     STUDENT_WITH_MAJOR.zip

Potential Pitfalls of Nested CDTs

  • If Cascade=CascadeType.ALL  in the XSD JPA annotation, the CDT will write back to the child CDT with every save - if the child CDT is not properly filled in, the lookup table data will be wiped out. This could be useful in a situation where you have a dependent table that is not a lookup - for example, if a student has an associated address and is deleted, then the address would also need to be deleted.

Cascading and its Effects in Appian CDTs

None of these settings are inherently good or bad. It depends on what you're trying to do. Persist and Remove can be very useful to simplify writes to the DB to apply changes by not having to create separate logic to handle deletes. Refresh has the least associated risk and should always be used for reference data. Before making a decision,  be aware of the effects each of these settings has on your data.

  • CascadeType.REFRESH - when writing into the database, nested CDTs with this property will ignore any changes. Retrieving the CDT’s final value from the output will overwrite the nested CDTs values with the latest value from the DB.
  • CascadeType.PERSIST / CascadeType.MERGE - when writing into the database, nested CDTs with this property will be updated with the new values. If the nested CDT entries don’t exist, they will be created.
  • CascadeType.REMOVE - when writing into the database, nested CDTs that were removed will be deleted from the DB.
  • CascadeType.DETACH - should not be used by itself, but only as part of ALL. See here for further information on this setting: http://www.concretepage.com/hibernate/example-cascadetype-detach-hibernate
  • CascadeType.ALL - enables all cascadeType operations.

When to nest and when to flatten CDTs

  • If pulling back CDTs in SAIL when the form is refreshed, then there will be no stale data dangers if there is a nested CDT structure.
  • Create a single transaction so that when updating data, both the parent and child elements are locked and updating occurs within the same transaction. For example, ensure that the child elements are not removed in a separate transaction while they’re also being updated.
  • Stale data occurs when a piece of data changes in the database, but is not updated in Appian. For example, a lookup table of European countries - an entry maintains the same key, but the value of Czechoslovakia changes from to Czech Republic, and a new entry is added for Slovak Republic. If the process instance was started before the database update and has a nested CDT with the value of the European countries lookup, it will show Czechoslovakia despite the database containing Czech Republic. To prevent this - opt to design for short-lived processes that retrieve from the database often.

Using Lists of Primitives

Using a list of primitives - instead of a real lookup table with an id - is a bad idea for multiple reasons:

  • Causes bad table structure
  • Hard to query
  • Not expandable/enhanceable -- it only exists in the context of the parent, doesn’t stand on its own, just a foreign key to the parent.  
  • May lead to poor relationship mapping

Enforcing Foreign Keys with Flat CDTs

When using flat CDTs, Appian will not create any foreign key relationships when publishing datastores and creating tables automatically. The lack of foreign keys causes lookups to not perform well due to lack of indices in the data model.

There are two ways to manage this

  • Do not rely on Appian to create the database. DB structures should not be created by/within Appian but by using DDL scripts.
    • Create the tables and views using SQL with the appropriate foreign key relationships
    • Create XSDs to map to the existing tables using JPA annotation
    • The SQL script would be used to migrate to other environments
  • Amend the DDL generated by Appian
    • Create XSDs, create the CDTs, verify the datastore
    • Download generated DDL from Appian
    • Manually add alter table statements to add constraints in the DDL
    • Use the DDL to generate the database in the development environment, then generate a SQL script to migrate to other environments