Syncing Third Party Updates to a series of Database Tables

RE: Syncing Third Party Updates to a series of Database Tables

Richard Nolan — Fri, 05 Apr 2024 17:12:35 GMT

I see two streams of relatively tedious work:
1) Identifying the changes.

2) Synching the changes.

I think there are ways to short-path the first one, either through a tracking table updated by the stored procedures, or a view that is aware of Appian's last poll and shows the keys that have been added since last poll.

Part two is where I think Appian could provide a lift.

From our perspective, there are only a peak of 120 updates a month, but the data model is terribly distributed. Even having something that synchs the records based on the top-level parent, and transcends the relationships to synch children or reference tables.

RE: Syncing Third Party Updates to a series of Database Tables

Shubham Aware — Fri, 05 Apr 2024 17:09:27 GMT

It's a great idea. I'll draft up a product enhancement suggestion as well.

RE: Syncing Third Party Updates to a series of Database Tables

Stefan Helzle — Fri, 05 Apr 2024 17:08:53 GMT

You already run a stored procedure that could keep track of all the updates. Why not write these to a separate table. Then a scheduled process in Appian could manage these update by syncing the modified records.

RE: Syncing Third Party Updates to a series of Database Tables

Richard Nolan — Fri, 05 Apr 2024 17:07:46 GMT

I'm going to submit a product enhancement on this issue, as it seems like it should be on their queue if the focus on Synched Records continues to be a significant component of their low-code framework.

RE: Syncing Third Party Updates to a series of Database Tables

Shubham Aware — Fri, 05 Apr 2024 17:01:14 GMT

Yeah that would be really nice. If Appian releases that feature. Try this workaround if possible

To identify unsynced records in an record, you could create a rule that compares the completion time of the previous poll with the timestamps of records in each table. This rule would recognize the discrepancy between synced and unsynced values in the underlying tables. You could automate this process to run periodically or trigger it manually as needed.

RE: Syncing Third Party Updates to a series of Database Tables

Richard Nolan — Fri, 05 Apr 2024 16:26:10 GMT

That was my initial feeling to, however at 50+ tables, and record sync not descending the relationship structure, that's a lot of asynchronous triggers to the Appian Sync Smart Service.

Given the table is partially synchronized before the polling tool examines the table, I also need some way of identifying the "un-synched" records. Probably by tracking the previous poll's completion time and looking at the timestamp of the records in each table.

It would be really nice if there was a service that recognized the discrepancy between synched records and un-synched values in the underlying tables.

RE: Syncing Third Party Updates to a series of Database Tables

Shubham Aware — Fri, 05 Apr 2024 16:20:39 GMT

For me it looks good.

I implemented similar solutions few days back.

RE: Syncing Third Party Updates to a series of Database Tables

Richard Nolan — Fri, 05 Apr 2024 16:15:01 GMT

My Record Types are based on those same tables, so I see my worst case scenario as:

1) Schedule a process that periodically uses a!queryEntity against the tables (requires CDT configuration?). Say I get these results:

Table 1: PK Values {100,101,102}
Table 2: PK Values {1000,1001, 1002}
Table 3: PK Values {2000,2001,2002}
Table N: PK Values {10000,10001,10002}

2) Trigger a series of Synchronize Record Smart Services where the configuration looks like:

recordType!Table 1:{100,101,102},
recordType!Table 2: {1000,1001, 1002},
recordType!Table 3:{2000,2001,2002},
recordType!Table N:{10000,10001,10002},

Is there a better way?

RE: Syncing Third Party Updates to a series of Database Tables

Shubham Aware — Fri, 05 Apr 2024 16:07:42 GMT

You can automate the process by setting up periodic database queries to retrieve new entries, mapping the data to your record types, and then using Appian's record sync functionality to seamlessly integrate the new entries without manual examination or sync operations for each record type.

Refer for sync records

https://docs.appian.com/suite/help/24.1/Sync_Records_Smart_Service.html#:~:text=The%20Sync%20Records%20smart%20service,first%20configure%20a%20sync%20expression.