Data Sync Limit Strategy

RE: Data Sync Limit Strategy

Peter Lewis — Wed, 25 Oct 2023 20:04:43 GMT

Yeah a!writeRecords() would continue to work as long as your record type is available (ie the sync has not failed). That should work as an approach, but I'd still recommend limiting a!queryEntity() to only scenarios where you truly need the archived data.

RE: Data Sync Limit Strategy

Wen Huynh — Wed, 25 Oct 2023 19:05:57 GMT

Most likely, the centralized relationship table will exceed 4M, so we would like to stlil be able to query these 4M+ records. Is it appropriate to continue writing using a!writeRecords however reads will be converted to using a!queryEntity?

RE: Data Sync Limit Strategy

Peter Lewis — Wed, 25 Oct 2023 18:32:49 GMT

Hi Wen, I think it really depends on how you plan to use the data in the future. For instance, what kind of information will you need about the older data once you approach the row limit? We've seen some customers who have multiple record types - one for "active" cases that are being worked on and another record type for "archived" data. This could work because all of your process models could go against the active cases, while your separate archived data could just be used for reporting.

We're also actively working to expand the amount of data that we can sync and provide features that also make it easier to determine which data is relevant at high data volumes, so we'll keep you posted!

RE: Data Sync Limit Strategy

Wen Huynh — Wed, 25 Oct 2023 16:50:34 GMT

Data archival will eventually occur but we need to store some data for 25 years or longer.

RE: Data Sync Limit Strategy

Stefan Helzle — Wed, 25 Oct 2023 16:49:08 GMT

Difficult. Did you consider to implement a kind of data archival where you move data into a separate long term table structure?

Peter Lewis, this might be a use case you are interested in.