Hi,
What is the most efficient way to retrieve data from a large table (more than 4 million rows) in Appian?
When I use queryEntity to fetch data from this table, it takes a long time to execute, which negatively impacts Appian’s performance. How can I improve the data retrieval process?
queryEntity
My current approach to solve the issue:
I split the data retrieval into smaller batches. Specifically, I:
Created multiple records that point to the same source table.
Used the foreach function to query each record in batches of 5,000 rows.
foreach
Used the union function to combine the results from each batch.
union
Used the union function to combine the results from each query record.
Then, combined the results across all the records to get the complete dataset.
I’m concerned about future system complexity.Do I really need to create multiple records and change the system design just to retrieve all the data from a large table? Or is there a better, more efficient way to handle large data retrieval in Appian without adding complexity?
Thanks,
Discussion posts and replies are publicly visible
Mohammed Dyab said:What is the best way to get data from the database to Appian.
a!queryEntity or a!queryRecordType
And you are limited to 5k rows in data, and showing 5k elements on one screen will not be a great UX.
Thanks