Hi,
I have a WebAPI retrieving data from the Appian Database, which is hitting the 1GB payload limit. This means I need to retrieve my data in chunks (multiple calls).
local!entities: a!queryEntity( entity: cons!ABD_INITIATIVE_ENTITY_CONS, query: a!query( filter: a!queryFilter( field: "lastUpdated", operator: "=", value: todate(http!request.queryParameters.lastUpdated) ), pagingInfo: a!pagingInfo( startIndex: 1, batchSize: 50 ) ), fetchTotalCount: true ).data
This query retrieves the majority of the data but not all. How can I tell which data has not been retrieved and needs a separate call?
Thanks.
Discussion posts and replies are publicly visible
Hi Oliverc,
Try to pass the pagination in Request params, So that you can get the data in Chunks instead off all.
But if the request calls are not immediate any data insertion is done in between there is a chance of getting the same data in between two requests.
Hello Oliverc,
I doubt that your code is eliminating the duplicate entries. This happens when we use batchSize in paging info. Suppose my table is having 10 records having 2 records duplicate then batchSize:5 will return total 9 records only. If you provide batchSize:-1 then you will receive complete 10 records. Hence I feel like which ever records you are currently getting having unique values, considering if you don't need to have all the records to be fetched (including duplicate). If you want all the records then you might need to write a stored proc and control the data retrieval.
Hope this information helps you. If it helps then hit like.
Thank You.