Error in Query Entity due to large data in Database

Question

Hi All, 
 I tried to fetch data from Database using queryEntity in expression rule. 
 But due to large data in database(6278),it shows me the below error.

Could you please help me to sort this issue. 
 Thanks in advance.

Chris · Answer

As an example for Stewart's #2, this is typically how I will batch from a!queryEntity() when I need to return a data set larger than a single query can handle: 
 a!localVariables(
 local!batchSize: 3000,
 local!totalCount: 
 a!queryEntity(
 entity: cons!YOUR_DS_HERE,
 fetchtotalcount: true,
 query: a!query(
 pagingInfo: a!pagingInfo(1,1)
 )
 ).totalCount,
 local!batchCount: ceiling(local!totalCount/local!batchSize),

 a!flatten(
 a!forEach(
 items: 1+enumerate(local!batchCount),
 expression: a!localVariables(
 local!startIndex: fv!item+((fv!index-1)*local!batchSize)-(fv!index-1),
 a!queryEntity(
 entity: cons!YOUR_DS_HERE,
 fetchtotalcount: false,
 query: a!query(
 pagingInfo: a!pagingInfo(local!startIndex,local!batchSize)
 )
 ).data
 )
 )
 )
) 
 With average amounts of columns I can typically batch 5000 rows at a time, however if your column count grows you may have to lessen this value since the maximum is related to data size , not row count. 
 This example is designed for a database view which contains the necessary filters built in, however you can add your filters to the a!query() calls on lines 7 and 21 (add the same filter sets for each).

Error in Query Entity due to large data in Database

Top Replies