Memory threshold error while querying database

Question

Hi, 
 We have a requirement to fetch some records for a particular filter (service name from DB) and show it on the grid to the user. The issue is we have a large number of records in DB, which are more than 20+ lakhs and goes on increasing. 
 While fetching the data from table I am facing the memory threshold error. I am using batch size as (1,-1), because I need to show all the data in the grid. 
 so, My question is how many records Appian can handle & how many records can be fetched at a time. 
 NOTE: i am not using Record Type, directly fetching it from DB 
 Also can we query more than 20 lakhs record from a record type,

Dave Lewis · Answer

For what reason would a user intend to actually sift through 2 million rows? That would probably take up the height of the screen about 50,000 times. Your scroll bar would be a single pixel in width, and moving it a single pixel up or down would catapult your view thousands of rows in either direction, meaning there'd be almost no conceivable way of finding your place again if you ever lost it. 
 You should batch. You should have pages of data so your users can move to the page they want. At 100 records a page, which isn't an unreasonable number, your users could pick which of the 20,000 pages they wanted to view and find a particular row more than once. You query the first 100, then when you click on the next page, you dump the first 100 and query the second 100. 
 Appian recommends that you only use batchSize -1 in very rare circumstances, and won't let you do it at all in some instances, for a reason. To query from a record I think you are limited to a batchSize of 5000, and you should honestly be thankful that you are.

Memory threshold error while querying database

Top Replies