fetching data issues

I have a table with 100,000 records. In the development environment, I used queryEntity to retrieve the data without issues. However, in the production environment, the dataset is much larger, and performance has become a concern. I tried optimizing the query with selections and filters, but the large volume still causes problems. I considered using a stored procedure to handle the data retrieval with specific conditions. The downside is that any future changes to the database schema would require updating the stored procedure, leading to additional maintenance work. To address this, I implemented a paging strategy in Appian. I retrieve the first 100 records using pagingInfo and then loop through subsequent pages to fetch the rest of the data incrementally. This approach helps manage large datasets more efficiently within Appian. is there any approaches other than these , can you guys help me with this?