MySQL Cloud - Slow Database Queries with Large Datasets - Replacing queryEntity with Stored Procedure

We keep running into a timeout issue when paging queries on a large table. 

For example, we have single table with about 2,000,000 rows. When we query this table from Appian (via a queryEntity rule) it times out when we click the double arrows to view the data at the end of the gridField.

Error message from the logs:

18:33:37,968 INFO  [stdout] (ajp-/0.0.0.0:8009-40) Caused by: com.appiancorp.suiteapi.common.exceptions.AppianRuntimeException: com.appiancorp.suiteapi.common.exceptions.AppianException: Unexpected error executing query (type: [APPIANDATATYPE1234], query: [queryentity expression], order by: [[Sort[lastName asc]]], filters:[(status<> TypedValue[it=3,v=DL])]) (APNX-1-4164-028)
...
18:33:37,969 INFO [stdout] (ajp-/0.0.0.0:8009-40) Caused by: org.hibernate.exception.GenericJDBCException: could not execute query
...
18:33:37,969 INFO [stdout] (ajp-/0.0.0.0:8009-40) Caused by: com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to timeout or client request
...



When we convert this to a SQL query (SELECT * FROM APPIANDATATYPE1234_TABLE ORDER BY LastName ASC LIMIT 1799975, 25) it takes about 40 seconds to complete. The cloud typically gives only 10 seconds before it times the query out, and is not configurable last time I asked.

The issue (and solution) in regards to the SQL Query length is posted on this stack overflow page: https://stackoverflow.com/questions/4481388/why-does-mysql-higher-limit-offset-slow-the-query-down

 Implementing this solution in a SQL query results in only take 0.78 seconds:

SELECT t.* FROM ( SELECT ID FROM APPIANDATATYPE1234_TABLE WHERE Status <> 'DL' ORDER BY LastName LIMIT 1799975, 25) q JOIN APPIANDATATYPE1234_TABLE t ON t.ID = q.ID

We've had success in the past utilizing stored procedures to avoid other timeout limits from MySQL.

I was planning on replacing the a!queryEntity rule with a fn!executestoredprocedure() to call a stored procedure that uses a prepared statement with inputs that would be sent from Appian (i.e. pagingInfo.startIndex for OFFSET, pagingInfo.batchSize for LIMIT, and using SQL_CALC_FOUND_ROWS to pass into an INOUT for the totalCount  and convert the results into a datasubset.

Are there any issues concerning Best Practices in this regard? Has anyone run into a similar issue and what was a solution you've found that works? Are there other solutions here that I'm missing?

I'd like some feedback before I go down the rabbit hole of re-vamping this queryEntity rule into a stored procedure. Also the tables have been indexed so that's not the issue here.

Thanks in advance!