Writing 20K plus record from database to excel

Question

Hi Everyone, 
 
 I have a requirement where in need to fetch more than 20K records from the database and then store it in Excel file, 
 As of now we are querying all the data once and writing it into a single excel file, which is taking more than 10-15 mins. 
 
 Can anyone suggest how we can break this into multiple excel file using the process model i,e if there are 100 records in the databse then we should be able to get 10 excel file with 10-10 records in each file.

aloks0189 · Answer

Hi @gauravs0002 I agree with the points highlighted by Mike Cichy , 20K records are not that huge, and hence Appian should be able to export it in less amount of time. 
 I have couple of quick questions, to understand why it's taking that long for you to export the Data into excel. 
 
 Is that a table or a View, if it's a view, then how long it's taking to load the View? 
 How many columns are being exported as part of your export to excel? As you know, we should not only consider the number of rows but also the number of columns and the max amount of data any field may hold 
 Which Smart Service are you using to export the rows into excel? I would recommend, using Export Data Store Entity to Excel smart service, as Appian also recommends to use this OOTB Smart Service instead of using Excel Tools, in order to achieve better performance 
 
 Also it would be worth having a look into your Server configuration, because i have seen some cases where the server configuration was low and hence while performing any major job, server used to perform slower than expected. 
 Also, i would recommend to monitor the SQL Active Sessions, while performing this operation, just to make sure that, some other jobs / sessions are not active (as part of some other process) when you have started the export of data into Excel. 
 Hope this will help you in debugging and resolving the issue.

aloks0189 · Answer

Hi Gaurav, 
 You need to work with your server administrator to find the server configuration details. 
 Also related to monitoring the Active SQL sessions / active queries, it depends upon which database you are working with, like: 
 MySQL : It would be worth executing the following query: SHOW FULL PROCESSLIST; 
 /* This will show all the queries running for 5 seconds or more in MySQL */
SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST WHERE COMMAND != 'Sleep' AND TIME >= 5; 
 Oracle : You do have the option to monitor the session if you are using SQL Developer available in your machine by selecting the following options: 
 Tools > Monitor Sessions > In the Select Connection dialog box, select a connection to SYSTEM (or another account with full DBA privileges) > And also choose the Status as Active 
 This will show the list of SQL active session currently active / running. 
 Hope this will help.