187488 - no subject - Hi!\n I have a requirement wherein the

Hi!
I have a requirement wherein the client wants the raw data from database to be downloadable (CSV or EXCEL format) by the user. The problem is that the table is quite large (~80 columns. Client requirement as well). We've tried using the plugins available (cloud-certified) modules but we get stuck at around 4k rows. As I understood it, there is a limit on the size of the queried object as well as the processing time when using these smart services.

Short of doing a manual export (using MySQL Admin Console), is there a better way of doing/fulfilling this requirement?

OriginalPostID-187488

  Discussion posts and replies are publicly visible

Parents
  • Though there is a possibility to increase the limits I don't think that's suggested. An alternative way to overcome this limitation is to query in terms of batches. Even if we query the data in terms of batches, we can't guarantee that the process will complete within the time limits. And at the same time if the dataset is too large, the plugin won't handle it. I believe that altering the limits won't make much difference at times if the processing time increases or the threshold is reached in terms of data handled by Download Datasubset plugin.

    An alternative way you could think of is,

    Design a process which accepts the filters or datasubset from the report as input, and thereby populate the excel sheet with resultant data and finally sends the initiator(user who triggered the process) an email which contains the excel file as an attachment. Add a link or button on the interface (say 'Export to excel and send to user') which triggers the process by making use of the plugin 'Start Process Writer' at forum.appian.com/.../summary. This might not be elegant but will help you overcome the limitations of the Download Datasubset.

    And added to the above, instead of opting for sending an email, you can also design in such a way that the interface will surface a link to the report(similar to Download Datasubset plugin), but this slightly adds overhead in terms of implementation as this needs a database setup as well. I believe there are few examples of this on forum.
Reply
  • Though there is a possibility to increase the limits I don't think that's suggested. An alternative way to overcome this limitation is to query in terms of batches. Even if we query the data in terms of batches, we can't guarantee that the process will complete within the time limits. And at the same time if the dataset is too large, the plugin won't handle it. I believe that altering the limits won't make much difference at times if the processing time increases or the threshold is reached in terms of data handled by Download Datasubset plugin.

    An alternative way you could think of is,

    Design a process which accepts the filters or datasubset from the report as input, and thereby populate the excel sheet with resultant data and finally sends the initiator(user who triggered the process) an email which contains the excel file as an attachment. Add a link or button on the interface (say 'Export to excel and send to user') which triggers the process by making use of the plugin 'Start Process Writer' at forum.appian.com/.../summary. This might not be elegant but will help you overcome the limitations of the Download Datasubset.

    And added to the above, instead of opting for sending an email, you can also design in such a way that the interface will surface a link to the report(similar to Download Datasubset plugin), but this slightly adds overhead in terms of implementation as this needs a database setup as well. I believe there are few examples of this on forum.
Children
No Data