187488 - no subject - Hi!\n I have a requirement wherein the

Hi!
I have a requirement wherein the client wants the raw data from database to be downloadable (CSV or EXCEL format) by the user. The problem is that the table is quite large (~80 columns. Client requirement as well). We've tried using the plugins available (cloud-certified) modules but we get stuck at around 4k rows. As I understood it, there is a limit on the size of the queried object as well as the processing time when using these smart services.

Short of doing a manual export (using MySQL Admin Console), is there a better way of doing/fulfilling this requirement?

OriginalPostID-187488

  Discussion posts and replies are publicly visible

  • Additional Info: We used this Download DataSubset (Cloud) Component to make the raw data downloadable.
  • Hi John, generally it depends on the size of data you are trying to query from database. there are certain limits that are configured. please refer to forum.appian.com/.../Post-Install_Configurations.html
    altering these responses may increase the number of rows but not sure that it will return all the data if the threshold limit exceeds
  • Though there is a possibility to increase the limits I don't think that's suggested. An alternative way to overcome this limitation is to query in terms of batches. Even if we query the data in terms of batches, we can't guarantee that the process will complete within the time limits. And at the same time if the dataset is too large, the plugin won't handle it. I believe that altering the limits won't make much difference at times if the processing time increases or the threshold is reached in terms of data handled by Download Datasubset plugin.

    An alternative way you could think of is,

    Design a process which accepts the filters or datasubset from the report as input, and thereby populate the excel sheet with resultant data and finally sends the initiator(user who triggered the process) an email which contains the excel file as an attachment. Add a link or button on the interface (say 'Export to excel and send to user') which triggers the process by making use of the plugin 'Start Process Writer' at forum.appian.com/.../summary. This might not be elegant but will help you overcome the limitations of the Download Datasubset.

    And added to the above, instead of opting for sending an email, you can also design in such a way that the interface will surface a link to the report(similar to Download Datasubset plugin), but this slightly adds overhead in terms of implementation as this needs a database setup as well. I believe there are few examples of this on forum.
  • Thanks for the tip guys! I found this plug-in on shared components (Export SQL and Report Data to EXCEL - V2 ) and it works for 10k rows. Only downside I see is that this module will write the document to document directory and does not initiate an automatic download. We're trying to work around that now and exploring the module further to test it's limits.

    Has anyone used this module(forum.appian.com/.../summary) comments on using this?