• State Not Answered
  • +1 person also asked this people also asked this
  • Replies 3 replies
  • Subscribers 5 subscribers
  • Views 5252 views
  • Users 0 members are here
Related Discussions
Home » Discussions » Plug-Ins

Excel Tools - Import CSV to Database - performance and limitations

peterh

I have a requirement to allow bulk import from an Excel file into the Appian DB. I'm exploring using the "Import CSV to Database" smart service from the Excel Tools shared component to import the file directly into a staging table with Varchar2(4000) columns matching the CSV file.  The file has ~75 columns and can have up to 100,000 rows.

Does anyone have any experience using this plugin? Any limitations or performance concerns that I should be aware of? The documentation is light on this node.

Thank you in advance

  Discussion posts and replies are publicly visible

  • 0
    Peter,
    This would definitely have some performance impact on the Appian server.
    Can you tell me if this is a one time job or an ongoing process?
    And do you have any data transformation between loading the data from CSV and saving it into DB?

    If it is one time job, you can possibly split the one file into multiple files and do the load.
    If you have Data transformation, you can use any ETL tool for this.

    If it is an ongoing process, probably you would need to find a different solution(Like an ETL job) so that it would not impact Appian Server performance.

    Let us wait and see other practitioner's opinion on this.
  • 0 in reply to sanjays0006
    , I am having a similar requirement and planning to use this smart service. It will be good to understand why this plugin will have server side impact. Can you please explain on that note ??
  • 0 in reply to mohamedt808
    mohamedt808 ,

    Below are my observations on why we should not handle large amount of data in Appian :

    This single process instance consuming a large amount of data for this process alone which might impact other process instances in the system.
    Huge data transactions might cause DB time out where you might have to continuously monitor this process model.Usually DB time out would be 10 seconds. I am not sure whether this transaction would be completed with in 10 seconds every time.

    In the above example, the user mentioned 75 ~ columns , data type : varchar2(4000) with 100,000 rows. This would be a huge data to process in Appian.

    My concern is only with the data size and not the plugin :)
© 2024 Appian. All rights reserved.
We want to hear from you! Submit a review on Gartner or G2.