Skip to main content
J
jasmithak0001
Known Participant
January 24, 2025
Question

Load data From S3 to Appian Tables

  • January 24, 2025
  • 8 replies
  • 0 views

What is the most efficient way to import a daily CSV with 1 million rows from s3 into an Appian? (On daily basis we have to truncate the table and load the new data).

    8 replies

    mathieud0001
    mathieud0001
    January 25, 2025

    Create a process that runs daily that:

    1. Downloads the file from S3 (via Amazon S3 Utilities plugin -  [mention:4d5b4266bad04afead7ca557089974a4:f7d226abd59f475c9d224a79e3f0ec07] )

    2. Imports the CSV into Database (via Excel tools plugin -  [mention:54d891d9b6af45b0b155ce764b815b3f:f7d226abd59f475c9d224a79e3f0ec07] )

    J
    jasmithak0001Author
    Known Participant
    January 25, 2025

    Yeah, That's the option but am thinking about the performance. so looking for other options. 

    mathieud0001
    mathieud0001
    January 25, 2025

    I think it'll be fine personally. Otherwise your other option is to have a service backed record with the method mentioned below.

    stefanhelzle0001
    stefanhelzle0001
    January 25, 2025

    Would it be an option to store that data in a database at Amazon and connect that to Appian?

    mathieud0001
    mathieud0001
    January 25, 2025

    You don't even need to load it in a DB if you can get this setup

    aws.amazon.com/.../

    Terms & ConditionsAccessibility statement