Skip to main content
jasmithak0001
Known Participant
Jan 24, 2025 10:18 PM UTC
Question

Load data From S3 to Appian Tables

What is the most efficient way to import a daily CSV with 1 million rows from s3 into an Appian? (On daily basis we have to truncate the table and load the new data).

    8 replies

    mathieud0001
    mathieud0001
    Genius
    Jan 25, 2025 8:43 AM UTC

    Create a process that runs daily that:

    1. Downloads the file from S3 (via Amazon S3 Utilities plugin -  [mention:4d5b4266bad04afead7ca557089974a4:f7d226abd59f475c9d224a79e3f0ec07] )

    2. Imports the CSV into Database (via Excel tools plugin -  [mention:54d891d9b6af45b0b155ce764b815b3f:f7d226abd59f475c9d224a79e3f0ec07] )

    jasmithak0001Author
    Known Participant
    Jan 25, 2025 2:01 PM UTC

    Yeah, That's the option but am thinking about the performance. so looking for other options. 

    mathieud0001
    mathieud0001
    Genius
    Jan 25, 2025 2:05 PM UTC

    I think it'll be fine personally. Otherwise your other option is to have a service backed record with the method mentioned below.

    stefanhelzle0001
    stefanhelzle0001
    Brainy
    Jan 25, 2025 9:00 AM UTC

    Would it be an option to store that data in a database at Amazon and connect that to Appian?

    mathieud0001
    mathieud0001
    Genius
    Jan 25, 2025 9:03 AM UTC

    You don't even need to load it in a DB if you can get this setup

    aws.amazon.com/.../

    Terms & ConditionsAccessibility statement