RE: Data upload by file

alexc0002 — Mon, 08 Jun 2020 10:14:10 GMT

Thx Lejanson.

I think it's a good idea.

RE: Data upload by file

legotx — Mon, 01 Jun 2020 22:06:46 GMT

Hey Alex,

I will suggest having a process where you read the file and put it in staging table before writing into your main tables. This way, you dont lose any data and your process model won't get the overloaded.

It depends on what you are importing also. If you are importing a CSV file, you can use the CSV to SQL plugin. I am not sure if this is still available.

After that you can have a process that pulls those in the database and write into your main tables - which is time triggered. You can have a status column on your staging table so you can keep track which data has already been processed.

Additionally, I would suggest having a threshold on the number of rows you want to process. I believe you can only do 1000 rows if I am not mistaken.

Hope this helps.

Lejanson