New Participant

Question

Import huge(~35 MB) CSV file into Database

6 replies
54 views

I have a file that have around 80000+ records and size is around 35MB.

I would like to understand the way to read the data in it but need paging to be part of the process as i want this to be inserted into DB.

csv to db

Employee

Try the "Import CSV to Database" smart service in the excel tools plugin

community.appian.com/.../excel-tools

It has the reputation to work well even for bigger files

jyotik5834Author

New Participant

Hi Soma,

Tried it, but it has the file size limitation.

Employee

Oh! Then can you also give a try using the "Load CSV to Database" smart service which is part of the same plugin

What is the maximum file size that smart service is accepting?

You can also try using readexcelsheetpaging(), if it is possible to convert the csv to excel

Participating Frequently

Hi [mention:57fc948ec4c44d7fbda6e5509555d197:e9ed411860ed4f2ba0265705b8793d05]

while using Load CSV to Database we have to check this

Employee

I tried uploading a CSV file with 640000 rows of data with the file size of 62 MB. I did not face any challenge using Import CSV to Database smart service.

Can you please clarify what is the difference we are having, so that we can try to solve your issue?

karumurua531442

Participating Frequently

Hi[mention:f2e446085ae74ff1abaff016e3349b60:e9ed411860ed4f2ba0265705b8793d05] I would not recommend pushing larger data files into the database in one go. You can split those into smaller chunks and push them to the database, which would also help to run queries faster with minimal load.