What is the Best way to write high volume of data into database

RE: What is the Best way to write high volume of data into database

Abhay Dalsaniya — Fri, 02 Feb 2024 17:40:16 GMT

It is definitely not a best practice to maintain large volume of records in draft/editable mode and save it in single transaction.

Mathieu Drouin — Thu, 01 Feb 2024 00:42:14 GMT

Not much you can do to speed up the writes but I am curious to know how many columns these records have.

I can write 1000 records from a JSON in 2-3 seconds.

Harshit Bumb (Appyzie) — Wed, 31 Jan 2024 16:19:19 GMT

There could be multiple ways of looking at it based on what is more important.

If the priority is to take less time, then I will advise changing the approach and letting the user save every record once they are done working on it. This will surely increase the clicks but will ensure that every data is saved instantly so the 2 minutes is reduced to almost negligible.
If the priority is to keep it performant and keep the user efforts minimal, then I will advice writing the data in batches of 100 and calling this process as an async process. You can add a refresh interval of 30 seconds on your grid data to auto-refresh and show the newly written batch of data