Bulk Upload of Data into Appian DB(Dealing with Lakhs of Records)

Is there any easy way where I can insert the data without affecting the Performance of the Server, and also I know that this would be done in batches and also in Off hours, Just wanted to check if there is any smart services or easy way to do this.

Hi,

You can also look for 'Import CSV to Database' Smart Service if you are fine to have a CSV file but this smart service has its own limitations as well so ultimately depends on your use case. It is pretty fast.

Thanks.
Hitesh

HI,

loading records more than 5000 is always a challenge which will effect the performance . Below points could be useful.

1. Please consider Import CSV to database as it does not required any data in CDT.
2.Do not do any data manipulation in Appian while loading the data from CSV file.
3. Do not consider data insert more than 5000 record , else it will show up in health check report as high impact.
5.Consider data upload in batch if you have more data like millions of record .
6.Consider loading high data using export feature of DB.
7.Consider data loading during off hours .
8.Use any external feature to load a data.
9.If you have any RPA tool use that to load data .
10.Consider the upload of such a large set of data monthly /weekly.

Please let us know what approach you have taken , as I also have faced the same scenario (40 K rows)and it always come as high risk in health check ,So we use RPA tool BluePrism in where we send mail with csv attachment to the blueprism and it will download the file and load the data in the Db.

Thanks

Manish

HI,

loading records more than 5000 is always a challenge which will effect the performance . Below points could be useful.

1. Please consider Import CSV to database as it does not required any data in CDT.
2.Do not do any data manipulation in Appian while loading the data from CSV file.
3. Do not consider data insert more than 5000 record , else it will show up in health check report as high impact.
5.Consider data upload in batch if you have more data like millions of record .
6.Consider loading high data using export feature of DB.
7.Consider data loading during off hours .
8.Use any external feature to load a data.
9.If you have any RPA tool use that to load data .
10.Consider the upload of such a large set of data monthly /weekly.

Please let us know what approach you have taken , as I also have faced the same scenario (40 K rows)and it always come as high risk in health check ,So we use RPA tool BluePrism in where we send mail with csv attachment to the blueprism and it will download the file and load the data in the Db.

Thanks

Manish