Bulk Upload of Data into Appian DB(Dealing with Lakhs of Records)

Hi harshav 2 use cases here:
1. When You want to deploy these scripts manually into DB: I would like to go for generating SQL scripts online via existing service providers as mention by @brettf because this will be much quicker than doing it via Appian or other tools. But it's having a limitation I.e.: a user who is aware of these will only be able to dump the data into db (if it's a continuous deployment, such as weekly, monthly etc..) which means a business user won't be able to do this unless we (developer) train them.

2. When you want to deploy these scripts via interface: in this case you have various options available, where you browse the file and upload the rows into DB in following ways:
-- Using OOTB Smart Service
-- Using custom smart service such as excel tools.
-- using robotics tool such as Blue Prism

Limitation: if we use, smart services, we must not perform this operation specially during business hours and archive the process immediately once after successfully completion. I recommend, try processing it during weekends to avoid the impact on server.

If we use Blue Prism, upload will be done asynchronously, hence we cannot have assurance whether these data will be effected/inserted immediately (before next working days, so that business can use these data). Also you or your organisation need to baer the licensing cost of these robotics tool

Hope you will be able to choose the best approach for your use case.

Hi harshav 2 use cases here:
1. When You want to deploy these scripts manually into DB: I would like to go for generating SQL scripts online via existing service providers as mention by @brettf because this will be much quicker than doing it via Appian or other tools. But it's having a limitation I.e.: a user who is aware of these will only be able to dump the data into db (if it's a continuous deployment, such as weekly, monthly etc..) which means a business user won't be able to do this unless we (developer) train them.

2. When you want to deploy these scripts via interface: in this case you have various options available, where you browse the file and upload the rows into DB in following ways:
-- Using OOTB Smart Service
-- Using custom smart service such as excel tools.
-- using robotics tool such as Blue Prism

Limitation: if we use, smart services, we must not perform this operation specially during business hours and archive the process immediately once after successfully completion. I recommend, try processing it during weekends to avoid the impact on server.

If we use Blue Prism, upload will be done asynchronously, hence we cannot have assurance whether these data will be effected/inserted immediately (before next working days, so that business can use these data). Also you or your organisation need to baer the licensing cost of these robotics tool

Hope you will be able to choose the best approach for your use case.

HI Alok ,

Just to correct a bit ,If you are using Blueprism , you can immediately show the response to business (with in few min ) through a webservice in Appian or via a mail .

Ragards

Manish

Hi Manish thanks for your response, just a bit focus on this point i.e. asynchronous call, as per the documentation 

Section: rule!BP_startProcess()

They have clearly mentioned that, synchronous call can easily be timeout because Blue Prism is expected to perform heavy operation. Hence its always a best practice to invoke Blue Prism flow asynchronously.

When it comes to , availability or querying about the latest data, that can only happen when the data is available in Appian DB, irrespective of, whether we make a service call. Also let's assume I configured an email, that will also trigger only when the dump is completed, because we will invoke Blue Prism flow asynchronously as per the best practices.

Hi Alok ,

I am agree with your above point ,I just wanted to say that on your point of assurance of data can be done on only next day .
Blue prism can send the processing status to Appian within few min and user can see it in Appian .The SLA will be just few min and user will come to know that the Blueprism processed the data successfully or failed.

Cheers

Manish

Agreed!