Appian Community
Site
Search
Sign In/Register
Site
Search
User
DISCUSS
LEARN
SUCCESS
SUPPORT
Documentation
AppMarket
More
Cancel
I'm looking for ...
State
Suggested Answer
+1
person also asked this
people also asked this
Replies
20 replies
Answers
4 answers
Subscribers
9 subscribers
Views
18726 views
Users
0 members are here
Share
More
Cancel
Related Discussions
Home
»
Discussions
»
General
Bulk Upload of Data into Appian DB(Dealing with Lakhs of Records)
harshav
over 7 years ago
Hi All,
I would be getting data from External system in an Excel sheet where I need to insert all the data into Appian DB
OriginalPostID-231862
Discussion posts and replies are publicly visible
Top Replies
brettf
over 7 years ago
+1
To tag onto colton's post above, I would recommend using an external tool to generate the SQL insert statements. The website sqlizer.io can take in an excel file and generate the code, as long as the excel…
0
harshav
over 7 years ago
Is there any easy way where I can insert the data without affecting the Performance of the Server, and also I know that this would be done in batches and also in Off hours, Just wanted to check if there is any smart services or easy way to do this.
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
0
Colton Beck
over 7 years ago
Could you generate SQL INSERT statements using Excel formulas? Inserting these records via a script will always perform better than kicking off Appian processes to ingest the data.
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
0
brettf
over 7 years ago
To tag onto colton's post above, I would recommend using an external tool to generate the SQL insert statements. The website sqlizer.io can take in an excel file and generate the code, as long as the excel file is formatted properly.
Cancel
Vote Up
+1
Vote Down
Sign in to reply
Verify Answer
Cancel
0
Colton Beck
over 7 years ago
Thanks, Brett! I didn't know this existed.
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
0
Chris
over 7 years ago
Good suggestions. Just to add, I do bulk updates nightly for 30,000 rows. Data is loaded into Appian via CSV file on the server, loaded into a CDT, CDT manipulated to retrieve unique keys then ran through a Write to DS node. This is off hours, but works fine and the only thing to watch out for is that you schedule this away from server checkpointing (notorious for creating errors when timing conflicts with large DB activity).
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
0
harshav
over 7 years ago
Thanks everyone I would be posting my approach which I have took soon
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
0
rp_balaji
A Score Level 1
over 6 years ago
in reply to
harshav
Hi harshav ,
Did you finalize any best approach to upload bulk data using Appian, if so can you please share your thoughts.
Regards,
Balaji.R
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
0
Alok Patro
over 6 years ago
Use the Appian DB import feature to import the data from excel to a table if the data structure in excel is in par with the table.
It makes your job quick, a lot easier and would not impact appian performance.
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Reject Answer
Cancel
0
rp_balaji
A Score Level 1
over 6 years ago
in reply to
Alok Patro
Hi Alok,
Thanks for your response, I agree with your point of using the DB import feature which is available but if in case there is a requirement of providing a user interface to the end user and the user may upload excel sheet with data around 10K to 20K rows then do we have any effective way of handling this in Appian or the existing excel smart service can handle this wisely?
Regards,
Balaji.R
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
0
hiteshd
Certified Lead Developer
over 6 years ago
in reply to
harshav
Hi,
You can also look for 'Import CSV to Database' Smart Service if you are fine to have a CSV file but this smart service has its own limitations as well so ultimately depends on your use case. It is pretty fast.
Thanks.
Hitesh
Cancel
Vote Up
0
Vote Down
Sign in to reply
Verify Answer
Cancel
>