Upload bulk data in Appian (Excel or CSV)

Question

We have a requirement where user wants to upload bulk data in Appian. A file upload option in an excel or CSV format is desired. Once uploaded, the file contents should be written to DB and later available for user to view data in a grid format. File can contain a maximum of 200+ columns & 10K rows. What is the best practice to address this requirement in Appian. Thanks in advance.

Stewart Burchell · Accepted Answer

Best practice is to split the upload into two phases: 
 
 upload the data to a "Staging" table - that is, load it without any indication of whether the content is valid or not, and then provide an acknowledgement to the User that the data has been uploaded and will be validated off-line 
 The "Staging" table will mirror the Excel file, and will have a few extra columns (primary key, a status to indicate if valid or not, validation error message, a "correlation" value that allows the system to communicate with the User who uploaded the data, and a column to track the status of the data (so that, at some point, it can be deleted from Staging as you don't want to keep it forever!) 
 Validate the data - build a validation process to determine that the uploaded data is fit for purpose. Understand whether the business case allows for data to be carried forwards either as individual rows or for the whole batch (i.e. if one row out of 10k rows has a validation error, should you accept the 9,999 and reject the one row, or do you have to reject the whole 10k batch?) 
 You'll need a correlation method to communicate to the User which upload you're referring to when providing validation errors, and a communication mechanism (e.g. email, or Task or some such, whatever makes sense for the role the User has) 
 Validated data should then be copied across to a target data model (this may be very similar to the Staging Table, but may also be very different, and this depends entirely on the data and the processing you want to conduct on that data)

Upload bulk data in Appian (Excel or CSV)

Top Replies