Bulk Legacy Document Migration into Appian

Bulk Upload

When you need to migrate a large number of documents into Appian, first consider whether you need to maintain metadata or associations with other data from the old system. Appian supports bulk uploads via the Appian Designer interface. The out of the box approach will be a significantly faster solution than designing and testing a migration process. This is a valid approach if the following conditions are met:

  • The number of documents in the migration is small enough that the manual effort is much faster than an automated effort.
  • There is no metadata associated with the documents (e.g.: you don't need to associate documents with records in a database).

Automating Document Migration

If your migration can't be accomplished using bulk upload, use an Appian process (or processes) to run the migration and track progress using a database table.

  • Review the available approaches for moving documents into Appian and the general guidance for transferring/processing large data sets.
  • Use a temporary database table to hold document metadata and to track the status of each migrated document. Update the status of the documents in the temporary table as documents are migrated.
  • Implement a configurable delay between document transfers and a configurable limit on the number of documents migrated in each batch to throttle the migration volume.
  • Performance test your migration solution on a lower level environment whose hardware and data are equivalent to production to determine the correct throttling numbers.
  • Use the start process smart service to spread the load across process execution engines.
  • If necessary, update the security on the document folders as the documents are migrated.
Practice Reason
Archiving Policy
  • Delete process instances immediately after completion.
  • Checkpoint and restart the engines after the migration to restore the engines to the proper size.
Reduce Manual Intervention
  • Incorporate sending status emails to provide updates on the migration.
Incorporate Validation Methods in Status Fields
  • Test sync failures as a part of your test plan to confirm that the document migration process is able to restart from where it left off in the event of a failure.
Process Optimization