Hi All,
Looking for the best way to move a large amount of documents from an external system into Appian. We are replacing an existing mainframe system, and with that we are doing a doc and data migration. These documents are broken out into a logical structure, and I tried to use the upload zip piece. But Appian will only allow for folders under 1GB. I tried to break out the folders into smaller folders, but even at 600mb a folder, the system was still getting choked up. They are in the cloud. Any ideas/approaches to executing a large document migration?
Discussion posts and replies are publicly visible
Utilizing an integration and migrating the documents with Appian processes will be more efficient than manually loading documents. For example, you can build a process in Appian that connects to a SFTP server with the documents. The process can parse documents/folders on the SFTP server and then store them in the appropriate KB/Folder structure on the Appian side based on your business requirements.
Automating the process will require you to monitor the process and potentially batch process so you don't overload system resources. I would suggest setting up some test cases in lower level environments to verify your migration process functions and performs well enough to run in production. If you have active users in your system you may also want to pause migration processes during core business hours and then let them run off hours until you have worked through all the migration documents.
There is also a playbook article on the topic that may give you some additional information on what to look out for when setting up an automated process.
https://community.appian.com/w/the-appian-playbook/105/bulk-legacy-document-migration-into-appian