Best way to migrate a large amount of documents (70-80gbs)

Hi All,

Looking for the best way to move a large amount of documents from an external system into Appian. We are replacing an existing mainframe system, and with that we are doing a doc and data migration. These documents are broken out into a logical structure, and I tried to use the upload zip piece. But Appian will only allow for folders under 1GB. I tried to break out the folders into smaller folders, but even at 600mb a folder, the system was still getting choked up. They are in the cloud. Any ideas/approaches to executing a large document migration?

  Discussion posts and replies are publicly visible

Parents
  • Certified Lead Developer

    I'd carefully consider whether you should use Appian for this bulk storage - Appian can do a lot of things well but I don't believe a bulk legacy document storage solution is really on-target for its intended use cases.  Will every one of the 80GB of legacy documents be used actively within the Appian application, for example?  If not, you should probably consider an external bulk file storage solution and perhaps interface this with your Appian system for flexibility as-needed.

Reply
  • Certified Lead Developer

    I'd carefully consider whether you should use Appian for this bulk storage - Appian can do a lot of things well but I don't believe a bulk legacy document storage solution is really on-target for its intended use cases.  Will every one of the 80GB of legacy documents be used actively within the Appian application, for example?  If not, you should probably consider an external bulk file storage solution and perhaps interface this with your Appian system for flexibility as-needed.

Children
No Data