Hi,
We have a daily batch process to load in files from a remote FTP server into the Appian server as Documents, and then process each Document.
To manipulate the Document, it appears Appian is loading the "entire" Document into memory when we use the readtextfromfile() function, as opposed to using a stream-based approach where only the streamed portion of the Document needs to take up the memory space. So the question is: What is the max file/Document size allowed?
I did search for the existing QnA about this, but didn't find a definitive answer. Some say it's 1G, and configurable. We are running on Appian cloud for the moment, but will probably move to on-premise installation with Amazon EC2 later. In either case, what is the max document size limit - relative to the host server's memory size, etc...?
Thanks for your help!
Discussion posts and replies are publicly visible
Thanks. Is my understanding correct that, in order to parse the file with Appian, the entire file string (up to 1G) needs to be loaded into memory? Is there a "stream based" option?
any update ?
From my point of view, you can upload documents into Appian, but it's not a document management system....
The more documents you have, the fewer resources Appian will have to work with.
I wouldn't recommend loading a 1GB into memory with readtextfromfile.
I usually try to stay within the 75MB limit imposed by the Integrations.
I agree with both of you , but I need somehow for an appian user to attach a heavy file in an appian interface , I am clear that we should not use appian document manager as official repository , but I don't see a viable way to not go through appian temporarily , before sending the document to the final repository.
Set up a temporary folder within Appian Document Manager to stage these large files.Let users attach files via the standard file upload function.Upon upload, trigger a process to:Retrieve the file from the staging area.Transfer the file to your designated external repository (e.g., Amazon S3).This way, Appian facilitates temporary uploads while keeping large files out of your permanent storage.
This is not possible, we are talking about files larger than 1GB, the appian interface does not allow such large files to be attached.
I have to try for file larger than 1GB.
Will keep you posted.