Number of documents effect the performance in system?

In live system, we have around 3.5 million documents stored in knowledge centres.

Will this effect the performance in System?

If we those documents to any FTP server, will performance of system increases ?

  Discussion posts and replies are publicly visible

  • Hi Rahul, I'm not good with Server Administration. But from Our experience I can suggest to remove the unused or unreferenced documents from knowledge center. The increase in Documents occupies more space. Some times to due to space issue. Search server responds very slow and cannot able to upload documents through file upload field.
  • As long as system resources are scaled appropriately and good design patterns are followed, I don't consider it problematic to have that many documents. Appian is designed to support that volume of usage, and I'm aware of several customers who have a similar (or greater) number of documents. (One scalability pattern that may be particularly worth considering is having an HA configuration, since this will allow for improved concurrency and throughput, with multiple nodes available to service read calls.)

    As you can imagine, the greatest strain on the system would be anywhere that you're asking the system to consider all 3.5 million documents at once. For example, if you had them all in a single KC and then opened that KC; accessing the All Objects view in Appian Designer without any type filters applied to filter out documents (since documents are, of course, Appian objects); or having a document picker that had the ability to suggest any of the 3.5 million documents. Of course, we continue to work to make these cases even faster too.

    In my experience, it's uncommon for there to be a use case where all 3.5 million documents are relevant to a single user. In practice, individual users may need to interact with a smaller subset of specific documents, and the fact that there happen to be that many total documents on the system doesn't affect the users.

    In terms of storing them externally (such as on an FTP server as you describe), that could be still be a good idea, but it depends on the specifics of your use case. It may make certain design patterns more complicated to implement, and it could make certain user interactions slower. Generally, it's going to be faster for the user to communicate directly with the server where documents are stored than it is to communicate with a server that's acting as an intermediary for the external server where the documents are stored. Though, by the same token, it may make it easier to design for the rest of your use cases if you've isolated your document-based use cases to a single part of your architecture.
  • The number of documents increase day by day, will it impact search server. It is nightmare if i want to search for any object in Live. I have seen in the documentation "The search server's use of disk and memory resources will scale with the amount of design objects, user activity, and rule executions".
    docs.appian.com/.../Search_Server.html
  • 0
    Certified Lead Developer
    in reply to Rahul Gundu
    Hi as per my understanding, Appian can handle this huge volume of documents but also not to forget, appian is not a content management tool and hence if you have such type of requirement where you want to store huge number of documents into Appian KC then I would recommend you to go for content management systems/tools such as Alfresco, Microsoft Azure etc.. in order to avoid Appian performance slowness
  • Actually, continued growth in the number of documents, specifically, would not significantly impact the search server, as it functions today. The search server is not currently involved in most document-related functionality.

    If the use case you're having trouble with is searching for objects, and you're experiencing slowness searching for objects in the Objects tab of Appian Designer, I would suggest searching for the objects using Quick Search docs.appian.com/.../Appian_Release_Notes.html .
  • If you don't need to use the Appian File System yo can upload your documents in an asynchronous process to evade the increase in residual space/storage.

    In my opinion, the best way to manage this kind of problems is the use of an external File System Manager via WebService or an FTP.