Application with large datasets

Certified Lead Developer

Hi All,

We have a new project coming up, it is expected to have almost around 600,000 records being created in a year. We have one existing application with let's say around 100,000 records using entity backed synced record. My concern here is that existing application performs very slow with 100,000 records and most of the times export to excel fails also with 20,000 records to be exported now if we talk about 600,000 cases I am not 100% sure to use Appian OOB records. 

I know we can create different records based on the personas and default source filters to make it work but what if customer wants to see all the records?

What could be the alternative approaches to deal with this?

Thanks!

  Discussion posts and replies are publicly visible

Parents
  • 0
    Certified Lead Developer

    My advice is to probe a define the unique business needs. Build for purpose instead of building one solution that is everything to everyone. Appian makes it too easy to have role-based views, etc.

    That said Appian is an "action-based" platform. From participating in workflows to creating new workflows. With that thought, why would anyone need to see hundreds of thousands of records with that context. Generally they should be seeing things that have the right context and/or are new so they can take action and make informed decisions.
    Like you said there are filters, paging, row-level security, etc. that can all limit how much data is shown. But if the goal is to just pull data into Appian to show all rows and allow export so that convos and work can be done outside Appian, is Appian where that should live since the data is living outside the platform, then pulled into Appian, just to be pulled out again?

    Just some food for thought...

Reply
  • 0
    Certified Lead Developer

    My advice is to probe a define the unique business needs. Build for purpose instead of building one solution that is everything to everyone. Appian makes it too easy to have role-based views, etc.

    That said Appian is an "action-based" platform. From participating in workflows to creating new workflows. With that thought, why would anyone need to see hundreds of thousands of records with that context. Generally they should be seeing things that have the right context and/or are new so they can take action and make informed decisions.
    Like you said there are filters, paging, row-level security, etc. that can all limit how much data is shown. But if the goal is to just pull data into Appian to show all rows and allow export so that convos and work can be done outside Appian, is Appian where that should live since the data is living outside the platform, then pulled into Appian, just to be pulled out again?

    Just some food for thought...

Children