Appian Community and Appian Academy are being upgraded. As a part of the upgrade, Appian Community is currently in read-only mode, and user registration is disabled until August 3. We apologize for any inconvenience this may cause, but a more secure, stable, and performant Community experience is coming soon!

The new Appian Community launches August 3, followed by Appian Academy on August 7. During the migration, Appian Community Edition, Appian Academy, Documentation, Certifications, Instructor-led Customer Training, Partner Sales Training & Accreditation, and Forum (for Appian Partners and Customers only) will remain available.

Large Dataset Query

Certified Associate Developer

I am looking for suggestions on the best and fastest way to query data that has about 50,000 rows of data.  My use case is data collection for inventory purposes where an operator will use a handheld device and scan and item_id barcode, the application retrieve a description and then they can scan a qty.  The reason for the retrieval is to verify a good part number, also to occasionally verify the barcode to the product.  The data currently resides in a SQL database.  We are on Appian 22.3 Cloud.

  Discussion posts and replies are publicly visible

Parents
  • 0
    Certified Lead Developer

    These sound like standard use cases for a!queryEntity() - the only time you'd have issues is if you're trying to retrieve several thousand rows of data all at once, but the use cases you list here don't seem to indicate a need to do that.

    For instance, querying an item from a 50,000 row DB to match a single scanned bar code, I would not expect to be any sort of difficult thing for a normal query filtering by, for example, a barcode number field.

  • Agree with Mike - by filtering under a!queryEntity() you should only need to retrieve 1 or 0 rows to determine if the item is present (and get the details on it).  Aside from [most often] not being necessary, retrieving 50,000 rows in batches of < 1 MB, during each scan, would be a nightmare for performance.

    We have one MSSQL table with 7 million rows which is queried 2,000+ times a day for a subset of 0 to ~10 rows each time, runs like a champ.

Reply
  • Agree with Mike - by filtering under a!queryEntity() you should only need to retrieve 1 or 0 rows to determine if the item is present (and get the details on it).  Aside from [most often] not being necessary, retrieving 50,000 rows in batches of < 1 MB, during each scan, would be a nightmare for performance.

    We have one MSSQL table with 7 million rows which is queried 2,000+ times a day for a subset of 0 to ~10 rows each time, runs like a champ.

Children
No Data