Query Rules vs a!queryEntity

Question

We are a cloud customer and have recently been experiencing data connection time out errors. When this occurs, we have to manually restart the node where the disruption occurred. As a result it often results in duplicate tasks. So far that hasn't been a problem. Once the user completes one of the tasks, the other disappears. Our concern is if this happens during a critical point in our process. To address this issue, the cloud support team recommended increasing the timeout setting from 10 sec to 30 sec. I have been doing some research and noticed that in a couple of these cases, (this has happened 4 times now in the current month), the disruption occurred when evaluating a query rule (the old type that has since been deprecated). Our applications contain nearly 200 of the old query rules. From one of the discussions I found on this topic, ( https://community.appian.com/discussions/f/rules/10265/query-rule-vs-a-queryentity ) it appears as though the old query rule may be bringing back all the data in the table to Appian before filtering for the smaller result set. If this is the case, it's possible that this may be causing some of our time out issues. We have been making a conscious effort to update these old query rules whenever we have to update a process that uses them, but with nearly 200 of the currently in use, it's not practical to update all at this time. We are looking at them now, however, to see which ones may be the most critical to update - and start on those. 
 Just wanted to share this with the community, and see if anyone else has had a similar experience.

Mike Schmitt · Answer

In my old longtime project, we began implementaton after Temo was mainstream (thank goodness) but before either SAIL or queryEntity were commonly used (or even present). To wit, we ended up with over 500 legacy query rules in the system. Particularly difficult was that since query rules are not at all flexible, we ended up with dozens upon dozens of nearly identical query rules that differed just by one parameter or something else. 
 For yours that are timing out routinely - are they querying from views? The only time I've seen consistent in-process query timeout issues was queries which are pulling data back from complex views which look at ever-growing data. The solution to this is multi-pronged, including reassessing the view and its efficiency (and addressing whether a data falloff can be added, i.e. principal items in the view older than X years are no longer included, etc). Additionally, it could help to cherrypick which Query Rules you convert to Query Entities, and finetune those to query efficiently when possible (pulling back only selected columns, a constrained page size, etc). 
 A suggestion for your current approach: the transition from Query Rules to queryEntities can be eased by having the QRs abstracted behind an expression rule (where the expression rule has the same inputs as the QR, calls it, and returns the same output). The thinking here is, one of the issues we had on that project was legacy long-life process instances wherein certain breaking query rules were 'hardcoded' essentially. In a few places the original rules were wrapped in an expression rule anyway, and in those cases the internal logic of the expression rule could simply be swapped out with the new QE rule call. For other places, we had to phase out the QR calls while making sure to leave the legacy rules sitting around until legacy instances could complete. 
 So my current suggestion (if you feel it's needed) might be: for QRs where you don't yet have time to develop and test equivalent QE rules, create expression rule wrappers and use those to replace the QRs in all other locations of your processes/forms/etc. With this you could potentially reduce (or eliminate) the legacy instances relying directly on the legacy rules, and then you can take your time with replacing the internal logic of these with new QE logic.

Mike Schmitt · Answer

The reasoning behind expression rule wrappers around all legacy query rules is, basically, you could reduce your dependence on the legacy objects to nothing other than the expression rules calling them (instead of hardcoded into process models / process instances / forms / etc), in a manner that would basically allow you to quickly do 1:1 replacements of the QR call with its new Expression Rule wrapper, save and deploy. Then when you're ready to replace a particular QR or set of related QRs, you don't need to hunt down all old places that used it, or worry (as much) about whether you'll be breaking a running process instance. 
 For the "data falloff" I mentioned - if you have a view that scales up in size with the growth of production data, filtering (regardless of which query you use) won't help; even pulling back a single row directly from the database can grow super slow when a view is allowed to grow unbounded. The only thing that noticeably helps with this (in my experience, and as an admitted SQL novice) is filtering within the definition of the view itself - i.e. filtering out all deactivated entries, entries older than X rolling date, etc. This can make the difference between a view that's pulling back a million entries, versus a view that's pulling back 10,000 *relevant* entries.

judym598 · Answer

Got it now - that makes sense as far as the new Expression Rule wrapper. As to the data falloff - I've done that with some of our view definitions, but will definitely take another look. Thanks so much for your help!