Retrieving huge amount of data

RE: Retrieving huge amount of data

Mike Schmitt — Wed, 13 Jan 2021 14:33:29 GMT

Check whether any of the dateTime values in the DB are coming back as "0000-00-00 00:00:00" or similar to this. I've seen tthis happen before and it breaks the Appian query. On the other hand, NULL values in the dateTime column should work fine from within Appian. This is my best guess without being provided more detail as to what error you're getting or what the appian logs say.

RE: Retrieving huge amount of data

akshayg0006 — Wed, 13 Jan 2021 00:42:02 GMT

Mike SchmittCan you suggest what exact change i need to do in DB so that i don't face any issue while retrieving data

RE: Retrieving huge amount of data

akshayg0006 — Wed, 13 Jan 2021 00:39:26 GMT

I am building a dashboard for which I have around 8 date and time field, Some time am able to retrieve complete data without any error but sometime it throws an error. In date and time field some dates have NULL value. Can you suggest some solution.

RE: Retrieving huge amount of data

Mike Schmitt — Sun, 10 Jan 2021 14:38:16 GMT

That filter logic wouldn't work. You're close, though. Try this version instead:

a!queryFilter(
  field: "recordedon",
  operator: ">",
  value: today() - 10
)

RE: Retrieving huge amount of data

Stewart Burchell — Thu, 07 Jan 2021 09:23:03 GMT

You could also add a 'selection' to your query to only retrieve data from the columns you need, rather than retrieve everything (which is the default). This is a good principle - only retrieve the data you need in the context you need it. You can also constrain the number of rows retrieved, which it looks like you are, by setting the batchSize to a value other than -1 (which is 'everything').

RE: Retrieving huge amount of data

akshayg0006 — Wed, 06 Jan 2021 19:36:22 GMT

Anyway thanks for help made changes from database side to retrieve data for 10 days.

RE: Retrieving huge amount of data

akshayg0006 — Wed, 06 Jan 2021 19:09:59 GMT

Thanks but still using date filter am getting

Error as Expression evaluation error at function a!queryEntity: An error occurred while retrieving the data.

But using filters on other fields am able to retrieve data

a!queryFilter(
field: "recordedon",
operator: "in",
value: index(ri!recordedon > = {today()- 10},"recordedon",{})
),

Is there any other logic that can be used to get data for last 10 days. If so please provide the code

RE: Retrieving huge amount of data

Stewart Burchell — Wed, 06 Jan 2021 17:18:29 GMT

Your Query Entity can include a queryFilter() where you can compare a date that is in your data with today()-10, for example.

RE: Retrieving huge amount of data

akshayg0006 — Wed, 06 Jan 2021 16:49:53 GMT

Can you help me with like how to retrieve data from db for last 10 days

RE: Retrieving huge amount of data

akshayg0006 — Wed, 06 Jan 2021 16:49:14 GMT

Thanks seems so many date field are used so that issue..

RE: Retrieving huge amount of data

Stewart Burchell — Wed, 06 Jan 2021 14:32:35 GMT

You need to examine the Appian logs to find out the exact root cause of the error. Look in the current stdOut log. Given that you've already said the dataset is huge it may be volume-related, but check the logs to be sure, there are other reasons why this error occurs.