Performance of queryEntity and custom picker for large dataset

A Score Level 1

Hello,

I'm trying to configure a custom picker on a large dataset,  ~7100 values.

The first problem I'm encountering is that the queryEntity() by itself takes ~8.5 seconds to complete.
May this be due to the fact that the CDT of the DSE has nested CDTs? I'm confused, because the query completion in MySQL is almost instantaneous. What's taking so long?

The second problem is that, even by loading the entire dataset in a local variable, the custom picker I configured is extremely slow during search operations, to the point that it triggers the browser's checks for unresponsive Javascript.

I'm using the following rule as filter, passing my labels and values:

=with(
  local!matches: where(
    a!forEach(
      items: ri!labels, 
      expression: search( ri!filter, fv!item)
    )
  ),
  a!dataSubset(
    data: index( ri!labels, local!matches), 
    identifiers: index( ri!identifiers, local!matches)
  )
)


Question 1) If I refactor my CDT not to contain nested types, what can I reasonably expect as a loading time?
Question 2) Is a custom picker expected to work on 7k values, and if so, how can I improve its performance?

I'm trying to understand if my efforts will ever make the picker usable, or if maybe these numbers are too high and I should change approach for having my users select values.


I can provide additional information if needed.

Thanks in advance.

Parents
  • What kind of nested CDT do you have? If you have a nested 1:M, this can have potential performance problems because it queries the parent data and queries for each of the children values separately: see the CDT Guidance in the documentation. Also, do you actually need the nested data in your query or can you just select columns from the parent like Mike suggested?

Reply
  • What kind of nested CDT do you have? If you have a nested 1:M, this can have potential performance problems because it queries the parent data and queries for each of the children values separately: see the CDT Guidance in the documentation. Also, do you actually need the nested data in your query or can you just select columns from the parent like Mike suggested?

Children
No Data

 Discussion posts and replies are publicly visible