Performance of queryEntity and custom picker for large dataset

A Score Level 1

Hello,

I'm trying to configure a custom picker on a large dataset,  ~7100 values.

The first problem I'm encountering is that the queryEntity() by itself takes ~8.5 seconds to complete.
May this be due to the fact that the CDT of the DSE has nested CDTs? I'm confused, because the query completion in MySQL is almost instantaneous. What's taking so long?

The second problem is that, even by loading the entire dataset in a local variable, the custom picker I configured is extremely slow during search operations, to the point that it triggers the browser's checks for unresponsive Javascript.

I'm using the following rule as filter, passing my labels and values:

=with(
  local!matches: where(
    a!forEach(
      items: ri!labels, 
      expression: search( ri!filter, fv!item)
    )
  ),
  a!dataSubset(
    data: index( ri!labels, local!matches), 
    identifiers: index( ri!identifiers, local!matches)
  )
)


Question 1) If I refactor my CDT not to contain nested types, what can I reasonably expect as a loading time?
Question 2) Is a custom picker expected to work on 7k values, and if so, how can I improve its performance?

I'm trying to understand if my efforts will ever make the picker usable, or if maybe these numbers are too high and I should change approach for having my users select values.


I can provide additional information if needed.

Thanks in advance.

 Discussion posts and replies are publicly visible