Skip to main content
June 16, 2023
Question

Quiring millions of data

  • June 16, 2023
  • 4 replies
  • 0 views

Millions of data in query entity is possible to query for fetching data

    4 replies

    peter.lewis
    Employee
    June 16, 2023

    Can you give more context to why you are asking? What are you hoping to do with the data?

    June 17, 2023

    Just asking for knowledge purpose

    harshitb6843
    June 17, 2023

    Depends on the data size. Even if you can, you shouldn't as it invites problems. 

    stewart.burchell
    June 19, 2023

    If you play this scenario through you'll see why this is not a good idea:

    • if you're fetching a million rows to show to a user that user couldn't deal with looking at and working on a million rows of data. So you'd probably want to give them a mechanism to either filter that data down to a more meaningful subset and/or fetch and present a small subset of the data into the user interface. Either way, you're now working with a small subset of the data, not the full 1 million rows
    • if you want to process the 1 million rows automatically (i.e. without any user being involved) you'd quickly realize that your environment has a limited amount of resource (memory, processing power) and you'd either process the data in small batches or if it truly needs to be dealt with in a single atomic transaction, offload it to a more appropriate solution