Skip to main content
Participating Frequently
Jun 16, 2023 5:32 PM UTC
Question

Quiring millions of data

Millions of data in query entity is possible to query for fetching data

    4 replies

    peter.lewis
    Employee
    Jun 16, 2023 8:47 PM UTC

    Can you give more context to why you are asking? What are you hoping to do with the data?

    Participating Frequently
    Jun 17, 2023 5:11 AM UTC

    Just asking for knowledge purpose

    harshitb6843
    Brainy
    Jun 17, 2023 6:50 AM UTC

    Depends on the data size. Even if you can, you shouldn't as it invites problems. 

    stewart.burchell
    Employee
    Jun 19, 2023 6:25 AM UTC

    If you play this scenario through you'll see why this is not a good idea:

    • if you're fetching a million rows to show to a user that user couldn't deal with looking at and working on a million rows of data. So you'd probably want to give them a mechanism to either filter that data down to a more meaningful subset and/or fetch and present a small subset of the data into the user interface. Either way, you're now working with a small subset of the data, not the full 1 million rows
    • if you want to process the 1 million rows automatically (i.e. without any user being involved) you'd quickly realize that your environment has a limited amount of resource (memory, processing power) and you'd either process the data in small batches or if it truly needs to be dealt with in a single atomic transaction, offload it to a more appropriate solution