Quiring millions of data

Certified Associate Developer

Millions of data in query entity is possible to query for fetching data

  Discussion posts and replies are publicly visible

Parents
  • If you play this scenario through you'll see why this is not a good idea:

    • if you're fetching a million rows to show to a user that user couldn't deal with looking at and working on a million rows of data. So you'd probably want to give them a mechanism to either filter that data down to a more meaningful subset and/or fetch and present a small subset of the data into the user interface. Either way, you're now working with a small subset of the data, not the full 1 million rows
    • if you want to process the 1 million rows automatically (i.e. without any user being involved) you'd quickly realize that your environment has a limited amount of resource (memory, processing power) and you'd either process the data in small batches or if it truly needs to be dealt with in a single atomic transaction, offload it to a more appropriate solution
Reply
  • If you play this scenario through you'll see why this is not a good idea:

    • if you're fetching a million rows to show to a user that user couldn't deal with looking at and working on a million rows of data. So you'd probably want to give them a mechanism to either filter that data down to a more meaningful subset and/or fetch and present a small subset of the data into the user interface. Either way, you're now working with a small subset of the data, not the full 1 million rows
    • if you want to process the 1 million rows automatically (i.e. without any user being involved) you'd quickly realize that your environment has a limited amount of resource (memory, processing power) and you'd either process the data in small batches or if it truly needs to be dealt with in a single atomic transaction, offload it to a more appropriate solution
Children
No Data