Why do the KDBs take up more space in memory than on disk?

The documentation on Checkpointing says this: "When using multiple gateways for an engine, a KDB file normally consumes a much larger amount of RAM than the size of the KDB file on disk, as all transactions are maintained both in RAM and on disk." I understand that the KDB in memory in this case would be larger than the KDB memory in the case where there is only a single gateway for an engine, because in the case of a single gateway the KDB in memory would not maintain all the transactions. But why would the KDB in memory in the case of multiple gateways consume a much larger amount of RAM than the size of the KDB on disk if all transactions are maintained both in RAM and on disk. Wouldn't these transactions take up the same amount of space in memory as on disk?...

OriginalPostID-48198

OriginalPostID-48198

  Discussion posts and replies are publicly visible

Parents
  • John, perhaps the phrasing in the documentation could be better, but in general, data size in memory is much larger than on disk. If you add the fact that in multiple gateway architecture you keep the transaction log in memory (for performance of gw synchronization) the discrepancy between kdb on disk and in memory grows even larger. I do not exactly know why the data on disk is smaller, but I assume it has to do with the fact that not only data but the code manipulating that data is also loaded into memory. I will follow up with our engineers to find out if my assumption is correct.
Reply
  • John, perhaps the phrasing in the documentation could be better, but in general, data size in memory is much larger than on disk. If you add the fact that in multiple gateway architecture you keep the transaction log in memory (for performance of gw synchronization) the discrepancy between kdb on disk and in memory grows even larger. I do not exactly know why the data on disk is smaller, but I assume it has to do with the fact that not only data but the code manipulating that data is also loaded into memory. I will follow up with our engineers to find out if my assumption is correct.
Children
No Data