Processing large amount of data in process variables

There are rare cases when we have to process a large amount of data in Appian, such as when reading from the health check log file hc-application-objects.csv; this file could easily contain about 4 MB of data. 

Questions:

  1. Is there any limit to how much data process variables contain or is it just dependent on the RAM hardware?
  2. I am planning on creating a process that reads the data from the log files, stores in PV and writes into the database and delete this process from memory right away. Are there any other options? For example, I did some research and there is a plugin that reads from a CSV file and writes directly into a database table without bringing the data into memory; however, this plugin is not cloud approved. 

FYI: I already checked this link https://community.appian.com/w/the-appian-playbook/201/using-process-sizing-for-engine-capacity-planning and all the playbook process sizing related links. 

  Discussion posts and replies are publicly visible