We want to retrieve data from a another system using a service backed record approach. While doing this we saw a 5MB size data limit restriction for the full sync. Also we are not able to provide any batchNumber so as to get full data(as we are not getting this parameter to be passed to the actual API).
Discussion posts and replies are publicly visible
There are very strict data limits in Appian to prevent us from breaking the system. And these limit can and should not be raised.
Synced data is stored inside Appian in a separate database which is only accessible through the record. If you want to persist that data to a SQL DB, then, first sync that data to this DB and then put the Appian record on top of it.
Can you give more context on what the data looks like that you're trying to sync? How many rows / columns of data do you have? How long does the web service call take to retrieve your data? Also where are you seeing a limit reached - is it when performing the API call or attempting to sync your record type?
One approach I have seen before is to do batching or parsing the data after receiving the result and just writing a subset of the data at a time. However, this typically works only in limited circumstances where the data is larger than the limits but not significantly larger, so we'd need more context on what your data looks like and how you plan to use this.