We are currently performing maintenance on Appian Community. As a result, discussions posts and replies are temporarily unavailable. We appreciate your patience.
I'm trying to parse a large data file into our sql database. I have found the import csv to database smart service which would work with the "|" delimiter of the dat file except it has metadata as its first and last lines (not a csv header). I have uploaded the file into an appian document object, but I can't figure out how to remove those lines so it works with the smart service.
I should also note the dat file is coming from an external datasouce and cannot be altered prior to ingestion.
Discussion posts and replies are publicly visible
1st thing I would try is to load the file as is in the DB and just delete/ignore the 1st and last rows.
Otherwise, you might be able to do something with this plugin:
Hi,
Thank you for the suggestion. It does seem to be a way to accomplish what I would like to do. However I am running into hitting max heap usage. What is the correct way to increase heap size in appian? I've tried looking and there isn't a lot of directly helpful information.
You probably want to find a way to split this in multiple files.
Unfortunately that cannot be done. How would I increase the heap instead?
This kind of things is not something Appian is made for. Hitting walls when trying to force it is expected.