Read a CSV File

RE: Read a CSV File

Yogi Patel — Wed, 12 Jun 2024 03:42:11 GMT

As suggested by Stefan Helzle , you can parse the CSV file manually.

Sample code to read and parse the CSV file manually:

a!localVariables(
  local!fileData: readtextfromfile(
    txtFile: ri!csvDocument,
    preserveLineBreaks: true
  ),
  local!fileLines: split(local!fileData, char(10)),
  a!forEach(
    items: local!fileLines,
    expression: if(
      fv!isFirst,
      {},
      a!localVariables(
        local!splitItem: split(fv!item, ","),
        /*Create Records or CDT  */
      )
    )
  )
)

Hope this helps.

RE: Read a CSV File

nksnksnks — Tue, 11 Jun 2024 19:44:16 GMT

Thank you Stefan

RE: Read a CSV File

Stefan Helzle — Tue, 11 Jun 2024 19:00:38 GMT

That is a very common problem with CSV files. There is no defined standard.

If your file is on the smaller side, you could try to use the "text file reader" plugin to read and parse that file manually.

If you have a larger file, you could try to use another plugin to first convert that CSV into JSON and back to CSV. This way it could be more uniform.

RE: Read a CSV File

nksnksnks — Tue, 11 Jun 2024 17:56:45 GMT

I'm uploading a CSV using import csv to database smart service, but the thing is some rows of the data containing double quotes in the values. for example: employee1;"1";"name ""appian""";

so, I'm getting an error like java.lang.IllegalStateException: IOException reading next record: java.io.IOException: (line 175) invalid char between encapsulated token and delimiter. Is there any way to fix this or some workarounds?

RE: Read a CSV File

Stefan Helzle — Tue, 11 Jun 2024 17:45:17 GMT

Similar issue can mean a lot. Let's take the guesswork out. Can you share some details?

RE: Read a CSV File

nksnksnks — Tue, 11 Jun 2024 17:34:25 GMT

Hi sai,

I'm facing similar issue. how did you resolve this or any suggestions to work on Stefan Helzle

RE: Read a CSV File

saikiranp0001 — Tue, 26 Jul 2022 06:32:51 GMT

Hi Stefan,

Good Day!!

After converting the file to UTF-8 the issue i mentioned got fixed but there is one another issue i am seeing now.
If i use "" in any of the column then the node errors out and throws below error.

java.lang.IllegalStateException: IOException reading next record: java.io.IOException: (line 3) invalid char between encapsulated token and delimiter

Could you please suggest me on this.

Thanks,
Sai Kiran Pannala.

RE: Read a CSV File

saikiranp0001 — Fri, 15 Jul 2022 11:01:30 GMT

HI Stefan,

It worked when i converted the CSV file with UTF-8 encoding and then run it. Thanks alot for the quick help.

RE: Read a CSV File

Stefan Helzle — Fri, 15 Jul 2022 06:07:37 GMT

I think the plugin expects the data to be encoded in UTF-8. Please check that.

RE: Read a CSV File

saikiranp0001 — Fri, 15 Jul 2022 06:01:22 GMT

HI Stefan,

Apologies if my statements were unclear.

I am not exporting data from APPIAN.
Basically i am getting data in a CSV file through SFTP integration from an external source. We need to consume that file into pur APPIAN database.
While consuming the file i am using a smart service(Export CSV to Database) from Excel Tools plugin. And this smart service or plugin is unable to read few special characters as mentioned above.

RE: Read a CSV File

Stefan Helzle — Fri, 15 Jul 2022 05:54:37 GMT

So, you export data from Appian into a CSV and import it, where? Appian uses UTF-8 to export the data. Is the other DB set up to store unicode? Does the import read the data in unicode?