Compare .CSV file

rakshandak over 5 years ago

Do we have any smart service or plugin in appian to compare two .csv files?

Discussion posts and replies are publicly visible

0 Shanmukha
A Score Level 1
over 5 years ago

Hi Raksha,

AFAIK, There is no such plugin in available.

If possible could you please explain your scenario bit further then we can try to solve it with work around. Some times data can be dump into list of CDT and then we can try to compare then as well. so if we get to know the scenario then we can provide some work around( if any according to your scenario)
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 rakshandak over 5 years ago in reply to Shanmukha

Hi raghvendra,

I have 2 .csv files with huge number of records(in lacks and more than it).

These .csv files have same columns and almost have same data but values in some columns might be changed.

So I need to identify the records whose value have been changed.

Is there any function or any way to compare 2 .csv files?

Thanks,

Rakshanda
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Shiva Valusa
A Score Level 2
over 5 years ago in reply to rakshandak

Hi,

AFAIK there is no plug in available in Appian to compare two CSV files directly.

But please check following plug in (CDT Diff Utilities) once, this might be helpful for your case.
forum.appian.com/.../summary
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 jerrys790 over 5 years ago in reply to Shanmukha

Are data coming from the database for those .csv files?
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 rakshandak over 5 years ago

@jerry: we are getting these. Csv from an onother system
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 rakshandak over 5 years ago in reply to rakshandak

But yes we are going to import it into databases and then we can compare but again there is huge number of records to compare
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 anshuk0001 over 5 years ago

You can parse the CSV files using Parse CSV Document smart service in the CDT's. Use the cdtdifference function to compare the values that have been changed.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Sowrabha Rajashekar over 5 years ago in reply to rakshandak

If the data is huge, I would suggest to do the comparison in Database, using a Stored Procedure, which will result you in the mismatched rows directly. When you use CDT Diff , you have to loop through the whole data, which again time consuming. Also you need to process the result[attributeName, previousValue=, currentValue=], to display which rows have been changed.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Larry Nguyen over 5 years ago

Does this comparison have to be done in Appian? Also are you doing this more than once or is this a one time effort followed up by a one-time data import? If this is a one time effort and you do not need to do it in Appian, you could use the Windows command line to run the following command:

fc filename1.csv filename2.csv > outfile.txt

Source: stackoverflow.com/.../33523588

If you need a more repeatable solution, you'll probably have to develop your own plugin. I don't think the implementation would be too terribly difficult in Java. It could be memory intensive based on your dataset though.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 ankurs718 over 5 years ago

Hi,
I will recommend to use SP(Stored Procedure) to compare the data as these are huge. it will fast and you won't face performance issue as well.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel