How to write only unique entries in the table

Question

Hi team, 
 I am capturing data from process metrics report and storing to oracle db as it's business requirement. 
 Now I just want to check is there any inbuilt functionality of Appian which only writes the rows in table which has been not inserted in the previous run ? 
 if not is there any suggested way you recommend that can be followed to achieve the same ?

Harshitha Mangamuri · Answer

Hi, We have to query the data with a!queryEntity() / a!queryRecordType() from the database and check if the data is present or not. By checking that you can write only data which is not present to the database.

Harshitha Mangamuri · Answer

Try to create a process model which can be used as a subprocess and with the help of looping in it with the start index and batch size as 1000 (for example) fetch the data and then continue the looping process till you fetch all the data which can solve the issue which is caused due to the high volume.

Mathieu Drouin · Answer

Would need to know more about your requirements to give a proper solution but one way to do it could be to calculate a unique composite key or a hash from key portions of the data. This way you can rerun your inserts without inserting duplicates.