What's the best and optimized way to copy the whole content of View_Legacy F

What's the best and optimized way to copy the whole content of View_Legacy From DB_Legacy to Table_Appian in DB_AppianBusiness and/or Is there a component that is able to compare 2 table structures and give as output only the differences?
- Both DataSource are mapped in Appian and available through the use of DataStore.
- View_Legacy belongs to a Legacy System.
Periodically the source View_Legacy is updated by an External System; that's why we need to compare the content of the two structures and update Table_Appian according to the current values of View_Legacy.
Thanks in advance!

OriginalPostID-149520

OriginalPostID-149520

  Discussion posts and replies are publicly visible

Parents
  • @patrick Hi, a scheduled process that makes use of a Query DB node with a query which inserts the unique records from View_Legacy (but aren't present in Table_Appian) into Table_Appian, should do the work and infact this works faster. The frequency of the scheduled process should be set considering the updates from external system. It can be run frequently (for example two hours a day) provided if you are not expecting huge volumes and a straight forward insertion of unique records. Also bear in mind that more frequency and huge volume of records can together consume a significant system of resources, thereby affecting the performance.

    Also a wrapper process model can be built around this scheduled process to trigger the updates on need basis and expose the same as an 'Action' to the Administrators.

    The only tradeoff of this approach is that, you need to write a query which is platform dependent. If you prefer to stay away from writing a sql query, probably you can think of an alternative like this - Build a new view that contains records from View_Legacy but are not in Table_Appian. In the scheduled process, retrieve the records from new view and run a 'Write to data store entity' smart service to insert the results into Table_Appian. Again your query rules should handle worst case scenarios probably generated because of huge volume of records as a result of updates.

    Added to the way, I don't think there is a OOTB or shared component already available, but it depends on the way how best we can make use of features in Appian.

    Let's see if other Appian practitioners could come up with much better solution(s).
Reply
  • @patrick Hi, a scheduled process that makes use of a Query DB node with a query which inserts the unique records from View_Legacy (but aren't present in Table_Appian) into Table_Appian, should do the work and infact this works faster. The frequency of the scheduled process should be set considering the updates from external system. It can be run frequently (for example two hours a day) provided if you are not expecting huge volumes and a straight forward insertion of unique records. Also bear in mind that more frequency and huge volume of records can together consume a significant system of resources, thereby affecting the performance.

    Also a wrapper process model can be built around this scheduled process to trigger the updates on need basis and expose the same as an 'Action' to the Administrators.

    The only tradeoff of this approach is that, you need to write a query which is platform dependent. If you prefer to stay away from writing a sql query, probably you can think of an alternative like this - Build a new view that contains records from View_Legacy but are not in Table_Appian. In the scheduled process, retrieve the records from new view and run a 'Write to data store entity' smart service to insert the results into Table_Appian. Again your query rules should handle worst case scenarios probably generated because of huge volume of records as a result of updates.

    Added to the way, I don't think there is a OOTB or shared component already available, but it depends on the way how best we can make use of features in Appian.

    Let's see if other Appian practitioners could come up with much better solution(s).
Children
No Data