We are working on a project wherein we pull data from an external application an

@gopinathreddyr To the best of my knowledge, I would like to suggest few things from the performance perspective based on which you could formulate either a new approach or you could continue with the approach devised by you recently and possibly some alterations if needed:

1. Please do bear in mind that delegating the functionality of the process completely to the complex expression rules is not a right practice. Please bear in mind that there is a limit for the execution time of the expression rule. If you write a complex expression rule that does all the work for you by replacing the process, I would like to suggest to check if the expression rule's processing time falls within the execution time limits by testing with the maximum amount of data. If it doesn't, this isn't a right idea as per my knowledge.

The documentation at https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html#USRX-0032 complements the above explanation. If you still think that the expression is that complex and handles lot of calculations, I would further like to split the functionality by creating new expression rules.

Additionally I would also like to suggest to refer https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html#USRX-0028 if you are employing queries in the expression or process as most of the times database read/write operations would be the culprits. In our case, writing just 1000 records into a table (flat, no relationships defined) that has 6 columns has been identified as a risk.

2. If possible try to delegate the logic to database(for example making comparisons by making use of the views etc) as much as possible. But also make sure that the database objects also performs faster executions which would need you to focus on the joins, indexes etc.

3. Never completely depend on process or completely on a single expression rule that does the entire work. Both should go hand in hand and make best use out of it by combining them. Make use of batching technique, let's say there are 1000 operations, make sure that your expression rule handles just 200(an arbitrary value, should be based on the complexity of your rule) at a time(maintain this in a constant so that you can increase the batch size if you think that operations are taking less time and also you can bring down the batch size if the batches are consuming more than the expected execution time.) and your process should be able to loop back until all the batches are processed.

4. Not sure if you are using MNI extensively in the current use case, but if so, make sure that the subprocesses are triggered by using messaging. https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html#DSGN-0032 should give you an good idea.

I guess that implementing the above scenarios might delay the process completion time, but ideally their prime focus is on performance. I would like to suggest to focus on the performance rather than keeping the Appian engines busy just for faster execution of few things. Also the above design considerations will ideally let you handle any batch size and any complex operations if you control the batch size effectively. These are the things which I have learnt so far from my previous experiences as well as the learnings from the Health Check results and I guess even Appian recommends the same. If you haven't gone through the Health Check already, I would like to suggest to go through https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html# and this would let you know the right way to a better extent.

@gopinathreddyr To the best of my knowledge, I would like to suggest few things from the performance perspective based on which you could formulate either a new approach or you could continue with the approach devised by you recently and possibly some alterations if needed:

1. Please do bear in mind that delegating the functionality of the process completely to the complex expression rules is not a right practice. Please bear in mind that there is a limit for the execution time of the expression rule. If you write a complex expression rule that does all the work for you by replacing the process, I would like to suggest to check if the expression rule's processing time falls within the execution time limits by testing with the maximum amount of data. If it doesn't, this isn't a right idea as per my knowledge.

The documentation at https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html#USRX-0032 complements the above explanation. If you still think that the expression is that complex and handles lot of calculations, I would further like to split the functionality by creating new expression rules.

Additionally I would also like to suggest to refer https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html#USRX-0028 if you are employing queries in the expression or process as most of the times database read/write operations would be the culprits. In our case, writing just 1000 records into a table (flat, no relationships defined) that has 6 columns has been identified as a risk.

2. If possible try to delegate the logic to database(for example making comparisons by making use of the views etc) as much as possible. But also make sure that the database objects also performs faster executions which would need you to focus on the joins, indexes etc.

3. Never completely depend on process or completely on a single expression rule that does the entire work. Both should go hand in hand and make best use out of it by combining them. Make use of batching technique, let's say there are 1000 operations, make sure that your expression rule handles just 200(an arbitrary value, should be based on the complexity of your rule) at a time(maintain this in a constant so that you can increase the batch size if you think that operations are taking less time and also you can bring down the batch size if the batches are consuming more than the expected execution time.) and your process should be able to loop back until all the batches are processed.

4. Not sure if you are using MNI extensively in the current use case, but if so, make sure that the subprocesses are triggered by using messaging. https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html#DSGN-0032 should give you an good idea.

I guess that implementing the above scenarios might delay the process completion time, but ideally their prime focus is on performance. I would like to suggest to focus on the performance rather than keeping the Appian engines busy just for faster execution of few things. Also the above design considerations will ideally let you handle any batch size and any complex operations if you control the batch size effectively. These are the things which I have learnt so far from my previous experiences as well as the learnings from the Health Check results and I guess even Appian recommends the same. If you haven't gone through the Health Check already, I would like to suggest to go through https://forum.appian.com/suite/help/7.10/Appian_Health_Check.html# and this would let you know the right way to a better extent.