Inconsistency between time taken for write to db node to complete and the timing shown in the 'data-store-details.csv' log

Hello Everyone,

I have a query regarding the time of Write to Datastore operation reported in the log file "data_store_details.csv"
This log file breaks the total time for the write to DB query to execute in "Mean Prepare Time", "Mean Execute Time" and "Mean Transform time". Now, for one example - on one environment I am working on - the durations reported for these are 5ms, 16ms and 5ms respectively - for a total of 26ms of Mean Total time.

Now, I observed the time taken for the "Write to Datastore" node to complete in the process history of the process instance. It shows the time as 1.04 seconds. This indicates the Write to datastore node took 1.04 seconds to complete. However, from the log file - I see that the actual database operation took a total of 26ms.

So, why is there some...

OriginalPostID-265602

  Discussion posts and replies are publicly visible

Parents
  • @aloks176, In Process History->Process Nodes, we can find the duration for node completion, it clearly shows the time taken for the "Write to DB" node to complete. I agree with other points mentioned by you, the Obtain connection to DB, Get a JDBC statement - are the "Prepare" phase. The "Execute the statement" is the Execute phase and getting the results back is the transform phase.

    Appian logs do tell us what durations each of these phases took.
    But it does not tell us the duration for the remaining activities (the ones you mentioned - 1.Create/Load configuration 2.Build SessionFactory, 3.Build Session 4.Access the Service 5. Close the Session).

    Also, I am certain that a lot of post processing also happens.
    I tried a few things:
    1. I made the CDTs in which the output was being saved as "Hidden". I observed that I can save a few milliseconds per CDT
    2. If I simple do not save the output of write to DB node, I can save even more - which is desirable only if you don't care about the output of the write to DB node.

    @stefanh791, yes - I am also beginning to think that node exec time cannot be related to DB exec time. But, at-least if I had some confirmation about what post processing happens after the "transform" phase, it would have been helpful.
    A client I am working with is interested to know this.
    And I also find it surprising that a raw sql query takes 330 ms but the "Write to DB" node takes 770ms.
    I understand that Appian has layers of abstractions over a raw database query, and those take the remaining time. But, we are expecting a breakdown of them, or at-least a confirmation that all of this post-processing happens.
Reply
  • @aloks176, In Process History->Process Nodes, we can find the duration for node completion, it clearly shows the time taken for the "Write to DB" node to complete. I agree with other points mentioned by you, the Obtain connection to DB, Get a JDBC statement - are the "Prepare" phase. The "Execute the statement" is the Execute phase and getting the results back is the transform phase.

    Appian logs do tell us what durations each of these phases took.
    But it does not tell us the duration for the remaining activities (the ones you mentioned - 1.Create/Load configuration 2.Build SessionFactory, 3.Build Session 4.Access the Service 5. Close the Session).

    Also, I am certain that a lot of post processing also happens.
    I tried a few things:
    1. I made the CDTs in which the output was being saved as "Hidden". I observed that I can save a few milliseconds per CDT
    2. If I simple do not save the output of write to DB node, I can save even more - which is desirable only if you don't care about the output of the write to DB node.

    @stefanh791, yes - I am also beginning to think that node exec time cannot be related to DB exec time. But, at-least if I had some confirmation about what post processing happens after the "transform" phase, it would have been helpful.
    A client I am working with is interested to know this.
    And I also find it surprising that a raw sql query takes 330 ms but the "Write to DB" node takes 770ms.
    I understand that Appian has layers of abstractions over a raw database query, and those take the remaining time. But, we are expecting a breakdown of them, or at-least a confirmation that all of this post-processing happens.
Children
No Data