Skip to main content
I
itsm6335
September 12, 2023
Question

looping in writeTodataStoreEntity

  • September 12, 2023
  • 11 replies
  • 0 views

Hi,

I am getting huge amount of data and needs to write it in database. Its taking so much time 

Have used 

a!writeTodataStoreEntity(dataStoreEntity:cons!APM_DSE_VALUES, valueToStore: local!data)

In local!data  am calling rule

Can someone help me how to convert it in forEach loop or in batches.

Please help with code

    11 replies

    stewart.burchell
    stewart.burchell
    September 12, 2023

    Are you able to quantify what "a huge amount of data" is? The problem with batching up the writes is that an individual iteration may fail, and that might be a problem if you need the overall persistence of your dataset to be transactional (i.e. all or nothing). I'd try and address the performance issue in a single write first before looking for alternate design options. Are you able to determine where the time is being spent in the write? Is it in the database? Is it in the network? Is it in Appian itself?

    I
    itsm6335Author
    September 12, 2023

    Thanks [mention:7628077af4b9463e9533a897fad2351f:e9ed411860ed4f2ba0265705b8793d05] for replying

    By Huge amount I mean its in 50-60 lakhs of rows.

    Its taking around a minute to execute it.

    In Healthcheck report its showing issue with datastore.

    So the solution which i can think of is to write in batches or by looping.

    stefanhelzle0001
    stefanhelzle0001
    Brainy
    September 12, 2023

    It seems like you do this in an interface. Storing such volumes of data in an interface is a bad idea and will not scale. I suggest to try to find a different solution.

    Terms & ConditionsAccessibility statement