Web Services with Record Types - Cannot Sync record where API called for more than one Input

Question

Hi, 
 I need some ideas. 
 I am accessing external data using an API and building a record, type - Web Service to store customer information. The API requires a group name to return customer details. 
 Example:

Company A &rarr; 20,000 customers

Company B &rarr; 30,000 customers

When you pass Company A , it returns the customer details for that company. 
 Now I am facing a challenge. When i call only Company A it goes fine. when i introduce foreach for company B then it is not possible as the sync limit is 1000 by Appian. 
 I am using the Batch by Sequential Values method. In my integration, I have set paging to return 1,000 records per call. error message i get: "The Record Data Source cannot return more than 1,000 records in a single batch. Batch size was 2,000." 
 Note: I cannot reduce the API limit below 1,000 records because we have many companies, and doing so would significantly increase the synchronization time. 
 sample Code: a!localVariables(
 /* Call the integration and pass the batchNumber rule input */
 local!companyName: { "ABC", "DEF" },
 a!forEach(
 items: local!companyName,
 expression: a!localVariables(
 local!integrationResponse: rule!GET_Data_Integration(
 batchNumber: ri!batchNumber,
 CompanyName: fv!item
 ),
 if(
 local!integrationResponse.success,
 /* Return the body; Appian continues calling until an empty list is returned */
 local!integrationResponse.result.body,
 /* Error handling */
 if(
 tointeger(
 index(
 local!integrationResponse.result,
 "statusCode",
 0
 )
 ) = 416,
 {},
 local!integrationResponse.error
 )
 )
 )
 )
) 
 
 Any thoughts or different possibility to achieve this?

Melvin George · Accepted Answer

Perfect Thanks Peter

Peter Lewis · Answer

One approach you could take is getting a little creative using the Batch by Cursor or Token option. If you use the batch sequentially option, the only thing that gets passed to each call is an incrementing integer. But in the batch by cursor option you determine information that gets passed to the next batch. In many cases this is used to pass the next page URL or something like that, but you can actually pass anything you want.

For example, with each batch you could pass the highest value for the previous batch and which integration was called. Then, you could use that information in your next batch to determine how to filter and which integration to call.

It definitely takes a little wrangling of your expression, but is totally supported and should allow you to meet your needs. Here's a basic example / pseudocode of what it might look like:

a!localVariables(
  /* Suppose that in the previous batch info you provide */
  /* the highest primary key value and which integration */
  /* is called. In this example, I'm simply formatting it */
  /* like this: <PK value>|<Integration name> */
  /* so an example could look like "123456|Integration One" */
  local!splitBatchInfoParts: if(
    a!isNullOrEmpty(ri!previousBatchInfo),
    null,
    split(ri!previousBatchInfo, "|")
  ),
  local!part1: index(local!splitBatchInfoParts, 1, null),
  local!part2: index(local!splitBatchInfoParts, 2, null),
  local!integration: if(
    or(
      a!isNullOrEmpty(local!part2),
      local!part2 = "Integration One"
    ),
    rule!Integration_One(minId: local!part1),
    rule!Integration_Two(minId: local!part1)
  ),
  ...
)

Stefan Helzle · Answer

Yeah, the issue is that call the API two times, for each company and then combine the result. This makes 2000. You will have to develop the logic in a way that it first fetches all data for the first company, and then for the second.

Web Services with Record Types - Cannot Sync record where API called for more than one Input

Top Replies