Would like be able to run a!toJson on a CDT where blank fields are suppressed

What's the actual problem here? Assuming that sometimes an attribute as a value and sometimes it doesn't then the recipient of the integration call will expect to receive messages with this attribute present, and if the value is empty that is still valid, isn't it? Are you able to elaborate on why you're trying to exclude blank attributes from your JSON message?

It's really more of a limitation of the downstream integration.

I could be doing an update on a single field of an existing entity of a downstream system with an integration.  Ideally, I would just pass the single field as part of that entity's CDT JSON structure, rather than all the fields from the CDT for that given entity.

Providing a blank value for an existing field could potentially be seen by the downstream integration as a removal of the field rather than just the fact that I'm just not setting it.  This is the main issue.  I don't control the integration, so I am trying to handle things with Appian.  If the field isn't passed at all by Appian, it is ignored by the downstream Integration.  

As an alternative, one could arguably pass *everything* on the CDT instead of the single field, but that's basically providing more data than necessary, and isn't really a correct reflection of the update to the entity -- it's a single field I'm updating, any other fields I'm passing are extra noise and shouldn't really be updated *again*.

I don't think suppressing blank fields in JSON is that unusual an ask, is it?

It's really more of a limitation of the downstream integration.

I could be doing an update on a single field of an existing entity of a downstream system with an integration.  Ideally, I would just pass the single field as part of that entity's CDT JSON structure, rather than all the fields from the CDT for that given entity.

Providing a blank value for an existing field could potentially be seen by the downstream integration as a removal of the field rather than just the fact that I'm just not setting it.  This is the main issue.  I don't control the integration, so I am trying to handle things with Appian.  If the field isn't passed at all by Appian, it is ignored by the downstream Integration.  

As an alternative, one could arguably pass *everything* on the CDT instead of the single field, but that's basically providing more data than necessary, and isn't really a correct reflection of the update to the entity -- it's a single field I'm updating, any other fields I'm passing are extra noise and shouldn't really be updated *again*.

I don't think suppressing blank fields in JSON is that unusual an ask, is it?

Anyone?  Bueller?  Bueller?

We had to create a workaround for some behavior where toJson added a "Z" to pur date values. This is not exactly out of standard, but some JSON libraries commonly used do not support this. We made up a recursive expression which turns the CDT into a map while keeping our own date format.

Following this pattern you can decide for yourself whether to add a field to the outgoing map. This map can then be converted to JSON in the integration.

I did spend some time looking at this.

Using CDTs is a no-go. The structure is already concrete when you design the CDT, so you'll get those attributes generated in a CDT instance and carried forwards when you cast to JSON.

Using a Dictionary has more possibilities, but there are still constraints with this method. You would have to write an if() statement around every attribute that has the possibility of not being included in the final payload, but you're backed into a corner that looks like this:

a!localVariables(
  local!payload: { 
    { firstName: ri!firstName },
    {lastName: ri!lastName},
    if(
      fn!isnull(ri!dob),
      {},
      {dob: ri!dob}
    )
  },
  a!tojson(
    local!payload
  )
)

...where output looks like this:

[{"firstName":"Stewart"},{"lastName":"Burchell"}]

(when 'dob' is passed as a null value)

I think you need to go back to first principles. What is the 'contract' of the service that you're calling? (this might be expressed in a document e.g. MS Word, or as a technical Schema). The contract should say what the specific attributes that are required or optional, what type they should be, whether they're single values or an array of values, even what are the valid values. It should also define what the expected behaviour is e.g. "If you pass an empty value in Attribute A then this will cause that value to be set to null"

As a follow-on from this you could construct a generic implementation that generates messages with name/value pairs, only if the attribute contains a non-null/blank value. To future proof this, you can dynamically extract all of the attribute names from a CDT then, with that list of names index() into the CDT to get the corresponding value, generate a {name: xxx, value: yyy} where thee value is no-null/blank, and then transform to JSON:

a!localVariables(
  /* local!attributes contiand the name of each attribute */
  /* construict by creating an instance of the target CDT, */
  /* make a string version of it, remove all of extraneous characters */
  /* and then split on the comma character*/
  local!attributes: split(
    reduce(
      fn!substitute,
      fn!tostring(
        'type!{urn:com:appian:types:SJB}SJB_customer'()
      ),
      { "[", "]", "=", " " },
      null
    ),
    ","
  ),
  a!tojson(
    a!forEach(
      items: local!attributes,
      expression: if(
        or(
          isnull(fn!index(ri!customer, fv!item, null)),
          
        ),
        {},
        {
          name: fv!item,
          value: fn!index(ri!customer, fv!item, null)
        }
      )
    )
  )
)

This way if your CDT every changed then this would break. But (referring to my previous response) your target  system would need to be able to receive the payload in this specific format - a JSON array of name/value pairs.

The services we are calling are internal to our organization, but are developed by a different team, and I don't have a lot of control over their APIs.  It's a bit of a hodgepodge of things, unfortunately.  In some cases passing an empty string will result in the removal of a field's data.  In other places, one needs to pass an "endDate" which is essentially a deletion. 

I'm a bit at their mercy.

Sounds like the real solution is to fix the way the APIs are designed and implemented! 

It has just occurred to me (although you'll almost certainly not like it - I know I don't!) that you could generate a full-fat JSON based upon your CDT (so it contains all of the attributes you might ever need to send in your JSON message), and then implement a post-processing rule that treats the JSON as a string (which it is!) and excise the attribute(s) you don't want sent based upon whatever are the rules e.g. if a given attribute is null and that is going to cause the API to delete the current value, then simply edit the string and remove that attribute. I'm sure if you experimented with a mixture of the text functions (find, mid, search, substitute etc.) you could implement a deterministic method of extracting the empty values and splicing the message back together so that it is still valid JSON.

(Yuck! I know, right? But it is an option...you'd need to know the rules about what's valid at each API and perform the necessary manipulation as required...)

Yeah, my situation pretty much sucks.  And yeah, some kind of post-processing on the JSON String is probably the best thing I can do within Appian.

I'm still gathering info as to how the rules even work -- I don't know that the downstream system even has that information for me.  Ugh.

I am going to mark "Verify Answer" on your reply, but I'm not happy about it!

If anything revolutionary occurs to me I'll let you know...but I feel your pain!