Most efficient way to remove duplicates in an array of CDT or Map based on a specific key?

Question

I need to check a specific field in an array of cdt or map for duplicates, and return an array of cdt or map without duplicates on that field. For example: CDT Array: 
 
 { type!example( 
 email: "hello@gmail.com", 
 type: 1, 
 date: 01/01/2022 
 ), 
 type!example( 
 email: "hello11@gmail.com", 
 type: 1, 
 date: 03/02/2023 
 ), 
 type!example( 
 email: "hello@gmail.com", 
 type: 2, 
 date: 01/02/2023 
 ) 
 } 
 
 In the above case, I want to return an array with unique email fields. Since the first and third item in the array have the same email address "hello@gmail.com", I only want to return the first one.

Mathurambika M · Accepted Answer

Hi, Following Stefan's suggestion, I have employed the 'unique' function and 'forEach' loop to retrieve the CDTs. Please refer to the code snippet below: a!localVariables(
 local!testData: {
 {
 email: "hello@gmail.com",
 type: 1,
 date: today()
 },
 {
 email: "hello11@gmail.com",
 type: 1,
 date: today()
 },
 {
 email: "hello@gmail.com",
 type: 2,
 date: today()
 }
 },
 local!uniqueEmails: union(
 touniformstring(local!testData.email),
 touniformstring({})
 ),
 a!forEach(
 items: local!uniqueEmails,
 expression: index(
 index(
 local!testData,
 wherecontains(
 tostring(fv!item),
 touniformstring(index(local!testData, "email", {}))
 ),
 {}
 ),
 1,
 {}
 )
 )
)

Mike Schmitt · Answer

walkers said: This is the result I want: "j ust keep the first copy of any given unique email 
 That's good news then, because it should be fairly simple to implement. 
 In pseudocode: 
 
 get unique emails from all entries 
 cycle through unique emails 
 grab the array of original entries matching the current unique email 
 return the first such array match 
 
 This is basically what Mathurambika M has already written above, FWIW.

Amaan Shekh · Answer

try this -> a!localVariables(
 local!data: {
 'type!{urn:com:appian:types:DCT}DCT_temp'(
 type: 1,
 email: "212@gmail.com",
 date: today()
 ),
 'type!{urn:com:appian:types:DCT}DCT_temp'(
 type: 2,
 email: "211@gmail.com",
 date: today()
 ),
 'type!{urn:com:appian:types:DCT}DCT_temp'(
 type: 3,
 email: "212@gmail.com",
 date: today()
 ),
 'type!{urn:com:appian:types:DCT}DCT_temp'(
 type: 1,
 email: "213@gmail.com",
 date: today()
 )
 },
 local!descardDuplicateEmailValue: a!forEach(
 items: local!data,
 expression: index(
 local!data,
 index(
 /*find where else our current email resides*/
 wherecontains(fv!item.email, local!data.email),
 /*take only first one*/
 1,
 ""
 ),
 ""
 )
 ),
 union(
 local!descardDuplicateEmailValue,
 local!descardDuplicateEmailValue
 )
)

Mike Schmitt · Answer

walkers said: What would be the best way to return the list of cdt where any duplicates on the email field have been removed? 
 What exactly are you expecting as the result of this? If the emails are duplicate, but the other items in each CDT are not necessarily the same, what is the expected behavior? Would it just keep the first copy of any given unique email and discard the data contained in any duplicates even if it's different? Or some other potential approach, which you haven't specified?

Most efficient way to remove duplicates in an array of CDT or Map based on a specific key?

Top Replies