a!pickerFieldUsersAndGroups() error

groupFilter (Group): When present, only users or groups which are members of this group are suggested.

Are you using a constant of type Group as the filter?

a!pickerFieldUsersAndGroups(
  groupFilter: cons!T_GROUP
)

no

it is good with users selection but not select groups

Not sure I follow, can you show me the code?

a!localVariables(
  local!pick,
  local!data:a!queryEntity(
    entity:cons!LAM_DSE_LEAVES,
    query:a!query(
      filter:a!queryFilter(
        field:"username",
        operator:"=",
        value:local!pick
      ),
      pagingInfo:a!pagingInfo(
        startIndex:1,
        batchSize:-1       

      )
    )
  ),
 a!pickerFieldUsersAndGroups(
    label:"picker",
    value:local!pick,
    saveInto:local!pick
  )
  
  
)

when I select user so not error but during selecting group it gives error

a!localVariables(
  local!pick,
  local!data:a!queryEntity(
    entity:cons!LAM_DSE_LEAVES,
    query:a!query(
      filter:a!queryFilter(
        field:"username",
        operator:"=",
        value:local!pick
      ),
      pagingInfo:a!pagingInfo(
        startIndex:1,
        batchSize:-1       

      )
    )
  ),
 a!pickerFieldUsersAndGroups(
    label:"picker",
    value:local!pick,
    saveInto:local!pick
  )
  
  
)

when I select user so not error but during selecting group it gives error

cannot apply [EQUALS] to field [assignee] when comparing to value [TypedValue[it =5,v=414].

what operator should I used for it

This error would always be seen when you first load your form because you're trying to filter on the value of local!pick which would always be empty at first, and the filter value can't just be empty without taking other steps.  In this case you probably want your query to return nothing until a value is given for local!pick, so you could wrap the entire query in an if() statement checking whether local!pick is blank. 

Additionally, if local!pick is expected to resolve to a username (as that's what you'll be querying), you should probably just use a user picker field, not the "user and group" picker, because if a group is selected, the query will fail (or at least not return anything meaningful).

but I want to select both users and groups in my filter

when i select user or group in both case it gives error.

a!localVariables(
  local!pick,
  local!user,
  a!pickerFieldUsersAndGroups(
    label:"picker",
    value: local!pick,
    saveInto: {
      local!pick,
      a!save(
        local!user,
        'type!{urn:com:appian:types:T_USER_OR_GROUP}T_USER_OR_GROUP'(
          userOrGroup: local!pick
        )
      )
    }
  )
)

Here are the 2 text representations for a user and a group.

how should I make...

'type!{urn:com:appian:types:T_USER_OR_GROUP}T_USER_OR_GROUP'(
userOrGroup: local!pick
)

After selecting user or group from picker.

It is not applied in filter 

filter:a!queryFilter(
field:"userOrGroupName",
operator:"=",
value:local!pick
),

It gives Error:

a!queryFilter(
  field: "username",
  operator: "=",
  value: tostring(local!pick)
)

put a tostring in your queryFilter