Is there any way to hide the pages from the site according to the user group?

Yes. While configuring your site page your define the visibility condition.

For your case it should work. Check whether the constant type is mapped with group, mapped group has the user.

Try without using without parameter names like isusermemberofgroup(loggedInUser(),cons!UMS_ASSIGNEE_GROUP).

If not create a separate rule and check whether the rule is giving appropriate output. So that you can have what the expression is returning.

Also check the security of the rules folder.

I tried isusermemberofgroup( username:loggedInUser(), groupId:cons!UMS_ASSIGNEE_GROUP ) in expression rule, it is returning false value. But still the page is visible.

If you are testing by using system administrator login it will not effect the security settings.

While testing use basic user. Not the System administrator

I created a new user for testing. But access is denied to my application. How to give access to user that I have created?

Please add the created user in the group(which has access to your application) and also provide security for application and other appian objects.

As the best practice, we assign the security to the groups.

Find out the group(in your case cons!UMS_ASSIGNEE_GROUP) which has access to the application, add the newly created user to that group.

Configure the security for the Site page object (Action/Record/Report) with the same group (minimum access should be Viewers)

Set the security to the Site with the same group (minimum access should be Viewers)

Note: When you configure the groups or security settings login as System Administrator account. While testing login as newly created user. So that you can have the clear visibility.

As the best practice, we assign the security to the groups.

Find out the group(in your case cons!UMS_ASSIGNEE_GROUP) which has access to the application, add the newly created user to that group.

Configure the security for the Site page object (Action/Record/Report) with the same group (minimum access should be Viewers)

Set the security to the Site with the same group (minimum access should be Viewers)

Note: When you configure the groups or security settings login as System Administrator account. While testing login as newly created user. So that you can have the clear visibility.