Block Reassignment option(Appian's default functionality) from Task

 I want to block the Reassignment functionality when a task is assigned to a team/group. I have used the default functionality available in Appian as mentioned below:-

This is working fine after setting the privilege as "No Privilege".

But I have provided a Reassignment Related Action functionality using which a user can Reassign the task back to the Original Group. In process model that is triggered by Reassignment Related Action, I have used "Reassign Task" smart Service to reassign the task back to original group. This smart service overrides the "No Privilege" setting and Reassignment option/button is available on the task to end user.

Please guide if someone can help me and suggest a way to control this overriding feature of "Reassign Task" smart Service.

Thanks in Advance!.