How can i add the rich text display field for the header cells?

Header cells can only accept plaintext for their labels, unfortunately.

The only workaround I know of is to leave the labels in the header cell row blank, then make a top row of data to function as a "fake" header cell - these would take rich text of any sort you like.  AFAIK it's a bit clumsy of a workaround though because I don't think you can completely hide the box that originally contains the actual labels (it might just appear pretty small).  But if your need is great enough then you can try this approach.

Depending on your requirements, this might be an option: https://appian.rocks/2023/04/17/formatting-labels-in-appian/