Box Layout Font Size Change

There isn't any direct solution to this but I have a work around for you.
You can use two card layouts below each other and that can give you the look and feel of box layout and also give you more controls on label, size, color etc.
Refer the code snippet below.

{
  a!cardLayout(
    style: "#F0F0F0",
    contents: a!richTextDisplayField(
      labelPosition: "COLLAPSED",
      value: {
        a!richTextItem(
          text: "Label",
          size: "MEDIUM"
        )
      }
    )
  ),
  a!cardLayout(
    
  )
}

There isn't any direct solution to this but I have a work around for you.
You can use two card layouts below each other and that can give you the look and feel of box layout and also give you more controls on label, size, color etc.
Refer the code snippet below.

{
  a!cardLayout(
    style: "#F0F0F0",
    contents: a!richTextDisplayField(
      labelPosition: "COLLAPSED",
      value: {
        a!richTextItem(
          text: "Label",
          size: "MEDIUM"
        )
      }
    )
  ),
  a!cardLayout(
    
  )
}

I need the collapsible feature as well

That can be very well achieved. Add a dynamic link on above card to set a variable to true when false and vice-versa every time you click on it. Then use the same variable in "showWhen" of the below card.

What you have to do in that case is set the height of the Card Layout to AUTO, that way, when the content disappears, it also shrinks to match the 0 content, or you can make the whole lower card disappear.