There are Pico 4 button controllers that can do this: PJ2-4B-GWH-P01/P02/P03 are proper 4 button scene controllers. All 4 buttons can set a scene for any devices in any rooms to any levels and each button can control a different set - there is no dedicated off button. Each of these is a pre-engraved - but you can easily disassemble swap the button faces from other picos. Unfortunately the custom engraved pico's (PJ2-4B-GWH-EL2) do not come with this style of controller - in my opinion they should.

In this case though - for what the OP requested - a standard cheaper PJ2-4B-GWH-L31 will do everything needed.