I completely agree the Alexa-RR2 command set is lacking. BUT, it is possible to make rooms, floors, or areas of your home (not individual devices) controllable via "on" and "off" commands. You just need to create a virtual button that controls the set of lights and make sure the button is set as a toggle. You then expose this virtual button to Alexa with a custom name like "Kitchen lights" or "outside lights" Then "Alexa, turn on kitchen lights" will work as desired, and "turn off kitchen lights" will as well.
I set up a number of these virtual buttons with every possible set of lights i might ever want to control, so that Alexa will properly do "Alexa, turn off everything", "Alexa, turn on upstairs lights", "Alexa, turn off outside lights"... all work fine. So you should NOT have to do "Alexa, turn on All Off". You just haven't set your programming and your Connect App settings properly.