Backing out from a panel

When I’m working on panels within panels, I find myself double-clicking on the canvas thinking it’s going to take me back just one panel. Instead, I’m taken back to the page level. I realize you can click on the panel navigation at the top but it sort of takes you out of context, especially since we now have the ability to see the objects right outside of the panel. Perhaps there’s a shortcut to get back to the page level if you’re deep into panels like cmd-double-click.