Do you know, that… ?

To move the drawing view within the sheet, You do not have to hold his frame.

only need to, that hold ALT and you can move the drawing view by clicking anywhere within this view.