You can let users drag the drawing area by setting diagram.Behavior = Pan or PanAndModify. Margin is inherited from base Control class and we don't have any built-in code handling it, if you need to set Margin for anything else our developer will check why it breaks auto-scroll.

That seems to be related to keyboard focus. When you click the diagram it requests focus in order to receive keyboard events, and apparently the scroll viewer brings focused components into view automatically (scrolling them to top-left position). Upon next clicks, when the diagram is already focused, it no longer jumps to the origin point. It also doesn't jump if Focusable is disabled -

diagram.Bounds = new Rect(0, 0, 2000, 2000);
diagram.Margin = new Thickness(200, 200, 0, 0);
diagram.Focusable = false;
diagram.AutoScroll = true; 

So you might disable Focusable while in your app's drag mode, or otherwise call diagram.Focus from code before entering drag mode, followed by ScrollX/Y assignments that restore scroll position to original one.

Otherwise non-zero Margin doesn't seem to cause problems with hit-testing and drawing, all's fine in our test after the diagram gets the keyboard focus or if focus is disabled.

Regards,
Slavcho