If using JavaApplet mode, set the ShapeLibraryLocation property to a URL where the applet can download the shape definitions from. Both for JavaApplet and ImageMap mode, add a ShapeLibrary.LoadFrom() call to the page_Load handler, or add the code that defines the custom shapes there, inside an if (Shape.FromId(...) == null) statement. The latter is required since custom shapes definitions are not saved in the viewstate, and might be lost once IIS unloads the Diagramming.dll assembly after some period if inactivity. Another possibility is to load / define the shapes upon application startup in Globals.asax.

I hope that helps,
Stoyan