Page Index Toggle Pages: 1 Send TopicPrint
Normal Topic Out of Memory Exception when using CompositeLayout with a large number of partitions (Read 340 times)
net_developer
YaBB Newbies
*
Offline


I Love MindFusion!

Posts: 1
Joined: Jul 9th, 2021
Out of Memory Exception when using CompositeLayout with a large number of partitions
Jul 9th, 2021 at 2:31pm
Print Post  
I’m am looking for help on an issue we have with the library MindFusion.Diagramming.Wpf (Version 3.5.5) with .NET Framework 4 in a 32-bit process.

We are trying to render a large diagram with about 1000 nodes and 700 edges. As you can see by the number of edges, not all nodes are connected to the same graph. Instead, we have about 200 separate graphs.

The issue arises when we try to arrange the diagram with a CompositeLayout consisting of a SpringLayout as the MasterLayout and a FractalLayout as the SubtreeLayout. We use a custom algorithm for the partitioning of the CompositeLayout which creates about 700 partitions. The number of partitions is so high since we plan to add more leaf nodes to these partitions.

After calling CompositeLayout.ArrangeAsync(), the Application “hangs” for ~10 minutes. Mindfusion seems to intermittently perform some operation in the UI thread and continuously allocate more memory.

When the total application memory hits ~1.3 GB, the app crashes with the following stack trace:

OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
--------------------------------------------------------------------------------

*** Stacktrace ***
   at System.Collections.Generic.Dictionary`2.Resize(Int32 newSize, Boolean forceNewHashCodes)
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at A.YT..ctor(IEnumerable I)
   at A.ET.SI(IEnumerable I)
   at A.ET..ctor(Diagram I)
   at A.NT.K()
   at A.MF.I(ST I, DT V)
   at MindFusion.Diagramming.Wpf.Layout.Layout.VV(ST I, Diagram V, Boolean K, Boolean O)
   at MindFusion.Diagramming.Wpf.Layout.SpringLayout.XK()
   at MindFusion.Diagramming.Wpf.Layout.SpringLayout.HK(Diagram I, IEnumerable V)
   at MindFusion.Diagramming.Wpf.Layout.SpringLayout.PR(Diagram I, IEnumerable V)
   at MindFusion.Diagramming.Wpf.Layout.CompositeLayout.HO(ST I, IM V)
   at A.PV.SI(IM I)
   at A.PV.FV.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
ion(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at A.PV.UV.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
ion(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at MindFusion.Diagramming.Wpf.Layout.CompositeLayout.FG.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
ion(Task task)
   at MindFusion.Diagramming.Wpf.Layout.Layout.GL.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
ion(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()


I investigated what Operation is being performed in the UI thread and it seems to be related with removing diagram items:

     …
     MindFusion.Diagramming.Wpf.dll!MindFusion.Diagramming.Wpf.ItemCollectionBase<Sys
tem.__Canon>.Remove(System.__Canon item)      Unknown
     MindFusion.Diagramming.Wpf.dll!A.ET.OI(A.ZT I)      Unknown
     MindFusion.Diagramming.Wpf.dll!A.ET.XI(A.ZT I)      Unknown
     MindFusion.Diagramming.Wpf.dll!A.PV.FV.MoveNext()      Unknown##
      …

When I disable the custom partitioning algorithm and just set CompositeLayout.CustomPartion to an empty list, the issue disappears.

Can you please explain why this issue occurs and how to avoid it?
  
Back to top
 
IP Logged
 
Slavcho
YaBB Moderator
*****
Offline


tech.support

Posts: 2532
Joined: Oct 19th, 2005
Re: Out of Memory Exception when using CompositeLayout with a large number of partitions
Reply #1 - Jul 12th, 2021 at 9:04am
Print Post  
Quote:
CompositeLayout which creates about 700 partitions


This kind of defeats the purpose of using composite layout, which is to run the polynomial-complexity layout algorithms on smaller graphs, and here the master graph is on the large side. Generally you'd want both the inner and outer graphs to be as small as possible, so partitions close to the sqrt of all items, e.g. 30 partitions x 30 nodes for diagram with 900 nodes. Anyway I've tested the sample project with some time measurements and MasterLayout replaced with SpringLayout instance -

Code
Select All
async private void layoutButton_Click(object sender, RoutedEventArgs e)
{
	var start = DateTime.Now;

	// Reset styles
	foreach (DiagramNode node in diagram.Nodes)
	{
		node.ClearValue(ShapeNode.BrushProperty);
		node.ClearValue(ShapeNode.StrokeProperty);
	}

	foreach (DiagramLink link in diagram.Links)
	{
		link.ClearValue(DiagramLink.BrushProperty);
		link.ClearValue(DiagramLink.StrokeProperty);
	}

	await ArrangeOnce();

	diagram.ResizeToFitItems(5);

	border2.BorderBrush = Brushes.Transparent;
	border3.BorderBrush = Brushes.Red;

	MessageBox.Show((DateTime.Now - start).ToString());
}

async private Task<bool> ArrangeOnce()
{
...

    var masterLayout = new SpringLayout();
    //masterLayout.CrossingLinksCost = 0;
    masterLayout.LayoutNode = OnMasterLayoutNode;
    masterLayout.LayoutLink = OnMasterLayoutLink;
    layout.MasterLayout = masterLayout;

...
    return await layout.ArrangeAsync(diagram);
} 



A 32-bit build with non-async code completed successfully in 12 seconds for 500 partitions by 2 nodes and in 1 second for 31 partitions by 33 nodes. Async code takes about double that time, I guess overhead comes from the more frequent visual tree updates happenning in LayoutNode / LayoutLink handlers. Still that's far from your 10 minutes - I guess check if some specific SpringLayout properties you are using aren't leading to that, or some slower code in LayoutNode / LayoutLink event handlers.

In release history I can see an "ArrangeAsync method of CompositeLayout now also applies MasterLayout asynchronously" item for v3.5.3, so also check if you aren't on older version than that. I've tried with current 3.6.3 assemblies.

Regards,
Slavcho
Mindfusion
  

CompositeLayout.zip ( 2532 KB | 5 Downloads )
small_master_graph.png ( 239 KB | 6 Downloads )
small_master_graph.png
large_master_graph.png ( 249 KB | 5 Downloads )
large_master_graph.png
Back to top
 
IP Logged
 
Slavcho
YaBB Moderator
*****
Offline


tech.support

Posts: 2532
Joined: Oct 19th, 2005
Re: Out of Memory Exception when using CompositeLayout with a large number of partitions
Reply #2 - Jul 12th, 2021 at 9:06am
Print Post  
and this shows the async times.
  

small_master_async.png ( 205 KB | 8 Downloads )
small_master_async.png
large_master_async.png ( 189 KB | 5 Downloads )
large_master_async.png
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1
Send TopicPrint