This bug appears only in load-balanced / remote events setups for CMS versions between 7.5 and 9.0.2 (released September 14th 2015, in which it was fixed). It’s reported as bug CMS-1405. This memory leak is relatively slow. Because of the way that ASP.NET handles cache, the application’s memory allocation will remain constant while the cache portion of it will continuously decrease ending up in an application with no cache at all.
Here’s a quick patch that fixes the issue for sites running this code:
using EPiEvents = EPiServer.Events;
public class PatchInitializer : IInitializableModule
public void Initialize(global::EPiServer.Framework.Initialization.InitializationEngine context)
context.InitComplete += context_InitComplete;
void context_InitComplete(object sender, EventArgs e)
var eps = EPiEvents.Providers.EventProviderService.Instance;
var emvField = eps.GetType().GetField("_messageValidator", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic);
if (emvField == null)
throw new InvalidOperationException("Could not locate _messageValidator variable; this patch might not be valid for the current application.");
var emv = emvField.GetValue(eps);
var timerField = emv.GetType().GetField("_sequenceCheckTimer", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic);
if (timerField == null)
throw new InvalidOperationException("Could not locate _sequenceCheckTimer variable to patch; this patch might not be valid for the current application.");
var timer = (ITimer)timerField.GetValue(emv);
throw new InvalidOperationException("Timer already enabled; this patch might not be valid for the current application.");
timer.Enabled = true;
public void Preload(string parameters)
public void Uninitialize(global::EPiServer.Framework.Initialization.InitializationEngine context)
After leaving your application running for some time, take a full user dump of the application (using through Task Manager/Right Click w3wp.exe/Create Dump File, ProcDump -ma and the like).
Open it up in WinDBG, and load the SOS extension. Being lazy, I first let the .loadby sos clr attempt loading and fail. It fails because the dump I analyze was taken off an Azure server, which has the system loaded on the D:-drive. Copy the path, sans the first character which I change to a C, and the SOS extension loads properly.
0:000> .loadby sos clr
The call to LoadLibrary(D:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos) failed, Win32 error 0n126
"The specified module could not be found."
Please check your debugger configuration and/or network access.
0:000> .load c:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos
Then I call !dumpheap -stat . After crunching for a while, it will write out a table with all the different types of objects in your application, sorted by the biggest space-takers in the bottom:
0:000> !dumpheap -stat
MT Count TotalSize Class Name
... and close to the bottom, you'll find ...
000007f8f37e98a0 3204485 102543520 EPiServer.Events.Providers.EventSequence
In this dump, I found more than 3 million instances of each of these types. Together, they take up 102+153 MB of memory. You would typically not expect any more instances than the number of remote events that could be emitted within a 5 second time span. That number is well below 3 million! That’s 255 MB wasted!