Methods of .NET self-annihilation part 1: Frequent Web Application Restarts

There are many different ways you may inadvertently kill your web application’s presence online. Here I try to list the major reasons an IIS site running .NET may underperform.

The trickiest cases to prevent are the time bombs. That code that runs really well in development, but starts misbehaving when put in production. Not immediately of course, but after a few months, making it really hard to track down which change brought the site under. I will get to those, but not in this post.

This post is the first in a series of posts. It describes a class of problems with the very descriptive name:

Frequent Web Application Restarts

The obvious case when this happens is when w3wp.exe changes PID. This is not the entire truth: the observant developer have probably seen that an application can restart without the worker process launching again under a new process id. This happens for example when you change web.config.

An application can restart without having to change the worker process because what really happens is that its Application Domain is unloaded and a new one replaces it within the w3wp.exe process. You see, it does not map to an application; it maps to the concept Application Pool, and as such, it may run multiple applications. Each application run in their own Application Domain. Microsoft’s Scott Forsyth blogged about this almost 9 years ago, it’s still true.

For troubleshooting this class of problems, my colleague Mattias Lövström created a module that you just drop into your /bin/-folder and it will generate an Event Log message each time the application restarts, whether or not it brings the application pool with it.

Application restarts are bad because they drop all caches and forces a reinitialization of all components. This is costly because for the duration of the restart, your application will serve absolutely zero requests, all while incoming requests fill up the thread pool workers and possibly the queue waiting to be served.

Tools

Install Mattias Lövström’s Application Restart Detector. It will emit an Event Log message when the Application Domain unloads, even if the new application re-launches in the same process.

If your w3wp.exe process crashes, especially predictably/frequently, explore ExceptionTrace.exe, a part of the same Mattias’ StackDump package.

AppPoolSettings_2Also make sure to configure the Advanced Settings for your application pool so that it writes a log message on recycle events. Open up IIS Manager and set all of the items under “Generate Recycle Event Log Entry” to true. This helps you track if and why your application restarts.

 

Leave a Reply

Your email address will not be published. Required fields are marked *