I was helping out a colleague with a nasty ASP .NET problem the other day. The problem we were dealing with was related to an application pool that became irresponsive once in a while. The system event log indicated some problems, but we couldn't figure out how the events logged were related to the problem itself - as usual, the event descriptions were less than helpful. At that time, the only thing I could suggest was looking up Google groups and the MS KB for any articles that could provide insight into the strange thing happening.
Several days later, I followed up with the guy and he told me they found the root cause. I am posting it here with his permission.
So, the root cause was related to the application pool being recycled too often (every 20 min. or so). Given that it was a production Web server under quite a high load, the situation was like this:
- IIS was still trying to shut down the old instance still dealing with a queue of requests
- While the old instance was still running, the new instance of the pool couldn't be started
- As a consequence, new requests were plainly rejected with a non-descriptive ASP .NET error
Morale: Watch out for IIS application pool recycle and health settings. When improperly configured, these can cause you a lot of trouble.