Lately I have been experiencing that the mod-mono-server2 process running on the server hosting the video upload web-service (I blogged about this in my last post) has been consuming a lot of memory. Once, it even reached the point where all memory (1.5GB) of the server was consumed and the Linux OOM-killer killed the mod-mono-server2 process. At this point apache failed to restart it. If you are interested you can see the kernel log here and the apache log here. The mono version I am using is mono-apache-server2 1.9.1-2 (Ubuntu 8.10 Intrepid).
I googled the issue and found some information on the mod_mono page:
Under high load, mono process consumes a lot of memory, website stops responding
These symptoms have been reported, but their underlying causes are not known. Set the MonoAutoRestartMode, MonoAutoRestartRequests, MonoMaxActiveRequests, and MonoMaxWaitingRequests directives as described earlier to limit the lifetime of the mono process and to restrict the concurrency happening in the server.
The above describes my issue pretty well, except for the “under high load” part. The video transcoding service is still in beta and is only used by one customer so far. It serves around 500 requests a day, which is not a lot, so I wouldn’t expect this kind of behavior. My guess is that every time a file is uploaded using the UploadFile webmethod additional memory is allocated, but not properly released/reused by mono.
As a workaround for this I will have the mod-mono-server2 process restart restart every 6th hour. This can be done by adding the following lines to Update: This did not work, caused my site to hang entirely every 6th hour. /etc/apache2/mods-enabled/mod_mono.conf:
-
MonoAutoRestartMode Time
-
MonoAutoRestartRequests 00:06
Hopefully this will limit the memory usage, for now. I would appreciate feedback from anyone who have experienced similar problems on mono.
