Ouch, mommy. Around noon today, I noticed my Web server becoming increasingly sluggish. Telnet to 110 and 25 were working fine, but Apache was non-responsive, and SSH was giving me the ol' silent treatment as well. After a seriously lengthy remote reboot process, I logged in and had a look at
top, and there was a mass of
apache processes glued to the top of the readout.
In the last couple of days I've been getting a bunch more music submissions and comments from the station than usual, so I had a quick look at the logs for the EpiphanyRadio site -- and noticed that today's file was frigging enormous. I hadn't been running Webalizer stats on the site since the most recent rewrite, so I ran the last couple of months of logs, and daily usage was the normal 100 visitors, 3000 hits or so per day -- all the way up to today. Today's stats around noon were 4000 visitors, 200,000 hits or so (Note: The figure of 7000 I had here earlier for visitors was actually the 'pages' column in the report).
I was figuring a nice DDOS like we had on the mail server awhile back, so I just threw up a static page in place of the Ajaxey single-page Web-app that's normally there. The site is a full-on Dojo app that does this continuous polling to keep the number of listeners and currently playing track updated on the page, and rotates the banners and album cover images.
I was on my way out the door to my nephew's birthday party, but I did have a quick look at the mirror servers for the stream, and oddly, they were all showing full-to-capacity usage. All 80 128Kbps connections were full -- a hell of a lot of actual people actually listening to the stream.
So, after supper tonight, I added a custom log directive to the server config to start grabbing the referers and sticking them in a separate file. Tailing that dude, now I can see that all this lovely traffic is from EpiphanyRadio showing up on Digg (1117 diggs as of now). Wa-hoo!
Guess I got off pretty easy -- with the static page, even the rinky dink box it's running on is basically idling. I'll post a final note on what the stats looks like when the wave totally passes.
Update: Must be the much-vaunted Digg Effect -- it looks like EpiphanyRadio is also sitting in the top ten on del.icio.us/popular.