Sunday, July 28, 2013

Apache zombie process problems.

The Apache server is being a dick. It keeps dying with unkillable zombie processes; these don't serve data, but they do keep their hold on port 80. The only workaround is to reboot the server. This is, of course, ridiculous. I've put a ticket in with Linode. Anyone got ideas? It's just the standard Ubuntu apache2 package.

Update: Apache was causing kernel oopses. (Things that make you go "wtf.") Linode suggest using kernel 3.9-linode instead of 2.6-linode (the default for Ubuntu 10.04, which our box runs on). We'll see how that goes next reboot. Might even fix the white-screening blog too.

Friday, July 12, 2013

Wiki slow, we're being hammered.

The wiki has been really slow all day. The Apache server and the Squids appear quite happy with life - but the load balancer is reporting huge traffic spikes. More information as I have it.

(The blogs are unaffected, as they're served directly from the Apache server.)

Update: Here's the load balancer connections per second. (The gaps are when logging is killed to increase throughput or something.) Note the bit where we go from a few tens of queries a second up to 800 a second.