Stability Issue

Soooo.... Did someone piss off an old gypsy chick last night? Maybe around midnight?

I was mining my backup pool for much of the day yesterday... only hit and miss connecting here for work.

What's frustrating is that these issues always seem to occur on Mondays. The "weird 1 satoshi orders" problem happened on a Monday morning, and these connection issues happened on a Monday too. Perhaps I am just placing undue weight on things that happen on Mondays because we can fix issues that happen on Saturdays or SUndays quickly.

The disks are barely responding at 1MB/s, despite all the performance optimizations made over the weekend. It seems that, once we fix one problem, another process that was bottlenecked by the last performance blocker just speeds up to use the disks at full capacity once again. There doesn't seem to be an end to this, and I don't know why all of a sudden this issue has happened now.

OK, I found a problem.

For some reason, someone had set the server to poll all daemons every five seconds to confirm that we have the latest blocks. Since most daemons send blocks immediately without polling, that was unnecessary. I reduced the polling frequency to 120s and it reduced CPU usage from 50% to 6%. Disk usage also seems to be far lower.

Let's see how this works. It seems like today will be a banner day if we can keep the system stable, as we are averaging more than seven cents.