Website issues earlier today

Steve Sokolowski avatar
Steve Sokolowski
Archived posts: 4559
Joined: Aug 27, 2014

Website issues earlier today

by Steve Sokolowski · · Post #27890

There was a period of website issues earlier in the morning. The issues were caused by three problems.

The first issue is that the monitoring system for the website failed to trigger, for an unknown reason. The mining server monitors have reliably triggered when there have been issues with the mining servers. The failure of the monitoring system caused Chris not to notice there was an issue.

The second issue is that a lot of people were entering invalid passwords, perhaps in some sort of attempted hack. The system performed as designed, banning the users, but the ban code had only been tested to be secure, not fast. We released code this evening to make banning users who enter lots of invalid passwords 100 times faster.

The third issue may be that there is a hardware fault developing in the website's hypervisor. If the fault recurs, we now have an extra computer that we can move the website to without having to wait for additional servers to arrive.

As far as we know, the system is back to normal operation. During the entire time, the mining servers functioned as normal, and therefore no money was lost and no share corrections are needed.
beridzee
Archived posts: 4
Joined: Jan 31, 2021

Re: Website issues earlier today

by beridzee · · Post #27891

Steve Sokolowski wrote: There was a period of website issues earlier in the morning. The issues were caused by three problems.

The first issue is that the monitoring system for the website failed to trigger, for an unknown reason. The mining server monitors have reliably triggered when there have been issues with the mining servers. The failure of the monitoring system caused Chris not to notice there was an issue.

The second issue is that a lot of people were entering invalid passwords, perhaps in some sort of attempted hack. The system performed as designed, banning the users, but the ban code had only been tested to be secure, not fast. We released code this evening to make banning users who enter lots of invalid passwords 100 times faster.

The third issue may be that there is a hardware fault developing in the website's hypervisor. If the fault recurs, we now have an extra computer that we can move the website to without having to wait for additional servers to arrive.

As far as we know, the system is back to normal operation. During the entire time, the mining servers functioned as normal, and therefore no money was lost and no share corrections are needed.

website is down, how long does it take to recover it?