Miners are centralizing into huge farms
Posted: Tue Jul 09, 2019 5:48 pm
Earlier in the weekend, I posted a review of the July 3-4 meeting that we held to discuss Prohashing's path forward over the next few months. The purpose of the meeting was primarily to answer the following question: why, given that almost everything else is exactly the same, are there fewer customers now than there were a year and a half ago?
After a lot of discussion, we realized that we had enough evidence to conclude the answer is that miners are centralizing into huge farms, owned by fewer people. This development should be alarming to community members and coin developers who are constantly pursuing decentralization of hashrate.
The logic for this conclusion was simple: if all of the initial conditions of an experiment are the same, then the outcome will be repeatable. If the outcome is different, then it follows that one of the initial conditions must also be different. We researched an exhaustive list of initial conditions and found that they are all the same except for one. There is either a cause that have never appeared in any Google search results about mining, or mining has become increasingly centralized, making pools less necessary.
In this article, I'll present a list of all the conditions we identified and why we believe that none of them have changed. Maybe you'll be able to post a reply and point out a condition that we're missing. Here's what we think has not changed:
-----------------------------------
The system is not as capable
In December 2017, shortly before peak users and hashrate were hit, we closed the site to new registrations. We parallelized the mining servers so that the system's capacity increased from 12,000 simultaneous miners to about 240,000 miners. Then, on the day that NiceHash lost its money, we released the update that allowed us to accept all the customers who wanted to join us.
The number of workers increased to about 20,000 in December 2017 and January 2018, and then fell back to 12,000 now - exactly the same amount as there had been, despite the system having twenty times the current capacity.
The system has more bugs
This condition is perhaps the most weird. Two years ago, there were days where the system would go offline for hours at a time, or not record shares, so that Chris would have to perform share corrections on a daily basis. Share recording would fall behind by hours, and at one point there were hundreds of support tickets backlogged.
One change I'm 100% confident has not occurred is that the system has more bugs than it did two years ago. On Sunday night, the number of support tickets submitted was exactly zero, the first time that had ever occurred. The state of the system now, compared to two years ago, is not even comparable in terms of stability.
There are fewer features
If the system had removed features that customers used, then it would be understandable that those customers might decide to leave. However, there aren't any features that were removed, and we added a number of features like grouping. As demonstrated in many recent polls, there aren't any new features that are in great demand. litecoinpool.org, which has not changed in years and which offers very basic functionality, maintained or grew its hashrate at the same time ours remained stable.
The market has shrunk
Last week, hashrate for the bitcoin network finally rebounded to its all-time high that had been set during the last cycle, and litecoin hashrate is in a similar position. Despite that, hashrate in both algorithms at Prohashing is about half what it was at the time of peak hashrate the last time around. The market did not shrink, but the hashrate is elsewhere.
Profitability is too low
To eliminate the possibility that low profits were deterring customers from mining with us, we started scrypt and SHA-256 miners at several pools to compare profits earned by mining at the pools. We focused on litecoinpool because their hashrate was so high (at 40 TH/s), and determined that our miners earned 1% more mining here than they did mining at litecoinpool. Even if we were only able to equal their profitability, it would still be logical that miners would prefer the pool where where they can be paid in any coin.
The site has poor marketing
One of the conclusions we drew from the July 3-4 meeting was that Prohashing has poor marketing. We decided to budget $50,000 to attack this problem head-on, by producing a lot of content and positioning these forums as a widely-used and viable alternative to the censored bitcointalk.org forums. Someone is going to come onboard to produce "how to" YouTube videos for every type of miner. But this lack of marketing can't be a changed factor because we never performed much marketing in the first place. In fact, the charts page is largely the same as it appeared during the higher-visitation period, except recent work on performance meant that it took about twice as long to load back then.
Mining rigs have changed and become incompatible
An interesting idea that was brought up was that new mining rigs are incompatible with the system. We did receive some support tickets from customers about rigs that didn't perform as well, but those reported issues were addressed and there have been no additional reports since then. Additionally, testing performed by us with newer miners, like the Antminer E3, failed to reveal any incompatibilities. Furthermore, since the cycle is just getting started, many mining manufacturers do not have to funds, or have not yet had the time, to design and release new hardware.
-----------------------------------
Given that we eliminated all of the factors we could identify as having remained unchanged since the last cycle, we needed to search for things that were both reasonably probable and which we had not already considered. The only answer that remained was that mining is centralizing into large mining farms. If the price of electricity is rising or if margins are getting squeezed, it may be that locating miners in huge warehouses is now the only way to survive. These farms, of course, would be solo mining because they have enough hashrate to eliminate the possibility of never finding any blocks due to variance.
If correct, then the implications of this theory are disturbing. The theory implies that 60% of the hashrate owned by independent miners during the last cycle has now gone offline, replaced by huge mining farms. In the recent Bitcoin Cash fork, we saw how two large pools colluded to execute a 51% attack against the network, which caused mega-exchange Coinbase to lock its wallet while it performed an investigation into why transactions has rolled back. In that case, it was obvious who had executed the attack. Now, the evidence demonstrates that hashrate has moved to private mining farms that anonymously submit blocks to the networks, and which for many coins may be more powerful than all of the known pools.
Fortunately, we have more than enough income to prosper even at current levels, and will be able to continue our push to improve our services. However, others should be as concerned as we are about the worrisome trend of miners moving to independent, anonymous farms, where they are stripped of any real accountability while gathering more hashrate every day.
After a lot of discussion, we realized that we had enough evidence to conclude the answer is that miners are centralizing into huge farms, owned by fewer people. This development should be alarming to community members and coin developers who are constantly pursuing decentralization of hashrate.
The logic for this conclusion was simple: if all of the initial conditions of an experiment are the same, then the outcome will be repeatable. If the outcome is different, then it follows that one of the initial conditions must also be different. We researched an exhaustive list of initial conditions and found that they are all the same except for one. There is either a cause that have never appeared in any Google search results about mining, or mining has become increasingly centralized, making pools less necessary.
In this article, I'll present a list of all the conditions we identified and why we believe that none of them have changed. Maybe you'll be able to post a reply and point out a condition that we're missing. Here's what we think has not changed:
-----------------------------------
The system is not as capable
In December 2017, shortly before peak users and hashrate were hit, we closed the site to new registrations. We parallelized the mining servers so that the system's capacity increased from 12,000 simultaneous miners to about 240,000 miners. Then, on the day that NiceHash lost its money, we released the update that allowed us to accept all the customers who wanted to join us.
The number of workers increased to about 20,000 in December 2017 and January 2018, and then fell back to 12,000 now - exactly the same amount as there had been, despite the system having twenty times the current capacity.
The system has more bugs
This condition is perhaps the most weird. Two years ago, there were days where the system would go offline for hours at a time, or not record shares, so that Chris would have to perform share corrections on a daily basis. Share recording would fall behind by hours, and at one point there were hundreds of support tickets backlogged.
One change I'm 100% confident has not occurred is that the system has more bugs than it did two years ago. On Sunday night, the number of support tickets submitted was exactly zero, the first time that had ever occurred. The state of the system now, compared to two years ago, is not even comparable in terms of stability.
There are fewer features
If the system had removed features that customers used, then it would be understandable that those customers might decide to leave. However, there aren't any features that were removed, and we added a number of features like grouping. As demonstrated in many recent polls, there aren't any new features that are in great demand. litecoinpool.org, which has not changed in years and which offers very basic functionality, maintained or grew its hashrate at the same time ours remained stable.
The market has shrunk
Last week, hashrate for the bitcoin network finally rebounded to its all-time high that had been set during the last cycle, and litecoin hashrate is in a similar position. Despite that, hashrate in both algorithms at Prohashing is about half what it was at the time of peak hashrate the last time around. The market did not shrink, but the hashrate is elsewhere.
Profitability is too low
To eliminate the possibility that low profits were deterring customers from mining with us, we started scrypt and SHA-256 miners at several pools to compare profits earned by mining at the pools. We focused on litecoinpool because their hashrate was so high (at 40 TH/s), and determined that our miners earned 1% more mining here than they did mining at litecoinpool. Even if we were only able to equal their profitability, it would still be logical that miners would prefer the pool where where they can be paid in any coin.
The site has poor marketing
One of the conclusions we drew from the July 3-4 meeting was that Prohashing has poor marketing. We decided to budget $50,000 to attack this problem head-on, by producing a lot of content and positioning these forums as a widely-used and viable alternative to the censored bitcointalk.org forums. Someone is going to come onboard to produce "how to" YouTube videos for every type of miner. But this lack of marketing can't be a changed factor because we never performed much marketing in the first place. In fact, the charts page is largely the same as it appeared during the higher-visitation period, except recent work on performance meant that it took about twice as long to load back then.
Mining rigs have changed and become incompatible
An interesting idea that was brought up was that new mining rigs are incompatible with the system. We did receive some support tickets from customers about rigs that didn't perform as well, but those reported issues were addressed and there have been no additional reports since then. Additionally, testing performed by us with newer miners, like the Antminer E3, failed to reveal any incompatibilities. Furthermore, since the cycle is just getting started, many mining manufacturers do not have to funds, or have not yet had the time, to design and release new hardware.
-----------------------------------
Given that we eliminated all of the factors we could identify as having remained unchanged since the last cycle, we needed to search for things that were both reasonably probable and which we had not already considered. The only answer that remained was that mining is centralizing into large mining farms. If the price of electricity is rising or if margins are getting squeezed, it may be that locating miners in huge warehouses is now the only way to survive. These farms, of course, would be solo mining because they have enough hashrate to eliminate the possibility of never finding any blocks due to variance.
If correct, then the implications of this theory are disturbing. The theory implies that 60% of the hashrate owned by independent miners during the last cycle has now gone offline, replaced by huge mining farms. In the recent Bitcoin Cash fork, we saw how two large pools colluded to execute a 51% attack against the network, which caused mega-exchange Coinbase to lock its wallet while it performed an investigation into why transactions has rolled back. In that case, it was obvious who had executed the attack. Now, the evidence demonstrates that hashrate has moved to private mining farms that anonymously submit blocks to the networks, and which for many coins may be more powerful than all of the known pools.
Fortunately, we have more than enough income to prosper even at current levels, and will be able to continue our push to improve our services. However, others should be as concerned as we are about the worrisome trend of miners moving to independent, anonymous farms, where they are stripped of any real accountability while gathering more hashrate every day.
