All of a sudden the reject share rate went up rapidly...

Discuss issues with mining rigs and connectivity issues with experienced Prohashing miners.
Forum rules
Welcome to the mining rig and connectivity support forum!

This forum is for discussing issues with mining rigs and connectivity issues with experienced PROHASHING miners. This is a great place to ask questions about connecting specific hardware to PROHASHING.

Remember, PROHASHING employees do not closely monitor the forums like we do the official support channels, so this forum's purpose is to connect you with other PROHASHING miners who have experience with similar hardware/issues. If you have connectivity issues you are unable to resolve here on the forum, please submit a ticket through the official support channels.

For the full list of PROHASHING forums rules, please visit https://prohashing.com/help/prohashing- ... rms-forums.
User avatar
Steve Sokolowski
Posts: 4585
Joined: Wed Aug 27, 2014 3:27 pm
Location: State College, PA

Re: All of a sudden the reject share rate went up rapidly...

Post by Steve Sokolowski » Thu Sep 08, 2022 7:41 pm

DeafEyeJedi wrote: Thu Sep 08, 2022 6:27 pm
Steve Sokolowski wrote: Thu Sep 08, 2022 6:21 pm
DeafEyeJedi wrote: Thu Sep 08, 2022 6:11 pm Hello again... anyone else notice their rigs running ETHash getting all shares rejected as of 15 minutes ago or so? All three are in the reds. I’ve also tried updating and rebooting to no avail. Hope this is all temporarily. Thanks!
We are aware of this issue and are currently investigating. It likely has something to do with the new ETH network clients that were released today to prepare for the Merge.
Copy that. Thanks, Steve!
We resolved the issue successfully. Because it's so late, I'll report exactly what happened tomorrow. Mining is back online, and sorry for the inconvenience!
User avatar
DeafEyeJedi
Posts: 32
Joined: Wed May 25, 2022 11:38 pm
Location: USA

Re: All of a sudden the reject share rate went up rapidly...

Post by DeafEyeJedi » Thu Sep 08, 2022 8:09 pm

Steve Sokolowski wrote: Thu Sep 08, 2022 7:41 pm
DeafEyeJedi wrote: Thu Sep 08, 2022 6:27 pm
Steve Sokolowski wrote: Thu Sep 08, 2022 6:21 pm

We are aware of this issue and are currently investigating. It likely has something to do with the new ETH network clients that were released today to prepare for the Merge.
Copy that. Thanks, Steve!
We resolved the issue successfully. Because it's so late, I'll report exactly what happened tomorrow. Mining is back online, and sorry for the inconvenience!
Thanks so much, Steve! Looking forward to your initial report tomorrow. Happy Mining!
Baseball and Mining. HaCkint0sh and Crypto. Magic Lantern never dies! :geek:
User avatar
Steve Sokolowski
Posts: 4585
Joined: Wed Aug 27, 2014 3:27 pm
Location: State College, PA

Re: All of a sudden the reject share rate went up rapidly...

Post by Steve Sokolowski » Fri Sep 09, 2022 6:56 am

DeafEyeJedi wrote: Thu Sep 08, 2022 8:09 pm
Steve Sokolowski wrote: Thu Sep 08, 2022 7:41 pm
DeafEyeJedi wrote: Thu Sep 08, 2022 6:27 pm

Copy that. Thanks, Steve!
We resolved the issue successfully. Because it's so late, I'll report exactly what happened tomorrow. Mining is back online, and sorry for the inconvenience!
Thanks so much, Steve! Looking forward to your initial report tomorrow. Happy Mining!
Thanks for your patience.

We discovered that the geth developers removed a field in their API calls that allow pools to determine how many are in, and how much value there is, of transactions in the next block. The removal of the field caused our daemon management software to fail to send new block notifications to the mining servers.

After some time, Vance restarted the mining servers to prepare for Autolykos2's release. At that point, because no blocks were being received from ethash daemons, the coins went into error because their DAGs couldn't be computed. The DAGs couldn't be computed because it wasn't known which block to compute the DAG for.

That issue was resolved quickly, but an additional problem was discovered that required an hour to troubleshoot. There was a bug in the daemon managers where, when a coin's host was changed (as Steven did when he reverted to the previous version of geth,) the new server's daemon manager would not retrieve the data from the database and therefore would not be aware that the coin had been moved to that server. Restarting the daemon managers on all servers resolved that issue.

I added a ticket and assigned to Vance that, before the end of the month, he will deploy a fix to the daemon managers so that, in the rare case that a coin is moved between servers, the daemon managers will now refresh from the database the new coin locations and respond accordingly without needing to be restarted.
User avatar
DeafEyeJedi
Posts: 32
Joined: Wed May 25, 2022 11:38 pm
Location: USA

Re: All of a sudden the reject share rate went up rapidly...

Post by DeafEyeJedi » Fri Sep 09, 2022 1:56 pm

Steve Sokolowski wrote: Fri Sep 09, 2022 6:56 am
DeafEyeJedi wrote: Thu Sep 08, 2022 8:09 pm
Steve Sokolowski wrote: Thu Sep 08, 2022 7:41 pm
We resolved the issue successfully. Because it's so late, I'll report exactly what happened tomorrow. Mining is back online, and sorry for the inconvenience!
Thanks so much, Steve! Looking forward to your initial report tomorrow. Happy Mining!
Thanks for your patience.

We discovered that the geth developers removed a field in their API calls that allow pools to determine how many are in, and how much value there is, of transactions in the next block. The removal of the field caused our daemon management software to fail to send new block notifications to the mining servers.

After some time, Vance restarted the mining servers to prepare for Autolykos2's release. At that point, because no blocks were being received from ethash daemons, the coins went into error because their DAGs couldn't be computed. The DAGs couldn't be computed because it wasn't known which block to compute the DAG for.

That issue was resolved quickly, but an additional problem was discovered that required an hour to troubleshoot. There was a bug in the daemon managers where, when a coin's host was changed (as Steven did when he reverted to the previous version of geth,) the new server's daemon manager would not retrieve the data from the database and therefore would not be aware that the coin had been moved to that server. Restarting the daemon managers on all servers resolved that issue.

I added a ticket and assigned to Vance that, before the end of the month, he will deploy a fix to the daemon managers so that, in the rare case that a coin is moved between servers, the daemon managers will now refresh from the database the new coin locations and respond accordingly without needing to be restarted.
Fantastic updates. Good to hear. I appreciate the follow up, Steve!
Baseball and Mining. HaCkint0sh and Crypto. Magic Lantern never dies! :geek:
Post Reply