Status as of Monday, July 31, 2017

Here's a brief status update as of today.

• The support ticketing system is having issues with sending out duplicate E-Mails and not processing messages again. I directed Chris to scrap the system and replace it with similar software. I don't know when that will happen.
• Yesterday, I got to what I think is a releasable system with the new share recording code. I did some performance testing and discovered that interprocess communication was reduced by a factor of 200, leading to CPU usage reductions of 110x. Not only will this improvement be seen in preventing the database from getting behind, some of the reduced CPU load will be in the mining server as well, improving coin selection until we can get that separated into a separate program too.
• One topic that was suggested yesterday is a "dynamic fee" assignment algorithm. When shares get delayed too much, fees would increase, reducing profitability slightly until the backup of shares is processed. The advantage of this is the system would only need to respond after a few minutes of heavy load, and profitability would only be reduced until the database catches up below the threshold. Another plus is that the money earned could get put into paying someone to fix the issues themselves, which might actually make miners more in the end. Comments are welcome.
• Chris ran a projections spreadsheet that provides an optimistic outlook for the business's future, almost 10 times higher than I estimated earlier. I'm going to have a discussion with Chris about what changed, because this spreadsheet seems to indicate we can now make more profit by hiring one or more people.
• Another "comments welcome" issue is the idea of how much testing should be put into the new code. Right now, it cannot handle a lot of rare edge cases, like what happens if every single share inserter process is offline, or if a block takes more than 10 seconds to propagate across the network. Since there are already issues with the system, the big question is at what point a release should occur because the changes will result in a better state than currently exists.

The sounds very positive. I would vote for brining sooner the update and the extreme case test scenarios could be examined after the initial release. If extreme edge potential bugs are later found a subsequent release could be made. Even if it meant an increased chance of downtime cause by rare scenarios in the meantime

Regarding increase fees during extensive server load;
- a heavy X11 load would only increase fees on X11 and vise versa?
- personally I would vote against fee adjustments at least until we see how the new upgraded pool release performs under heavy load

100x times fast sounds great *fingers crossed*

A possibility to handle those burst NH rentals (bots): Is is possible to delay wamp data by amount x? With this bots would be useless.
And your normal NH renter does not get hurt

well, obviously this software improvement sounds fantastic, so you got me interested there.

not really sure what you mean by "Another plus is that the money earned could get put into paying someone to fix the issues themselves, which might actually make miners more in the end." but on the topic of share delay related dynamic fees, this would have to be algo specific of course, scrypt miners should not be penalized for x11 backlog and vise versa, maybe you should look into splitting the two stratums in the future due to the incoming hashrate over the next few months, this will help with system stability. whether its running virtually or on other hardware, ultimately i believe they should be split, but that's a whole nother issue. as it stands i don't really see how dynamic fees based on share backlog helps, miners doing their share at correct diffs would be penalized with greater fees because other miners are low diff submitting and causing a backlog which happens live.

on the topic of how much testing should be done, i would recommend a server maintenance downtime schedule until pool is where it should be. pre scheduled downtime would not piss people off as they would know about it in advance and are open to the idea of regular maintenance to overall improve service. many online services do this to maintain server stability and profitability, gameservers for example. it can go for 30 mins all the way to 8 hours, and people would just come back and get back to business as if nothing happened after the maintenance window closes.

my 2 litoshi for the moment, thanks steve.

Is the elec calculation ... ever gonna get fixed?

GenTarkin wrote:Is the elec calculation ... ever gonna get fixed?

unimportant as of right now

vhmanu wrote:100x times fast sounds great *fingers crossed*

A possibility to handle those burst NH rentals (bots): Is is possible to delay wamp data by amount x? With this bots would be useless.
And your normal NH renter does not get hurt

Yes! I also like this idea and had suggested a variation of it seeing how the Nicehash rental market has becoming dominated by bots tied to real-time pricing data.

I believe hashrate volatility and its "abuse" can be defeated and/or smoothed out by only displaying and outputing only
= "average profitability over 1 hour" for both pool algo and all coins.

It seems that very large amounts of hashrate exist outside of machine miner owners (such as on nicehash), and if those hashrates are being used in unstructured and "abusive" manners, then only broadcasting "average hourly" profitability data for pool and coins might level all the nonsense out.

encouraging aggressive and desructive mining behavior will end the game all too quickly for all of us.

These ideas are kind of an expansion of Greg Harding's original suggestion to take out API data when traffic is high.

I believe this solution if implemented would result in:
- less hashrate volatility due to less jumps in aggressive bot driven rentals (which account for significant hashrate traffic and spikes)
- more profit due to overall mining pool being stabilized from the input side
- higher profit margins mean more income for pool operator
- less profits for nicehash as it is in their interest to see people rent hash at higher rates
- encourage better mining etiquite and lay the stage for a long lasting ecosystem based on stability, NOT aggressiveness

I don't have the proof or the background, but i think overall in for all large systems, optimal rates for growth are achieved via stability and equilibrium of input and output stages.

I don't see why the realm of mining wouldn't also be governed by the same law of numbers that seem to be universal.

As the mining operators, you guys actually have a role and potential responsibility to create conventions now that will lead to a long term industry that doesn't eat itself up.

My 2 satoshis.

Agree with Greg / wkying, I think limiting the API data or delaying it during peak traffic situations would likely help to deal with them. The shorter spikes in profitability would get 'smoothed over' as wkying suggested, thus keeping the bots in check and keeping hashrate reasonable. I just don't see how a dynamic fee is going to change anything, because afaik these bots renting hash aren't calculating mining pool fees correct? So unless whoever is controlling said bot decides to completely shut things down, that bot is going to keep renting the same hash regardless. And thus, only the pool operators really win in that scenario. The API data suggestion would at least keep the bots from being as aggressive, which would keep the spikes in check. But beyond that, I'd certainly hope any dynamic fees would be incurred on the specific algo and not across the board.

wkying79 wrote:

vhmanu wrote:100x times fast sounds great *fingers crossed*

A possibility to handle those burst NH rentals (bots): Is is possible to delay wamp data by amount x? With this bots would be useless.
And your normal NH renter does not get hurt

Yes! I also like this idea and had suggested a variation of it seeing how the Nicehash rental market has becoming dominated by bots tied to real-time pricing data.

I believe hashrate volatility and its "abuse" can be defeated and/or smoothed out by only displaying and outputing only
= "average profitability over 1 hour" for both pool algo and all coins.

It seems that very large amounts of hashrate exist outside of machine miner owners (such as on nicehash), and if those hashrates are being used in unstructured and "abusive" manners, then only broadcasting "average hourly" profitability data for pool and coins might level all the nonsense out.

encouraging aggressive and desructive mining behavior will end the game all too quickly for all of us.

These ideas are kind of an expansion of Greg Harding's original suggestion to take out API data when traffic is high.

I believe this solution if implemented would result in:
- less hashrate volatility due to less jumps in aggressive bot driven rentals (which account for significant hashrate traffic and spikes)
- more profit due to overall mining pool being stabilized from the input side
- higher profit margins mean more income for pool operator
- less profits for nicehash as it is in their interest to see people rent hash at higher rates
- encourage better mining etiquite and lay the stage for a long lasting ecosystem based on stability, NOT aggressiveness

I don't have the proof or the background, but i think overall in for all large systems, optimal rates for growth are achieved via stability and equilibrium of input and output stages.

I don't see why the realm of mining wouldn't also be governed by the same law of numbers that seem to be universal.

As the mining operators, you guys actually have a role and potential responsibility to create conventions now that will lead to a long term industry that doesn't eat itself up.

My 2 satoshis.

A superb post!

In case Steve is not in favor of a permanent delay in the profit data feed, perhaps we could either:

- Set as a temporary measure until the system has loads of headroom to spare
or
- Have the delay kick in once critical performance metrics are exceeded and drop again as the system recovers

Bravo to you guys for the great recommendations, and cheers to Steve for reaching out to us for ideas!

It seems to me everyone is in agreement the Bots need to be dealt with! Do whatever it takes to get the Bots out of the pool and see how it runs after that.. A quick easy fix is just delay the live profits by 30 minutes or whatever it takes.. Then re-assess the situation!