Status as of Monday, March 16

Here's today's status:

• Chris successfully added all coins for all markets and all exchanges. My understanding is that the number of mining coins is now around 150. Some of the coins immediately went into error because they were misconfigured, so as Chris fixes those errors we will have even more choices.
• Orphan rates are very high right not because I discovered an issue with block submission. Miners mining a coin should immediately be switched to a different coin when a block is found, rather than possibly submitting the same block twice, which will result in an orphaned block. I fixed this issue and it should result in a significant reduction in orphan rates, and therefore an increase in profitability, once Chris deploys the change.
• This weekend, I made some enhancements to the website that Chris has yet to release. The font is changed to be more consistent, I added more documentation, and I modified some of the charts. "Last found blocks" should perform more quickly now, and "recent shares" has been replaced with "average hashrate for the past five minutes." I discovered that "recent shares" was implemented incorrectly anyway and was double-counting shares (in the chart, not for payment) in some cases, resulting in extremely high numbers of shares where many coins were mined. This will be fixed when Chris gets around to certifying the release.
• Merge mining was the most difficult thing I ever did in my entire life; it took about 140 hours to get anything working at all because there is no documentation on it. I worry that adding multiple coin merge mining could result in another bottomless pit of work that results in little benefit. For example, Leafcoins only provide 0.03 cents and would require us to add more work restarts. If we didn't issue restarts and only provided new blocks for these merge-mined coins when the primary coin has a new block, we would only earn 1/3 of that. I'd be surprised if multiple merge mined coins could increase profitability by even 1%, and the company might lose money due to slippage. Still, I'll investigate this at rootdude's request and see whether it is feasible or not.
• I'd be interested in hearing more about this issue of miners who can't deal efficiently with work restarts. It seems to me that we could add a new password argument, something like "s=0.03," where s is the "spin up time" until miners reach full speed. The argument would allow the software to assign miners to coins that result in the best hashrate. If there are no miners available with a lower "spin up time," then your earnings would be docked slightly to compensate other miners for the profitability decrease caused by that. It's also possible that this setting could be deduced automatically by the pool. Such a setting would allow the pool to automatically compute the optimal choice between lost money due to work restarts and lost money due to not mining the most profitable coin.
• At this point, we need to keep the changes small for several reasons. First, now that we have reached feature-completion, we don't want to introduce bugs that could cause a danger to stability, and changes require a lot of testing. More importantly, we have reached a point where we need to turn to marketing so that we can prove whether the pool will be successful or not. With the clock ticking until the money runs out, results need to be obtained within 3-6 months to justify the opportunity cost of not obtaining jobs working for someone else. Once we can obtain the hashrate required to prove the pool's potential, we can go back to aggressive development, especially in the x11 area. The "spin up time" suggestion may be important enough to take time away from marketing to implement immediately, however, depending on user comments.

Here's one vote in favor of the spinup time config.

kires wrote:Here's one vote in favor of the spinup time config.

Kires, if you had to guess, how long do you think it takes for your miners to spin up after a restart? Is it 0.2 seconds? 0.5?

Steve Sokolowski wrote:Here's today's status:

• Chris successfully added all coins for all markets and all exchanges. My understanding is that the number of mining coins is now around 150. Some of the coins immediately went into error because they were misconfigured, so as Chris fixes those errors we will have even more choices.
  
  Nice - I already see a lot more interesting coins popping up to the top of the profitability list which is cool.
• Orphan rates are very high right not because I discovered an issue with block submission. Miners mining a coin should immediately be switched to a different coin when a block is found, rather than possibly submitting the same block twice, which will result in an orphaned block. I fixed this issue and it should result in a significant reduction in orphan rates, and therefore an increase in profitability, once Chris deploys the change.
  
  Dig.
• This weekend, I made some enhancements to the website that Chris has yet to release. The font is changed to be more consistent, I added more documentation, and I modified some of the charts. "Last found blocks" should perform more quickly now, and "recent shares" has been replaced with "average hashrate for the past five minutes." I discovered that "recent shares" was implemented incorrectly anyway and was double-counting shares (in the chart, not for payment) in some cases, resulting in extremely high numbers of shares where many coins were mined. This will be fixed when Chris gets around to certifying the release.
  
  Double Dig.
• Merge mining was the most difficult thing I ever did in my entire life; it took about 140 hours to get anything working at all because there is no documentation on it. I worry that adding multiple coin merge mining could result in another bottomless pit of work that results in little benefit. For example, Leafcoins only provide 0.03 cents and would require us to add more work restarts. If we didn't issue restarts and only provided new blocks for these merge-mined coins when the primary coin has a new block, we would only earn 1/3 of that. I'd be surprised if multiple merge mined coins could increase profitability by even 1%, and the company might lose money due to slippage. Still, I'll investigate this at rootdude's request and see whether it is feasible or not.
  
  Probably true. After mining at GMC for 10 days, I didn't see much benefit from coins mined beyond DOGE. VIA and PTC get pounded hard by merged-mining dumps. Don't spend too many CPU cycles considering doing the extra work, my eyeballs tell me the benefit wouldn't be worth the effort.
• I'd be interested in hearing more about this issue of miners who can't deal efficiently with work restarts. It seems to me that we could add a new password argument, something like "s=0.03," where s is the "spin up time" until miners reach full speed. The argument would allow the software to assign miners to coins that result in the best hashrate. If there are no miners available with a lower "spin up time," then your earnings would be docked slightly to compensate other miners for the profitability decrease caused by that. It's also possible that this setting could be deduced automatically by the pool. Such a setting would allow the pool to automatically compute the optimal choice between lost money due to work restarts and lost money due to not mining the most profitable coin.
  
  
  Now this is a novel new approach to cutting down the work restarts. From the console on my side, when we do one of these quick switches, BFGMiner does a flush-work on my side. On my rigs with 4 cubes, the re-initialization of each die happens reasonably quickly and the rig spins back up in a reasonable period of time. However, 4 of my rigs are 6 cubes - meaning the flushes take 50% longer, and often I see that a coin switch occurs during a re-initialization meaning the rigs sit initializing longer than a share can be submitted = lost mining time. I definitely want my rigs mining coins above and beyond the going rate for LTC, but there's definitely a point where I'm losing valuable shares and thus making far less than I could just mining straight coins. Right now, it's really looking like a toss-up between the profitability of mining straight LTC (and having average luck) and mining low DIFF coins and switching. Have you guys given any thought to allowing us to select the coins we want to mine and only switching to those coins when it makes sense to? In other words, I'd select from the 150 odd coins available and only have those in the rotation? That way I could judge for myself what to mine and allow the pool to make it's normal decisions on that subset? What'd I'd likely do is select LTC of course, along with the other moderately difficult coins... perhaps resulting in a higher average hashrate.
• At this point, we need to keep the changes small for several reasons. First, now that we have reached feature-completion, we don't want to introduce bugs that could cause a danger to stability, and changes require a lot of testing. More importantly, we have reached a point where we need to turn to marketing so that we can prove whether the pool will be successful or not. With the clock ticking until the money runs out, results need to be obtained within 3-6 months to justify the opportunity cost of not obtaining jobs working for someone else. Once we can obtain the hashrate required to prove the pool's potential, we can go back to aggressive development, especially in the x11 area. The "spin up time" suggestion may be important enough to take time away from marketing to implement immediately, however, depending on user comments.

I definitely appreciate the hard work you guys put into the features here. I have quite a few mining friends that I can tap for hashrate if we can boost the profitability that the miners perceive mining here. Everytime I look at poolpicker.eu, I'm reminded that coinking reports the highest profitability most days, but in practice they pay out far less, for whatever reason. I have always believed that they are siphoning hashes and/or coins on the back end...

Regards,
rootdude

I don't know everything about how Coinking reports the numbers they do, but I do know one of the answers as to why they promise too much. Their software doesn't buy and sell coins in real time. They report to PoolPicker what a coin is worth at the time it was mined, but they "aggregate" the payout coins together. If, for example, someone wants to be paid a lot of Darkcoins, then they wait until they have enough requests and make a large buy. In the meantime, the price of Darkcoins often rises, causing losses to miners. Furthermore, slippage occurs when you make large buys like this, costing more and further reducing revenue. We execute buys and sells every time the value gets above 1 / (fee fraction), so at Cryptsy (where fees are 0.25%), that equals 1/0.0025 = 400 satoshi. That's the minimum amount to buy so that we don't lose excess money to fees. By buying every time the owed balance rises above this amount, we reduce the risk of price rising too much.

I think the more interesting part is the work restart issue. The c= password parameter is probably better intended for enthusiasts who want to support only one coin. We could easily make it support settings like c=litecoin,darkcoin,netcoin,etc..., but I don't think that's a good design. First, if the sole purpose of adding this feature is for profitability, then I don't think the right approach is requiring customers to figure out the right settings. Specifying coins, for example, requires knowing their block times, but we already have the block times for every coin, so why make customers do it twice?

Second, the system can figure out better than the customers can what the best settings are anyway. I talked with Chris about how to do this and I realized that it's possible for the system to determine this work restart delay. The method is to look at how the expected number of submitted shares differs from the actual number of submitted shares for a number of different coins. After 10 minutes of mining, miners will have a history across many different coins, and the shortest block times will result in the greatest difference between the two figures. We can compute exactly how long it takes for a miner to spin up more accurately than a human can.

Once we have the spin-up time, we can then adjust the profitability for all the coins based upon it. If the block time for a coin is 15s, then the expected time until the next block is about 10s (you need to look at statistics to see why this is). A miner with a 1s spin-up time would then be expected to lose 10% in profitability for that coin. The loss will be only 2.5% for a 60s block time coin. We can adjust each coin's profitability by the losses caused by spin-up time and determine exactly which coin is profitable for your miner, all the time. In the case above, a coin with a 60s block time can be 5% less profitable than a 15s block time coin, but it would still be selected for you.

Customers don't need to care about work restarts or even understand what they are; they just know they'll make more because the system will take it into account.

Steve Sokolowski wrote:

kires wrote:Here's one vote in favor of the spinup time config.

Kires, if you had to guess, how long do you think it takes for your miners to spin up after a restart? Is it 0.2 seconds? 0.5?

I can only guess by what I see in BFGMiner's readout, and that has always looked to me like it takes well over a second, maybe even as much as five seconds to get up to speed. Since that's an order of magnitude above the range you're suggesting, I can only assume that my guess is wrong. Is there any way to capture bfgminer's mashies at .1 sec resolution, maybe pipe it's output to a txt file? I'll give it a shot if you can point me at the software or the config file tweaks. I'm no guru, but I know my way around config files and vi.

(I use vi because it's better than emacs) /troll

kires wrote:

Steve Sokolowski wrote:

kires wrote:Here's one vote in favor of the spinup time config.

Kires, if you had to guess, how long do you think it takes for your miners to spin up after a restart? Is it 0.2 seconds? 0.5?

I can only guess by what I see in BFGMiner's readout, and that has always looked to me like it takes well over a second, maybe even as much as five seconds to get up to speed. Since that's an order of magnitude above the range you're suggesting, I can only assume that my guess is wrong. Is there any way to capture bfgminer's mashies at .1 sec resolution, maybe pipe it's output to a txt file? I'll give it a shot if you can point me at the software or the config file tweaks. I'm no guru, but I know my way around config files and vi.

(I use vi because it's better than emacs) /troll

Unfortunately, the only way to figure this out would be to run BFGMiner in debug mode and then run statistical analysis on it, like I'm suggesting that we will do in this next release. It will take too long for you to do it now for it to be worth the time.

I'm just taking a guess, so if it actually takes 5s, then these companies are ripping people off. It's great to advertise that your miners get 100Mh/s when there is five seconds of zero hashing every time a block is found. If you're mining litecoins, then your hashrate is actually something like 96Mh.

I wasn't aware this issue was so serious. We can't make you earn 100% of the promised hashrate, but we can mathematically prove that you are mining the most profitable coin for your miners once we get this implemented in two weeks.

Here's an idea; just spitballing here. Would it be of any benefit to the pool if I were to give you remote access to one of the titans for a day or two, so you could experiment with various settings and configurations, and see what works best? PM me if you think it'd be worthwhile.

kires wrote:Here's an idea; just spitballing here. Would it be of any benefit to the pool if I were to give you remote access to one of the titans for a day or two, so you could experiment with various settings and configurations, and see what works best? PM me if you think it'd be worthwhile.

This would be great, thanks. But we're not ready yet with any code to test this. I'll send you a PM on Saturday or Sunday once we get to the point where we are ready to test.

I wanted to add to the observations about Coinking above. I determined today that their posted rates are slightly lower than what would be paid by us if we had zero orphans. I wonder if the rate they are publishing is what the pool would pay out in perfect conditions, which are never actually present because orphans happen?