$50 reward - Help with merge mining

[Steve Sokolowski]

I'm working on enabling merge mining for Prohashing. I've spent 45 hours trying to get the dogecoin daemon to accept a merge mined block, with no success. I'm posting my progress in this post, in the hopes that someone who has experience in merge mining can figure out what is wrong. I'll pay the first person $50 who can tell me what is wrong with what I'm doing. If there are multiple issues, then I'll split the reward amongst all the helpers.

I simplified the procedure by removing the parts of the algorithm that are irrelevant.

Here is the procedure I used:

1. Get the latest block from the litecoin testnet and store its data in memory.
2. Call getauxblock against the dogecoin testnet. Since this example is only going to merge mine dogecoins, we ignore "chainid" and store only "hash" in memory. "Target" is obtained by calling getblocktemplate, because we need difficulty and other things from the full template for calculating payouts. "Target" in getauxblock and in getblocktemplate are reversed, so the appropriate conversion is made.
3. When a block is found for the litecoin testnet, check to see whether the target is less than the dogecoin testnet's target. If so, we call getauxblock again, passing the "hash" exactly as provided in step 2, without any modification, and the serialized block data as the second parameter. The help for the command states that the parameters are "getauxblock [hash] [auxpow]."

The result is that the litecoin blocks are always accepted, and the dogecoin blocks are always rejected with the following errors:

Code: Select all

2014-10-09 02:37:45 ERROR: Aux POW merkle root incorrect
2014-10-09 02:37:45 ERROR: AUX POW is not valid

Here is an example "auxpow" serialized block that is submitted to the dogecoin damon. I annotated it as I think is correct, but keep in mind that the annotations could be incorrect and you shouldn't assume that I have identified the correct things to insert or the correct order. In the real submission, there are no spaces or characters between the separated sections.

Litecoin coinbase transaction:

Code: Select all

02000000010000000000000000000000000000000000000000000000000000000000000000ffffffff52034e5106062f503253482f04c93c3754fabe6d6d50868b4be2c645d5b763f8b2f137e87e48ed7154db9e7a5083a4d4582fdfbeec010000000000000008f8000006190000000c2f50726f68617368696e672f000000000100f2052a010000001976a914c7045a47e3aff57c28a728c7339150efc879555488ac000000000a50726f68617368696e67

Double-SHA-256 hash of the litecoin block header:

Code: Select all

0309e093c93a7560ecde4128c13371ee4778bd195a03681e57aeacd694094600

The length of the merkle branch from the litecoin block, which is the same as the branch sent out in the stratum protocol. Because this litecoin block has no transactions, the length of the merkle branch is zero:

Code: Select all

00

The litecoin merkle branch, if there were one, would go here in a series of hashes. Since there are no transactions in the block other than the coinbase transaction, we append nothing here.

Code: Select all

[There is nothing here]

The "branch side mask" of the coinbase transaction, which is always zeroes:

Code: Select all

00000000

The auxiliary branch count, which is zero because we are only mining dogecoins in this example:

Code: Select all

00

The auxiliary branch index, which is also zero because we are only mining dogecoins:

Code: Select all

00000000

The block header of the litecoin block, in full:

Code: Select all

0000000254a8f234ca1f2448ca9a85dcade4697e1496fa46a22abe8ffb0cfc604a4d11371b31c4accb59203d465c70588f1fd68eddd93e25da83101bbaa68aa02244800454373c611d02324089d3403e

I'll also break down my understanding of what is supposed to be placed in the litecoin coinbase transaction to signify that we are merge mining dogecoins. Here is my understanding of the litecoin coinbase transaction's merge mining portion, which you can find embedded within the coinbase transaction printed above:

This string signifies that we are merge mining.

Code: Select all

fabe6d6d

The "hash" parameter obtained from the dogecoin daemon's getauxblock command, verbatim:

Code: Select all

50868b4be2c645d5b763f8b2f137e87e48ed7154db9e7a5083a4d4582fdfbeec

The following are used for when multiple merge-mined coins are being sought at the same time, but since we are only merge-mining dogecoins, this is a 4-byte 1 followed by a 4-byte 0.

Code: Select all

0100000000000000

Here are some of the things I tried and the references I used.

1. https://en.bitcoin.it/wiki/Merged_mining_specification seems to be the primary source on merge mining. However, I noticed that some of the examples in the document don't work with dogecoins. For example, the "block hash" in the auxiliary proof of work that is submitted to namecoin in that document (the second field) looks like the proof of work hash for the block, since it ends in a string of zeroes. Looking at that, I tried placing the scrypt proof of work hash in that field, but it didn't work.

My understanding of the "block hash" is that when you call getblock from a daemon, you provide the double SHA-256 hash of the block header, not the scrypt proof of work hash. The "block hash" is not the scrypt proof of work hash.

2. I tried reversing various hashes in the fields of the blocks on the theory that endianness was the problem, but 16 different permutations didn't work. I tried reversing the dogecoin auxiliary hash, the block hash, the merkle branch hashes (when there are transactions in the litecoin block, which there are not in this example), and even the block header of the litecoin block. None of these things worked. I couldn't find a permutation of reversed and non-reversed hashes that made any difference. Of course, it is possible that, since there are so many permutations, that I missed the correct one and the hashes are not in the correct endianness in the example.

3. At http://forum.namecoin.info/viewtopic.php?f=7&t=368, there is a poster who offers advice on how to submit merge mined blocks to getauxblock, although that information is specific to namecoin. I reviewed what I was doing and it appears to be identical to what he is suggesting.

4. After reviewing the documentation for what a merkle tree is, it took me an entire day to figure out what happens when there are an odd number of transactions in the tree. It turns out that the algorithm is to hash the nodes with themselves. Seeing this, I took the example above and I tried specifying the length of the "merkle branch" for the coinbase transaction as "01," and then provided the hash of the coinbase transaction as the only hash in the "merkle branch." The long-shot idea was that perhaps the dogecoin daemon was looking to hash the coinbase transaction with itself, and use that as the root of the tree. It still returned the same error.

5. In the litecoin coinbase transaction, the 44-byte merge mining part (fabe + "mm") is preceded by the length (44, or 2c) in some examples, but not in others. Apparently, this length is not necessary if the merge mining string is provided within the first 20 characters of the script, so I left it out in this example. However, in previous iterations, I added an additional byte of "2c" before the merge mining portion and it did not result in any difference in this error.

6. In these examples, I always assumed that the merkle branches are double-sha256 hashes, even for scrypt coins. All the documentation I read seems to indicate that in scrypt, the only difference is the algorithm used to verify work. From what I can tell, the rest of the block still is stored using SHA-256 hashes, as is the hash of the block headers and even the hashes of the transactions. If there is some difference between scrypt and SHA-256 in how the merge mining headers are stored, that could be a clue.


Thanks to anyone who is willing to try to point out what is wrong here. We have about 15 features ready for release and merge mining is the only one that is holding back the release. Your help is greatly appreciated.

[jimlite]

What is the stratum software? NOMP, python, etc

[Steve Sokolowski]

The mining software is custom. It uses the Python stratum interface created by Slush to communicate with clients. All of this described in the posts above was written from scratch.

[Chris Sokolowski]

We believe we have come to a solution, so while we test this solution the reward is no longer available.

Thank you to everyone who looked over the code.

[ChrisR]

Hey Steve,

would you like to share your solution ? As I'm having the same problem, but only with dogecoin and i'm not sure why..

[Steve Sokolowski]

Hi Chris,

The solution was extremely complicated and took about 120 hours to figure out. I honestly don't recall right now what the key was.

I'd love to help, but unfortunately, there wasn't any single issue with the system that led to the solution. It was just a matter of trying one thing at a time until I finally came to a solution that worked. There were at least five things wrong.

At this point, dogecoins are so unprofitable that it might not be worth the time. With the price of bitcoins falling, the entire viability of mining pools is in doubt at this point, so I wouldn't recommend starting on a new project.