Page 1 of 1

$100 to help figure out DASH transaction ids

Posted: Sun Feb 19, 2017 12:50 pm
by Steve Sokolowski
Updated: this issue has been resolved, and a description of the resolution is below. The reward is no longer available.

We are in the last steps of adding X11 mining to the pool. Everything else, like counting balances, modifying the website to support multiple algorithms, and checking shares, is working except for submitting blocks to DASH.

I'm writing a post to track the problems I'm encountering trying to successfully submit blocks to the DASH network. This is an ongoing issue that I've been working on for about 70 hours. I thought I would post it here to see if someone could assist.

The key problem is that the following line in the DASH client code is being triggered and printing to the logs (main.cpp, 3715, in dash-master):

Code: Select all

return state.DoS(100, error("CheckBlock(): hashMerkleRoot mismatch"),
                             REJECT_INVALID, "bad-txnmrklroot", true);
To debug this issue, I modified our software to print out the blocks that we were sending to submitblock. In one example, we mined a testnet block with the following hex:

Code: Select all

00000020001580da24043d2fe44bb9e36731432005c1f42b3cc28ab20cdeaca41b00000087da6c80c2c7a8034951e68bd845892b98291c7401857f362245fa03f300d80f07c4a958d13c011efca135000102000000010000000000000000000000000000000000000000000000000000000000000000ffffffff1f030660020425c4a95808980000017e0000000c2f364d55394439514652522f000000000240230e43000000001976a9142fe1230bd7859268277431bd2c15d1d92bc6b45f88ac40230e43000000001976a9148f95e6fae491957622ed7fc8021d01e48d7188fb88ac000000000a50726f68617368696e67
I also modified the latest checkin from dash-master to LogPrintf the data received in the submitblock RPC call. The logs demonstrated that the received hex matches the sent hex. The block in this example contains one transaction, the coinbase, and the coinbase transaction is:

Code: Select all

02000000010000000000000000000000000000000000000000000000000000000000000000ffffffff1f030660020425c4a95808980000017e0000000c2f364d55394439514652522f000000000240230e43000000001976a9142fe1230bd7859268277431bd2c15d1d92bc6b45f88ac40230e43000000001976a9148f95e6fae491957622ed7fc8021d01e48d7188fb88ac000000000a50726f68617368696e67
I confirmed that this transaction is present verbatim in the block above using a find/replace. I modified the DASH daemon to print out the value returned by .ToString() for the transaction, and it prints:

Code: Select all

CTransaction(hash=2c8d760a35, ver=2, vin.size=1, vout.size=2, nLockTime=0)
    CTxIn(COutPoint(0000000000000000000000000000000000000000000000000000000000000000, 4294967295), coinbase 030660020425c4a95808980000017e0000000c2f364d55394439514652522f, nSequence=0)
    CTxOut(nValue=11.25000000, scriptPubKey=76a9142fe1230bd7859268277431bd)
    CTxOut(nValue=11.25000000, scriptPubKey=76a9148f95e6fae491957622ed7fc8)
I confirmed that the "scriptPubKey" values match the values we provided in our code.

Next, I computed the hash of the transaction using our code, which is doublesha(trans). I made sure to do so at the last possible moment. The output equals the hash at transaction creation time, indicating that nothing is modifying the transaction before it is sent to the DASH daemon. This hash is 87da6c80c2c7a8034951e68bd845892b98291c7401857f362245fa03f300d80f, which you'll see in the block header from the submitted block is also the merkle root of the block because there is only one transaction. I tried reversing the byte order of this hash but it made no difference with the hashMerkleRoot mismatch problem.

It was suggested to me that the cause of this problem could be that the proof of work being submitted was too low, but I reviewed the DASH code and determined that the check for proof of work occurs before this line. I also modified our code to send random nonces so that the proof of work would be invalid, and the error message changed, indicating that if there were a proof of work error, it would occur before this problem.

After more investigation, I determined that the DASH daemon does not compute the same hash for this transaction as we do. At main.cpp, line 3713 (or so), I added a line:

Code: Select all

LogPrintf("hash: %s, root2: %s\n", block.hashMerkleRoot.GetHex(), hashMerkleRoot2.GetHex());
And it prints:

Code: Select all

hash: 0fd800f303fa4522367f8501741c29982b8945d88be6514903a8c7c2806cda87, root2: 2c8d760a3582c7fffbe882c4c754708f228c3d1649955f2df2bd31e240e54ce2,
The "hash" is the merkle root of the one transaction I provided in submitblock, as expected. I compared the second output to the output from CTransaction.ToString() and noticed that they are the same (at least the first few characters that are printed), indicating that the DASH daemon computes these hashes differently than I do. I then used decoderawtransaction in the dash-cli, and it also prints this value as the "txid." Next, I tried multiple modifications of the hashing algorithm in our code to see if I could result in a matching hash. I tried doublesha(trans), sha(trans), x11(trans), doublex11(trans), and all four of these against the transaction in reversed byte order. None of those matches the txid or hash the DASH daemon computes for the transaction.

Finally, I again tried submitting blocks to the litecoin daemon and all other daemons, and they still accept doublesha(trans) as the txid, as they always have. I then thought that a diff between litecoin and dash-master might be useful, but there are far too many changes to make sense of it.

At this point, the issue appears to be that I don't have a clear understanding of how DASH computes a transaction id. One step I thought I would try is trying to get the DASH client to serialize a transaction after it had been deserialized, to see if what comes out is the same as what I provided in submitblock. If it is different, then I could compare the bytes that are different and try to find why and where they are modified. Unfortunately, I've been trying to figure out how the "<<" operators and streams work in the C++ code for the past six hours with no luck.

If someone can help me understand how to follow the C++ code to figure out how to serialize a transaction, that would be greatly appreciated and would reduce the time required on Monday.

------

If someone can go further and explain exactly what algorithm is being used and what is being hashed in the above example to compute the DASH transaction id of this transaction, I'll pay $100. The correct answer will include three facts: what algorithms are being used by DASH to hash the coinbase transaction in the example, the hex of the transaction DASH is hashing (because the endianness or something else may be different), and what caused our hash to be different from the dameon's hash.

If you are helping for payment, make the first correct post that we can reproduce. We'll pay you immediately after we verify what we're doing wrong. Our judgment on who (if anyone) contributed to resolving the issue and therefore gets paid is final.

Re: $100 to help figure out DASH transaction ids

Posted: Sun Feb 19, 2017 8:53 pm
by tungfa

Re: $100 to help figure out DASH transaction ids

Posted: Sun Feb 19, 2017 10:40 pm
by Steve Sokolowski
tungfa wrote:check:
https://www.dash.org/forum/threads/100- ... ost-114283

tx to Udjin and Chaeplin
:D
Note: this is a possible answer. To anyone reading this, we welcome all help, but if this answer proves correct, then any answers posted after this are ineligible for rewards other than our extreme gratitude.

I need to go to bed now, so I'll check this out in the morning. Thanks!

Re: $100 to help figure out DASH transaction ids

Posted: Mon Feb 20, 2017 8:23 am
by Steve Sokolowski
Thanks for the help. I verified that the answer is correct. I can now submit blocks to the daemon, and Chris will begin deployment of the beta test server as soon as he finishes the Debian setup.

The solution indicates that there are extra bytes appended to the transaction, so I investigated why those bytes were there. It turns out that they were added by a feature we added called "tx_comments." There are some coins which require that a "comment" be added to the end of the transaction, presumably to indicate the reason for a payment or something like that. For our comments, we simply added the ASCII representation of the word "Prohashing," which the developers of those coins were able to use to track how many blocks we mined. This seems to be a new feature that was not present in the original bitcoin code at the time Litecoin was forked.

About half of scrypt coins require this field. Other coins do not support it, so there is code to remove the extra bytes for coins that do not. The way we can determine whether the comment is needed or not is because these coins, except for DASH, reject blocks stating that the transaction is invalid. When that happens, we remove the comment and try again.

DASH is the first coin we found that accepts transactions with these comments, but then removes them and computes a txid with the comment removed. Since hashes are one-way functions, we were not able to determine how to get the hash to match up because DASH doesn't log what it did in any way.

As a suggestion to the DASH developers to prevent such a huge amount of time from being lost in the future, perhaps I might suggest modifying the DASH client to clarify this error explicitly. Consider either:
  • Continue to reject the block but with a new error message, like "transaction has unnecessary data appended to it," instead of the current "hashMerkleRoot mismatch," which is correct but misleading
  • Retain the current behavior but simply add a line to the logs stating that the txid of this transaction will be computed by dropping the unnecessary data that was appended to it
tungfa, thanks for your help for this answer! You deserve every penny. If it's not too much trouble, you can automate your payment, retain your privacy, and save us exchange and tx fees by doing the following:
  • Go to the prohashing.com main site and click "register" in the upper right
  • Enter a username and password
  • In the "payout proportions" screen that appears a few seconds later, scroll down to "payout proportions"
  • Add the coins (or dollars) you want to be paid in, the payout addresses, and set the percentages of payment
  • When you are done, post the username here so we can credit the account
Thanks again! You pulled forward the release of X11 mining by at least two weeks. If there are any regular customers like Critterdog reading this who were awaiting X11 mining, please join me in thanking tungfa for his hard work and dedication to this task.

Re: $100 to help figure out DASH transaction ids

Posted: Mon Feb 20, 2017 9:58 am
by tungfa
happy that worked !!

just to be clear :
UdjinM6
deserves all credit and Funds here ! I only posted to make this visible to others (outside of Dashforum)
https://www.dash.org/forum/members/udjinm6.811/

his Dash Address is:
XsV4GHVKGTjQFvwB7c6mYsGV3Mxf7iser6

If u have no dash ping me again and i sign into that page and such
talked to udjin - he is totally fine with me collecting this on here for him

https://keybase.io/tungfa
https://www.dash.org/team/
Philipp Engelh....

Re: $100 to help figure out DASH transaction ids

Posted: Mon Feb 20, 2017 10:35 am
by excelerator
Thanks tungfa!

Re: $100 to help figure out DASH transaction ids

Posted: Mon Feb 20, 2017 11:00 am
by Steve Sokolowski
tungfa wrote:happy that worked !!

just to be clear :
UdjinM6
deserves all credit and Funds here ! I only posted to make this visible to others (outside of Dashforum)
https://www.dash.org/forum/members/udjinm6.811/

his Dash Address is:
XsV4GHVKGTjQFvwB7c6mYsGV3Mxf7iser6

If u have no dash ping me again and i sign into that page and such
talked to udjin - he is totally fine with me collecting this on here for him

https://keybase.io/tungfa
https://www.dash.org/team/
Philipp Engelh....
We'll create an account, so you don't have to do anything. The reason I want to do that is because the system will automatically purchase the necessary DASH and issue it with the payouts tomorrow morning at 6:17am EST.

Thanks again!

Re: $100 to help figure out DASH transaction ids

Posted: Mon Feb 20, 2017 11:21 am
by tungfa
Steve Sokolowski wrote:
tungfa wrote:happy that worked !!

just to be clear :
UdjinM6
deserves all credit and Funds here ! I only posted to make this visible to others (outside of Dashforum)
https://www.dash.org/forum/members/udjinm6.811/

his Dash Address is:
XsV4GHVKGTjQFvwB7c6mYsGV3Mxf7iser6

If u have no dash ping me again and i sign into that page and such
talked to udjin - he is totally fine with me collecting this on here for him

https://keybase.io/tungfa
https://www.dash.org/team/
Philipp Engelh....
We'll create an account, so you don't have to do anything. The reason I want to do that is because the system will automatically purchase the necessary DASH and issue it with the payouts tomorrow morning at 6:17am EST.

Thanks again!
great
tx man
:D

Re: $100 to help figure out DASH transaction ids

Posted: Tue Feb 21, 2017 8:31 am
by Steve Sokolowski
tungfa wrote:
Steve Sokolowski wrote:
tungfa wrote:happy that worked !!

just to be clear :
UdjinM6
deserves all credit and Funds here ! I only posted to make this visible to others (outside of Dashforum)
https://www.dash.org/forum/members/udjinm6.811/

his Dash Address is:
XsV4GHVKGTjQFvwB7c6mYsGV3Mxf7iser6

If u have no dash ping me again and i sign into that page and such
talked to udjin - he is totally fine with me collecting this on here for him

https://keybase.io/tungfa
https://www.dash.org/team/
Philipp Engelh....
We'll create an account, so you don't have to do anything. The reason I want to do that is because the system will automatically purchase the necessary DASH and issue it with the payouts tomorrow morning at 6:17am EST.

Thanks again!
great
tx man
:D
This payment was made at https://prohashing.com/explorer/Dash/58 ... eff243393/.

Re: $100 to help figure out DASH transaction ids

Posted: Wed Feb 22, 2017 3:45 am
by tungfa
Steve Sokolowski wrote:
tungfa wrote:
Steve Sokolowski wrote:
We'll create an account, so you don't have to do anything. The reason I want to do that is because the system will automatically purchase the necessary DASH and issue it with the payouts tomorrow morning at 6:17am EST.

Thanks again!
great
tx man
:D
This payment was made at https://prohashing.com/explorer/Dash/58 ... eff243393/.
udjinm6 : well received $ and all happy
tx guys
:D