Status as of Monday, September 25, 2017

Discussion of development releases of Prohashing.
Post Reply
User avatar
Steve Sokolowski
Posts: 4124
Joined: Wed Aug 27, 2014 3:27 pm
Location: State College, PA

Status as of Monday, September 25, 2017

Post by Steve Sokolowski » Mon Sep 25, 2017 6:45 pm

There are two issues of note this evening: a new release, and DDoS attacks.
  • Chris will issue a new release tonight. It addresses two issues that he thinks are responsible for a lot of the customer support tickets with vague complaints.
  • First, he found that some mining software has a "worker name" textbox, which actually appends the worker name to the username, like "username.worker name" without the user's knowledge. This caused failures to authenticate. Since periods are already prohibited in usernames, it is simple to split the usernames on periods and use the part after the "." as a worker name. However, "n=" names will still override any mining software that provides usernames like this, and we won't document this feature to reduce confusion.
  • Second, I modified the behavior when the server detects a duplicate worker name. We still need to enforce the unique worker name problem, because I don't know what will happen on the UI if we allow names to be duplicated. Now, instead of disconnecting the user for violating uniqueness, an error message is displayed instead and the miner's name is replaced with a UUID. The error remains until the miner replaces the name.
Another issue that Chris is investigating is DDoS attacks. Apparently, someone's website is being DDoS attacked, and it turns out that our system is one of many performing these attacks. Further investigation reveals that while we paid a lot of attention to making sure nobody could steal any money, we never considered that someone could put code in a coin daemon to send SYN floods to a site. It is possible that the network connectivity issues that have plagued the system for months may be due to this attack. It also explains why it cost an additional $210 for bandwidth last month - the floods seem to be programmed to occur at specific times and we've never been looking at those times.

Chris is trying to track down which coin is causing the problem. Exchanges that are going offline may be having troubles with this coin too. Once he figures it out, he'll pay out balances for that coin, delete it, and then we should see a dramatic improvement in network connectivity. Our goal will be to 51% attack this coin to destroy the network and inform exchanges to delist it, so that we can make a statement to people that this sort of behavior is unacceptable.
simonjbcmm
Posts: 10
Joined: Sun Aug 27, 2017 6:23 pm

Re: Status as of Monday, September 25, 2017

Post by simonjbcmm » Mon Sep 25, 2017 6:50 pm

You and Chris are truly very talented individuals. Thank you for your hard work and attention to detail. It is a privilege to have an account with your prohashing service.

With appreciation,
Jonathon
Alex
Posts: 5
Joined: Sun Sep 10, 2017 11:12 am

Re: Status as of Monday, September 25, 2017

Post by Alex » Mon Sep 25, 2017 6:51 pm

You mentioned "programmed to occur at specific times" would it be helpful if you knew when our miners switched to their respective backup pools or I am going down the wrong path?
vinylwasp
Posts: 98
Joined: Mon Oct 31, 2016 3:42 am
Location: Singapore

Re: Status as of Monday, September 25, 2017

Post by vinylwasp » Tue Sep 26, 2017 12:37 am

Steve Sokolowski wrote:There are two issues of note this evening: a new release, and DDoS attacks.

Another issue that Chris is investigating is DDoS attacks. Apparently, someone's website is being DDoS attacked, and it turns out that our system is one of many performing these attacks. Further investigation reveals that while we paid a lot of attention to making sure nobody could steal any money, we never considered that someone could put code in a coin daemon to send SYN floods to a site. It is possible that the network connectivity issues that have plagued the system for months may be due to this attack. It also explains why it cost an additional $210 for bandwidth last month - the floods seem to be programmed to occur at specific times and we've never been looking at those times.

Chris is trying to track down which coin is causing the problem. Exchanges that are going offline may be having troubles with this coin too. Once he figures it out, he'll pay out balances for that coin, delete it, and then we should see a dramatic improvement in network connectivity. Our goal will be to 51% attack this coin to destroy the network and inform exchanges to delist it, so that we can make a statement to people that this sort of behavior is unacceptable.
Steve, the first thing to do is to stop allowing any of your coin daemon servers to initiate direct connections to unknown hosts on the internet. That's a fairly standard egress rule for most enterprises. If they need connectivity use an internal web proxy/email relay/dns relay.

Syn attacks are very old school, if they're attacking a website, they've probably hardcoded one of the many versions of LOIC, HOIC or Slowloris available on GitHub into the source code.

The big question is whether you're DoSing a single website or even worse, you're part of a bot-net which is being used (sold) to DDoS multiple websites. If you're sure it's just one, the domain name or IP address is probably hardcoded into the binary or in a config file somewhere (assuming they're not too sophisticated and haven't encrypted it) so you might try using a suitable binary search tool to look for it.

Happpy hunting.
User avatar
Steve Sokolowski
Posts: 4124
Joined: Wed Aug 27, 2014 3:27 pm
Location: State College, PA

Re: Status as of Monday, September 25, 2017

Post by Steve Sokolowski » Tue Sep 26, 2017 8:08 am

vinylwasp wrote:
Steve Sokolowski wrote:There are two issues of note this evening: a new release, and DDoS attacks.

Another issue that Chris is investigating is DDoS attacks. Apparently, someone's website is being DDoS attacked, and it turns out that our system is one of many performing these attacks. Further investigation reveals that while we paid a lot of attention to making sure nobody could steal any money, we never considered that someone could put code in a coin daemon to send SYN floods to a site. It is possible that the network connectivity issues that have plagued the system for months may be due to this attack. It also explains why it cost an additional $210 for bandwidth last month - the floods seem to be programmed to occur at specific times and we've never been looking at those times.

Chris is trying to track down which coin is causing the problem. Exchanges that are going offline may be having troubles with this coin too. Once he figures it out, he'll pay out balances for that coin, delete it, and then we should see a dramatic improvement in network connectivity. Our goal will be to 51% attack this coin to destroy the network and inform exchanges to delist it, so that we can make a statement to people that this sort of behavior is unacceptable.
Steve, the first thing to do is to stop allowing any of your coin daemon servers to initiate direct connections to unknown hosts on the internet. That's a fairly standard egress rule for most enterprises. If they need connectivity use an internal web proxy/email relay/dns relay.

Syn attacks are very old school, if they're attacking a website, they've probably hardcoded one of the many versions of LOIC, HOIC or Slowloris available on GitHub into the source code.

The big question is whether you're DoSing a single website or even worse, you're part of a bot-net which is being used (sold) to DDoS multiple websites. If you're sure it's just one, the domain name or IP address is probably hardcoded into the binary or in a config file somewhere (assuming they're not too sophisticated and haven't encrypted it) so you might try using a suitable binary search tool to look for it.

Happpy hunting.
I think the better solution here is to make permanent what Chris did already. We'll simply count SYN packets, and if there are an unreasonable number coming from a username, then an E-Mail is sent to Chris notifying him of the problem. We're still waiting for that E-Mail, so no attacks have happened yet.

One of our goals is that every time we encounter a problem, we want to permanently fix it so that the system becomes more stable in the future. Since this guy pointed out the attacks on his website, it will now likely never be possible for a coin network to conduct SYN attacks because exchanges will also detect and discontinue the coin causing the attacks.

Unfortunately, we can't block outgoing connections because the way that coins work is that they discover new peers by asking existing peers for addresses of those peers. The only way we could block outgoing connections is if we were able to obtain a large list of initial peers, and guarantee that those peers would never go offline. Since we can't do that, we'll have to settle for the second best choice, which is detecting each type of attack as we encounter it and making sure that nobody will be able to use it again.
tomos81
Posts: 9
Joined: Thu Aug 03, 2017 7:05 am

Re: Status as of Monday, September 25, 2017

Post by tomos81 » Tue Sep 26, 2017 10:17 am

Steve thanks for this update, now nicehash accepting higher hashrate to rent, i dont care of error with duplicate names in the PH website earnings page :)
zafar177
Posts: 55
Joined: Mon Aug 21, 2017 4:38 pm

Re: Status as of Monday, September 25, 2017

Post by zafar177 » Tue Sep 26, 2017 10:37 am

Reading these posts make realise my lack of knowledge and your impressive expertise. Great work guys.
Kfedorek
Posts: 29
Joined: Sun Sep 10, 2017 6:33 pm

Re: Status as of Monday, September 25, 2017

Post by Kfedorek » Tue Sep 26, 2017 4:49 pm

Great work guys. That is very crazy about the ddos. What a crazy way to perform a ddos but very scary for you guys and us as well! Best of luck. Thanks again for the updates and tremendous work you guys put in.
pumaro
Posts: 26
Joined: Sun Aug 30, 2015 8:17 am

Re: Status as of Monday, September 25, 2017

Post by pumaro » Wed Sep 27, 2017 5:08 am

I point my miners through miningrigrentals which uses 3 possible locations to hash ; East, Central, and West is it possible that the duplicate names are coming from two locations at once, maybe load sharing or fail over causing this issue?

us-west01.miningrigrentals.com:3333
us-east01.miningrigrentals.com:3333
us-central01.miningrigrentals.com.3333

- Ron
Denis
Posts: 1
Joined: Thu Sep 28, 2017 3:12 pm

Re: Status as of Monday, September 25, 2017

Post by Denis » Thu Sep 28, 2017 3:30 pm

Dear Dev Team,

I am trying to get total hashpower suscribing "f_all_miner_updates" RPC. On first call I get all miners, more than 10. But next call return just 1 miner data. So mostly my hashpower value is wrong.

Best,
Den
Post Reply