Small details can quickly become a really big deal if they’re not looked after.
I recently suffered a network outage that oddly had little to do with technology and a lot to do with attention to detail: in this case using your Outlook calendar, or service desk, to not self-inflict a (partial) network outage.
If you’re an MSP or business, you have domain registrations. If you’re an IT service provider there is a good chance you have many customers, quite possibly with many domains that you are responsible for—and if you are not responsible for them, you need to be. I’m going to say right here, right now: domain registration and DNS go hand-in-hand. If you neglect this, it can turn vital business services (like a website and email) off. Not only does that hurt, it is down time that can be avoided.
So, this is the scenario: the website check I have running on one of my domains, failed; just up and failed. Everything on the server was fine—ping to the server, all the services, everything else all perfect. I’m using the world’s best RMM platform, but it sure seemed like a problem on the server—maybe Apache died?
A quick call to my friend, web dev, and open source guru Theodore (check out his epic networking blog ciscodude.net) and he’s like, “No dude, I didn’t do anything.”
Shortly after that event some friends started saying that email was bouncing from the domain, but not for all folks and not for any customers. OK… DNS and MX record problem, corruption of a record, cyberattack… Did someone change or break into the domain registration? Now I am thinking registrar or DNS for sure. I ping the domain and nothing. Google can’t find the domain, but email still seems to be working… sort of.
Time to visit the registrar’s portal. Well the news is not good: the domain had been suspended (not by the DOJ) because the contact information on the domain was old, and the contact email was dead so no notifications about having to be “verified” were received.
The big take away: review and update the contact information annually for the domain(s) you and your customer(s) use.
Despite the domain suspension all the customers could still send email and could receive email from each other. By all rights this should have stopped too, but it didn’t. The “rule” in violation (apparently) was: ICANN’s “2013 Registrar Accreditation Agreement (RAA)—The Whois Accuracy Program Specification of the 2013 Registrar Accreditation Agreement (RAA) requires registrars to validate and verify certain Whois data fields, which may include contacting you by phone, email, or postal mail. Registrars must suspend or delete domain names that are not timely verified.” Even though the domain was all paid, if your contact information is not correct and you need to be “verified.” And if you don’t respond in 15 days, that’s it, the domain is “un-plugged.”
I dug deeper into the email situation because it bugged me. If the domain is suspended it should not work, period—broken should be broken. So, that got me thinking about what all the customers and the suspended domain have in common? Answer: Office 365. Apparently Microsoft, because it is not a domain registrar, does not have to play by ICANN’s rules. I have a hypothesis on this.
If you are emailing Office 365 to Office 365, things work (even though they shouldn’t) despite a domain suspension. It would appear that the status of your domain means nothing to MSFT. I think there must be some Microsoft internal MX/DNS happening between Office 365 accounts. This is also interesting from a security perspective as it may mean that Office 365 to Office 365 communication happens inside Microsoft's cloud and the mail transfer may never go outside Microsoft's network until delivery to a client.
Also, inside Microsoft’s cloud the Office 365 to Office 365 communication may be encrypted via TLS and then HTTPS to the client endpoint device. That’s pretty cool. I would love to know if Office 365 to Office 365 communication would be end to end encrypted—that would have some interesting ramifications for HIPAA and protection of PII in transit. It may be ok (after all) to send credit card info in email, so long as it is Office 365 to Office 365.
That aside, one final word on all the DNS and registrar tomfoolery. For that vital contact information, use an email address that does not rely on a domain you have registered, such as Gmail or Outlook.com. If your domain is suspended (or some other urgent notification is required) and you update the domain contact email with the same domain email, you can’t receive the link to verify, and you have to phone the registrar tech support to get it sorted out. Simple really.
As I said at the start it’s the little things that can make all the difference.
Oh and a special thanks goes out to the EasyDNS folks (fellow Canadians) for their help and patience sorting out these suspension shenanigans.
Ian Thornton-Trump, CSA+, CD, CEH, CNDA is CTO at Octopi Managed Services Inc. Ian is an ITIL certified Information Technology (IT) consultant with more than 20 years of experience in IT security and information technology. He enjoys and maintains a strong commitment to the security community. From 1989 to 1992, Ian served with the Canadian Forces (CF), Military Intelligence Branch; in 2002, he joined the CF Military Police Reserves and retired as a Public Affairs Officer in 2013.
You can follow Ian on Twitter® at @phat_hobbit.