Thursday, July 30, 2009

Comments, Links

Over the past few weeks I've been hearing reports by people on ShanghaiExpat and Twitter that they couldn't access certain sites even through a proxy or VPN. I ran into the same problem recently over my SSH tunnel to a server in California: strangely, I could load Facebook and Twitter on my home machine but my eeePC couldn't get through.

Trying to get to the root of the problem, I opened the puTTY console and looked at what the SSH tunnel was doing. On my home computer, it was opening connections to the websites that I wanted. But on the eeePC it seemed to be opening connections to random IP addressed, regardless of what I typed into the URL bar. For example, I would type "", hit Return, and puTTY showed a connection being opened to some IP address; I copied the IP address and did a reverse DNS lookup in the shell, and found that it was some residential IP in New Jersey — definitely not Twitter!

So I suspected that it was a DNS problem. Typing "about:config" into Firefox's URL bar and then filtering by "dns" showed me the network.proxy.socks_remote_dns option set to false, which told me that Firefox was doing DNS lookups domestically before trying to open the webpage itself (accessing a website is a two-step process: ① finding the IP address of a URL, and then ② asking for the page itself from that IP address). Toggling that option to true fixed the problem and I was back on Facebook/Twitter seconds later.

I began to suspect that all of the people I heard complaining about access were having the same problem as me, though I don't know if the same solution will work for everybody. My suspicions were further confirmed today when I followed a tweet by shizhao to a blog post in Chinese by Antonio (yeah, me too) showing how the GFW is now hijacking DNS queries to a certain subset of sites that it wants to block. Before, the GFW had been blocking sites on step ②, returning the correct IP address and then resetting requests for the webpage. Now, it has been upraded to block access on step ①, returning random, incorrect IP addresses for blocked sites!

Confirming Antonio's post, here's a critical example:

See explanation below.

You can see that the first two DNS lookups, "nslookup", return different IP addresses for Twitter. The third command asks for a reverse DNS lookup on the second IP address, and the result is that this is an address that doesn't actually exist. If you continue to run nslookup over and over (just hit up arrow and Return) you will continue to get random, incorrect IP addresses. If you were a web browser, you would try to connect to one these addresses and get nothing.

Let me translate the conclusions to Antonio's post:

This new upgrade contributes to the worsening state of the Chinese domestic internet:

  1. When you surf to a blocked site the browser will show a "connection has timed out" message, which is even more deceptive than before.
  2. Right now the GFW doesn't use this method for all blocks (for example, the Chinese BBC site still gets the right IP) but it's clear that as this method gains acceptance it will be used more and more due to its effectiveness.
  3. This method trumps OpenDNS, making it even harder to reach blocked sites; you can still edit the /etc/hosts file manually but it's a pain, and setting DNS queries to happen through a proxy makes the process even more complicated:
    • HTTP and Socks4 (except Sock4/a) don't support DNS lookup by proxy, so that's out.
    • The IE family of browsers (including Maxthon and The World Browser) do local/domestic DNS queries by default, even with a SOCKS proxy configured. [This doesn't match my experience —Micah] DNS over the proxy can be forced with software like Sockscap.
    • Firefox also does local DNS queries by default; the solution is to set network.proxy.socks_remote_dns to true. If you've installed FoxyProxy and are using a Socks proxy you can go into settings and check "Perform DNS lookups over proxy" [Or something like that; translating back from the Chinese. —Micah].

Back to my own commentary, as far as I know this upgrade happened a month or two ago.

UPDATE: Commenter Nart has been seeing this happen for a while now. It must have only recently been applied to the more popular sites that caught my attention. See his weblog:


At Jul 30, 2009, 8:26:00 PM, Anonymous Eric said:

Hmmmm. Is there a Mac/Windows divide here? I haven't been having any similar problems using my MacBook.

At Jul 30, 2009, 9:22:00 PM, Blogger Micah Sittig said:

It's possible. It comes down to a browser issue, so if you're using Safari then maybe this isn't an issue. Or it's possible that your Firefox was already configured correctly like mine happened to be on my desktop box.

At Jul 31, 2009, 6:04:00 AM, Anonymous Nart said:

They will often MITM dns:

At Jul 31, 2009, 1:24:00 PM, Blogger Micah Sittig said:

Nart: Thanks! I'll update the post with those links.

At Jul 31, 2009, 5:43:00 PM, Anonymous Anonymous said:

Wow! Thanks for this post! I changed the configuration in Firefox on my Mac OS X platform and I am good to go with my webserver tunnel to the States! No more hunting for webproxys to use Facebook. You are a technical super slooth.

At Jul 31, 2009, 10:59:00 PM, Blogger martha said:

I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.


At Aug 1, 2009, 12:44:00 PM, Blogger Richard said:

I am having an amazingly similar problem on my Mac. It is a BIG problem on Safari - can't access blogspot, wordpress or Wikipedia, but no problem with Twitter, Facebook and even China Digital Times. Firefox on my Mac is fine. Only Safari is screwed up, and dumping the cache isn't helping. I moved away from China a week ago, and feel like I took it home with me.

At Aug 1, 2009, 6:06:00 PM, Anonymous Terry said:

Brilliant!! I can now speedily access Facebook through my proxy rather than taking a very slow and circuitous route through JonDo.

Thank you Micah!!!

At Aug 1, 2009, 7:31:00 PM, Blogger martha said:

I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.


At Aug 1, 2009, 9:27:00 PM, Anonymous Anonymous said:

starting happening for me june 2 or 3rd, right when the 20th anniversary blocks started. i am on a mac, see the problem in both firefox and safari, thru witopia vpn

At Aug 8, 2009, 1:38:00 AM, Blogger Antonio said:

Hey, this is the Antonio:) Sry I lost track of you 'coz I transfered my blog to Wordpress, but hey here you are.
And about the DNS issue of IE, IE did run DNS lookup locally(i.e. not proxy-remotely) first by default, and because it get a seems-nothing-wrong result from GFW, no chance it will do it again. My guess is that your IE get the "right" result from a previously cached lookup.
奥巴马说:Change! 胡总书记说:别折腾.lol


Post a Comment

Post a Comment

« Home