Troubleshooting Akamai: How to geolocate an IP Address

Akamai provides enormous value by accelerating content delivery around
the world. However, as I described a few months back, Akamai relies on
the DNS server
 that processes the request to determine where to
accelerate traffic. Large, global corporations often have complex DNS
configurations that can cause Akamai to accelerate content to the wrong
location. For example, they may have users that are located in Europe
but resolve DNS in the United States.

When troubleshooting these issues, it is often necessary to geolocate an
IP address to figure out whether there is a DNS configuration problem.
End users generally don’t know the details of how they access the
internet, and the IT personnel who would know are often hard to reach or
may not know exactly how a specific end user is set up. In these cases,
it’s very helpful to take information about the user’s IP address, their
DNS server, and the IP address of the Akamai edge server they are
hitting and geolocate them to see if there is a mismatch.

Take geolocation databases with a big grain of salt

Geolocating is more art than science. There are many databases that
will attempt to geolocate an IP address, but they can often be error
prone. For example, many of them are based on who registered the IP
address or block and have no bearing on the real physical location.

Let’s take an example: 60.254.175.30. If I use a geolocation service
like www.maxmind.com to look up the location
of this IP address, it will tell me that it is in Cambridge,
Massachusetts. Well, that is very nice of Maxmind to tell us this, but
one thing I can tell you is that this server is very, very far from
Cambridge.

However, using a combination of techniques, it is often possible to get
a pretty good sense of where the IP address is. Let’s give it a try.

Start with a traceroute

The first step is to run a traceroute. For example:

C:Usersjrothmanshore>tracert 60.254.175.30

Tracing route to 60.254.175.30 over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  172.29.0.5
  2    <1 ms    <1 ms    <1 ms  172.29.0.2
  3     1 ms    <1 ms    <1 ms  216.52.51.1
  4     1 ms     1 ms     2 ms  69.25.73.93
  5     1 ms     1 ms     2 ms  core2.po2-bbnet2.bsn.pnap.net [63.251.128.82]
  6     6 ms     6 ms     6 ms  ge11-0-1d0.mcr1.cambridge-ma.us.xo.net [216.55.4.9]
  7    16 ms    14 ms    14 ms  vb1020.rar3.nyc-ny.us.xo.net [216.156.0.25]
  8    16 ms    15 ms    15 ms  te-3-0-0.rar3.washington-dc.us.xo.net [207.88.12.74]
  9     *       14 ms    13 ms  207.88.14.165.ptr.us.xo.net [207.88.14.165]
 10     *        *        *     Request timed out.
 11    92 ms    91 ms    89 ms  ae-5.r21.sttlwa01.us.bb.gin.ntt.net [129.250.4.182]
 12   272 ms   251 ms   248 ms  as-2.r21.osakjp01.jp.bb.gin.ntt.net [129.250.3.86]
 13   249 ms   239 ms   265 ms  as-1.r21.newthk02.hk.bb.gin.ntt.net [129.250.2.149]
 14   229 ms   274 ms   244 ms  xe-3-2.r00.newthk02.hk.bb.gin.ntt.net [129.250.3.167]
 15   323 ms   343 ms   321 ms  203.131.245.110
 16   317 ms   326 ms   321 ms  60.254.175.30

Trace complete.

Look at hostnames

One very convenient thing about many ISP’s and backbone providers is
that they tend to name machines based on where the machines actually
are. While the first several IP addresses in the list above don’t tell
me much, at the 6th hop, we hit a machine that indicates that it is in
Cambridge. The 7th hop implies New York City, and the 8th is Washington
DC.

If you use a little creativity and imagine routes through the world, you
can make some good guesses about the next couple of hops. I’m pretty
confident that the “sttlwa” in #11 is Seattle Washington. From there, I
would guess that the “osakjp” in #12 is Osaka, Japan. Knowing I am now
in the Asia would lead me to conclude that the “.hk.” in hops #13 and #14 are Hong Kong.

Looks like we are in China. Indeed, very far from Cambridge, MA.

Look at the latencies

Another clue in reading the hostnames is the latency at each of the
hops. I am working out of the Boston area, and in general, I find
latency from traceroutes to different parts of the world to be pretty
consistent:

  1. Eastern US takes 15-30 milliseconds
  2. Western US takes 40-70 milliseconds
  3. Europe takes 80-100 milliseconds
  4. Asia takes 200-300 milliseconds

If I compare this with my guesses based on the host names above, I will
see that they work out pretty well. Washington, DC was 16 milliseconds,
which matches my experience for East coast locations. Seattle was 91
milliseconds, a little higher than I normally expect from the west
coast, but not too far off. When I see that latency than jumps up into
the 200 millisecond range at hop #12, it’s a good bet we are in Asia,
and that matches my guess of Osaka, Japan based on the host name.

Check geolocation databases for the last few hops

While the geolocation databases are often pretty far off for the end IP
address, they are much more accurate when used for the nodes of the last
mile ISP’s themselves, which tend to be regional entities.

My preferred sites for lookups is
www.maxmind.com, which provides 25 free
lookups a day. If I try looking up hop #15 (203.131.245.110) in MaxMind
– the one just before my target – I see that it is in Hong Kong and is
managed by the HKNet Company Limited.

All Sources Agree

In this case I now have confirmation from three separate sources that my
target IP address is in China, probably Hong Kong. The hostnames of my
path indicate that we crossed the United States, went over the Pacific
through Japan, and into China. Latency matches my past experience for
an IP address in China. And my geolocation database shows one of my
last few hops in China.

So remember, when geolocating an IP address, don’t just trust what a
database says about last hop. Make sure that the latency numbers and
host names support the theory, and check the last few hops which tend to
be more accurate.

This entry was posted in Uncategorized and tagged , , , , , . Bookmark the permalink.

3 Responses to Troubleshooting Akamai: How to geolocate an IP Address

  1. Phillip says:

    you could add to this to have users ping whoami.akamai.com. The address returned will be the edge server they are reaching. Geo-locate that address and you will know if they are going to the correct edge server.

    • Sounds cool… but the link just hangs for me. Does this work in all environments?

      • Actually, whoami.akamai.com just resolves to the IP address Akamai thinks you are. If you run your own DNS resolver (as I do), it’ll likely be your own IP address. If you use your ISP’s resolver, it’ll usually be that resolver’s IP address. And if you use an anycast public DNS resolver like Google Public DNS, OpenDNS, Quad9, or Cloudflare (1.1.1.1), it’ll usually be the actual underlying IP address of the specific server that anycast IP routes to.

        The IP address shown by whoami.akamai.com is what Akamai will generally use to geolocate you and decide which node to serve you out of.

        If you want a web-based tool to diagnose Akamai geolocation, http://whatismyip.akamai.com/advanced will tell you which Akamai server you’re hitting along with where Akamai thinks you are. If it doesn’t match what you’d expect to see, looking up the IP address of whoami.akamai.com can still be helpful to troubleshoot why Akamai may not be serving a logical address up to you.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s