A hop-by-hop read of two traceroutes from a RackWorks VPS to Cloudflare: six hops, under a millisecond, and what that changes for a site or app running behind Cloudflare.
この記事は現在英語のみでご利用いただけます。
Most hosting pages tell you their network is fast. This one shows the trace. Below are two traceroutes run from a RackWorks VPS in Southfield, Michigan to Cloudflare's public resolvers, 1.1.1.1 and 1.0.0.1. Both arrive in six hops, and both come back in well under a millisecond.
Here is what every line means, why the number is as low as it is, and what it does (and does not) change for a site or application running behind Cloudflare.
The Raw Data
traceroute to 1.0.0.1
1 _gateway (23.170.200.97) 0.519 ms
2 agr2.sfld1-mi.networkedge.net (74.80.183.197) 0.381 ms
3 core11.sfld1-mi.networkedge.net (23.170.200.80) 0.253 ms
4 core10.sfld2-mi.networkedge.net (104.244.208.104) 0.283 ms
5 * * *
6 one.one.one.one (1.0.0.1) 0.380 ms
traceroute to 1.1.1.1
1 _gateway (23.170.200.97) 0.541 ms
2 agr2.sfld1-mi.networkedge.net (74.80.183.197) 0.343 ms
3 core11.sfld1-mi.networkedge.net (23.170.200.80) 0.221 ms
4 core10.sfld2-mi.networkedge.net (104.244.208.104) 0.297 ms
5 static.det-ix.net (209.124.52.26) 1.132 ms
6 one.one.one.one (1.1.1.1) 0.553 ms
A traceroute works by sending packets with a deliberately short lifespan. Each router along the way decrements the lifespan by one, and the router where it hits zero sends back an error. That error tells you who the router is and how long the round trip took. Repeat with a lifespan of 1, 2, 3 and so on, and you get a map of the path.
Hop by Hop
Hop 1: the gateway (23.170.200.97)
This is the default gateway for the subnet our VMs live on: the first router a packet touches after it leaves the host. It belongs to our upstream network provider, not to us, and it sits in the same facility as our hardware. Half a millisecond here is mostly the time it takes the packet to leave the virtual machine, cross the hypervisor, and reach the physical network.
Hop 2: aggregation (agr2.sfld1-mi)
The agr prefix is short for aggregation. An aggregation router collects traffic from many customer-facing ports and hands it up to the core. sfld1-mi is the site code: Southfield, Michigan, location 1.
Hop 3: core (core11.sfld1-mi)
The upstream's core router at the same Southfield site. Core routers carry the full routing table and make the decision about which way a packet leaves the network. Notice the time actually dropped compared to hop 2. That is normal: each router answers traceroute at its own speed, so the numbers do not have to climb in a straight line.
Hop 4: core (core10.sfld2-mi)
A second core router, named for a second Southfield site. The time is still under 0.3 ms, which tells you this is a metro-distance link, not a trip out of town.
Hop 5: the exchange (static.det-ix.net)
This is the interesting one. The address 209.124.52.26 is registered to the Detroit Internet Exchange. It is an address on the exchange's shared switching fabric, and a router answering from it is, almost certainly, Cloudflare's own router answering from its port on that fabric. In plain terms: this is the moment our upstream hands the packet to Cloudflare, across the exchange, without a transit carrier in between.
Two things look odd here and are not problems:
- The
* * *on the 1.0.0.1 trace. Many large networks rate-limit or ignore traceroute probes on their routers. The packet went through; the router just declined to talk about it. - Hop 5 is slower than hop 6. Routers forward traffic in hardware at line rate, but generating a traceroute error reply is handled by the router's much slower control-plane CPU, and it is the lowest priority job on the box. Hop 6 is answered by Cloudflare's servers, which reply promptly. So 1.132 ms at hop 5 and 0.553 ms at hop 6 is exactly what you would expect.
Hop 6: Cloudflare (one.one.one.one)
The destination. 0.380 ms to 1.0.0.1 and 0.553 ms to 1.1.1.1, round trip. Both addresses are anycast, meaning the same IP is announced from hundreds of Cloudflare locations and your packet lands at whichever one the routing picks. You can confirm which one answered with a single request:
curl -s https://1.1.1.1/cdn-cgi/trace | grep colo
colo=DTW
DTW is Cloudflare's Detroit location. So the packet never left the metro.
Why Sub-Millisecond Is Significant
Light in optical fiber covers roughly 200 km per millisecond. A 0.38 ms round trip is 0.19 ms each way, which puts a hard ceiling of about 38 km of fiber between the VPS and the Cloudflare server that answered, before you subtract any time spent inside routers. In practice that means the whole path stays within the Detroit metro, and most of the budget is switching, not distance.
Six hops matters for a related reason. Every network boundary a packet crosses is a place where it can be queued, rerouted, or dropped when that network has a bad day. Here there is exactly one boundary between our upstream and Cloudflare, and it is a local exchange port.
The alternative looks like this: a provider whose upstream is not on the exchange hands the same packet to a transit carrier, and that carrier exchanges traffic with Cloudflare wherever it happens to meet them, which can be Chicago. Our looking glass measures Chicago at about 10 ms round trip from here. Same destination, same Cloudflare, and one path is more than 20 times longer than the other.
If you run these yourself a few times, you will occasionally see a reply at 8 or 10 ms. That is ICMP being deprioritized somewhere on the path, not the path getting longer. The minimum across several runs is the number that describes the route. Across ten pings from our own host, the minimum to both resolvers was 0.41 ms.
What Detroit-IX Is, and Why the Building Matters
An internet exchange is, physically, a set of large switches in a data center. Networks that want to swap traffic with each other each run a fiber into that switch and agree to hand traffic across it directly, instead of paying a transit carrier to carry it for them. The result is fewer hops, fewer middlemen, and shorter paths between the networks that are connected.
The Detroit Internet Exchange (Detroit-IX) has 90+ connected networks, Cloudflare among them. It is on-site at 123Net DC1 in Southfield, which is the facility where RackWorks hardware is colocated. Being in the same building means the distance from our upstream to the exchange is a cross connect measured in meters, not a leased circuit across town. That is how you get from the VPS to a Cloudflare server in under half a millisecond.
To be precise about our role: RackWorks is not a Detroit-IX member, and we do not peer on the exchange ourselves. Our connectivity comes from a Detroit-IX connected upstream, and hop 5 is that upstream handing traffic to Cloudflare over the exchange. RackWorks does hold its own ASN, AS402908, but we have no allocated IP block yet, so we are not announcing our own prefixes today. What you get as a customer is Detroit-IX connected upstream routing, and the trace above is what that looks like in practice.
What This Means If You Run a Site Behind Cloudflare
When you put a site behind Cloudflare's proxy (the orange cloud), visitors connect to a nearby Cloudflare location, and Cloudflare connects back to your server, the origin, for anything it cannot serve from cache. That second leg, Cloudflare to origin, is the one this trace describes.
- Dynamic pages and API calls. Anything uncached (logged-in pages, checkout, API responses, admin panels) makes the origin trip on every request. For visitors whose nearest Cloudflare location is Detroit, that trip adds well under a millisecond. The time your application spends generating the response will dwarf it.
- Cache misses and revalidation. When the cache is cold or an object expires, the refill comes from your origin. A short origin leg keeps the first visitor after a purge from paying a visible penalty.
- TLS to the origin. With Full (strict) SSL, Cloudflare opens encrypted connections to your server. Every new connection costs a handshake of one or more round trips. At sub-millisecond round trips, that overhead disappears into the noise.
- Cloudflare Tunnel. If you use
cloudflaredinstead of opening ports, the tunnel connects out to nearby Cloudflare locations. From here, that means Detroit, a few hops away.
It also helps in the other direction, for traffic your server starts:
- DNS. Point your resolver at 1.1.1.1 and 1.0.0.1, and uncached lookups return at LAN speed. For an application that makes many outbound calls, that adds up.
- Services hosted on Cloudflare. A large share of SaaS APIs, webhooks, and object storage endpoints sit behind Cloudflare. The TCP and TLS setup to those services terminates in Detroit. What happens behind Cloudflare's edge after that depends on the service.
What It Does Not Do
This is a measurement of one path, and it is worth being clear about its limits.
- It does not beat physics for distant visitors. A visitor in Los Angeles connects to Cloudflare in Los Angeles, and the request still has to reach Detroit. The last stretch into our network is short; the distance across the country is not.
- A traceroute shows one direction. Return traffic from Cloudflare to us can take a different path. Routing on the internet is frequently asymmetric.
- It is not a guarantee for every destination. Networks connected to the exchange get this kind of path. Networks that are not connected get routed through transit like anywhere else.
- Your application is still the bottleneck. A 400 ms database query is a 400 ms database query. A fast network removes excuses; it does not remove slow code.
Check It Yourself
You do not have to take our word for any of this. From a RackWorks VPS, run:
traceroute 1.1.1.1
mtr -rwc 20 1.1.1.1
curl -s https://1.1.1.1/cdn-cgi/trace | grep colo
Before you buy, the RackWorks looking glass runs latency tests from the same network against destinations around the world. For the background on why we picked this facility in the first place, read Why I Colocated in Detroit.
If your site already sits behind Cloudflare, the origin you put behind it is the part you control. See KVM VPS plans in Detroit, or LXC containers if you want something lighter.