event correlation by BGP is such a difficult task, if BGP4 can carry Lat/Lon information of each hop(even ibgp box) in announcement(1 time for each prefix per path/day i.e.), things can be much easier.
Submarine Cables Add Resilience But Paths Still Matter
|Last month, I had the honor of opening the second day of the Submarine Networks World 2013 conference at the Marina Bay Sands in Singapore. Using a handful of recent examples, I demonstrated that, while new submarine cables certainly contribute to physical diversity and hence increased Internet resiliency, cables alone do not necessarily reduce traffic latency. As we explained in our recent blog, when trying to understand performance across the global Internet, paths matter.|
We say that paths matter because while there are many tools for monitoring end-to-end latencies of Internet connections, understanding the path that traffic takes through the global Internet is vital to understanding the root cause of performance issues and in designing an effective Internet strategy for your target markets.
As I told the audience in Singapore, Renesys is not in the submarine cable industry. We are an Internet measurement company. Our unique ability to detail submarine cable events (failures and activations) comes as a fortunate side-effect of our extensive data sets and the capabilities we have built on top of them for observing structural changes in the global Internet. These changes can dramatically impact performance, sometimes for the better, sometimes for the worse. The most significant events are often those involving submarine cables.
Submarine cables are carefully engineered to minimize cost and latency. However, Internet routing–which determines the actual paths taken by traffic–plays by a different set of rules. Though the shortest physical path may be across a submarine cable, routing policy may steer the traffic a different way. When routing policy does align with physical infrastructure, latencies can drop dramatically, as in the following two examples.
In August, Tonga, an island nation of 100,000 people, got its first submarine cable, allowing it to forgo expensive and slow Internet satellite service. How expensive? In the telecom business, prices are typically quoted in terms of Mbps per Month (megabits per second per month or MM). In the competitive US market, this cost can be below $3/MM. For satellite service in the Pacific, the cost is around $3,000/MM — 1,000 times more expensive. How slow? Take a look at the following graphic that uses our latency measurements to illustrate Tonga’s Internet connectivity transition from satellite (600ms minimum) to submarine cable (40ms minimum), capturing the cable activation that occurred at 05:56 UTC on 5 August 2013.
ACE cable activation in Sierra Leone
|Funded in part by the World Bank, another recent cable activation was France Telecom’s ACE (“Africa Coast to Europe”) cable serving West Africa. With an additional US$15 million contributed by the government of China, the ACE cable came ashore in Sierra Leone and was activated in February in a ceremony attended by the country’s president. His Minister of Information and Communications called it a “landmark day in Sierra Leone.”|
This summer, an online publication in Sierra Leone questioned why Internet service from SierraTel, the country’s incumbent, to various government offices, including the Ministry of Information and Communications, was still slow. While there can be many reasons for slow Internet access, we can see that SierraTel may not be getting the maximum benefit of the new cable. In the diagram below, we compare the activations of SierraTel and Afcom, the “largest Internet and IP service provider in Sierra Leone” according to their website.
The downward shifts in both latency distributions mark the moments SierraTel (13:00 UTC, 21 February 2013) and Afcom (19:06 UTC, 20 February 2013) began routing traffic over the ACE cable. However, while parts of Western Europe can reach Afcom in less than 100ms, it can take over three times longer to reach SierraTel. Unfortunately, the in-bound path to SierraTel is still via the same satellite services in use prior to the ACE cable activiation.
This scenario suggests SierraTel is potentially misconfigured; asymmetrically using satellite for inbound traffic and submarine cable for outbound traffic. This is similar to the misconfiguration we saw in Cuba when ETECSA activated the ALBA-1 submarine cable in January, but this error was corrected a couple days later. We suspect a similar situation with SierraTel, which would explain the reports of continued slow Internet access.
Submarine Cable Industry Feeling the Pinch
|Advances in the state of the art of submarine cable technologies, such as Coherent technology, are enabling upgrades to existing cables that can increase capacity by orders of magnitude, such as the upgrade of NTT’s Pacific Crossing-1 to 100G this summer.|
At the same time that submarine cables are increasing in capacity, the growth of international bandwidth demand has been slowing due to advancements in distributed content delivery networks. These networks position content closer to users for faster delivery, reducing the need to reach overseas hosting centers via submarine cables.
An increase in supply combined with a decrease in demand makes it much more difficult to build a business case for a new submarine cable project. However, with this year’s revelations concerning National Security Agency (NSA) interception, a new motivating factor for future cables may be emerging: national sovereignty.
Following the news that her email correspondence had been intercepted by the NSA, Brazilian President Dilma Rousseff has announced plans to bypass United States for Brazilian traffic destined for other countries (Brazil, like most Latin American countries, connects to the global Internet through submarine cables that land in Miami, Florida).
National sovereignty may be the trump card that overcomes the one-two punch of increasing capacity and decreasing demand, and justify the installation of submarine cables to serve political interests. Brazil has come to realize that Internet paths matter, albeit for security as opposed to performance issues.
I concluded my talk in Singapore with a few examples of Internet hair-pinning in the Far East. Hair-pinning is the phenomenon in which Internet traffic gets unnecessarily routed through a faraway city, a common occurrence that clearly illustrates that the Internet is not designed for efficiency or performance. In other words, you cannot simply use Telegeography’s excellent Submarine Cable Map in the same way you might use a street map for determining likely Internet traffic paths. The following example traceroute illustrates an extreme case of hair-pinning, where traffic from Singapore to another part of Singapore first traveled through Los Angeles, California, over 13,000 kilometers away!
Nearly every provider in the world has idiosyncrasies like these. They can make global Internet performance very difficult to understand, let alone monitor. It’s Renesys’ business to help our customers understand these issues so they can effectively plan their Internet strategy and monitor the delivery of their services, and it is why we say paths matter.