On June 8, the internet panicked for an hour. Amazon, Twitch, Reddit, Twitter and Spotify, among many others, were inoperative due to a problem with the Fastly CDN. The content distribution network (CDN) fixed the problem in just under an hour, but this temporary drop was proof that the internet is in the hands of a few . A oligopoly in full rule.
Under most of the tools that we use daily, there are three companies that manage their servers: Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform. These large corporations also work from the hand in hand with CDNs so that, ultimately, the internet can work quickly and efficiently. For example: AWS may have the content of millions of pages stored in the United States, but the goal is that anyone can access that content without waiting. Regardless of where in the world you are. That is what the CDNs take care of. How? Through a network of servers distributed throughout the world that are responsible for serving copies of the content we demand as quickly as possible. In addition, they help to avoid certain attacks (such as DDoS) and avoid possible overloads.
The problem with CDNs – and one that explains the oligopoly of the internet – is the small number of companies that provide these services. Fastly, Cloudflare and Akamai have practically conquered the market and serve thousands of pages. This means that, if one of them falls, those thousands of websites fall with it. Precisely what happened on June 8, which highlighted how fragile the infrastructure that supports the internet can be.
The fall of the pages put in the spotlight the oligopoly of the Internet but also it happened the same with Fastly. The CDN responsible for the problem became in a few hours one of the most talked about companies in networks. The visibility was given by an internal error, but that did not prevent its market value from rising (at least for a few hours).
Image: Unsplash.
It's not easy to compete with Fastly
Fastly is now one of the most visible faces of CDNs. And as a result of his incident, the debate has arisen: How is it possible that the entire internet is in the hands of a few? Shouldn't we change the strategy so that a security breach of a CDN does not suppose the fall of thousands of pages? It is not that easy.
“In addition to AWS, Microsoft and Google, there are several other actors that have tried to sneak into that ecosystem: IBM with Bluemix, Alibaba,” explained Diego Suárez, CTO of Transparent Edge Services, a technology company that has the only Spanish commercial CDN. “The problem with competing here is size: it takes a lot of investment in machines and professionals to provide a service that is competitive against everything that the three giants contribute.” It is logical that the entry of new players in the market could even lower market prices, but, as Diego Suarez indicates, “it is complicated.” “A new competitor is very late in the race.”
Something similar to what we see in the cloud business happens to the CDN industry: a global, competitive service requires a lot of work and a lot of investment, and the profit margin in the CDN business is tight. To this is added one more reason, which is trust in the brand . Most of those who contract a CDN do it because they want absolute assurance that it will not fall and they are guided a lot by “what the majority use”, but every CDN is exposed to fall at some point.
CDNs, over time, have become an indispensable technology for the internet to work. They have become visible when one of the large ones has fallen, knocking down part of the web worldwide, but they go unnoticed 99.99% of the time , that is, when they are available and do that the webs load more quickly, agilely and safely.
A plan B, necessary and unlikely
For Corinne Cath, a researcher at the Internet Institute of the University of Oxford, the centralization of the internet and the oligopoly in the business is explained by the few incentives for clients to close deals with more than one CDN. “That way we would have the option of reducing the risks of having a situation like last week,” he told Hipertextual.
A situation that not only affected private companies, but also the official website of the United Kingdom Government. For this reason, Cath proposes that rules be imposed between governments that force to have a plan B in the event that a CDN suffers a drop in service. “Measures can be encouraged so that governments impose this obligation on themselves and have more than one CDN contracted.”
Much more difficult would be to try it in the market through the laws, he acknowledged. You cannot force a private company to diversify its contracts and, if you want to bet on a single supplier, you have every right to do so. However, in the case of government websites or those related to public services, the fall of the website could have a direct effect on the population (such as immigration or health-related procedures) if the gap is not solved. quickly.
Image: Unsplash.
The controversy of Cloudflare, the CDN that competes with Fastly
However, beyond the consequences of a practically global drop in a CDN, for Cath there is another more important red flag within the oligopoly that sustains the internet today. In 2017, Cloudflare was in the eye of the hurricane for suspending The Daily Stormer page. The American neo-Nazi website, behind the fateful Charlottesville demonstration, was the reason why the CRC had to break one of its most important values.
Cloudflare had previously stated that it is not in their hands to judge the content of the pages they manage and that their functions are far removed from those of a hosting company. However, The Daily Stormer publicly said that the company supported the ideology of the publication. Of course, Cloudflare couldn't allow his image to tie in with Nazi ideology. So they cut their losses and decided to suspend the account, which finally remained offline due to the DDoS attacks it received.
Beyond this specific case, the misadventure between Cloudflare and The Daily Stormer showed that a CDN can act as a kind of “guardian” of the internet. Although, as in the case of Cloudflare , that's not your intention, you could do it. “Is it correct that they suspend a page for clients even if we do not agree with their ideology? We have a problem when a company can make these types of decisions “, said Corinne Cath.
“These companies can make these decisions with political ramifications about what information is available to whom and in a different way than what happens on social networks like Twitter. Because in this case we are not talking about users, but about customers who pay for a service. “.
Corinne Cath, researcher at the University of Oxford Internet Institute,
The difficulty of ending the oligopoly of CDNs around the world
Having a decentralized internet would have advantages. Greater competition in the market and surely an optimization of prices, as well as a lower risk that a problem in a service causes the fall of hundreds of pages. At this time, the use of a CDN is practically mandatory to guarantee a fast and accessible website, recalled Diego Suárez. The alternative could be to set up your own infrastructure . “It is very complicated due to the level of knowledge necessary for it and the necessary infrastructure to have,” he said.
For now, there is no indication that the market will change so suddenly that the oligopoly will end. The only initiatives that intend to break this structure are companies like Transparent Edge Services.
An oligopoly in which the price of the premium CDN is very high “which therefore can only be used by companies with high resources, and low cost CDN, which are cheap but do not offer support, which means that the customer must have a high degree of specific technical knowledge to be able to handle them, something that is not easy at all, “explained Suárez. The objective of the Spanish company is to offer premium CDN services at more affordable prices.
Photo by Leon Seibert on Unsplash
What we needed to realize the fragility of the internet
The collapse of hundreds of web pages due to a human error in Fastly was fixed in less than an hour, but those minutes of chaos were enough to expose how an oligopoly that everyone knows works. The one in which a few have almost total control of the internet. No matter how large and complex it may seem, in the end everything hangs by a fine thread.
Reactions to how the CDN ecosystem works and the centralized internet model has also put the companies behind that oligopoly at the center of the conversation. Of course Fastly. On the same day the service fell, the company's shares rose; That can be explained by the sudden interest of some investors who were aware that day of the power that Fastly concentrates.
“ It has been a tremendous brand push . Practically nobody knew who Fastly was until this Tuesday, and many people have discovered that there is a company that is holding a good part of the internet. It practically never falls. Not only that: when it has fallen, the fall has lasted less than an hour “, said the Director of Technology of Transparent Edge Services.
However, some experts point out that over time, some customers may switch CDNs after failure. Although the company explained that the fall was caused by an unintended failure by a user, for some what happened can generate mistrust. Only time will tell if the incident will have an effect for Fastly beyond those minutes of despair over the worldwide downfall of the internet.
