{"id":4038,"date":"2017-03-23T00:00:00","date_gmt":"2017-03-23T00:00:00","guid":{"rendered":""},"modified":"2025-06-16T08:42:41","modified_gmt":"2025-06-16T15:42:41","slug":"lighting-up-network-innovation","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/lighting-up-network-innovation\/","title":{"rendered":"Lighting up network innovation"},"content":{"rendered":"\n

Driving up network speed, reducing cost, saving power, expanding capacity, and automating management become crucial when you run one of the world\u2019s largest cloud infrastructures. Microsoft has invested heavily in optical technology to meet these needs for its Azure network infrastructure. The goal is to provide faster, less expensive, and more reliable service to customers, and at the same time, enable the networking industry to benefit from this work. We\u2019ve been collaborating with industry leaders to develop optical solutions that add more capacity for metro area, long-haul, and even undersea cable deployments. We have integrated these optical solutions within network switches and manage them through Software Defined Networking (SDN).<\/p>\n\n\n\n

Our goal was to provide 500 percent additional optics capacity at 10x reduced power, a fraction of the previous footprint at a lower cost than what\u2019s possible with traditional systems. Microsoft chose to ignore the chicken and egg problem and create a demand for 100 Gbps optics in a stagnant ecosystem unable to meet the demands of cloud computing. In this blog, we explain the improvements we\u2019ve made and where we\u2019re boldly heading next.<\/p>\n\n\n\n

Optical innovation leadership<\/h2>\n\n\n\n

We began thinking about how to more efficiently move network traffic between cloud datacenters, both within metro areas and over long distances around the world. We homed in on fiber optics, or \u201coptics,\u201d as an area where we could innovate, and decided to invest in our own optical program to integrate all optics into our network switching platforms.<\/p>\n\n\n\n

What do we mean when we talk about optics? Optics is the means for transmitting network traffic between our global datacenters. Copper cabling has been the traditional means of carrying data and is still a significant component of server racks within the datacenter. However, moving beyond the rack at high bandwidth (for example, 100 Gbps and more) requires optical technologies. Optical links, light over fiber, replace copper wires to extend the reach and bandwidth of network connections.<\/p>\n\n\n\n

Optical transmitters \u201cpush\u201d pulses of light through fiber optic cables, converting high-speed electrical transmission signals from a network switch to optical signals over fiber. Optical receivers convert the signals back to electrical at the far end of the cable. This makes it possible to interconnect high-speed switching systems tens of kilometers apart in metro areas, or thousands of kilometers apart between regional datacenters.<\/p>\n\n\n\n

To connect devices within the datacenter, each device has its own dedicated fiber. Since the light\u2019s optical wavelength, or color, is isolated by the fiber, the color used to make the connections can be reused on every connection. By using a single color, optic manufacturers can improve costs of high-volume manufacturing. However, single-color has a high fiber cost, particularly as distances increase beyond 500 meters. Although this cost is manageable in intra-datacenter implementations where distances are shorter, the fiber used for inter-datacenter connections, metro and long-haul, is much more expensive.<\/p>\n\n\n\n

Cost<\/h2>\n\n\n\n

We focused on optics because the cost can be 10x the cost of the switch port and even more for ultra-long haul. We began by looking for partners to collaborate on ultra-high integration of optics into new commodity switching platforms to break this pattern. Simultaneously, we developed open line systems (OLS) for both metro and ultra-long-haul transmission solutions to accept the cost-optimized optical sources. Microsoft partnered with several networking suppliers, including Arista, Cisco, and Juniper, to integrate these optics with substantially reduced power, a very small footprint, and much lower cost to create a highly interoperable ecosystem.<\/p>\n\n\n\n

\"Closed<\/figure>\n\n\n\n

Figure 1. Traditional closed line system<\/em><\/p>\n\n\n\n

In the past, suppliers have tried to integrate optics directly into switches, but these attempts didn\u2019t include SDN capabilities. SDN is what enables network operators to orchestrate the optical sources and line system with switches and routers. By innovating with the OLS concept, including interfaces to the SDN controller, we can successfully build optics directly into commodity switches and make them fully interoperable. By integrating optics into the switch, we can easily manage and automate the entire solution at a large scale.<\/p>\n\n\n\n

With recent OLS advances, we\u2019re also able to achieve 70 percent more spectral efficiency for ultra\u2013long haul connections of our datacenters between distant regions. We have drastically cut costs with this approach, more than doubling the capacity between datacenter when combining lower channel spacing with new modulation techniques that offer 150\u2013200 Gbps per channel. <\/p>\n\n\n\n

\"Open<\/figure>\n\n\n\n

Figure 2. Open line system (OLS)<\/em><\/p>\n\n\n\n

Power<\/h2>\n\n\n\n

In cloud-scale datacenters, power usage is a major consideration and can limit overall capacity of the solution. As such, we developed a new type of inexpensive 100 Gbps colored optic that fits into a tiny industry standard QSFP28 package to cover distances within metro area and the metro OLS that\u2019s needed to support it. This solution completely replaces expensive long-haul optics for distances up to 80 km.<\/p>\n\n\n\n

Due to miniaturization, integration, and innovations in both ultra\u2013long-haul and metro optics, network operators can take advantage of these new approaches and use a fraction of the power (up to 10x less power) while giving customers up to 500 percent more capacity. We\u2019ve expanded capacity and spectral efficiencies at lower overall cost in both capital expenditure and operating expenses than our current systems.<\/p>\n\n\n\n

Space<\/h2>\n\n\n\n

The physical equipment necessary to connect datacenters between regions takes a large amount of space in some of the most expensive datacenter realty. Optical equipment often dominates limited rack space. The equipment necessary to connect datacenters within a region can also require a large amount of space and can limit the number of servers that can be deployed.<\/p>\n\n\n\n

By integrating the metro optics and long-haul optics into commodity switching platforms, we\u2019ve reduced the total space needed for optical equipment to just a few racks. In turn, this creates space for more switching equipment and more capacity. By miniaturizing optics, we\u2019ve reduced the overall size of the metro switching equipment to half of its previous footprint, while still offering 500 percent more capacity.<\/p>\n\n\n\n

\"Inphi's<\/figure>\n\n\n\n

Figure 3. Inphi\u2019s ColorZ\u00ae product\u2014large-scale integration of metro-optimized optics into a standard switch-pluggable QSFP28 package<\/em><\/p>\n\n\n\n

Automation<\/h2>\n\n\n\n

Microsoft is focused on simplicity and efficiency in monitoring and maintenance in cloud datacenters. We recognized that a further opportunity to serve the industry lay in full automation of our optical systems to provide reliable capacity at scale and on demand.<\/p>\n\n\n\n

Monitoring for legacy systems can\u2019t distinguish optical defects from switching defects. This can result in delays in diagnosing and repairing hardware failures. For the optical space, this has historically been a manual process.<\/p>\n\n\n\n

We saw that we could solve this problem by fully integrating optics into commodity switches and making them accessible with our SDN monitoring and automation tooling. By driving an open and optimized OLS model for optical networking equipment, we\u2019ve ensured that the proper interfaces are present to integrate optical operations into SDN orchestration. Now automation can quickly mitigate defects across all networking layers, including service repair, with end-to-end work flow management. The industry benefits from this because optics monitoring and mitigations can now keep pace with cloud scale and growth patterns.<\/p>\n\n\n\n

Industry impact<\/h2>\n\n\n\n

Microsoft has incorporated all these technologies into the Azure network, but the industry at large will benefit. For example, findings from ACG Research<\/a> show that the Microsoft metro solution will result in a more than 65 percent reduction in Total Cost of Ownership. In addition, the research demonstrates power savings of more than 70 percent over 5 years.<\/p>\n\n\n\n

Several of our partners are making available the building blocks of the Microsoft implementation of open optical systems. For example:<\/p>\n\n\n\n