Originally published in EETimes

As humans, we are primed to think in a linear way—even when we are thinking about AI. If we want to double the accuracy of AI models, you might reasonably assume we need to double the size of the AI training set and double the compute power. But as Peter Desantis, senior VP of Utility Computing Products at AWS, explained at AWS re:Invent last year, reducing AI model loss (another term for reducing AI’s inaccuracy) does not scale in a linear way with compute power. To halve AI model loss, we will need one million times more compute.

Compute for AI is energy intensive. The world’s energy grids are already straining to meet the demands of power-hungry data centres. Looking at the global picture, data centers consume 2% of the world’s energy supply, but this is set to double by 2026.

The scale of the AI challenge ahead

Clearly, we are on a collision course. Our ability to scale clean, renewable power is being outstripped by AI’s insatiable appetite for energy. Nations committed to net-zero emissions goals are facing tough choices: invest in nuclear, abandon emissions targets or fall behind in the AI race.

AI workloads require vast computational power. A rack of standard servers might run on 7 kW, while an AI rack can require over 100 kW. As AI accelerates, we can expect even higher energy costs for consumers, the rising price of compute stifling innovation and slower performance gains for AI.

The approach to this seemingly intractable problem that has attracted the most investment and attention is to increase grid capacity. Some operators and regional policymakers are exploring the nuclear option, with hyperscale operators including Google, Microsoft and Meta considering recommissioning old nuclear plants like Three Mile Island or placing small modular reactors next to hyperscalers. There is even research into putting solar arrays in space and beaming power back to Earth.

But even herculean efforts to increase grid capacity will move slower than AI’s potential to accelerate. If we want to accelerate AI, we must recognise that increasing grid capacity can only ever be part of the solution.

Rethinking AI infrastructure from first principles

We must do far more compute for far less power. Balancing that equation requires investment in grid capacity twinned with investment in energy-efficient data centre infrastructure.

Every aspect of the modern data centre needs to be reconsidered from first principles. At re:Invent 2024, AWS touted their new Graviton processors as improving power efficiency by up to 60%. The continued investment by AWS, Arm, NVIDIA and others to develop more efficient processors is to be applauded. But we need to look beyond compute clusters and across the whole data centre for efficiency and performance gains.

Another significant efficiency gain—one that simultaneously unlocks higher performance compute—is rearchitecting the energy-intensive technologies used for networking.

Network switches are the workhorses of every data centre. They route packets of data back and forth between GPUs and CPUs in data centres. These switches account for 20% of every data centre’s energy use. Conventionally, they have an optoelectronic architecture, which converts data signals between optical and electronic formats.

Incoming signals carried via optical fibres are converted into electronic signals by photodiodes. The electronic signal is then processed to determine its destination, then re-converted into light for transmission.

This process is highly inefficient. An electronic-to-optical conversion is required to switch every single data packet and is performed every time data is routed from server A to server B. Each conversion for each packet switch consumes power, limits throughput and introduces delay in a network process where every nanosecond counts.

Companies like Cisco and NVIDIA, who produce today’s state-of-the-art network switches, are working hard to optimise energy efficiency. However, they are all working with a legacy optoelectronic architecture that is fundamentally inefficient—and has comparatively few gains left in terms of performance.

Today, state-of-the-art network switches consume between 1 and 2 kW per unit and introduce an average latency of one microsecond. Across the thousands of network switches in a data centre, switching trillions of packets each day, this adds up to an enormous energy demand.

A passive breakthrough in optical networking

Reinventing networking to be exponentially more energy efficient is critical if we hope to see the continued acceleration of AI. The central challenge here is to eliminate the need for optoelectronic conversion entirely.

The field of optics has the answer. A fully optical network switch that can route data passively without the need for electronic signals bypasses conversion entirely. But how can a data packet’s destination be determined without any electronic signal?

There are a number of materials that have particularly interesting photonic properties. Take nonlinear photonic crystals, like lithium niobate, for example. This material is well-known in the quantum computing field where it is commonly used to entangle pairs of photons because it can be “tuned” to passively control the wavelengths/particles of light.

These “tunable” properties mean nonlinear photonic crystals have tremendous potential in network computing. Today’s network architectures send packets of data comprising the data itself and a packet header identifying the destination for the packet. By separating the header information and putting it onto a separate unique wavelength determined by the packet destination, and then sending this “control signal” in parallel with the data signal, opportunities open to eliminate the electronics.

By re-architecting network switches around nonlinear photonic crystals, it is possible to add the control signal to the data signal, effectively encoding routing information in the data packet without ever leaving the optical domain. The result is a hyper energy-efficient, ultra high-speed switch that operates entirely within the light spectrum.

Light is, of course, supreme when it comes to speed—and it does not require as much power to generate as an electrical signal. Accelerating AI can be achieved if, instead of using electronics to control light, we instead use light to control light. This reduces the energy demands of the network exponentially, while also massively increasing throughput capacity and reducing latency. In a nutshell, it unlocks the ability to do much more compute with far less power, far faster.

This new generation of fully optical, passive network switches measure latency in nanoseconds, not microseconds. Energy consumption of the network switch is not just reduced—it is almost eliminated. If every network switch was replaced with a fully optical passive one, we would immediately reduce the world’s total data centre energy use by almost 20%, while increasing compute performance by more than 50×.

Investing in the photonic future of AI

Photonics is a domain that has tremendous potential to increase energy efficiency and performance of AI architecture—from the network switch to specialised computation.

Only by twinning the investment in scaling grid capacity with investment in developing new energy-efficient AI infrastructure will we have the energy to harness AI’s potential to reshape our world.