Introduction
The insatiable demand for interconnect bandwidth driven by emerging computing applications like Artificial Intelligence (AI) and Machine Learning (ML) has created a widening gap between in-package and off-package I/O bandwidth. Traditional electrical interconnects struggle to scale due to power and density constraints, making it challenging to address this gap. Optical interconnects, on the other hand, offer a promising solution with their inherent advantages of high bandwidth density, low power consumption, and longer reach capabilities. This tutorial focuses on the role of photonics in bridging the in-package and off-package I/O interconnect gap through the use of optical I/O chiplets and links.
Scaling Challenges for Off-package I/O
The growth of interconnect bandwidth requirements has outpaced the scaling of traditional electrical I/O interfaces. As shown in Figure 1, the gap between in-package and off-package I/O bandwidth continues to widen, creating a bottleneck for data transfer between chips and systems. Optical I/O chiplets aim to bridge this gap by providing high-bandwidth, energy-efficient, and scalable interconnect solutions.
Optical I/O Chiplet System Architecture
Optical I/O chiplets can be integrated into advanced package architectures to enable high-bandwidth die-to-die (D2D) interconnects between sockets, boards, or racks. As illustrated in Figure 2, an optical I/O chiplet acts as a bridge between the electrical D2D interfaces of two separate packages or sockets, facilitating data transfer through single-mode optical fibers. This architecture leverages an external multi-wavelength laser source, which acts as an optical power supply for the optical communication channels.
Building Blocks: Microring Modulators and Filters
The core building blocks of optical I/O chiplets are microring modulators and filters. Microring modulators (Figure 3) are compact devices that behave as tunable optical notch filters. They are doped with PN junctions, and applying voltages across these junctions changes the number of carriers in the modulator waveguide, effectively shifting the notch wavelength. By coupling the microring to a bus waveguide, specific wavelengths of light can be selectively modulated or filtered, enabling wavelength division multiplexing (WDM) for high-bandwidth data transmission.
Microring filters (Figure 4) capture and route specific wavelengths of light into photodetectors, enabling the demultiplexing of WDM signals. By cascading multiple microring modulators and filters along a bus waveguide, dense WDM (DWDM) systems can be realized, where each microring acts as an independent optical communication channel.
Chiplet D2D Interfaces: Linear Drive and Retimed
Optical I/O chiplets can interface with different D2D electrical interfaces, broadly categorized as linear drive (LD) and retimed. In the LD approach, the optics act as a pass-through, forming part of the serializer/deserializer (SerDes) channel. In contrast, the retimed approach decouples the optics from the D2D interface, introducing a retimer and allowing for independent optimization of the optical and electrical interfaces.
For compute-intensive applications like AI and ML, the retimed optical I/O chiplet approach offers advantages in terms of energy efficiency, bandwidth density, and reach. It can leverage state-of-the-art D2D interface technologies, such as the Advanced Interface Bus (AIB) or the upcoming Universal Chiplet Interconnect Express (UCIe), to address the off-package I/O interconnect gap.
Retimed Optical I/O Chiplet Design: Shasta
To demonstrate the potential of retimed optical I/O chiplets, a prototype called "Shasta" was designed and fabricated. Shasta is a 4.096 Tb/s full-duplex retimed optical I/O chiplet, featuring 2.048 Tb/s transmit and 2.048 Tb/s receive bandwidth on both the optical and electrical interfaces. It employs DWDM with 8 wavelengths per port, each operating at 32 Gbps, and supports 16 electrical AIB Gen 1 channels running at 2 Gbps per channel.
Process Technology and Fiber Attach
Shasta was built in GlobalFoundries' 45SPCLO silicon-on-insulator (SOI) process, which enables monolithic integration of optics and transistors on a 300mm platform. Passive alignment edge-coupling with V-grooves simplifies the packaging process by self-aligning the optical fibers into position. This combination of monolithic integration and V-grooves significantly reduces the complexity of fiber attachment.
Electrical Interface Design
The electrical interface of Shasta leverages the AIB Gen 1 chiplet interface, which originated from Intel/Altera FPGAs for chiplet disaggregation and was subsequently open-sourced. The AIB interface is well-suited for advanced packages with high I/O counts and sub-55μm bump pitches, enabling high bandwidth densities over short reaches.
Shasta's AIB interface implements 16 channels, each operating at 160 Gbps (80 Gbps transmit and 80 Gbps receive) with over 1 Tbps/mm of raw bandwidth density. The interface employs a full-rate internal architecture at 2 Gbps, with forwarded clocks compliant with AIB specifications. A single AIB I/O cell design is used for data, clock, and sideband signals, providing flexibility in handling different spare bump configurations.
Optical Transceiver Design
The optical interface of Shasta comprises eight optical transceiver macros, each supporting 8 wavelengths for transmit and 8 wavelengths for receive, totaling 256 Gbps per port (Figure 7). Each transceiver macro is divided into transmit and receive slices, with each slice containing the necessary components for modulating, transmitting, filtering, and receiving data on a single WDM channel.
On the transmit side (Figure 8), a modulator driver circuit generates the drive waveform for the microring modulator, while a closed-loop WDM control system maintains the microring's alignment with the desired wavelength. The transmitter employs an improved "half-ridge" modulator design, offering better modulation efficiency and thermal isolation compared to traditional ridge modulators.
The receiver slices (Figure 9) include circuits for amplifying and processing the photocurrent from the drop-port photodetector, as well as a closed-loop WDM control system for aligning the microring filter to the desired wavelength.
Measurement Results
Shasta's design and performance were thoroughly evaluated through various measurements and tests. The transmitter IQ error correction circuit successfully removed IQ errors from the drive waveform, ensuring accurate modulation. The optical transmitter exhibited a total penalty of -7.8 dB, with no discernible difference between PRBS7 and PRBS31 patterns at 32 Gbps.
On the receiver side, the median optical sensitivity measured at the input fiber was -11.7 dBm OMA (Optical Modulation Amplitude), including losses from connectors, fiber attach, waveguides, microring drop loss, and photodetector responsivity. The median receiver circuit swing sensitivity was 35 μApp, indicating the required photocurrent swing for error-free operation.
Loopback link dynamics testing between a transmitter and receiver on the same chip demonstrated healthy eye margins at multiple data rates, with open eyes even at 36 Gbps (12% over specification). The link achieved a bit error rate (BER) of less than 10^-12 without forward error correction (FEC) or error correction coding (ECC).
In multi-wavelength duplex link testing, Shasta achieved full 4 Tbps full-duplex link throughput using an 8-port, 8-wavelength on-board laser module called SuperNovaâ„¢. Out of 16 total links (8 in each direction), 11 links had zero errors over 10^15 bits, while the worst-performing link exhibited a BER of 2.3e-14 with an eye opening of approximately 0.25 UI for a 10^-10 BER.
Power measurements revealed a total power consumption of 126.7 mW for the transmitter and receiver combined, resulting in a link energy efficiency of less than 4 pJ/b. The clocking circuitry was the largest power consumer, while the tuning power was highly dependent on environmental conditions due to the microring heater power output.
Comparison with Other Works
Shasta represents a significant advancement in terms of the number of simultaneous running WDM channels and the total achieved data rate per fiber while maintaining a native BER of less than 10^-12. Additionally, it demonstrates the lowest required laser power per wavelength compared to other state-of-the-art optical transmitters and receivers.
Conclusion
The ever-increasing demand for interconnect bandwidth driven by emerging computing applications like AI and ML has created a pressing need for scalable and energy-efficient off-package I/O solutions. Optical I/O chiplets employing a retimed interface and leveraging photonic technologies like DWDM and microring-based links offer a promising approach to bridging the in-package and off-package I/O interconnect gap.
The Shasta retimed optical I/O chiplet, designed and demonstrated by Ayar Labs, showcases the potential of this approach. By achieving full 4 Tbps full-duplex link throughput with an energy efficiency of less than 4 pJ/b, Shasta paves the way for future high-performance computing systems that can efficiently handle the massive data flows required by AI and ML workloads.
As the demand for interconnect bandwidth continues to grow, the integration of photonics and chiplet technologies will play a crucial role in enabling scalable and power-efficient data transfer between chips, packages, and systems. Optical I/O chiplets represent a significant step towards realizing the full potential of photonic interconnects in addressing the off-package I/O interconnect gap.
Reference
[1] Sun, C. (2024). Photonics for Die-to-Die Interconnects: Links and Optical I/O Chiplets in 2024 IEEE International Solid-State Circuits Conference (ISSCC).
Comentários