Dr. Vincent Lin — Director, Corporate R&D
The rapid evolution of Artificial Intelligence (AI) has surpassed the capabilities of traditional monolithic compute architectures. The industry is shifting toward a systemic approach, where large-scale distributed clusters of GPUs and AI accelerators function as a single, unified computational engine to support the next generation of trillion-parameter models.
Co-Packaged Optics (CPO) offers substantial potential for enhancing the interconnection efficiency in this architecture, but it also faces challenges such as bandwidth density mismatches, complex assembly, and precise fiber alignment. This article explores both the opportunities and challenges of adopting CPO in next-generation AI clusters.
Technology Trends in Scale-Up and Scale-Out Solutions
AI data center infrastructure is defined by two distinct scaling strategies: Scale-Up and Scale-Out.
- Scale-Up: Focuses on creating a unified, high-performance computing entity within a rack, and possibly beyond the rack. Implementations such as the GB200 NVL72 use low-loss copper cables and protocols including NVLink, UALink, or Scale-Up Ethernet to interconnect dozens of GPUs through NVSwitch.
- Scale-Out: Connects multiple AI PODs across the data center to distribute massive workloads. This layer typically employs InfiniBand or Ethernet over longer distances.
While Scale-Out networks are adopting CPO switches to mitigate signal loss and power consumption over distance, the next step involves integrating silicon photonics into the Scale-Up layer to move beyond the bandwidth-distance limitations of copper cables.
The Evolution of Optical Engine Bandwidth and Efficiency
Silicon photonics-based pluggable optical transceivers have been available since 2018, but optical engines can now be integrated closer to switch ASICs as Near-Packaged Optics or within the same package as Co-Packaged Optics. Shortening electrical paths increases bandwidth, lowers power consumption, and improves scalability.
In five years, advancements in electronic integrated circuits and photonic integrated circuits integrated in optical engines have delivered a 64-fold increase in bandwidth density alongside a fivefold improvement in energy efficiency.
| Solution Type | Energy Efficiency | Bandwidth Capability |
|---|---|---|
| Conventional pluggable transceivers | 20–30 pJ/bit | Up to 800G |
| On-board optics | About 20 pJ/bit | Up to 1.6T |
| Co-packaged optics | Below 5 pJ/bit | 3.2T–6.4T |
Challenges in CPO Implementation
1. Bandwidth Density Mismatches
A critical disparity exists between the edge bandwidth density of AI chiplets and current optical interconnects. AI chiplets using UCIe or OIF standards achieve about 3 Tbps per millimeter, while state-of-the-art CPO solutions achieve about 0.5 Tbps per millimeter. This sixfold gap increases the need for smaller PICs and more scalable fiber array units.
2. CPO Assembly Complexity
CPO assembly involves lasers, optics, fiber array units, and silicon ICs across different wafer nodes. Photonic FOPoP positions the PIC on top of the EIC using wafer-level fan-out processes and copper pillars. Photonic 3D places the PIC at the bottom and uses TSVs to connect to a top-mounted EIC, improving thermal management and supporting data rates beyond 200G per lane.
3. Optical Alignment and Coupling
Active Alignment uses real-time feedback to optimize optical coupling but is time consuming. The industry is moving toward detachable fiber array unit solutions to improve repairability and upgradeability, creating new challenges in sub-micron alignment accuracy, repeatability, and high-volume manufacturing.
4. Known-Good Optical Engines
To maintain yield in complex CPO assembly, optical engines must be qualified before final assembly. Double-side wafer-level testing simultaneously interfaces with one side through automated test equipment and the other through detachable optical sockets. Packaging configurations must be reinforced to withstand socket and probing forces.
Looking Ahead: The Path to Widespread CPO Adoption
The journey toward scaling AI infrastructure through CPO and heterogeneous integration is both promising and challenging. ASE is at the forefront of this evolution with advanced 3D packaging solutions for EIC and PIC integration.
Next-generation CPO solutions require significant improvements in bandwidth density and energy efficiency. Collaboration across the semiconductor ecosystem—from EIC and PIC designers to packaging specialists and system integrators—is critical to meet the performance, efficiency, and scalability demands of next-generation AI.
