40G Ethernet MAC & PCS IP Core
The 40Gbps Ethernet IP solution offers a fully integrated IEEE802.3-2015 compliant package for NIC (Network Interface Card) and Ethernet switching applications. As shown in the figure below, the 40Gbps Ethernet IP includes:
∑ 40Gbps MAC core
∑ 40Gbps (40GBase-R) PCS core
∑ Optional FEC Core (Available separately)
∑ Technology dependent transceiver wrapper for Intel and/or Xilinx FPGAs
∑ Statistics counter block (for RMON and MIB)
∑ MDIO and I2C cores for optical module status and control
A complete reference design using a synthesizable L2 (MAC level) packet generator/checker is also included to facilitate quick integration of the Ethernet IP in a user design. A GUI application interacts with the reference designís hardware elements through a UART interface (a PCIe option is also available). A basic Linux PCIe driver/API is also provided for memory mapped read/write access to the internal registers with the PCIe interface option. See Appendix A for details.
MAC and PCS cores are designed with 128-bit data path operating at 312.5MHz.
As the transceiver wrapper is included with the Ethernet IP solution, the line side directly connects the 10.3125Gbps FPGA transceivers to the optical module (QSFP+, CFP, etc).
Ethernet IP solution implements two user (application) side interfaces. The register configuration and control port is a 32-bit AXI4-Lite interface. Depending upon the application layer, user can select a 128-bit @ 312.5MHz or 256-bit @ 156.25MHz AXI-4 streaming bus to interface with the MAC block.
40Gbps Ethernet IP supports advanced features like per-priority pause frames (compliant with 802.3bd specifications) to enable Converged Enhanced Ethernet (CEE) applications like data center bridging that employ IEEE 802.1Qbb Priority Flow Control (PFC) to pause traffic based on the priority levels.
MAC Core Features
● Implements the full 802.3ba specification with preamble/SFD generation, frame padding generation, CRC generation and checking on transmit and receive respectively
● Implements 802.3bd specification with ability to generate and recognize PFC pause frames
● Implements reconciliation sublayer functionality with start and terminate control characters alignment, error control character and fault sequence insertion and detection
● Implements a 128-bit XLGMII interface operating at 312.5MHz for 40G EMAC
● Implements Deficit Idle Count (DIC) mechanism to ensure maximum possible throughput at the transmit interface
● Implements logic for padding of frames on the transmit path if the size of frame is less than 64 bytes
● Implements fully automated XON and XOFF Pause Frame (802.3 Annex 31A) generation and termination providing flow control without user application intervention. Non PFC mode only.
● Pause frame generation additionally controllable by user application offering flexible traffic flow control
● Support for VLAN tagged frames according to IEEE 802.1Q
● Support any type of Ethernet Frames such as SNAP / LLC, Ethernet II/DIX or IP traffic
● Discards frames with mismatching destination address on receive (Except Broadcast and Multicast frames)
● Supports programmable promiscuous mode to omit MAC destination address checking on receive EMAC
● Optional multicast address filtering with 64-bit HASH Filtering table providing imperfect filtering to reduce load on higher layers
● CRC-32 generation and checking at high speed using an efficient pipelined CRC calculation algorithm
● Implements logic for optional padding removal on RX path for NIC applications or forwarding of unmodified data to the user interface
● Optional discard of runt frames (less than 64 Byte) at the coreís reconciliation sublayer or forwarding of runt frames to the user application interface
● Implements logic for optional forwarding of the CRC field to user application interface
● Implements logic for optional forwarding of received pause frames to the user application interface
● Programmable frame maximum length providing support for any standard or proprietary frame length (e.g. 9K-Bytes Jumbo Frames)
● Status signals available with each Frame on the user interface providing information such as frame length, VLAN frame type indication and error information
● Implements programmable internal XLGMII Loop-back
● Implements statistics indicators for frame traffic as well as errors (alignment, CRC, length) and pause frames
● Implements statistics and event signals providing support for 802.3 basic and mandatory managed objects as well as IETF Management Information Database (MIB) package (RFC 2665) and Remote Network Monitoring (RMON) required in SNMP environments
streaming user application interface. The application interface is
designed for either a 128-bit interface operating @ 312.5MHz or a
256-bit interface operating @ 156.25MHz.
PCS Core Features
● Implements 40GBase-R PCS core compliant with IEEE 802.3ba Specifications
● Implements a 128-bit XLGMII interface operating at 312.5MHz for 40G Ethernet
● Implements 64b/66b encoding/decoding for transmit and receive PCS
● Implements 40G scrambling/descrambling using 802.3ba specified polynomial 1 + x39 + x58
● Implements Multi-Lane Distribution (MLD) across 4 Virtual Lanes (VLs) for 40Gbps operation
● Implements periodic insertion of Alignment Marker (AM) on the transmit path and deletion on the receive path
● Implements 66-bit block synchronization and Alignment Marker Lock machines as specified in 802.3ba specifications
● Implements skew compensation logic in order to realign all the virtual lanes and reassemble an aggregate 40G stream (with all 64b/66b blocks in the correct order)
● Implements lane reordering to support reception of any virtual lane on any physical lane
● Implements BIP-8 insertion/checking per Virtual Lane on transmit/receive respectively
● Implements Inter Packet Gap (IPG) Insertion/Deletion for Alignment marker and clock compensation while maintaining a minimum of 1 byte IPG
● Implements gear-box logic to convert 66-bit blocks to 40-bit for 40G PCS. The 40-bit operate at the 10.3125Gbps transceiver reference clock.
● Implements programmable internal XLGMII loop-back which directs traffic received from core's receive path back to transmit PCS
● Implements Bit Error Rate (BER) monitor for monitoring excessive error ratio. In addition to that implements various status and statistics required by the IEEE 802.3ba such as block synchronization status, AM lock status, lane deskew and lane reordering status and BIP-8 error counters per virtual lane.
Optional FEC Core
● Optional FEC core compliant with IEEE 802.3-2012 clause 74 Specifications is also available
Licensing and Maintenance
∑ NO yearly maintenance fees for upgrades and bug fixes
∑ Basic core licensing for a single vendor (either Xilinx or Intel) compiled (synthesized netlist) binary
∑ Option for vendor and device family agnostic source code (Verilog) license
The core utilization summary for the 40G Ethernet solution is given in following tables. The utilization numbers are best in class as compared to other available 40G Ethernet cores. The Ethernet solution has been fully verified on different hardware platforms for both Intel and Xilinx FPGAs and has also been verified for interoperability with other 40G capable devices.
40G Ethernet - Resource Usage for Xilinx Devices
40G Ethernet - Resource Usage for Intel Devices
∑ Ethernet wrapper design with:
o Top level MAC and PCS wrappers (source files, Verilog) for user specific customizations
o Compiled synthesizable binaries (Netlists) for MAC and PCS cores
o Compiled synthesizable binaries (Netlists) for L2 packet generator and checker
o Technology specific transceiver wrappers for the selected device family
o Source code RTL (Verilog) for RMON and Register-File blocks
∑ UART/PCIe interface based reference design with:
o Top level wrapper (source files, Verilog) for user specific customizations
o Compiled synthesizable binaries (Netlists) for the I2C and MDIO cores
o Compiled synthesizable binaries (Netlists) for the UART or PCIe host interface
∑ Encrypted RTL for MAC, PCS and packet generator/checker for simulation
∑ Constraint files and synthesis scripts for design compilation
∑ PCIe driver/API (source files, C) for Linux with the optional PCIe host interface
∑ GUI application (Linux only) for interfacing to the reference design
∑ Design guide(s) and user manual(s)
Ported/Validated Modules List
1. HiTech Global HTG-K800, HTG-K816 and HTG-830; Xilinx Virtex Ultrascale and Kintex Ultrascale FPGAs; Interface through FMC (HTG-FMC-X2QSFP+) and Z-Ray (HTG-ZR-X2QSFP+) QSFP+ modules.
2. HiTech Global HTG-K700; Xilinx Kintex-7 FPGA; Interface through FMC (HTG-FMC-X2QSFP+) QSFP+ module.