The story behind the X-Lab isn’t just about the race to 10GBASE-T or making sure that when every server or workstation that needs more than one gigabit port, there’s a device able to reliably supply that bandwidth. The X-Lab exists to pave the way for the future of networking. In more than one way, networking scales alongside of CPUs and Moore’s Law. As systems are able to process more data, they need to exchange those greater data loads more quickly in order to maintain real-time functionality. Also consider the role of virtualization in networking. As companies condense five or ten servers into one physical machine, the networking load of those old servers gets jammed into a single box. What might have been 2-4 Gb/s of bandwidth spread across each of ten systems now has to flow in and out of only one, and this requires some fresh infrastructure changes.
Since 1978, Intel has led the industry through multiple speed transitions, from 10 megabit Ethernet through 100 megabit and gigabit Ethernet and now on to 10 gigabit Ethernet (10GbE). The IEEE may be responsible for the creation and supervision of the underlying specifications, but someone has to get their hands dirty and do the years of costly hardware development and validation. A huge chunk of that work gets done by Intel.
...
“Twinville will be used on our fourth-generation 10GBASE-T adapter product,” said Pete. “Our first-generation card here at the top was a single-port, 10GBASE-T adapter. It used the 82598, had a third-party PHY, and it came in right at the top of the 25 W limit for PCI Express. The PHY itself burned approximately 14 W of that power budget and required an active cooling solution. That product was introduced in 2007. We then ‘upgraded’ that 82598 MAC by coupling it with a second-generation, 65 nm 10GBASE-T PHY. With that single-port solution, we were able to lose the active cooling. Losing active cooling is critical for LAN-on-motherboard solutions. You don’t want more fans…although blue LEDs might be nice! Anyway, we called this the WWF heatsink because it almost looks like the WWF logo. That design gave us enough surface area to dissipate the heat without a fan, and the total power was significantly lower—about 16 W.
Then, using a similar 65 nm device, we were able to use the same active [PHY] heatsink but go to a dual-port solution. Total power bumped up to approximately 20 W, which is still lower less power than the first-generation device, but with two ports. It uses the 82599 media access controller and has less board complexity. There are fewer power components, for example.
And now, moving to our 40 nm device, Twinville, you can see that the MAC is integrated with the PHY. Actually, it’s not just one physical interface—it’s two. Also, it supports three speeds—100 Mb, 1 G, and 10 G—whereas the others only supported two speeds. And it’s expected to consume about 10 W. The Twinville we have in testing now uses an active heatsink, but by production it’ll be passive, just like the second-gen card.”
...
“10BASE-T has been around for a long time, and it’s even been demonstrated to work over barbed wire. There’s that much signal-to-noise ratio margin, even on a very lossy channel like barbed wire. It’s 6 V peak to peak, best-case, and the pulses are very wide. It takes a lot of bad things happening in the channel for a receiver not to see it.
100BASE-TX requires you to do some funky things to the signal. Instead of the two-level signal in 10BASE-T, there’s a three-level signal. If you look at the signal energy, it’s a 2 V peak to peak system, so there’s less power, but all of this scrambling and pulse shaping gets it to work. In 1998, that all wasn’t very straightforward, but with modern signal processing, it’s pretty easy to do 100BASE-TX.
Now, for gigabit, you start to get into some magic. There is more noise power than signal power, meaning we have a negative signal-to-noise ratio. That means if there’s a lot of background noise—like in this room now—and if we get that hammering noise so loud that you can’t hear me, that’s like a gigabit Ethernet noise environment. In 2000, 2001, there were some signal processing techniques applied, some special encoding and decoding that, at a high level, means you’re taking a best guess. You know what you’re sending out, and the receiver knows that there are certain expected combinations that will be coming back. So the system takes a best guess, to put it crudely, at what that data is. Better than 1 in 10-10 times, it makes the right guess.
But gigabit sucked up a lot of power and required, for the time, a lot of gates. At the gigabit inflection point, you started to have more gates than analog circuitry because we’re sending highly-encoded analog signals—those wiggly things with an amplitude and everything else. To encode and decode that properly requires a lot of logic gates. For 10GBASE-T, you just carry that concept to the next level. If gigabit is a whisper in a rock concert, 10GBASE-T would be like a whisper in a nuclear blast. It’s that much more noise power compared to the signal power. But with today’s digital signal processing techniques, you can make a signal have more apparent power. That’s one way to think of it. Again, the ratio of analog content to gates in 10GBASE-T is—wow. It’s very significant, with much, much more digital than analog content. This is good because it suddenly becomes very Moore’s Law-friendly, plus you get the advantage of power savings as you go to each new process node. In the lab here, we have 90, 60, and 40 nm technologies. The power savings associated with each generation has been key for our 10 Gb NIC products and the broader 10GBASE-T deployment in servers and switches.”
...
As you wonder at all of that cabling, think about your feet. In particular, think about your feet dragging across a carpet, then touching the light switch with your fingertip. Friction followed by static electricity buildup and discharge, right? Now imaging snaking those miles of cables through walls and crawlspaces, the friction causing electrostatic buildup on the cabling jackets. What do you suppose might happen when someone goes to plug that cable into a patch panel? Maybe nothing…or maybe not.
“I was a cable discharge skeptic until we actually pulled cables into the lab and I got zapped by a cable, “ says Pete.” Guys were working next door to us, pulling cable through conduits and onto racks and trays. So you pull the cable, you plug it into a port—well, the switch is going to provide a passive ground for that charge. We’re verifying the immunity of a networking device—a port—to that type of a discharge. It’s different than the typical ESD testing we do anywhere else in Intel or at any semiconductor company. Those involve looking at the movement of people, like across a carpet, or the movement of machines building up a charge as they operate. But this is a special type of ESD that really only appears in the networking world.”
...
Intel doesn’t charge for this support. Anybody who buys Intel silicon can get these test results for their product. The University of New Hampshire has its Interoperability Test program, and it’s the most renowned lab in this space, but, according to my tour hosts, the consortium fees can add up fast for a low-margin manufacturer.
“So we’ll provide all that support plus review your design and suggest what components to use on magnetics or resistors or whatever,” said Matt. “We may not always be the lowest-cost solution, but we bring a lot to the table.”
Fascinating read if you're into that kind of thing
