Slog - Sohrab's weblog

What's Moore's law got to do with the network?

Traditional wisdom is that today's network devices (routers, load balancers, firewalls,...) greatly benefit from specialized dedicated hardware to process network traffic. This is an artifact of the explosive growth in network demand in the late 80's, and early 90's when the ability of processing such high volumes of traffic out paced the capability of general purpose computing servers. There is however a disruptive effect brewing to this well established industry norm through the cumulative effect of Moore's law over this 16 year stretch, coupled with new multi-core chip architectures of today.

If the CPU's used in today’s servers are now fast enough to process tomorrow's traffic needs, then cost factors (driven by economies of scale) would suggest a shift away from expensive specialized hardware back to processing network traffic on general purpose servers. For example, building on the multi core / multi threads of the UltraSPARC T1, the Niagara 2 with its 64 threads of execution, dual 10G Ethernet on chip and advanced cryptographic support at wire speed has many of the capabilities necessary to build a network processing device from general purpose server components. Imagine the effect this has on a software router now capable of forwarding data in 64 parallel threads of execution and benefiting from the low latency of having dual 10G NIC's on a CPU. All this begs the question: Has Moore's law brought us to a point where we can replace most of the expensive, dedicated, network devices by a general purpose server?

There is a serendipitous consequence of running these devices on a general purpose server.  It opens up an important opportunity for us to run these network devices on a general purpose Operating System.

Since there is an overhead to running such performance sensitive network devices on a general purpose Operating System like Solaris, there is an argument in favor of using a special purpose Network Operating system over that of a general purpose one. We should however consider the trade-offs to any resultant loss in performance against the flexible environment offered by a general purpose Operating system. Common belief is that a general purpose operating system promotes a faster pace of innovation by offering a more accommodative environment for developers to build and debug network based applications/services. It also offers IT deployment a way of managing costs more effectively through a rich, standardized and flexible format for command and control. As an example consider the case where a general purpose Operating System would make it easier to not only increase utilization of the hardware by consolidating multiple "network devices", on the same hardware platform, but also reduce costs by providing an easier and more standardized way to manage the life-cycle (patching, upgrading,...) of these services.

I feel there is one other important redeeming quality in using a general purpose Operating System that merits mentioning. Advances in virtualization technologies on modern general purpose Operating Systems provide game changing flexibility to general purpose OS's. Multiple network devices (routers, web servers, firewalls, load balancers,...) can reside either cooperatively, or, through virtualization, as isolated identities on the same physical hardware machine. Stateful migration capabilities being built into virtualization technologies, can further optimize the resource management & utilization of physical hardware by these software network devices. Undoubtedly network devices will greatly benefit from such capabilities in a general purpose Operating System.

Eventually this comes down to the realization that an environment like networking can greatly benefit from the economies of volume and scale of a general purpose platform. All this is predicated on the belief that the cumulative effect of Moore's law and performance improvements that are made to general purpose OS's will provide a reasonable rate at which we can process network traffic.

Finally, it comes down to the question of which Operating System? Of the open source Operating Systems, FreeBSD and NetBSD are strong contenders due to their robust quality and the work put into improving network performance. Linux derivatives are also considered due to their familiarity and wide distribution/deployments.

For me, Solaris has some unique characteristics besides the numerous positive attributes like robustness, supportability,... that I feel position it well as a dominant player in this space.

* Project crossbow on Open Solaris, is now capable of providing a holistic approach to virtualization, by extending it to support the network. By, providing a class of service (amongst other attributes), critical performance can be tied to a "lane" with more resources.

http://opensolaris.org/os/project/crossbow/

* Solaris Containers can be used with IP instances, and the above "virtual" NICs, to create virtual network machines.
http://opensolaris.org/os/community/zones/

I have little doubt in my mind that all this coupled with a strong community around open source will foster a cottage industry of innovation that will very likely change the face of networking as we know it. I have helped initiate an Open Solaris project http://opensolaris.org/os/project/vnm, in an effort to help discuss and foster such a change. There is a virtual router sub-project that leverages a lot of the thoughts discussed already and I expect to have more sub-projects mushroom subsequently. It's really disruptive times like these when I feel most alive and appreciate the industry that we are a part of. 

Posted at 03:53PM Jun 13, 2007 by sohrab in Sun  |  Comments[0]