Virtualization is a trend in the development of Solaris. Its already rich set of
CPU and memory controls have been refined in Solaris Containers (or Zones) to fully virtualize workloads. ZFS introduced the concept of pooled storage to virtualize data management, and now Project Crossbow has introduced a new way to virtualize networking.
Project Crossbow fully virtualizes the Solaris network stack. Rather than a
one-size-fits-all approach, network interface cards (NICs) can now be virtualized
into one or more virtual NICs (VNICs). These VNICs can then be individually configured and tuned to take advantage of the physical NIC's in-hardware capabilities and workload needs.
The stack is even further virtualized by Etherstubs, the ability to create in-software
switches to which VNICs can be assigned. Branded vWire, or "network in a box," you can create networks that act like real physical networks but exist entirely in software.
The idea is that you could create, for instance, 100 Solaris Containers on a system, each
with a VNIC connected to an Etherstub, to form a complete functioning network of virtual servers that looks and feels like a real network but is entirely in software on one box.
Beyond just virtualizing network components, Crossbow has re-envisioned IP Quality of Service (IPQoS). For any interface, you can define "flows," which describe some type of traffic. A flow might be an entire interface or perhaps only HTTP and HTTPS traffic, for example. Resource controls can be applied to these flows, such as traffic priority (low, medium, high), CPU binding and, most importantly, bandwidth limitations. With Crossbow you have the ability to limit an interface to only, say, 10 Mbps or perhaps limit only SMTP traffic to 40Mbps so it doesn't overwhelm a gigabit link. Moreover, these flows can also be audited (logged) for monitoring and reporting purposes.
Crossbow introduces an important new distinction between interfaces and data link...
To continue reading for free, register below or login
To read more you must become a member of SearchDataCenter.com
s.
Traditionally, ifconfig has been the end-all, be-all of networking commands. But
the command has been so completely over-burdened with new functionality that Sun's
development team decided to introduce a new command that would handle data link
administration, appropriately named "dladm." This new command is used for managing
physical interfaces, creating VNICs and Etherstubs, creating and managing WiFi links or
port aggregations ("trunking" or "teaming"), etc. The idea is that you create and
manage data links with dladm and then interact with them as usual via ifconfig. Therefore, to use a VNIC, you use dladm to create a new VNIC from a physical
NIC, then use ifconfig to plumb and assign IP information to the VNIC,
just as you would any traditional NIC.
Simple Crossbow use case
The most basic use of Crossbow's capabilities would be to replace traditional virtual interfaces with VNICs. Most Unix operating systems have allowed you to associate multiple IP addresses with a single physical link (NIC) in the following way.
This might be done in order to support different virtual servers. The problem is that these virtual interfaces (:1, :2) share the same attributes as the parent link. They will appear to come from the same MAC address, they use the same tunings, they are on the same VLAN, etc.
If we instead use VNICs, we could simplify the configuration and provide far more flexibility.
You can see in this example that using dladm show-phys, we can view all our
physical interfaces (NICs) and use those to create VNICs which are
then configured using ifconfig just as though they were traditional network
interfaces. Even though more commands are involved, consider how much cleaner
the end result is. Also notice that each VNIC has its own MAC address, can
be assigned an individual VLAN ID ("VID" above), etc.
New levels of control
Crossbow introduces the concept of link properties. Using properties, we can
set a maximum bandwidth limit (in megabits per second), assign its processing
to specific CPUs, modify its processing priority or modify its tagging behavior.
In this example, we've downgraded vnic0's priority to medium, limited its processing
to CPU's 0 and 1, and capped its bandwidth to 10 Mbps. All of these parameters
are dynamic and can be changed at any time.
While having per data link control is great, Crossbow lets us get even more granular
with the flowadm command. A flow is defined as a particular classification of
traffic, typically identified by IP address, port, transport (IP, UDP, etc.) or
RFC 2474 DS-Field. We can associate priority and bandwidth properties to these flows.
This gives us simple IPQoS-like functionality without the painful configuration.
In this example, we've created two flows. The first is for the vnic0 link we created earlier, and it defines a flow for HTTP traffic, which has a medium priority and is limited to 200 Mbps. The second flow is for MySQL traffic, which isn't rate limited and is given high priority.
This combines to give us a multiple levels of control over how our data links are used
and provides a fine-grained capability to partition network capacity.
Additionally, dladm and flowadm can work in harmony with the Solaris Extended Accounting facility to provide auditing data; however, that topic exceeds the scope of this article. See the command man pages for details.
Empowering virtualization
Crossbow's ability to virtualize network interfaces and control how those data links are
used really comes to its full potential when used in conjunction with virtualization
technologies such as Solaris Containers, xVM (aka Xen) or VirtualBox. Unlike other
operating systems, VNICs provide a single, uniform way to manage network virtualization for all three technologies. Because VNICs act like real network interfaces, you provide full network capabilities to each virtual environments, yet maintain full auditing and resource control capabilities independent of the individual implementation.
Another of Crossbow's features are Etherstubs, virtual switches that can be used
to form internal private networks entirely in software. This is extremely powerful
because we can simulate complete network topologies on a single box. To use
them, we simply create a new Ethersub and create VNICs, which bind to the Etherstub
instead of a physical interface.
Here we've created an Etherstub named "vswitch" and then created new VNICs
that are attached to it. We could then provide these VNICs to our virtual environments
as a private network.
Things can get really interesting when you create multiple Etherstubs and
route between them. In the above example, for instance, we could assign both vnic2 and vnic3 to an xVM instance and then enable routing to allow a Solaris Zone on vnic5 access to the public network. The possibilities are endless. I've personally
used this capability as a way to prototype new network topologies using a variety
of experimental routing protocols.
I hope this article has given you a glimpse into the amazing capabilities Crossbow has to offer. With just a few easy commands, you can create all the virtual network interfaces you could ever want, mimic complex network topologies, audit link activity, assert fine-grained bandwidth controls, and much more. No more messing around with TUN/TAP drivers, no more fussing with complex IPQoS configurations -- just a powerful and generic tool that allows you to rethink modern network design.
ABOUT THE AUTHOR: Ben Rockwood is the director of systems at cloud computing infrastructure company Joyent Inc.. A Solaris expert and Sun evangelist, he lives just outside of Silicon Valley, Calif., with his smokin' hot wife Tamarah and their three children. Read his blog at cuddletech.com.
What did you think of this feature? Write to SearchDataCenter.com's Matt Stansberry about your data center concerns at mstansberry@techtarget.com.
