Slicing a Partition out of a running T2000

Servers don't fall from trees, nor are they cheap! But these developers keep coming for more... you know what I'm talking about, right?

Well, if you have Sun's new Coolthreads Servers, you're in good shape. Sun has released it's new partitioning technology called Logical Domains (LDOM for short), and you can start slicing and dicing your server right away, using both LDOM's to create your new virtual servers and Solaris Containers for a finer and more dynamic resource control.

First, you have to check if you have all the stuff you need to play with LDOMs, and also get the software. You have to start here.

Once you have both the LDOM Manager and the Security Toolkit installed, you are set to go. Now we have to start by turning our physical server into a Logical Domain, by shrinking it to use up fewer resources, thus freeing space for our partitions.

# pkgadd SUNWldm
# svcadm ldmd start

Now we have a functioning Logical Domain Manager, and we can start issuing control commands:

# ldm list

We just have 1 domain thus far, using all of the available resources. Now lets start by adding support for a couple of services we are going to host in this Control
Domain, like the Disk Server, a Virtual Switch and a Virtual Console:

# ldm add-vdiskserver primary-vds0 primary

The Disk Server will be used to provide virtual devices to our guest domains, who will be using Virtualized I/O through our Control Domain

# ldm add-vswitch net-dev=e1000g0 primary-vsw0 primary

This Virtual Switch will enable layer 2 communications between guest domains. We need one of these for each network we are going to create inside this machine.

# ldm add-vconscon port-range=5000-5100 primary-vcc0 primary

The consoles for the domains will be available as a service in the Control Domain, and the port assignment is sequential within the range defined, meaning our first guest domain will use port 5000.

Now we'll just state how much resources we are going to assign to this Primary domain. Everything else will be available in a free pool of resources.

# ldm set-crypto 1 primary
# ldm set-vcpu 4 primary
# ldm set-mem 1g primary

We just configured 1 MAU, 4 vCPUs and 1GB of RAM. This should be more than enough for the Primary Domain, as we are going to deploy our applications on the guest domains.

# ldm add-config initial

We have now saved this configuration into the machine's Service Processor, and can now restart the server to activate the initial partitioning scheme and start the Virtual Terminal services

# shutdown -i6 -g0 -y
# svcadm enable vntsd

As the machine boots, it will come up as a 4-CPU, 1GB RAM system. We can now proceed to define our first guest domain, using some of the available resources:

# ldm create domain1
# ldm set-vcpu 2 domain1
# ldm set-mem 2g domain1

So this will be a 2-CPU, 2GB RAM virtual machine

# ldm add-vnet vnet1 primary-vsw0 domain1

This domain will connect to the virtual switch we had setup previously

# ldm add-vdiskserverdevice /dev/dsk/<some_device> vol1@primary-vds0
# ldm add-vdisk vdisk0 vol1@primary-vds0 domain1

We need to provide a device to be used as the boot disk of the new domain. We can choose from whole disks (internal or external), slices, ZFS volumes, or even disk images on files. It all depends on the purpose, the available resources, and your mileage. Either way, you end up with a virtual disk associated to the new domain. For now it should be clean, and we need to install Solaris (or linux) on it, but you can copy/clone existing disk images from other domains, so that you can speed up even further the setup of a new partition.

This new domain is unusable until you bind the resources (reserving them for exclusive use of this domain, even when it is not running).

# ldm bind domain1

You can choose to save this configuration with a different name, and have as many configs you like, to keep it dynamic

# ldm add-config config1

To start he domain, we just need to issue a simple command:

# ldm start domain1

We can now go to that domain's console, and boot (existing disk image), or jumpstart (net install) the OS. We can obviously set some variables up like the boot device, auto boot preference, and others. For more information, just grab the Admin Guide.

And just remember that you can dynamically add or remove vCPUs from a running domain, using these simple commands:

# ldm {add-vcpu|remove-vcpu} <number> domain1