The art of virtualization : virtualization for mortals

ABSTRACT
In this entry I will demonstrate how to migrate a Solaris Zone running
on T5220 server to a new  T5220 server using ZFS as file system for
this Zone.
Introduction to Solaris Zones

Solaris Zones provide a new isolation primitive for the Solaris OS,
which is secure, flexible, scalable and lightweight. Virtualized OS
services  look like different Solaris instances. Together with the
existing Solaris Resource management framework, Solaris Zones forms the
basis of Solaris Containers.

Introduction to ZFS

ZFS is a new kind of file system that provides simple administration,
transactional semantics, end-to-end data integrity, and immense
scalability.
Architecture layout :

Prerequisites :

The Global Zone on the target system must be running the same Solaris release as the original host.

To ensure that the zone will run properly, the target system must have
the same versions of the following required operating system packages
and patches as those installed on the original host.

Packages that deliver files under an inherit-pkg-dir resource
Packages where SUNW_PKG_ALLZONES=true
Other packages and patches, such as those for third-party products, can be different.

Note for Solaris 10 10/08: If the new host has later versions of the
zone-dependent packages and their associated patches, using zoneadm
attach with the -u option updates those packages within the zone to
match the new host. The update on attach software looks at the zone
that is being migrated and determines which packages must be updated to
match the new host. Only those packages are updated. The rest of the
packages, and their associated patches, can vary from zone to zone.
This option also enables automatic migration between machine classes,
such as from sun4u to sun4v.


Create the ZFS pool for the zone
# zpool create zones c2t5d2
# zpool list

NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
zones   298G    94K   298G     0%  ONLINE  -

Create a ZFS file system for the zone
# zfs create zones/zone1
# zfs list

NAME          USED  AVAIL  REFER  MOUNTPOINT
zones         130K   293G    18K  /zones
zones/zone1    18K   293G    18K  /zones/zone1

Change the file system permission
# chmod 700 /zones/zone1

Configure the zone
# zonecfg -z zone1

zone1: No such zone configured
Use 'create' to begin configuring a new zone.

zonecfg:zone1> create -b
zonecfg:zone1> set autoboot=true
zonecfg:zone1> set zonepath=/zones/zone1
zonecfg:zone1> add net
zonecfg:zone1:net> set address=192.168.1.1
zonecfg:zone1:net> set physical=e1000g0
zonecfg:zone1:net> end
zonecfg:zone1> verify
zonecfg:zone1> commit
zonecfg:zone1> exit
Install the new Zone
# zoneadm -z zone1 install

Boot the new zone
# zoneadm -z zone1 boot

Login to the zone
# zlogin -C zone1

Answer all the setup questions

How to Validate a Zone Migration Before the Migration Is Performed

Generate the manifest on a source host named zone1 and pipe the output
to a remote command that will immediately validate the target host:
# zoneadm -z zone1 detach -n | ssh targethost zoneadm -z zone1 attach -n -

Start the migration process

Halt the zone to be moved, zone1 in this procedure.
# zoneadm -z zone1 halt

Create snapshot for this zone in order to save its original state
# zfs snapshot zones/zone1@snap
# zfs list

NAME               USED  AVAIL  REFER  MOUNTPOINT
zones             4.13G   289G    19K  /zones
zones/zone1       4.13G   289G  4.13G  /zones/zone1
zones/zone1@snap      0      -  4.13G  -
Detach the zone.
# zoneadm -z zone1 detach

Export the ZFS pool,use the zpool export command
# zpool export zones

On the target machine
 Connect the storage to the machine and then import the ZFS pool on the target machine
# zpool import zones
# zpool list

NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
zones   298G  4.13G   294G     1%  ONLINE  -
# zfs list

NAME               USED  AVAIL  REFER  MOUNTPOINT
zones             4.13G   289G    19K  /zones
zones/zone1       4.13G   289G  4.13G  /zones/zone1<
zones/zone1@snap  2.94M      -  4.13G  -

On the new host, configure the zone.
# zonecfg -z zone1

You will see the following system message:

zone1: No such zone configured

Use 'create' to begin configuring a new zone.

To create the zone zone1 on the new host, use the zonecfg command with the -a option and the zonepath on the new host.

zonecfg:zone1> create -a /zones/zone1
Commit the configuration and exit.
zonecfg:zone1> commit
zonecfg:zone1> exit
Attach the zone with a validation check.
# zoneadm -z zone1 attach

The system administrator is notified of required actions to be taken if either or both of the following conditions are present:

Required packages and patches are not present on the new machine.

The software levels are different between machines.

Note for Solaris 10 10/08: Attach the zone with a validation check and
update the zone to match a host running later versions of the dependent
packages or having a different machine class upon attach.
# zoneadm -z zone1 attach -u

Solaris 10 5/09 and later: Also use the -b option to back out specified patches, either official or IDR, during the attach.
# zoneadm -z zone1 attach -u -b IDR246802-01 -b 123456-08

Note that you can use the -b option independently of the -u option.

Boot the zone
# zoneadm -z zone1 boot

Login to the new zone
# zlogin -C zone1

[Connected to zone 'zone1' console]

Hostname: zone1

All the process took approximately five minutes

For more information about Solaris ZFS and Zones

ABSTRACT

This blog entry provides technical insight into the benchmark of the MySQL database on Xen Virtualization environment based on the xVM Hypervisor

Introduction to the xVM Hypervisor

 xVM hypervisor can securely execute multiple virtual machines simultaneously, each running its own operating system, on a single physical system. Each virtual machine instance is called a domain. There are two kinds of domains. The control domain is called domain0, or dom0. A guest OS, or unprivileged domain, is called a domainU or domU. Unlike virtualization using zones, each domain runs a full instance of an operating system.

Introduction to the MySQL database

 MySQL database is the world's most popular open source database because of its fast performance, high reliability, ease of use, and dramatic cost savings.

Tests Objective:

The main objective is to bring an understanding on how MySQL behaves within a virtualized environment, using UFS or ZFS file system

Tests Description:

We built a test environment by using a Sun X4450 MySQL 5.4 was installed on OpenSolaris 2009_06 because of the OS built-in integration with the xVM Hypervisor . A separate set of performance tests was run with MySQL data placed on a SAN disk. xVM guest OS is OpenSolaris 2009_06 .

When running under xVM the server resources ( cpu, memory) were divided between the dom0 and domU guest OS.

 dom0 - 2 vcpu and 2GB RAM

domU -  4 vcpu and 6GB RAM

• We used paravirtualized domU operating system in order to get the best performance.


• We chose to analyze the performance behavior for InnoDB storage engine due to its high popularity.


• We chose to analyze the performance behavior for two file systems (ZFS and UFS) in order to check which file system performs better for MySQL .

SysBench was used as loading tool to test base performance for each configuration.

The tool is simple to use, modular, cross-platform and multi-threaded. It also and can give a good feeling regarding the performance for a simple database use.  

Hardware configuration:

Server :SUN X4450 ,with 2X 2.9GHz dual-core CPU,8GB RAM , 2 X 146 GB internal disks.

Storage :StorageTek 6140 - configured RAID 0+1, directly attached to the server.

Software: MySQL 5.4, OpenSolaris 2009_06 .

The SysBench script

sysbench --test=oltp --mysql-table-engine=innodb --oltp-table-size=10000000 --mysql-socket=/tmp/mysql.sock --mysql-user=root prepare sysbench --num-threads=8 --max-time=900 --max-requests=500000 --test=oltp --mysql-user=root --mysql-host=localhost --mysql-port=3306 --mysql-table-engine=innodb --oltp-test-mode=complex --oltp-table-size=80000000 run

We tested it with different number of threads  4 ,8 ,16 ,32  (--num-threads=8 )  

The benchmark layout

After the creation of OpenSolaris 2009_06 in domU we attached the SAN storage

Attached the file system to the guest

xm block-attach para-opensolaris phy:/dev/dsk/c0t600A0B8000267DD400000A8D494DB1A6d0p0 3 w

Verified access to the file system from the guest

root@para-opensolaris:~# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
     0. c7d3 <DEFAULT cyl 4096 alt 0 hd 128 sec 32>
        /xpvd/xdf@3
     1. c7t0d0 <DEFAULT cyl 3915 alt 0 hd 255 sec 63>
        /xpvd/xdf@51712

zpool create -f xvmpool c7d3

root@para-opensolaris:~# zpool list
NAME      SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool    29.8G  10.6G  19.2G    35%  ONLINE  -
xvmpool  117.94G  89.5K  117.94G     0%  ONLINE  -

The first result of running this benchmark on UFS

The first result of running this benchmark on ZFS

The results after match ZFS recored size to block size and limiting  ZFS ARC size

zfs create -o recordsize=16k xvmpool/mysql

set zfs:zfs_arc_max = 0x10000000   in /etc/system

The results after disable ZFS cache flush ( We have battery backed cache) 

set zfs:zfs_nocacheflush = 1   in /etc/system

Conclusion

After ZFS tuning we were able to receive the same results as UFS  

Thus, we can benefit from ZFS extra features like snapshot and clone.

For more information about ZFS and OpenSolaris

COMSTAR is a software framework that enables you to turn any
OpenSolaris host into a SCSI target that can be accessed over the
network by initiator hosts. COMSTAR breaks down the huge task of
handling a SCSI target subsystem into independent functional modules.
These modules are then glued together by the SCSI Target Mode Framework (STMF).

COMSTAR features include:

    * Extensive LUN Masking and mapping functions
    * Multipathing across different transport protocols
    * Multiple parallel transfers per SCSI command
    * Scalable design
    * Compatible with generic HBAs

COMSTAR is integrated into the latest OpenSolaris

In this entry I will demonstrate the integration between COMSTAR and ZFS

Architecture layout :

You can install all the appropriate COMSTAR packages

# pkg install storage-server
On a newly installed OpenSolaris system, the STMF service is disabled by default.

You must complete this task to enable the STMF service.
View the existing state of the service
# svcs stmf
 disabled 15:58:17 svc:/system/stmf:default
Enable the stmf service
# svcadm enable stmf
Verify that the service is active.
# svcs stmf
 online 15:59:53 svc:/system/stmf:default
Create a RAID-Z storage pool.
The server has six controllers each with eight disks and I have
built the storage pool to spread I/O evenly and to enable me to build 8
 RAID-Z stripes of equal length.
# zpool create -f  tank \
raidz c0t0d0 c1t0d0 c4t0d0 c6t0d0 c7t0d0 \
raidz c1t1d0 c4t1d0 c5t1d0 c6t1d0 c7t1d0
raidz c0t2d0 c4t2d0 c5t2d0 c6t2d0 c7t2d0 \
raidz c0t3d0 c1t3d0 c5t3d0 c6t3d0 c7t3d0 \
raidz c0t4d0 c1t4d0 c4t4d0 c6t4d0 c7t4d0 \
raidz c0t5d0 c1t5d0 c4t5d0 c5t5d0 c7t5d0 \
raidz c0t6d0 c1t6d0 c4t6d0 c5t6d0 c6t6d0 \
raidz c0t7d0 c1t7d0 c4t7d0 c6t7d0 c7t7d0 \
spare c0t1d0 c1t2d0 c4t3d0 c6t5d0 c7t6d0 c5t7d0

After the pool is created, the zfs utility can be used to create 50GB ZFS volume.
# zfs create -V 50g tank/comstar-vol1

Create a logical unit using the volume.
# sbdadm create-lu /dev/zvol/rdsk/tank/comstar-vol1
Created the following logical unit :
GUID DATA SIZE SOURCE
--------------------------- ------------------- ----------------
600144f07bb2ca0000004a4c5eda0001 53687025664 /dev/zvol/rdsk/tank/comstar-vol1
Verify the creation of the logical unit and obtain the Global Unique Identification (GUID) number for the logical unit.
# sbdadm list-lu
Found 1 LU(s)
GUID DATA SIZE SOURCE
-------------------------------- ------------------- ----------------
600144f07bb2ca0000004a4c5eda0001 53687025664 /dev/zvol/rdsk/tank/comstar-vol1
This procedure makes a logical unit available to all initiator hosts on a storage network.
Add a view for the logical unit.

# stmfadm add-view GUID_number
Identify the host identifier of the initiator host you want to add to your view.
Follow the instructions for each port provider to identify the initiators associated with each
port provider.

You can see that the port mode is Initiator

# fcinfo hba-port
       HBA Port WWN: 210000e08b91facd
        Port Mode: Initiator
        Port ID: 2
        OS Device Name: /dev/cfg/c16
        Manufacturer: QLogic Corp.
        Model: 375-3294-01
        Firmware Version: 04.04.01
        FCode/BIOS Version: BIOS: 1.4; fcode: 1.11; EFI: 1.0;
        Serial Number: 0402R00-0634175788
        Driver Name: qlc
        Driver Version: 20080617-2.30
        Type: L-port
        State: online
        Supported Speeds: 1Gb 2Gb 4Gb
        Current Speed: 4Gb
        Node WWN: 200000e08b91facd
        Max NPIV Ports: 63
        NPIV port list:
Before making changes to the HBA ports, first check the existing port
bindings.
View what is currently bound to the port drivers.
In this example, the current binding is pci1077,2422.
# mdb -k
Loading modules: [ unix genunix specfs dtrace mac cpu.generic
cpu_ms.AuthenticAMD.15 uppc pcplusmp scsi_vhci zfs sd sockfs ip hook
neti sctp arp usba qlc fctl  
fcp cpc random crypto stmf nfs lofs logindmux ptm ufs sppp nsctl ipc ]
> ::devbindings -q qlc
ffffff04e058f560 pci1077,2422, instance #0 (driver name: qlc)
ffffff04e058f2e0 pci1077,2422, instance #1 (driver name: qlc)
Quit mdb.
> $q
Remove the current binding, which in this example is qlc.
In this example, the qlc driver is actively bound to pci1077,2422.
You must remove the existing binding for qlc before you can add that
binding to a new driver.

 Single quotes are required in this syntax.
# update_drv -d -i 'pci1077,2422' qlc
Cannot unload module: qlc
Will be unloaded upon reboot.
This message does not indicate an error.
The configuration files have been updated but the qlc driver remains
bound to the port until reboot.
Establish the new binding to qlt.
Single quotes are required in this syntax.
# update_drv -a -i 'pci1077,2422' qlt
  Warning: Driver (qlt) successfully added to system but failed to
attach
This message does not indicate an error. The qlc driver remains bound
to the port, until reboot.
The qlt driver attaches when the system is rebooted.
Reboot the system to attach the new driver, and then recheck the
bindings.
# reboot
# mdb -k

Loading modules: [ unix genunix specfs dtrace mac cpu.generic
cpu_ms.AuthenticAMD.15 uppc pcplusmp scsi_vhci zfs sd sockfs ip hook
neti sctp arp usba fctl stmf lofs fcip cpc random crypto nfs logindmux
ptm ufs sppp nsctl ipc ]
> ::devbindings -q qlt
ffffff04e058f560 pci1077,2422, instance #0 (driver name: qlt)
ffffff04e058f2e0 pci1077,2422, instance #1 (driver name: qlt)
Quit mdb.
 > $q

You can see that the port mode is Target

# fcinfo hba-port

        HBA Port WWN: 210000e08b91facd
        Port Mode: Target
        Port ID: 2
        OS Device Name: /dev/cfg/c16
        Manufacturer: QLogic Corp.
        Model: 375-3294-01
        Firmware Version: 04.04.01
        FCode/BIOS Version: BIOS: 1.4; fcode: 1.11; EFI: 1.0;
        Serial Number: 0402R00-0634175788
        Driver Name: qlc
        Driver Version: 20080617-2.30
        Type: L-port
        State: online
        Supported Speeds: 1Gb 2Gb 4Gb
        Current Speed: 4Gb
        Node WWN: 200000e08b91facd
        Max NPIV Ports: 63
        NPIV port list:

Verify that the target mode framework has access to the HBA ports.
# stmfadm list-target -v

Target: wwn.210100E08BB1FACD
Operational Status: Online
Provider Name : qlt
Alias : qlt1,0
Sessions : 0
Target: wwn.210000E08B91FACD
Operational Status: Online
Provider Name : qlt
Alias : qlt0,0
Sessions : 1
Initiator: wwn.210000E08B89F077
Alias: -
Logged in since: Thu Jul 2 12:02:59 2009

Now for the client setup :

On the client machine verify that you can see the new logical unit
# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 CD-ROM       connected    configured   unknown
c1                             scsi-bus     connected    configured   unknown
c1::dsk/c1t0d0                 disk         connected    configured   unknown
c1::dsk/c1t2d0                 disk         connected    configured   unknown
c1::dsk/c1t3d0                 disk         connected    configured   unknown
c2                             fc-private   connected    configured   unknown
c2::210000e08b91facd           disk         connected    configured   unknown
c3                             fc           connected    unconfigured unknown
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok
You might need to rescan the SAN BUS in order to discover the new logical unit
# luxadm -e forcelip /dev/cfg/c2
# format
Searching for disks...
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@0,0
1. c1t2d0 <FUJITSU-MAV2073RCSUN72G-0301-68.37GB>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@2,0
2. c1t3d0 <FUJITSU-MAV2073RCSUN72G-0301-68.37GB>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@3,0
3. c2t210000E08B91FACDd0 <SUN-COMSTAR-1.0 cyl 65533 alt 2 hd 16 sec 100>
/pci@7c0/pci@0/pci@9/SUNW,qlc@0/fp@0,0/ssd@w210000e08b91facd,0
Specify disk (enter its number):
You can see the SUN-COMSTAR-1.0 in the disk properties
Now you can build storage pool on top of it
# zpool create comstar-pool c2t210000E08B91FACDd0
Verify the pool creation
# zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT
comstar-pool 49.8G 114K 49.7G 0% ONLINE -
After the pool is created, the zfs utility can be used to create a ZFS volume.
#  zfs create -V 48g comstar-pool/comstar-vol1

For more information about COMSTAR please check the COMSTAR  project on OpenSolaris

Lots of cool staff  JavaFX,  Virtualization , Cloud computing , Mysql

I will have two sessions in the event :

1. Virtualization from the Desktop to the Enterprise

2. Sun in front of the cloud.

I am looking forward to see you 

Yesterday Sun hosted the Israeli Association of Grid Technology (IGT) event "IGT Cloud Computing WG Meeting" at the Sun office in Herzeliya. During the event Nati Shalom, CTO GigaSpaces, Moshe Kaplan, CEO Rocketier , Haim Yadid, Performance Expert, ScalableJ, presented various cloud computing technologies,There were 50 attendees from a wide breadth of technology firms.

For more information regarding using Sun's cloud see http://www.sun.com/solutions/cloudcomputing/index.jsp .

meeting agenda :

Auto-Scaling Your Existing Web Application Nati Shalom, CTO, Gigaspaces

 

In this session, will cover how to take a standard JEE web application and scale it out  or down dynamically, without changes to the application code. Seeing as most web applications are over-provisioned to meet  infrequent peak loads, this is a dramatic change, because it enables growing your application as needed, when needed, without paying for unutilized resources.  Nati will discuss the challenges involved in dynamic scaling, such as ensuring the integrity and consistency of the application, how to keep the load-balancer in sync with servers' changing location, and how to maintain affinity and high availability of session information with the load balancer. If time permits, Nati  will show a live demo of a Web 2.0 app scaling dynamically on the Amazon cloud.

 

How your very large databases can work in the cloud computing world?
Moshe Kaplan, RockeTier, a performance expert and scale out architect
Cloud computing is famous for its flexibility, dynamic nature and ability to infinite growth. However, infinite growth means very large databases with billions of records in it. This leads us to a paradox: "How can weak servers support very large databases which usually require several CPUs and dedicated hardware?"
The Internet industry proved it can be done. These days many of the Internet giants, processing billions of events every day, are based on cloud computing architecture such and sharding. What is Sharding ? What kinds of Sharding can you implement? What are the best practices?

Utilizing the cloud for Performance Validation
Haim Yadid, Performance Expert, ScalableJ
Creating Loaded environment is crucial for software performance validation. Execution of such a simulated environment required usually great deal of hardware which is then left unused during most of the development cycle. In this short session I will suggest utilizing cloud computing for performance validation. I will present a case study where loaded environment used 12 machines on AWS for the duration of the test. This approach gives much more flexibility and reduces TCO dramatically. We will discuss the limitation of this approach and suggest means to address them.

 

Logical Domains offers a powerful and consistent methodology for creating virtualized server environments across the entire CoolThreads server range:

   * Create multiple independent virtual machines quickly and easily
     using the hypervisor built into every CoolThreads system.
   * Leverage advanced Solaris technologies such as ZFS cloning and
     snapshots to speed deployment and dramatically reduce disk
     capacity requirements.

In this entry I will demonstrate the integration between Ldom and ZFS

Architecture layout

Downloading Logical Domains Manager and Solaris Security Toolkit

Download the Software

Download the zip file (LDoms_Manager-1_1.zip) from the Sun Software Download site. You can find the software from this web site:

http://www.sun.com/ldoms

 Unzip the zip file.
# unzip LDoms_Manager-1_1.zip
Please read the REDME file for any prerequisite
The installation script is part of the SUNWldm package and is in the Install subdirectory.

# cd LDoms_Manager-1_1

Run the install-ldm installation script with no options.
# Install/install-ldm

Select a security profile from this list:

a) Hardened Solaris configuration for LDoms (recommended)
b) Standard Solaris configuration
c) Your custom-defined Solaris security configuration profile

Enter a, b, or c [a]: a

Shut down and reboot your server
# /usr/sbin/shutdown -y -g0 -i6

Use the ldm list command to verify that the Logical Domains Manager is running
# /opt/SUNWldm/bin/ldm list


NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-c-- SP 32 16256M 0.0% 2d 23h 27m

Creating Default Services

You must create the following virtual default services initially to be able to use them later:
vdiskserver – virtual disk server
vswitch – virtual switch service
vconscon – virtual console concentrator service

Create a virtual disk server (vds) to allow importing virtual disks into a logical domain.
# ldm add-vds primary-vds0 primary

Create a virtual console concentrator (vcc) service for use by the virtual network terminal server daemon (vntsd)
# ldm add-vcc port-range=5000-5100 primary-vcc0 primary

Create a virtual switch service
(vsw) to enable networking between virtual network
(vnet) devices in logical domains
# ldm add-vsw net-dev=e1000g0 primary-vsw0 primary

Verify the services have been created by using the list-services subcommand.

# ldm list-services

Set Up the Control Domain

Assign cryptographic resources to the control domain.
# ldm set-mau 1 primary

Assign virtual CPUs to the control domain.
# ldm set-vcpu 4 primary

Assign memory to the control domain.
# ldm set-memory 4G primary

Add a logical domain machine configuration to the system controller (SC).
# ldm add-config initial

Verify that the configuration is ready to be used at the next reboot
# ldm list-config

factory-default
initial [next poweron]

Reboot the server
# shutdown -y -g0 -i6

Enable the virtual network terminal server daemon, vntsd
# svcadm enable vntsd

Create the zpool

# zpool create ldompool c1t2d0 c1t3d0

# zfs create ldompool/goldimage

# zfs create -V 15g ldompool/goldimage/disk_image


Creating and Starting a Guest Domain

Create a logical domain.
# ldm add-domain goldldom

Add CPUs to the guest domain.
ldm add-vcpu 4 goldldom

Add memory to the guest domain
# ldm add-memory 2G goldldom

Add a virtual network deviceto the guest domain.
# ldm add-vnet vnet1 primary-vsw0 goldldom

Specify the device to be exported by the virtual disk server as a virtual disk to the guest domain
# ldm add-vdsdev /dev/zvol/dsk/ldompool/goldimage/disk_image vol1@primary-vds0

Add a virtual disk to the guest domain.
# ldm add-vdisk vdisk0 vol1@primary-vds0 goldldom

Set auto-boot and boot-device variables for the guest domain
# ldm set-variable auto-boot\?=false goldldom
# ldm set-var boot-device=vdisk0 goldldom

Bind resources to the guest domain goldldom and then list the domain to verify that it is bound.
# ldm bind-domain goldldom
# ldm list-domain goldldom

NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     4G       0.2%  15m
goldldom         bound      ------  5000    4     2G

Start the guest domain
# ldm start-domain goldldom
Connect to the console of a guest domain
# telnet 0 5000
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
Connecting to console "goldldom" in group "goldldom" ....
Press ~? for control options ..

{0} ok

Jump-Start the goldldom

{0} ok boot net - install
We can login to the new guest and verify that the file system is zfs

# zpool list
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool  14.9G  1.72G  13.2G    11%  ONLINE  -
Restore the goldldom configuration to an "as-manufactured" state with the sys-unconfig command

# sys-unconfig
This program will unconfigure your system.  It will cause it
to revert to a "blank" system - it will not have a name or know
about other systems or networks.
This program will also halt the system.
Do you want to continue (y/n) y

Press ~. in order to return to the primary domain

Stop the guest domain
# ldm stop goldldom
Unbind the guest domain

# ldm unbind  goldldom
Snap shot the disk image
# zfs snapshot ldompool/goldimage/disk_image@sysunconfig

Create new zfs file system for the new guest
# zfs create ldompool/domain1

Clone the goldldom disk image
# zfs clone ldompool/goldimage/disk_image@sysunconfig ldompool/domain1/disk_image

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
ldompool 17.0G 117G 21K /ldompool
ldompool/domain1 18K 117G 18K /ldompool/domain1
ldompool/domain1/disk_image 0 117G 2.01G -
ldompool/goldimage 17.0G 117G 18K /ldompool/goldimage
ldompool/goldimage/disk_image 17.0G 132G 2.01G -
ldompool/goldimage/disk_image@sysunconfig 0 - 2.01G -

Creating and Starting the second  Domain


# ldm add-domain domain1
# ldm add-vcpu 4 domain1
# ldm add-memory 2G domain1
# ldm add-vnet vnet1 primary-vsw0 domain1
# ldm add-vdsdev /dev/zvol/dsk/ldompool/domain1/disk_image vol2@primary-vds0
# ldm add-vdisk vdisk1 vol2@primary-vds0 domain1
# ldm set-var auto-boot\?=false domain1
# ldm set-var boot-device=vdisk1 domain1
# ldm bind-domain domain1
# ldm list-domain domain1
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
domain1 bound ------ 5001 8 2G

Start the domain
# ldm start-domain domain1

Connect to the console
# telnet 0 5001
{0} ok boot net -s

Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Booting to milestone "milestone/single-user:default".
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface vnet0...
Configured interface vnet0
Requesting System Maintenance Mode
SINGLE USER MODE

# zpool import -f rpool
# zpool export rpool
# reboot

Answer the configuration questions

Login to the new domain and verify that we have zfs file system

# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
rpool 14.9G 1.72G 13.2G 11% ONLINE -

Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters.

For this setup we will use the following software packages:

1. Ganglia - the core Ganglia package

2. Zlib - zlib compression libraries

3. Libgcc - low-level runtime library

4. Rrdtool - round Robin Database graphing tool

5. Apache web server with php support

You can get the  packagers ( 1-3)  from sunfreeware (depending on your architecture -  x86 or SPARC)

Unzip and Install the packages

1. gzip -d ganglia-3.0.7-sol10-sparc-local.gz



   pkgadd -d ./ganglia-3.0.7-sol10-sparc-local


2. gzip -d zlib-1.2.3-sol10-sparc-local.gz 


   pkgadd -d ./zlib-1.2.3-sol10-sparc-local 


3. gzip -d libgcc-3.4.6-sol10-sparc-local.gz


   pkgadd -d ./libgcc-3.4.6-sol10-sparc-local

  

4. You will need pkgutil from blastwave in order to install rrdtool software packages

  
/usr/sfw/bin/wget http://blastwave.network.com/csw/unstable/sparc/5.8/pkgutil-1.2.1,REV=2008.11.28-SunOS5.8-sparc-CSW.pkg.gz 

  
gunzip pkgutil-1.2.1,REV=2008.11.28-SunOS5.8-sparc-CSW.pkg.gz 

  
pkgadd -d pkgutil-1.2.1,REV=2008.11.28-SunOS5.8-sparc-CSW.pkg

  


  

Now you can install packages with all required dependencies with a single command:

/opt/csw/bin/pkgutil -i rrdtool

  

5. You will need to download Apache ,PHP and Core libraries from Cool stack

Core libraries used by other packages

bzip2 -d CSKruntime_1.3.1_sparc.pkg.bz2



pkgadd -d ./CSKruntime_1.3.1_sparc.pkg

  

Apache 2.2.9,  PHP 5.2.6

 bzip2 -d CSKamp_1.3.1_sparc.pkg.bz2



 pkgadd -d ./CSKamp_1.3.1_sparc.pkg


The following packages are available:


  1  CSKapache2     Apache httpd


                    (sparc) 2.2.9


  2  CSKmysql32     MySQL 5.1.25 32bit


                    (sparc) 5.1.25


  3  CSKphp5        PHP 5


                    (sparc) 5.2.6


Select package(s) you wish to process (or 'all' to process


all packages). (default: all) [?,??,q]:1,3

 

Select the 1 and 3 option

Enable the web server service

svcadm enable svc:/network/http:apache22-csk 

 

Verify it is working

svcs svc:/network/http:apache22-csk



STATE          STIME    FMRI


online         17:02:13 svc:/network/http:apache22-csk

  

 Locate the Web server  DocumentRoot

grep DocumentRoot /opt/coolstack/apache2/conf/httpd.conf

 

DocumentRoot "/opt/coolstack/apache2/htdocs" 

  

Copy the Ganglia directory tree

cp -rp /usr/local/doc/ganglia/web  /opt/coolstack/apache2/htdocs/ganglia

 

Change the rrdtool path on  /opt/coolstack/apache2/htdocs/ganglia/conf.php

from /usr/bin/rrdtool  to /opt/csw/bin/rrdtool

 

 

Start the gmond daemon with the default configuration

/usr/local/sbin/gmond --default_config > /etc/gmond.conf

 

Edit /etc/gmond.conf  ,change  name = "unspecified"  to name="grid1"  (This is our grid name.)

Verify that it has started : 

ps -ef | grep gmond 

nobody  3774 1 0 16::57:41 ? 0:55 /usr/local/gmond

 

In order to debug any problem, try:

/usr/local/sbin/gmond --debug=9

 

Build the directory for the rrd images

mkdir -p /var/lib/ganglia/rrds

chown -R nobody  /var/lib/ganglia/rrds  

  

Add the folowing line to /etc/gmetad.conf

data_source "grid1"  localhost 

  

Start the gmetad daemon

/usr/local/sbin/gmetad

  

Verify it -->

 ps -ef | grep gmetad

nobody  4350     1   0 17:10:30 ?           0:24 /usr/local/sbin/gmetad
 

To debug any problem

  
/usr/local/sbin/gmetad --debug=9



  
Point your browser to:  http://server-name/ganglia



  




  


  


  


  

The integrated GigaSpaces GlassFish solution with its components is captured in the following diagram :

 

   SLA Driven deployment environment:

The SLA Driven deployment environment is responsible for hosting all services in the network. It basically does match making between the application requirements and the availability of the resources over the network. It is comprised of the following components:

• Grid Service Manager - GSM – responsible for managing the application lifecycle and deployment


• Grid Service Container – GSC – a light weight container which is essentially a wrapper on top of the Java process that exposes the JVM to the GSM and provides a means to deploy and undeploy services dynamically.


• Processing-Unit (PU )– Represents the application deployment unit. A Processing Unit is essentially an extension of the spring application context that packages specific application components in a single package and uses dependency injection to mesh together these components. The Processing Unit is an atomic deployment artifact and its composition is determined by the scaling and failover granularity of a given application. It, therefore,  is the unit-of-scale and failover. There are number of pre-defined Processing Unit types :

• Web Processing Unit – Web Processing Unit is responsible for managing Web Container instances and enables them to run within SLA driven container environment. With a Web Processing Unit, one can deploy the Web Container as group of services and apply SLAs or QoS semantics such as one-per-vm, one-per-machine, etc.  In other words, one can easily use the Processing Unit SLA to determine how web containers would be provisioned on the network. In our specific case most of the GlassFish v3 Prelude integration takes place at this level.


• Data Grid Processing Unit – Data Grid is a processing unit that wraps the GigaSpaces space instances. By wrapping the space instance it adds SLA capabilities avliable with each processing unit. One of the common SLA is to ensure that primary instances will not be running on the same machine as the backup instances etc. It also determines deployment topology (partitioned, replicated), as well as scaling policy, etc. The data grid includes another instance, not shown in the above diagram, called the Mirror Service. The Mirror Service is responsible for making sure that all updates made on the Data Grid will be passed reliably to the underlying database.

 

• Load Balancer Agent – The Load Balancer Agent is responsible for listening to web-containers availability and add those instances to the Load Balancer list when a new container is added, or remove it when it has been removed. The Load Balancer Agent is currently configured to work with the Apache Load Balancer but can be easily set up to work with any external Load Balancer.

How it works:

The following section provides a high-level description of how all the above components work together to provide high performance and scaling.

 

• Deployment - The deployment of the application is done through the GigaSpaces processing-unit deployment command. Assigning specific SLA as part of the deployment lets the GSM know how we wish to distribute the web instances over the network. For example, one could specify in the SLA that there would be only one instance per machine and define the minimum number of instances that need to be running. If needed, one can add specific system requirements such as JVM version, OS-Type, etc. to the SLA . The deployment command points to to a specific web application archive (WAR). The WAR file needs to include a configuration attribute in its META-INF configuration that will instruct the deployer tool to use GlassFish v3 Prelude as the web container for this specific web application. Upon deployment the GlassFish-processing-unit will be started on the available GSC containers that matches the SLA definitions. The GSC will assign specific port to that container instance. .  When GlassFish starts it will load the war file automatically and start serving http requests to that instance of the web application.


• Connecting to the Load Balancer - Auto scaling -The load balancer agent is assigned with each instance of the GSM. It listens for the availability of new web containers and ensures that the available containers will join the load balancer by continuously updating the load-balancer configuration whenever such change happens. This happens automatically through the GigaSpaces discovery protocol and does not require any human intervention.


• Handling failure - Self healing - If one of the web containers fails, the GSM will automatically detect that and start and new web container on one of the available GSC containers if one exists. If there is not enough resources available, the GSM will wait till such a resource will become available. In cloud environments, the GSM will initiate a new machine instance in such an event by calling the proper service on the cloud infrastructure.


• Session replication - HttpSession can be automatically backed up by the GigaSpaces In Memroy Data Grid (IMDG) . In this case user applications do not need to change their code. When user data is stored in the HttpSession,  that data gets stored into the underlying IMDG. When the http request is completed that data is flushed into the shared data-grid servers.


• Scaling the database tier - Beyond session state caching, the web application can get a a reference to the GigaSpaces IMDG and use it to store data in-memory in order to reduce contention on the database. GigaSpaces data grid automatically synchronizes updates with the database. To enable maximum performance, the update to the database is done in most cases asynchronously (Write-Behind). A built-in hibernate plug-in handles the mapping between the in-memory data and the relational data model. You can read more on how this model handles failure as well as consistency, aggregated queries here.

Here is how you can run the demo of GlassFish 3.0 on Amazon cloud. 

Where should I start? The best way to get started is to run a demo applicaiton and see for yourself how this integration works. To make it even simpler we offer the demo on our new cloud offering. This will enable you to expereince how a full production ready environment which include full clustering, dynamic scaling, full high avliability and Session replication works in one click. To run the demo on the cloud follow the follwoing steps:

1. Download the GlassFish web scaling deployment file from here to your local machine.

2. Get the mycloud page and get your free access code – this will enable you to get access to the cloud.

 

 

3. Select the stock-demo-2.3.0-gf-sun.xml then hit the Deploy button (you first need to save the attached file in one your local machine) – The system will start provisioning the web application on the cloud. This will include a machine for managing the cloud, a machine for the load-balancer and machines for the web and data-grid containers. After approximately 3 minutes the application will be deployed completely. At this point you should see “running” link on the load-balancer machine. Click on this link to open your web-client application.

.

4. Test auto-scaling – click multiple times on the “running” link to open more clients. This will enable us to increase the load (request/sec) on the system. As soon as the request/sec will grow beyond a certain threshold you’ll see new machines being provisioned. After two minutes approximately the machine will be running and a new web-container will be auto-deployed into that machine. This new web-container will be linked automatically with the load-balancer and the load-balancer in return will spread the load to this new machine as well. This will reduce the load on each of the servers.

5. Test self-healing – you can now kill one of the machines and see how your web client is behaving. You should see that even though the machine was killed the client was hardly effected and system scaled itself down automatically.

Seeing what’s going on behind the scene:

All this may seam to be like a magic for you. If you want to access the machines and watch the web containers, the data-grid instances and the machines as well as the real time statistics you can click on the Manage button button. This will open-up a management console that is running on the cloud through the web. With this tool you can view all the components of our systems. You can even query the data using SQL browser and view the data as it enters the system. In addition to that you can choose to add more services, relocate services through a simple click of a mouse or drag and drop action.

 

For more information regarding using Glassfish see http://www.sun.com/software/products/glassfish_portfolio/

 

There are two ways of accessing the public API, the simplest is by writing a Java client using JMX, or alternatively for non-Java client programs using Web Services for Management (WS-MAN, JSR-262).

 

This example demonstrates the usage of the direct access to the read-only copy of the domain model. This example can be run against both Sun xVM Server or Sun xVM Ops Center.

This example performs the following functions:

• Configures the connection


• Performs security settings


• Opens the connection (locally or remotely)


• Queries the domain model for all the OperatingSystems objects and displays the value of their Hosts name


• Closes the connection

 

ServerClient.java



/**

 * Copyright 2006 Sun Microsystems, Inc. All rights reserved.

 * SUN PROPRIETARY/CONFIDENTIAL. Use is subject to license terms.

 */

import com.sun.hss.type.os.OperatingSystem;

import com.sun.hss.type.server.Server;

import com.sun.hss.type.virtserver.VirtServerContainer;

import com.sun.hss.type.virtserver.VirtServerOperatingSystem;

import com.sun.hss.type.xvmserver.XVMApplianceDetails;

import com.sun.hss.type.xvmserver.XVMServer;

import java.util.HashMap;

import java.util.Map;


import java.security.cert.CertificateException;

import java.security.cert.X509Certificate;


import javax.net.ssl.X509TrustManager;

import javax.net.ssl.TrustManager;

import javax.net.ssl.SSLContext;

import javax.net.ssl.SSLSocketFactory;


import javax.management.MBeanServerConnection;

import javax.management.remote.JMXConnector;

import javax.management.remote.JMXConnectorFactory;

import javax.management.remote.JMXServiceURL;

import javax.management.ObjectName;

import com.sun.xvm.services.guest.GuestServiceMXBean;

import com.sun.xvm.services.xvmserver.XVMServerServiceMXBean;

import com.sun.xvm.services.guest.GuestDetails;


public class ServerClient {


    static private JMXConnector connector = null;

    static private MBeanServerConnection mbsc = null;

    static private String hostname = null;


    /**

     * Simple SocketFactory that uses a trust manager that ALWAYS 

     * accept the certificate provided by the server we try to connect to.

     *

     * This is unsafe and should NOT be used for production code.

     */

    private static SSLSocketFactory getSocketFactory() {

        X509TrustManager tm = new AnyServerX509TrustManager();

        TrustManager[] tms = {tm};


        try {

            SSLContext sslc = SSLContext.getInstance("TLSv1");

            sslc.init(null, tms, null);


            SSLSocketFactory factory = sslc.getSocketFactory();


            return factory;

        } catch (Exception ex) {


            return null;

        }

    }


    /**

     * Small trustmanager that ALWAYS accepts the certificate provided 

     * by the server we try to connect to. 

     *

     * This is unsafe and should NOT be used for production code.

     */

    public static class AnyServerX509TrustManager implements X509TrustManager {

        // Documented in X509TrustManager


        public X509Certificate[] getAcceptedIssuers() {

            // since client authentication is not supported by this

            // trust manager, there's no certicate authority trusted

            // for authenticating peers

            return new X509Certificate[0];

        }


        // Documented in X509TrustManager

        public void checkClientTrusted(X509Certificate[] certs, String authType)

                throws CertificateException {

            // this trust manager is dedicated to server authentication

            throw new CertificateException("not supported");

        }


        // Documented in X509TrustManager

        public void checkServerTrusted(X509Certificate[] certs, String authType)

                throws CertificateException {

            // any certificate sent by the server is automatically accepted

            return;

        }

    }


    /**

     * Create a WSMAN connection using the given credentials

     * 

     * @param host

     * @param userName

     * @param userPass

     * @return MBeanServerConnection 

     */

    private static void setupConnection(String host, String user, String pass)

            throws Exception {

        try {

            int port = 443;


            String urlPath = "/wsman/ea/jmxws";


            Map env = new HashMap();


            // credentials for basic authentication with the server

            String[] creds = new String[2];

            creds[0] = user;

            creds[1] = pass;


            env.put(JMXConnector.CREDENTIALS, creds);


            // provide a specific socket factory to avoid the need to setup

            // a truststore

            env.put("com.sun.xml.ws.transport.https.client.SSLSocketFactory",

                    getSocketFactory());


            // Create JMX Agent URL over https

            JMXServiceURL url = new JMXServiceURL("ws-secure", host, port, urlPath);


            //    System.out.println("WSMAN client opening a connection with url " + url.toString());


            // Connect the JMXConnector

            connector = JMXConnectorFactory.connect(url, env);


            // Get the MBeanServerConnection

            mbsc = connector.getMBeanServerConnection();


        } catch (Exception ex) {

            System.out.println("Got an exception while trying to open a WSMAN connection : " + ex.toString());

            throw ex;

        }

    }


    public static void main(String[] args) {

        if ((args.length == 0) || (args.length > 3)) {

            System.err.println("Usage:  user password [target]");

            System.exit(1);

        }


        String userName = args[0];

        String userPass = args[1];


        hostname = "localhost";


        if (args.length == 3) {

            hostname = args[2];

        }


        try {

            // Open the WSMAN connection and get the MBeanServerConnection

            setupConnection(hostname, userName, userPass);


            // get details on these xVM servers

            serverService();


        } catch (Exception ex) {


            System.out.println("WSMAN client error : " + ex.toString());

            ex.printStackTrace();

            System.exit(1);


        } finally {

            // close connection if necessary

            if (connector != null) {

                try {

                    connector.close();

                } catch (Exception dc) {

                }


            }

        }

    }


    private static void serverService() throws Exception {


        try {

            XVMServerServiceMXBean xssmxb = ServerClientServices.getXVMServerService(mbsc, false);


            // get the list of xVM  servers

            ObjectName[] servers = xssmxb.getXVMApplianceDetailsObjectNames(null, null);


            if ((servers == null) || (servers.length == 0)) {

                System.out.println("No xVM server detected on " + hostname);

                return;

            }


            GuestServiceMXBean guestmxb = ServerClientServices.getGuestService(mbsc, false);


            // get details on these xVM servers

            for (int i = 0; i < servers.length; i++) {


                XVMApplianceDetails details = xssmxb.getXVMApplianceDetails(servers[i]);


                if (details != null) {

                    OperatingSystem os = details.getOperatingsystem();


                    Server svr = details.getServer();


                    VirtServerContainer vsc = details.getVirtservercontainer();


                    XVMServer xsvr = details.getXvmserver();


                    if (xsvr != null) {

                        System.out.println("xVM Server name = " + xsvr.getApplianceName());

                    }


                }


                // get guests on this xVM server

                ObjectName[] guests = guestmxb.getGuestObjectNames(servers[i], null, null);


                Java.util.Map<java.lang.String, Java.util.Set<java.lang.string>> map = null;

                GuestDetails guestDetails[] = guestmxb.getGuestDetails(guests, map);


                String s = guests[0].getCanonicalName();


                if ((guests == null) || (guests.length == 0)) {

                    System.out.println("No guest on this xVM server");

                }

                if (guestDetails != null) {

                    for (int k = 0; k < guestDetails.length; k++) {

                        VirtServerOperatingSystem virt = guestDetails[k].getVirtServerOperatingSystem();

                        System.out.println("guest hostname is " + k + ": " + virt.getHostname());


                    }

                }

            }

        } catch (Exception ex) {

            System.err.println("Got Exception while testing the xVM server service : " + ex.toString());

            throw ex;

        }


    }

}

ServerClientServices.java

public class ServerClientServices
{
   /**************************************************************************
    * getCacheManagerService
    *
    * @param mbsc
    * @return CacheManagerMXBean
    * @throws java.lang.Exception
    **************************************************************************/
   public static CacheManagerMXBean getCacheManagerService(MBeanServerConnection mbsc, boolean verbose)
          throws Exception
   {
       CacheManagerMXBean cmmxb = null;

       try {
           // build the objectname to access the service
           ObjectName cmson = new ObjectName(CacheManagerMXBean.DOMAIN +
                                             ":type=" + CacheManagerMXBean.TYPE);

           // verify that the service is currently deployed
           if (!mbsc.isRegistered(cmson)) {
               System.out.println("Cache Manager service is not registered : aborting");
               throw new Exception("MXBean for Cache Manager service not registered");
           }

           // create a proxy to access the service
           cmmxb = JMX.newMXBeanProxy(mbsc,
                                      cmson,
                                      CacheManagerMXBean.class,
                                      false);

           if (verbose) {
               System.out.println("Proxy for Cache Manager service : OK");
           }
           return cmmxb;

       } catch (Exception ex) {
           System.err.println("Got Exception while creating the Cache Manager service proxy : " + ex.toString());
           ex.printStackTrace();
           throw ex;
       }
   }

This document describes how to build Iscsi server based on Solaris platform on sun X4500 server.

On the target (server)

The server has six controllers each with eight disks and I have built the storage pool to spread I/O evenly and to enable me to build 8 RAID-Z stripes of equal length.

zpool create -f  tank \

raidz c0t0d0 c1t0d0 c4t0d0 c6t0d0 c7t0d0 \

raidz c1t1d0 c4t1d0 c5t1d0 c6t1d0 c7t1d0 \

raidz c0t2d0 c4t2d0 c5t2d0 c6t2d0 c7t2d0 \

raidz c0t3d0 c1t3d0 c5t3d0 c6t3d0 c7t3d0 \

raidz c0t4d0 c1t4d0 c4t4d0 c6t4d0 c7t4d0 \

raidz c0t5d0 c1t5d0 c4t5d0 c5t5d0 c7t5d0 \

raidz c0t6d0 c1t6d0 c4t6d0 c5t6d0 c6t6d0 \

raidz c0t7d0 c1t7d0 c4t7d0 c6t7d0 c7t7d0 \

spare c0t1d0 c1t2d0 c4t3d0 c6t5d0 c7t6d0 c5t7d0

After the pool is created, the zfs utility can be used to create 50GB ZFS volume.

zfs create -V 50g tank/iscsivol000

Enable the Iscsi service

svcadm enable iscsitgt

Verify that the service is enabled.

svcs –a | grep iscsitgt

To view the list of commands, iscsitadm can be run without any options:

iscsitadm

Usage: iscsitadm -?,-V,--help Usage: iscsitadm create [-?] 

iscsitadm create target -b /dev/zvol/rdsk/tank/iscsivol000 target-label

IGT2008 video and presentations are now available online:

http://video.new-app.com/customers/grid/IGT2008/main.html

Yesterday  I want to the  IGT 2008  event  In addition to exhibiting at the event, I  delivered a opening demo presentation and a hands-on

xVM server workshop

.