Let's configure zones. I have a recently released version of Solaris 10 (build 74L2a),
hostname: unixman, and
the 3 zones I'm creating are called: zunixman, zunixman1, zunixman2.
One of the commands that we need to become familiar with is: zonecfg
For example: zonecfg -z
Please note: There are things here that may not be necessary for you to
begin with, such as resource controls perhaps. Our searchable website
over at http://docs.sun.com has the documentation of zones available
and you should check it out for the complete list of things that might
be of interest
in your environment.
Essentially, you start by feeding attributes of what how you want your
zone configured, one
command at a time:
Install a zone:
So, we execute it just by running "nspin &". This default behaviour
creates one cpu-bound
thread. We do this in 2 of our zones (we don't have to, but I did for
the fun of it).
Then, in our global zone, we can observe the usage between zones by
using 'prstat' with '-Z'
argument.
We can also see what is the usage amongst various zones. The
bottom-most 4 lines show
you distribution of CPU% among the zones based on the work requirement
of each zone at this time.
What else can we do ? Assuming we DID NOT KNOW what nspin is, or where
it is located.... and
we were called in to troubleshoot "performance problems" on this system.
What would we do ?
Well, we can either use the traditional means of trying to find what
'nspin' is. Nothing
wrong with that. Go off and run "find / -name nspin -print" and you'll
see where 'nspin'
lives on the file systems. May not be enough though. What would be nice is
to get an answer to the question as to what is REALLY being done by
nspin... ?
Any guesses as to what new facility in Solaris 10 we could use ?
Those of you who thought about Dtrace should pat yourselves on the back. Yes, the
Dynamic Tracing
framework that is introduced in Solaris 10 is built to instrument the
kernel on "live" systems.
Being armed with this, we can identify an offending process (say,
'nspin' from our process
table) and run the following:
After I type that command, almost immediately the following is printed back:
This says that given the current conditions specified, the dtrace
consumer (the command called
"dtrace" up above) has matched over 2945 probes in the kernel. Now, after
this line is printed, nothing happens. But why ? Well, the fact that
nothing
is printed does not mean that nothing is happening. What is in fact
happening is
that the kernel has enabled various probes and, based on your command
above, is
working to count how many times this pid enters a function. You can think
of this as allowing the kernel to collect statistics about what its
doing on the behalf
of this process - and maintain those statistics that filter through the
rule in your
command line.
When sometime has gone by (a few seconds, minutes is usually enough) you
can hit 'ctrl-c' to
stop the 'dtrace' consumer. What you'll get as STDOUT is something like
the following:
Just for grins, imagine that you were interested in another process that
runs in the zone, say
a 'tcsh' process, with a pid of 25941, like so:
(from another shell window)
Now, you want to run the same Dtrace command only on this process. Run
the below command,
and get to your other shell (tcsh) and invoke some activity on it. You
would have
to invoke some activity on it, or on another process had it been the one
you're looking at,
if you really wanted to see what it does when it does what it should :-)
So here goes:
Now, this is quite an extensive list, but what you can gather from this
is that 'memset' was
the function that was entered the most. If you are a developer or a
Systems Administrator
trying to troubleshoot this process, you should ask yourself why is this
happening ? This
is the way to troubleshoot performance problems, and this is a small,
and not by all means,
an exhaustive way to go about it in Solaris.
Nice, huh ? There's plenty more. In fact, plenty is not even enough
to describe this. Go ahead, have fun. Start making accounts and
doing all other
sorts of things. Read up on zones at docs.sun.com and our other public
forums, as well as the BigAdmin site and the developer.sun.com site.
'till next time.
Use 'create' to begin configuring a new zone.
zonecfg:zunixman>
Now, for those of you who have tinkered with computers for a while, this
is a comfortable
prompt that will take, as input, various commands pertaining to
sculpting a zone as
per your desire and available system resources. If you are unsure of
what to do, type "?"
at the prompt and you'll get an equivalent of paging a SysOp and getting
some guidance as to
what your options are. Very similar to the way Cisco IOS, and other
configurable tools (SMC, LDAP,
Sun's Application server, MySQL) and the like, work.
create
set zonepath=/export/allzones_configs/zunixman
Note, this directory must be created manually.
set autoboot=true
add fs
set dir=/export/home
set special=/export/allzones_data/zunixman/export/home
Note, this directory also must be created manually.
You continue ...
set type=lofs
add options [rw,nodevices]
end
add net
set address=192.168.0.1
set physical=eri0
end
add rctl
set name=zone.cpu-shares
add value (priv=privileged,limit=20,action=none)
end
add attr
set name=comment
set type=string
set value="This is zone named zunixman"
end
At the end, type:
verify
commit
...to ensure that the configuration syntax is verified and saved. See ?
This *does* have
a database-like approach to things. Whoever said that "commit" was just
an Oracle database thing? :-)
Verify that everything was fine:
zoneadm -z zunixman verify
If errors appear, fix them.
zoneadm -z zunixman install
Check the status based on the response to the above command:
zoneadm -z zunixman list -v
(Note, in my case the zone is already running. At this point, the
STATUS would be different).
root@unixman:/: 4:15pm > zoneadm -z zunixman list -v
ID NAME STATUS PATH
17 zunixman running /export/allzones_configs/zunixman
If the status says "incomplete", there was a problem. After you fix the
problem, un-install
the zone first:
zoneadm -z zunixman uninstall
Then make the corrections specified in the message, and try the
'zoneadm -z zunixman install' command again.
Once any errors (if any were) are fixed, make it ready and boot it -
after its installed:
zoneadm -z zunixman ready
zoneadm -z zunixman boot
You will see a process that copies necessary file structures for the
zone (root and dev) directories
into the /export/home directory tree we had specified above.
This process may take a few minutes, so while its happening, here's a
trick that
will help you ease the deployment for zones. You can save all of the
zone attribute commands
into a file, and then just pass the name of the file every time you're
building a new zone (remembering,
of course, to make slight modifications to IP/hostname/zonepath/others,
as necessary) values
to reflect the new zones you're building.
On my machine, it looks like this, for the last zone I built, called:
zunixman2
Note that the indentation is not necessary; but it helps make the file
more readable,
as the pieces pertaining to various objects are input.
create
set zonepath=/export/allzones_configs/zunixman2
set autoboot=true
add fs
set dir=/export/home
set special=/export/allzones_data/zunixman2/export/home
set type=lofs
add options [rw,nodevices]
end
add net
set address=162.124.52.20
set physical=eri0
end
add rctl
set name=zone.cpu-shares
add value (priv=privileged,limit=20,action=none)
end
add attr
set name=comment
set type=string
set value="This is zone named zunixman2"
end
verify
commit
When all of your zones are built and booted, you can run 'zoneadm' to
see all of them, and their state.
root@unixman:/: 4:15pm > zoneadm list -vi
ID NAME STATUS PATH
0 global running /
14 zunixman1 running /export/allzones_configs/zunixman1
15 zunixman2 running /export/allzones_configs/zunixman2
17 zunixman running /export/allzones_configs/zunixman
root@unixman:/: 4:25pm >
To login to the zone, use 'zlogin', and in fact you should do it as root
for the
first time because that will allow you to create further accounts, and
ensure
the run-time environment is ok. Sorta like an old MUD game. Login,
look around, see if there's
anything you don't like.... Of course, there's no one to talk to, so
you're on your own. :-)
To login to the console of a zone, use: 'zlogin -z zone_name'
The first thing you'll notice is that the zone is sharing the kernel
space with the global
zone (the zone with the ID of 0 in the output of 'zoneadm list -vi'). One
thing you can check to validate if you are in a zone or not, is to check
for the 'zsched' process. If its in your process table (run: pgrep
zsched),
you're likely in a private (non-global) zone.
So you've configured the zones and now you want to run things inside the
zones.
In this example, we'll consider that you are using a binary called
'nspin', which was
bundled in the previous release of Solaris Resource Manager (that was
version 1.3
on Solaris 8 environments). Now, you don't have to use that binary,
but I am
only using it to illustrate it as a simple cpu hog.
So, I copy that binary to each of my zones from within the global zone.
The nice
thing is that I don't have to "go through" the global zone. I could scp
or ftp (use scp if
you can) the binary directly into the zone. Once the binary ends up in
the zone,
it can be seen from the global zone by running a "find ..." command with
the name of the binary,
but that's not our goal here. We *know* that the binary is in the zone
and now we
want to execute it.
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
26037 root 1136K 760K run 10 0 0:15:07 52% nspin/1
25987 root 1136K 1072K run 20 0 0:14:26 44% nspin/1
535 root 74M 63M sleep 59 0 1:53:37 0.5% Xsun/1
26148 root 4992K 4512K sleep 49 0 0:00:08 0.4% prstat/1
26244 root 4992K 4688K cpu0 59 0 0:00:00 0.4% prstat/1
26177 root 4768K 4336K sleep 59 0 0:00:01 0.1% prstat/1
721 root 7888K 2296K sleep 59 0 0:16:07 0.1% sdtperfmeter/1
728 root 2112K 800K sleep 59 0 0:16:42 0.1% rpc.rstatd/1
4559 root 77M 54M sleep 59 0 0:20:33 0.0% mozilla-bin/4
4750 root 7864K 3336K sleep 49 0 0:00:09 0.0% dtterm/1
21723 root 3768K 2760K sleep 49 0 0:00:00 0.0% xterm/1
26204 yan 8160K 2584K sleep 59 0 0:00:00 0.0% sshd/1
867 root 4064K 2880K sleep 59 0 0:01:26 0.0% nscd/24
777 root 9688K 6048K sleep 49 0 0:03:07 0.0% dtterm/1
23056 root 3736K 2960K sleep 59 0 0:00:04 0.0% nscd/25
2636 root 5440K 1152K sleep 59 0 0:02:22 0.0% ssh/1
26215 root 2944K 2368K sleep 59 0 0:00:00 0.0% tcsh/1
24596 root 9936K 6952K sleep 59 0 0:00:08 0.0% svc.startd/12
ZONEID NPROC SIZE RSS MEMORY TIME CPU ZONE
14 31 111M 71M 7.0% 0:15:41 53% zunixman1
17 30 106M 67M 6.7% 0:14:57 44% zunixman
0 136 656M 319M 32% 3:15:25 1.7% global
15 36 138M 86M 8.6% 0:00:32 0.0% zunixman2
Notice how there are 2 nspin's being reported, with with 1 LWP/thread
(the first top-most lines).
# dtrace -n 'pid$target:::entry{ @[probefunc] = count() }' -p 25987
You can substitute the process ID of your own process that you see fit
using "-p" above.
Here, what I've done is asked the kernel to fire (show) probes as
various events occur by
printing the name of the function being entered and the quantity of
times this takes place.
dtrace: description 'pid$target:::entry' matched 2945 probes
^C
docpu 4206
Which tells you that the process you were interested in, called 'docpu'
routine 4206 times.
The complete picture of this Dtrace example looks like this on my
workstation:
root@unixman:/export/home/isaac> dtrace -n 'pid$target:::entry{
@[probefunc] = count() }' -p 25987
dtrace: description 'pid$target:::entry' matched 2945 probes
^C
docpu 4206
root@unixman:/: 3:22pm > ps -fe | grep 25941
root 25987 25941 44 14:28:11 zoneconsole 20:22 ./nspin
root 25941 24696 0 14:27:37 zoneconsole 0:00 tcsh
root 26275 26215 0 15:22:11 pts/17 0:00 grep 25941
root@unixman:/: 3:22pm > dtrace -n 'pid$target:::entry{ @[probefunc] =
count() }' -p 25941
dtrace: description 'pid$target:::entry' matched 5980 probes
^C
_sbrk_unlocked 2
sbrk 2
_brk_unlocked 2
tcsetpgrp 4
__setcontext 4
setcontext 4
enthist 4
__sighndlr 4
unsleep_self 4
pchild 4
sigacthandler 4
__sigsuspend 4
sigsuspend 4
call_user_handler 4
_sigpause 4
sigpause 4
pjwait 4
sigdelset 4
__schedctl 4
setup_schedctl 4
copylex 4
pwait 4
srchx 4
setpgid 4
cond_signal 4
palloc 4
_libnsl_parent_atfork 4
atexit_unlocks 4
mutex_held 4
fork_lock_exit 4
stdio_unlocks 4
libc_parent_atfork 4
_postfork_parent_handler 4
pthread_rwlock_unlock 4
__fork1 4
suspend_fork 4
Dfix 4
continue_fork 4
isbfunc 4
rw_read_held 4
rw_wrlock_impl 4
rwlock_lock 4
stdio_locks 4
atexit_locks 4
libc_prepare_atfork 4
pthread_rwlock_wrlock 4
_prefork_handler 4
lastchr 4
unparse 4
_libnsl_prefork 4
fork_lock_enter 4
fork 4
pfork 4
putn 4
getsid 4
tcgetsid 4
job_cmd 4
GetSize 5
Refresh 5
tty_getchar 5
tty_gettabs 5
tty_cooked_mode 5
tty_geteightbit 5
tcgetattr 5
tty_getty 5
ResetInLine 5
Inputl 5
btell 5
lex 5
sched_next 5
atoi 5
short2str 5
setalarm 5
precmd 5
period_cmd 5
sched_run 5
watch_login 5
pendjob 5
check_window_size 5
postcmd 5
rmstar 5
continue_jobs 5
alias 5
savehist 5
unreadc 5
memcpy 5
memmove 5
Cookedmode 5
PastBottom 5
e_newline 5
times 8
_waitid 8
_waitpid 8
waitpid 8
set_parking_flag 8
rw_write_held 8
tglob 8
___errno 9
execute 9
syntax 9
_cerror 9
tputs 10
Itoa 10
printprompt 10
alarm 10
tcsetattr 10
so_write 10
MoveToChar 10
freelex 10
StrQcmp 10
any 10
setjmp 10
ClearDisp 10
tty_setty 10
s_strlen 12
s_strcpy 12
_syscall6 12
sigon 12
trim 13
freesyn 13
setq 13
set1 13
set 13
blkfree 13
findenv 15
getenv 15
assert_no_libc_locks_held 15
getsystemTZ 15
localtime_r 15
pthread_getspecific 15
tsdalloc 15
localtime 15
tprintf 15
e_insert 15
sigismember 15
__sigaction 15
sigaction 15
sigset 15
MoveToLine 15
offtime_u 15
str2short 15
RefPlusOne 15
c_insert 15
ltzset_u 15
set_zone_context 15
cfgetispeed 15
tty_getspeed 15
calloc 16
time 19
__time 19
_read 20
mutex_unlock 20
GetNextChar 20
mutex_lock_impl 20
Load_input_line 20
read 20
mutex_lock 20
readc 25
write 25
Rawmode 25
_write 25
strcmp 30
ioctl 33
flush 40
lmutex_lock 44
lmutex_unlock 44
s_strsave 49
value1 53
sighold 55
sigrelse 55
_save_nv_regs 61
free 83
malloc 101
pthread_sigmask 129
sigprocmask 129
sigaddset 129
__systemcall6 133
__lwp_sigmask 133
sigemptyset 135
block_all_signals 137
adrof1 140
s_strcmp 144
sigvalid 148
putraw 240
SetAttributes 245
putpure 275
memset 4100
Isaac
Thoughts on Thin Computing, Latency, Minimization, Solaris and Multiplicity ...of...it...all Isaac's musings
Thursday Feb 10, 2005
So you heard about zones and you'd like to tinker with them a bit.
Well, here are my notes
from what I recently did on my workstation. Hopefully you'll benefit
from what you read
about here; let me know. The things used here are results of my own observations. Definitely
consider visiting Sun's online documentation site for more specific details.
Posted by fdasfdsa on October 12, 2006 at 09:21 AM EDT #