I'm very happy to announce the release of a new version of the CPU Caps project. This new version is based on Solaris Nevada build 40 and comes with a number of changes made since last release:

• CPU caps should now be more accurately enforced.
• FSS shares now work under caps (e.g., capped zone with projects in FSS class)
• Added DTrace probe for when threads are placed on wait queues (sched:::wait) for better observability
• Maximum CPU cap value was raised from 65535 to 1000000
• CPU caps now work property with hires_tick set to 1
• Added new -N option to prstat to show unnormalized CPU usage percentages (i.e, 100% = 1 CPU)

Hopefully, more people will download and try this version on their systems now. As always, please forward your comments and/or questions to cpu-caps-beta@sun.com. The feedback received so far had been very positive.

If you ever wanted to limit CPU usage of some applications so that other applications can run faster or even waste some CPU cycles running the idle loop instead, then keep reading. If you always try to achieve maximum possible performance for all of your applications, then this feature is probably not for you, and you should pay a visit to the Performance Community on OpenSolaris :-)

Beta preview of upcoming Solaris feature called CPU caps was released today under brand new OpenSolaris Resource Management project web page. CPU caps can be set independently on projects and zones using two new resource controls. The unit used for these resource controls is a percentage of a single CPU. Processes can still migrate between different CPUs, but their overall CPU usage will be limited. If you want more details, go to the project web page.

I'm really excited about finally getting it released externally and hoping to get your feedback on how it does (or maybe it does not for some reason) fit your specific needs on cpu-caps-beta@sun.com. It would be really interesting if somebody tried this on a SunRay server (but please, do NOT run this in production yet!) and tell me how it behaves there and whether it can be used to keep bad behaving applications (or users) under control.

If you ever wanted to limit CPU usage of some applications so that other applications can run faster or even waste some CPU cycles running the idle loop instead, then keep reading. If you always try to achieve the maximum possible performance for all of your applications, then this feature is probably not for you, and you should pay a visit to the OpenSolaris Performance Community.

Beta preview of upcoming Solaris feature called CPU caps was released today under brand new OpenSolaris Resource Management project web page. CPU caps can be set independently on projects and zones using two new resource controls. The unit used for these resource controls is a percentage of a single CPU. Processes can still migrate between different CPUs, but their overall CPU usage will be limited. If you want more details, go to the project web page.

I'm really excited about finally getting it released externally and hoping to get your feedback on how it does (or maybe it does not for some reason) fit your specific needs on cpu-caps-beta@sun.com. It would be really interesting if somebody tried this on a SunRay server (but please, do NOT run this in production yet!) and tell me how it behaves there and whether it can be used to keep bad behaving applications (or users :) under control.

In pre-Zones era, the Fair Share Scheduler (FSS) was used to control how many CPU cycles are assigned by the system to active applications or workloads. The administrator could create projects to identify those workloads and assign CPU shares to them. The FSS scheduler, if used, would then guarantee that if these projects compete with each other for the same set of CPUs, each project would get a fraction of all CPU cycles that is proportional to the ratio between its number of shares and the total amount of shares given to all other active projects.

If processor sets were used, FSS would treat each processor set independently. That is, each processor set has its own total number of shares assigned to projects running on it. This is obvious because projects running on different processor sets do not (and can not) compete for the same CPUs.

In Solaris 10, when Zones were added to the mix, this behavior was preserved but we added as a "second level" (zone level on top of project level) scheduler. Shares can now be assigned to projects and zones. The more shares a zone has compared to other zones, the more CPU cycles will be assigned to projects that run inside it (if there's contention for CPU resources between the zones). The actual allocation of CPU cycles between projects in a zone is done the same way as before. Local zone administrator can divide CPU cycles between their projects, but cannot change the number of shares assigned to its zone, as this can only be done by the global zone administrator.

Now, if you want to use FSS to divide CPU cycles between zones, here's what you need to do. You can assign shares to zones permanently with zonecfg(1M), or change the assignment dynamically with prctl(1M). Both of these operations can be only done from the global zone. In the following example, zone "zone1" gets 15 shares statically assigned to it with zonecfg(1M):

global# zonecfg -z zone1
zonecfg:zone1> add rctl
zonecfg:zone1:rctl> set name=zone.cpu-shares
zonecfg:zone1:rctl> add value (priv=privileged,limit=15,action=none)
zonecfg:zone1:rctl> end
zonecfg:zone1> exit

This way, CPU shares will be automatically assigned to the zone at zone boot time. If the zone is already running, CPU shares can be assigned to it without rebooting by using prctl(1M):

global# prctl -r -n zone.cpu-shares -v 15 -i zone z1

To confirm that the number of CPU shares has changed, prctl(1M) can be used from within zone "z1":

z1# prctl -n zone.cpu-shares $$
process: 101439: zsh
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
zone.cpu-shares
        privileged         15       -   none                                 -
        system          65.5K     max   none                                 -

As you can see there's an upper limit (set to 65535) on how many CPU shares can be assigned to a zone or a project. The absolute values don't really matter because CPU shares are only used in comparison. Assigning 10 and 20 shares to two projects will have the same effect as if they would get assigned 10,000 and 20,000 shares.

One very important detail. The FSS scheduling class is not enabled by default. That is, projects or zones will ignore the CPU shares information if they are not running in the FSS class. The FSS scheduler can be set to be the default scheduling class for the whole system by using the -d option for dispadmin(1M) and rebooting:

global# dispadmin -d FSS
global# reboot

If reboot is not desireable, then all processes can be manually moved to the FSS scheduling class by using priocntl(1M):

global# priocntl -s -c FSS -i class TS
global# priocntl -s -c FSS -i class IA
global# priocntl -s -c FSS -i pid 1

Note that we explicitly avoid moving processes running in RT (Real-Time) and FX (Fixed) scheduling classes. To verify that all processes are now running in the FSS scheduling class, you can run the following command and look at the 4th column:

global# ps -cafe
...
    root 101050 101039  FSS  59 17:14:59 ?           0:00 /sbin/init 
    root 101169 101167  FSS  29 17:15:07 ?           0:00 /usr/lib/saf/ttymon
...

That's it, really. If you need more information about FSS, then a good place to start is FSS(7). Next time I'll talk about another way of dividing CPU time between zones by using the Resource Pools feature in Solaris 10.