One of the best kept secrets about Solaris is its real-time capabilities. The foundation for the real-time capabilities is the initial design of Solaris. The designers “wanted the kernel to be capable of bounded dispatch latency for real-time threads which requires absolute control over scheduling, requiring preemption at almost any point in the kernel, and elimination of unbounded priority inversions wherever possible.” [Eykholt 92]
Several key features in the original Solaris design have been enhanced over the years and additional functionality has added to Solaris to make its time-critical processing capabilities more compelling.
Original Design Features
Interrupts as threads
Most UNIX-like operating systems have for many years dealt with the issue of synchronizing data between interrupts and regular kernel processing by ensuring that interrupts simply did not happen while the shared data was manipulated. This was accomplished either by totally disabling interrupts or ensuring that interrupts could only occur for devices whose interrupt priority is above that of the device whose data is being changed. The side effect of this is to either inhibit interrupts for all devices or inhibiting them for all devices below a certain priority level. The act of enabling/disabling interrupts is also an expensive operation.
Every entity that executes code in Solaris whether as a part of a user process or running solely in the kernel is represented by a thread. In prior versions of SunOS, interrupts captured the currently running process on a CPU and executed using that process' data structures. In Solaris2, interrupts are converted into threads with their own data structures. Interrupts can block on kernel synchronization primitives. This enables the ability to protect data structures in the Solaris kernel with synchronization primitives rather than raising and lowering interrupt priority levels.
Fully preemptible kernel
Many UNIX and UNIX-like operating systems use the idea of preemption points. When a kernel entity executing code (whether for the kernel or on behalf of a user process) encounters a preemption point, it looks around to see if there is another entity waiting to run and if so, causes preemption to occur. The main issue with this approach is what happens between the preemption points? A kernel entity may still block for an indefinite time waiting for a resource. The approach often taken is to add even more preemption points but this has diminishing returns as more and more time is spent passing through the points. There are also limits as to where preemption points may be placed.
Solaris is fully pre-emptible for processes in the real-time class. If a RT process becomes runnable, it will immediately be placed on a CPU if its priority is higher than the thread running on that CPU.
Real-Time scheduling
A key design feature of Solaris 2 was that “the scheduling of tasks in the kernel should be deterministic. ... the scheduler should provide priority-based scheduling for user tasks, so that the time-critical application developer has control of the scheduling behavior of the system; the kernel should provide bounded dispatch latency, so that time-critical user tasks are not subjected to unexpected and undesirable delays; the kernel should be free from unbounded priority inversions.” {Khanna 92]
Solaris offers a variety of schedulers using the System V scheduling class interfaces. These interfaces allow each schedulable entity within the kernel to have a scheduler appropriate to its needs. Entities needing real-time response latencies can use the real-time scheduling class which offers two options: round-robin or FIFO scheduling.
Round-robin scheduling assigns each priority level a time quanta. Entities in that priority level run until that time quanta expires. If another entity of the same priority is ready to run, it will run until its quantum expires. If no other entity is ready to run at that priority, the current entity will continue to run.
FIFO scheduling means that there is no time quantum assigned and that the entity will run until it voluntarily gives up the CPU or an entity with a higher priority becomes ready to run. Two entities with the same priority will run one after the one when one voluntarily gives up the CPU.
Entities in the real-time scheduling class are immediately scheduled to run if they become runnable and are of higher priority than what is currently running. Entities not in the real-time class will suffer delays when they become runnable even if they are of higher priority than what is running.
Priority Inheriting Synchronization primitives
The primary synchronization primitive in the Solaris kernel is the mutex. Mutexes are a light weight mechanism favored over spin locks. They are adaptive in that if the entity owning the mutex is not currently running then the entity attempting to gain the mutex is put to sleep as there is no chance of gaining the mutex while the current owner is not asleep. Mutexes (and other primitives) offer the possibility of “priority inversion.” A typical priority inversion is when a high-priority entity desires a mutex owned by a low priority entity. If the low priority thread becomes runnable, and by inference would then release the mutex, but is blocked from running by another entity whose priority is less than the high priority entity but greater than the low-priority entity. A recent well-known occurrence of this occurred in the first Mars Rover where a housekeeping task on the Rover caused an inversion that inhibited functioning of the Rover's guidance software.
So as to avoid this problem, Solaris mutexes implement the basic priority inheritance protocol as described in [Sha 90]. When the high level entity blocks, all of the entities blocking it are given the high level entity's priority. When they cease to block the thread, their priorities revert to their previous level.
Key Features Added
Processor Sets
Processor sets provide the ability to select one or more CPUs and isolate them from the rest of the system in terms of what entities are scheduled to run on them. Once a set is created, all processes currently run on the CPUs are moved to other CPUs. Processes with the appropriate privilege may add themselves to the set and be scheduled to run on CPUs within the set. It is also possible to force and interrupts currently being delivered to CPUs within the set to move to other CPUs, this isolating processes running on CPUs in the set from the effect of interrupt handlers disrupting processing. Processes and or threads within the process may be bound to individual CPUs within the set.
POSIX Compliance
Solaris has almost full support for what was once known as POSIX 1003.1b (“realtime”) and full support for POSIX 1003.1c (“threads”). These are now part of the IEEE 1003.1, 2004 Edition standard. Support of requisite real-time features is indicated by support of various option groups (e.g., Thread Priority Inheritance). These options are also grouped into larger bundles, of which four exist relating to real-time and threads. These are “Real-time”, “Advanced Real-time”, “Real-time Threads” and “Advanced Real-time Threads”. Solaris satisfies all but one option group within the “Real-time” group. That group is prioritized I/O and it is not clear that any operating system that does I/O request queuing would satisfy. Solaris fully satisfies the “Realtime Threads” option. Support for “Advanced Real-time threads” is missing one option group while “Advanced Realtime” support is missing a number of option groups.
Note that POSIX compliance only indicates that set of interfaces and behaviors exist. It says nothing about how capable an OS is of meeting response time requirements.
Fixed Priority Scheduler
While not truly a
real-time feature, Solaris does offer a fixed priority scheduler to
aid in processing. Unlike the normal time sharing scheduling class,
the fixed priority class does not modify a process' priority as it
runs. The priority changes only if the process asks for the change
and has the appropriate privilege for the change. This provides an
option for those processes who were placed in the real-time class
only because of the need for fixed priorities.
REFERENCES
[Eykholt 92]
J.R. Eykholt, S.R.Kleiman, S.Barton, R.Faulkner, A. Shivalingiah, M. Sm0ith, D. Stein, J. Voll, M. Weeks, D. Williams, Beyond Multiprocessing ... Multithreading the SunOS Kernel, Proceedings USENIX Summer 1992
[Khanna 92]
S. Khanna, M. Sebree, J. Zolnowsky, Realtime Scheduling in SUNOS5.0, USENIX, Winter 1992.
{Kleiman 95]
S.Kleiman, J. Eykholt, Interrupts As Threads, ACM SIGOPS Operating System Review, Vol. 21, Issue 2, April, 1995, pp 21-26.
[Sha 90]
L. Sha, R. Rajkumar, J,P, Lehoczky, Priority Inheritance Protocols: An Approach to Real-Time Synchronization, IEEE Transactions on Computers, Vol. 39, No. 9, September, 1990.