I wonder who of all the smart people in Sun Labs + Engineering invented the time machine used to achieve this, but it's still great to know we're mentioned in the book of books - Ecclesiastes 2:20
Had a good laugh when stumbling over this one ;-)
I wonder who of all the smart people in Sun Labs + Engineering invented the time machine used to achieve this, but it's still great to know we're mentioned in the book of books - Ecclesiastes 2:20
Had a good laugh when stumbling over this one ;-)
And power corrupts, as we all know...
Thing is, the 20th century was the era when thinking independent thoughts, experimentation and pursuing knowledge was generally held in high esteem. There have been bitter lapses into totalitarism with horrible consequences of course, whether that'd be Nazism, Stalinism, McCarthy's anti-communism or others. But at all times, scientists, thinkers and do-ers had the possibility to evade politically motivated repression by choosing to go elsewhere - and would be received with open arms. For example, this special case in US immigration rules still allows bypassing visa requirements under certain circumstances. It's reasonable to say that for the most part of the last century, people were encouraged to experiment and research.
If you grew up in the 1970s / 1980s as I did, you might remember the days when your parents took you to that play farm where everyone got a few pieces of wood, hammers, nails plentiful, saws, drills and all sorts of tools to build that soapbox racer. The splinters of wood in your fingers, the throbbing thumb you hit with the hammer just when the last nail should go in or even the little cut when the saw slipped wouldn't bother you. The next day, you'd boldly brave the slope in your Ferrarini until you ran it into an obstable, it'd break and you'd get out all bruised but nonetheless happy. It was all part of the experience. As was, when you were a little older, being caught and fined by local police because you tuned your first motorcycle to go just a little bit faster than what it legally should ...
Or those days when you got to use your elder cousin's chemistry set. Remember what sort of a shock it was when you detonated the first electrolytic gas you made yourself and the bang was far louder than you'd expected ? How your eyes burned before because you got a bit too much of a whiff of the chlorine because you put too much salt into the water ? How your mother pushed back at you when the jeans had another hole etched in from nitric acid ? When you thought it might be a good show trying to get that magnesium strip to ignite in pure oxygen you created by heating permanganate, and it "ignited" a bit too well, blew the flask and the splinters cut your hand ?
Or the physics/electronics sets. Ah, that ozone smell from the home-made tesla generator ! It was a funny thing to throw a charged capacitor over to your unsuspecting friend, and see him dance after he "caught" it. And you got a bad cough when inhaling a bit too much of the fumes the tin solder (which was actually at least half lead anyway) gave off when you did that first circuit board of yours. Not to count the shocks and burns, those times you happened to touch mains power ...
And the computer - a fascinating device. You definitely did explore a bit of programming, typing in the program listings, or even - gasp - "hexcode" from Byte magazine. First you ape it, then you start to understand some of it, and you'd be proud once you've done the first "hack" of your own - even if it was no more than changing some program's startup banner to say "myself was here - gr33t1n95 !". Later at school, or maybe at College/Uni, there was that mythical beast of the network. Sniffing networks to get the teacher's password ... you were sent off for a day for that, but you eventually learned the IPX protocol as well.
And it was easy to get access to all these things. A good teacher actually noticed when you overstepped the 'care boundaries' and tried to get your hands onto the mercury flask in chemistry lessons (it was so marvellously heavy ...), and took you aside, telling you about the dangers and gave you a book to read about the stuff. Your mom got bothered about the hole in your jeans because that meant you wouldn't value them and wouldn't take care of yourself, not because it was some dangerous chemical substance that did it. Your dad took next week's pocket money when your electrical experiments had blown the flat's main fuse again - but you kept that electronics kit, of course. You'd be told to repaint the neighbour's garage door that got stained from the self-made gunpowder, and instructed to do that sort of thing somewhere else but in the courtyard behind the house. Punishment for my first hack at Uni was very simple - I was forced to become assistant system administrator, and the responsible prof would request progress reports on what sort of audit / penetration tests I did with what results.
But most important of all - you got a feeling of pride from all this. Your parents would smile at you when you told them how much fun that latest experiment was (although you knew better than to tell them it involved explosives), what you had learned. They'd tell you how proud they were of you because the teacher had put a special remark in your half-term report saying your interest in sciences/math/computing went beyond what's normally being taught. When you came back from playing football all bruised and battered, the question that mattered was whether your team had won it, and you'd beam in spite of the pains.
To sum it all up: Anything remotely interesting involved certain dangers. Not even the girls liked it if they'd be told off "go doing girls stuff" when there'd be the daring exploit that they wanted to be part of. Finding out where the boundaries between experienced, seemingly careless activity and recklessness lie was an essential part of late teens and young adulthood. It laid the basics for the study of sciences, math or computing that eventually got you where you are today. Most important, you weren't reproached for the pursuit of knowledge - you were reproached for having forgotten to take care when you did overstep. It was simply accepted that minor accidents would happen, and they were too insignificant to fret about.
Enter the 21st century. We've come to a state now where experimentation and pursuit of knowledge is considered a danger. To yourself - and your parents could decide to sue your teacher because of the existance of a flask of mercury at school, instead of giving thanks for telling you about the dangers involved with such things. To your parents, because the landlord might want to sue them for keeping dangerous substances at home since your chemistry kit contains sodium bicarbonate. You're a public enemy if you know anything about explosives, or worse, have actually detonated something self-made in the local park. If your friend happens to be Turkish, you're definitely a terrorist threat. You're a danger to the world, an official criminal, if you sniff network packets, no matter their content, and performing a portscan on your neighbours unsecured wireless network could get you jailed since he'll sue you for "hacking" if you tell him how insecure his configuration is. Hey, even your sports teacher could get into trouble if you came home exhausted, because your parents might sue since he dared to push you to the brink of collapse.
Don't authorities and parents see where this leads to ? It discourages experimentation, creativity and independent thinking if everything not predefined in a rulebook, not written down in a set of guidelines or explicitly sanctioned by the law will lead to a punishment likening a harmless activity to a heinous crime against humanity. With this mindset, noone can really claim to be surprised if the interest in technical occupations and scientific research is going down. If you are not allowed to feel proud about knowledge you acquired in a nontrivial and maybe even somewhat dangerous way, but instead are issued a behavioural order or even a criminal record, would you want to continue experimenting ? Eventually, even the biggest curiosity will be curbed. And of course, what's worse, even if you get support from your parents or teachers, you'll eventually run into legal pitfalls because some laws have been issued marking things as criminal activity that are anything but. And you can't go anywhere. The whole world seems to have fallen for that "security" prank, ban everything that's potentially dangerous.
At the same time, authorities and business leaders claim to be short of engineering and research talent. I do genuinely think they deserve it. And they're not short enough of it quite yet. Because a lawyer's salary is orders of magnitudes above that of a researcher, developer or teacher. And because companies still haven't learned the 40-year-old lesson from the mythical man month that the biggest demotivator to engineers is to define how to work instead of what to work on, and give career paths and prospects to managers that engineers can only ever dream about.
I genuinely hope that our society will get back to its roots, and encourage young people to experiment - because experimentation is the necessary first step to innovation. An overregulated environment that discourages experimentation by meticulously defining every rule of how things are to be done, where the only innovation explicitly encouraged is the one that defines a new set of rules for something that used to be unregulated, such an environment is doomed to stagnate.
And I do hope that Sun management will recognize the same. We don't need business processes that outline the correct way to perform every tiny step of any small item of work. We've come to be what we are because we dared to experiment, we dared to make mistakes. Rules made to prevent making mistakes can easily overshoot and stifle innovation.
huh, that was the first blog of mine attracting comments actually ... so I'll try to answer them as well as I may.
#include <ucontext.h>
#include <sys/types.h>
int can_alloca(size_t sz) {
char *buf = alloca(sz);
return (stack_inbounds(buf) && stack_inbounds(&buf[sz - 1]));
}
void myfunc_that_does_alloca(...) {
int use_alloca = can_alloca(sz);
if (use_alloca)
buf = alloca(sz);
else
buf = malloc(sz);
[ ... ]
if (!use_alloca)
free (buf);
}which is slightly more expensive than the first example shown, but might still be more lightweight than using malloc() since definitely no syscalls are involved and no "malloc bookkeeping" is needed.And now - I need to find out how to actually post replies to comments without needing to write another blog article. You, the readers, might eventually turn this "blogophobe me" into a real blogger - who'd have thought that ...
The C programming language offers a nifty feature for temporary memory allocation - alloca(3C). The advantage of alloca() over memory allocation via the heap is compelling:
Unfortunately, alloca() has one critical flaw - it cannot fail. Or rather, it will not fail gracefully. It always returns a buffer address. If the buffer was too large, stack memory runs out and nasty things will happen to your application:
From a diagnosability point of view, the crash will simply be a SIGSEGV or SIGBUS (invalid address / access to address that isn't mapped), and it will not necessarily happen close to the location in the code where alloca() was called. To complicate things further, the crash may be reproducible in one environment, while it may not occur in another. The reason for this is because default stacksizes are dependent on the user environment (see ulimit(1)), and/or because stacksizes for threads in a multithreaded application may differ (from the process' main stacksize and/or from each other's stacksize).
Well, in the past, and on many current operating systems, the answer is "you can't", or at best "it's so expensive to do it that you'd better use malloc()". This is so because your function would have to query 'its' stack boundaries before deciding whether to use alloca() or malloc(), and doing so involves some very unportable, low-level hacking (like setting a temporary signal handler and attempting to read a byte roughtly from the area where the buffer would end up, and checking whether that'd trigger a fault), and/or expensive calls to system-level monitoring interfaces (the use of procfs to find the limits of the mapping that e.g. a local variable address is found in). It's a very expensive operation, and hence defeats the purpose of alloca().
Solaris 10 to the rescue ! There are a few new interfaces in Solaris 10 which allow to do this check quickly and efficiently. The following test programm illustrates that:
#include#include #include #include #include #define RESERVE 8192 int main(int argc, char **argv) { int sz = atoi(argv[1]); char *buf; int done_alloca = 0; if (!stack_inbounds((char *)&sz - (sz + RESERVE))) { if ((buf = malloc(sz)) == NULL) { perror("too large for alloca(), and malloc() failed"); return(1); } } else { buf = alloca(sz); done_alloca = 1; } bzero(buf, sz); printf("got %u bytes, at addr %p, via %s()\n", sz, buf, done_alloca ? "alloca" : "malloc"); if (!done_alloca) free(buf); return(0); }
As you can see, it's pretty simple on Solaris 10 to check whether there's enough space on the thread stack to successfully do alloca() or not. And since stack_inbounds() does not perform any system calls, but only inspects the thread context, it's pretty fast as well.
The feature is already used in OpenSolaris - check the code in usr/src/cmd/ldapcachemgr/cachemgr.c how it allows to gracefully deal with previously impossible-to-address stack overflow issues.
Check the Jim Litchfield's blog for further details on stack boundary queries.
Courtesy also to my colleague I.Zymin, who talked me into evaluating this feature.
Blogging is a bit like writing a diary, and in fact many people just use a blog for exactly this purpose these days. Why don't I ?
Strangely enough, although I'm what most people would comfortably call a "computer geek" I'm still very conservative in my own use of technology. I do positively hate mobile phones, for the simple reason that they're providing so many features that I don't want to use, with no simple way to reconfigure the menu structure in such a way that it suits my taste and shows only the options/menu items that I actually want to use. In that sense, I do understand Apple's success with the iPod - and I'd never get a video iPod or a mobile phone with integrated radio and media player. I want a dedicated, targeted device that's efficient and simple at what it does (if it's shiny - the better), not an egg-laying, milk-giving woolly flying pig.
It's a bit similar with blogging. When you meet me, you might well find me chatty and openminded to a degree that it may seem embarrassing. But I'm not extroverted enough to put the story of my daily life online, for the simple reason that there are thoughts in my mind and things that I do which are, quite frankly, completely unrelated to work and not suitable for the general public to know. It's called "having a life". Some people are able to pursue computer-related activity, work or spare time, with an amount of dedication that I admit I admire, and who seem to be able to play the publicity card to the gain of both their crowd and themselves.
That's positively not the case for me. I'd rather be a wallflower. Yes, it's kind of stage fever, the thought that stepping out and becoming a messenger, a leader for something, and - gasp - a public figure is paralyzing to me.
So you know now why this blog isn't growing frequently. It's about things at work, and therefore bound by the constraints of "spare time at work". I do keep an old-style written diary for hobby, fun and things dear to me, but I haven't chosen the blog name ambiguous for no reason. Blogging for me is about letting people know the interesting things I work on, if time allows. My blog will never become an encyclopedia of sorts, as I think, like the iPod/cellphone example, blogging misses the point there. And my private life is that - private. I'm not really expecting this to change significantly. Some of the things that I do, like for example this article series about how to write Solaris Filesystem drivers, or the Crashdump Analysis book I wrote, or my activities with the Czech OpenSolaris users group are therefore not featured here but in other places - it's far easier working with the community directly, using mailing lists and wikis, than trying to attract people to your blog and telling them "watch this for the latest news on ...". I might incidentally braindump here about things noone asked for, but rather normally will do that in a targeted fashion, via interactive channels.
I rather want you to participate in the community, talk to the community, not to cheer (and/or bother) me ... so go to OpenSolaris.org, register yourself for the discussion lists - I promise you we'll get along very well there !
So why do I even tell you that ? Well, I was surprised today when I put my name (which isn't _that_ unique) into google.com, my blog came up at #3 - and that for a page that has been stale for a year ? My sense of duty kicked in to give a little explanation those who obviously read it about what (and what not) to expect here. Maybe more than there was in the last 12 months, but no promises ...
All the time when I wrote my blog entry about how Solaris/x86 "fakes" hostIDs, I had this deja-vu feeling of sorts. I finally found what this story reminded me of - this old DrFun cartoon.
Visionary :)Being new to the blogging community, I guess it's a polite thing to say a few words about myself before I overwhelm you with that stream of thoughts of mine.
As so many of my "learned" profession (I've graduated in Physics back in '97),
my path lead me away from research and into computing. Having done system
programming and administration on a HP/UX monoculture in the institute of
physics for several years, I shunned that HP offer and joined Sun instead
(hey, even after all these years of Sun not exactly doing well I'm still
happy about this decision)...
So from Sun Service to Solaris Sustaining I went over the next years, and
I'm still there - bugfixing Solaris, as my job description says I should
spend around 80% of my time on.
The big advantage of the Sustaining Engineering role is that you come to
see a lot of different places in the Solaris Operating Environment over
time. So I've worked on the SPARC kernel, various filesystems, threading,
VM, even OBP, and since we turned our focus back onto x86 also on the
x86/x64 kernel. If it weren't for the urgency-to-patch of escalated issues
that now and then interrupt the cosy "what bug can I fix today" state, life
would be paradise ...
There is a life outside of work; as far as that goes, I'm first and formost
into astronomy.
I've started stargazing as a kid - the first book I ever owned was titled
Eine Reise zu den Sternen (A journey to the stars), consisting of
little stories where grandpa explained the constellations to his grandson
over the course of a year, one at a time and always directing to the next
based on those having been shown previously. A wonderful book, it kept me
enthralled for a few months. I keep it to this day, and no it's not for
sale. Like none of the other Astro-gear that I own :)
I do like to travel and see foreign places. And I like good food & wine, if you happen to be German, Wine-Lover and in the Farnborough/UK area, contact me about having a bottle together.
But now, on to more interesting things !
A hardware ID and software querying for this is usually the
mechanism to implement this. The UNIX specifications actually contain
such a query mechanism - see gethostid. Comittees are very careful
about committing to anything, and therefore the OpenGroup's manpage
clearly says "The Open Group does not define the domain in which the
return value is unique"...
Now since we have OpenSolaris,
hostid generation doesn't need to remain secret anymore. Let's therefore
do a little tour of the OpenSolaris sourcecode and identify how the
hostid is generated. I'll start with the x86/x64 platforms, but might
eventually come to talk about OBP and SPARC and how even the OBP
.hostid command there can be fooled into reporting something
fancy like deaff001, as a reminder to those that believe
closedness to be a feature ...
A quick search on the
OpenSolaris CVS gives us the sourcecode for gethostid()
which is extremely simple:
44 long
45 gethostid(void)
46 {
47 char name[HOSTIDLEN+1], *end;
48 unsigned long hostid;
49
50 if (sysinfo(SI_HW_SERIAL, name, HOSTIDLEN) == -1)
51 return (-1);
52 hostid = strtoul(name, &end, 10);
53 if (end == name)
54 return (-1);
55 return ((long)hostid);
56 }
57
So this performs a call to a Solaris system call sysinfo.
There's more about how system calls are dispatched/defined here, here, and here, so I needn't go into the implementation details of the "glue code" at this place again.
Let's jump straight ahead to the kernel.
54 long
55 systeminfo(int command, char *buf, long count)
56 {
57 int error = 0;
58 long strcnt, getcnt;
59 char *kstr;
60
[ ... ]
105 case SI_HW_SERIAL:
106 kstr = hw_serial;
107 break;
[ ... ]
125 if (kstr != NULL) {
[ ... ]
132 if (copyout(kstr, buf, getcnt))
133 return (set_errno(EFAULT));
134 return (strcnt + 1);
135 }
So this just takes that string from a kernel global hw_serial.
Searching the sources for the definition gives us the first (but by far
not the last) attempt at obfuscation:
496 /* 497 * On x86 machines, read hw_serial, hw_provider and srpc_domain from 498 * /etc/bootrc at boot time. 499 */ 500 char architecture[] = "i386"; 501 char architecture_32[] = "i386"; 502 char hw_serial[11] = "0"; 503 char hw_provider[SYS_NMLN] = "";
Don't always trust comments to reflect what the code actually does. Some
may talk about how things were in a distant past, others may be wishful
thinking - use the source, Luke, and check what's really
going on.
At this point, I'll have to put the SPARC folks off to some later time;
sorry guys, you're not forgotten and exploring the bowels of Forth and
OBP surely is interesting, but my time is limited and blogs.sun.com
isn't the newsletter of WhatTheHack
(hmm, on second thought ...)
As far as the x86/x64 platforms are concerned, searching the source
for hw_serial finds us no place where this is assigned a
value to. But instead we find:
1336 /*
1337 * This is needed here to initialize hw_serial[] for cluster booting.
1338 */
1339 if ((i = modload("misc", "sysinit")) != (unsigned int)-1)
[ ... ]
1855 extern char hw_serial[];
1856 char *_hs1107 = hw_serial;
Hmm - somebody doesn't like descriptive variable names. And honestly,
of course, _hs1107 sounds much more trustworthy than a plain
"hw_serial". Anyway, we're now right at where we want to be - a sourcefile
without a single comment in it:
35 #define V1 0x38d4419a
36 #define V1_K1 0x7a5fd043
37 #define V1_K2 0x65cb612e
38
39 static int32_t t[3] = { V1, V1_K1, V1_K2 };
40
41 extern ulong_t _bdhs34;
42 extern char *_hs1107;
43
44 #define A 16807
45 #define M 2147483647
46 #define Q 127773
47 #define R 2836
48
49 #define x() if ((s = ((A*(s%Q)) - (R*(s/Q)))) <= 0) s += M
50
51 void
52 sysinit(void)
53 {
54 char *cp;
55 char d[10];
56 int32_t s, v;
57 int i;
58
59 s = t[1];
60 x();
61 if (t[2] == s) {
62 x();
63 s %= 1000000000;
64 }
65 else
66 s = 0;
67
68 for (v = s, i = 0; i < 10; i++) {
69 d[i] = v % 10;
70 v /= 10;
71 if (v == 0)
72 break;
73 }
74 for (cp = _hs1107; i >= 0; i--)
75 *cp++ = d[i] + '0';
76 *cp = 0;
77 _bdhs34 = (ulong_t)s + (ulong_t)&_bdhs34;
78 }
Nice and obfuscated. But all that this "driver" contains. With the source
open, one could simply put a hostID of your choice into hw_serial,
but then, as indicated, the whole point of the original implementation
seems to have been security by obscurity - never a good idea.
So, the details of this aside: This shows the hostID is simply muched
together from two out of the three 32bit values stored in the array
t[3] from the kernel module /kernel/misc/sysinfo.
But this isn't system-specific ! It doesn't attempt to query any
hardware-specific information to determine the value.
Then why on earth do you get a system-specific hostid out of this ? There's
more obfuscation around - but again none that the source wouldn't tell us about ...
54 # 55 # Avoid signing the sysinit modules 56 # They will be modified during installation, 57 # rendering any signature invalid. 58 #
Aha, so that's it - "modified during installation". The Solaris install process simply patches the binary - and it really patches up only the bare minimum - t[1] and t[2]. Compile a sysinit module and compare it with the one installed on your Solaris/x86 or Solaris/x64 system:
$ pwd
/share/bld/u/frankho/onnv-lowcarbfs/usr/src/uts/intel/sysinit/obj64
$ od -A x -t x4 sysinit | grep 38d4419a
00002c0 38d4419a 7a5fd043 65cb612e 00000000
$ od -A x -t x4 /kernel/misc/amd64/sysinit | grep 38d4419a
00002c0 38d4419a 571ad928 23a6fdc5 00000000
So the install process just patches up t[1] and t[2] to end up with something that's "hostspecific".
And even that's not true - just reinstall the same machine from scratch,
and you'll get a different host ID. I cannot show you the source for the
install program since the Solaris installation utilities are not yet
part of OpenSolaris, but for all practical purposes it's safe to assume
that the install process simply assigns a random number to be the
hostid !
Hmm - opens up the final question that I can answer from the
OpenSolaris sources: How can I upgrade a Solaris/x86 system then without
loosing the hostID ? The answer to that is so simple that I needn't even
show it - upgrade preserves the old kernel/misc/sysinfo file,
and patches up the new one so that the hostID will stay the same. We've
even made sure that BFU won't clobber the hostid:
336 # 337 # files to be preserved, ie unconditionally restored to "child" versions 338 # 339 preserve_files=" 340 kernel/misc/sysinit
So what do we learn out of all this:
SPARC, on the other hand, is a little bit of a different story. I'll tell it eventually, I promise. And yes, I'm bribeable with good German Riesling wine ...