Wednesday October 21, 2009

Still time to register for the OpenSolaris Security Summit!

Advanced registration for the OpenSolaris Security Summit that is going on in Baltimore, Maryland on Tuesday, November 3rd in conjunction with USENIX LISA 09 is open until October 26th. After that, you'll need to register on site, space permitting.

Why should you go?  This free summit will include some of the top people in the field of computer security and networking, including author and luminary Bill Cheswick!  This will be your chance to learn about technologies already shipping with the Solaris 10 Operating System as well as get a peak at what is coming in the future for OpenSolaris!

Did I mention this is free? While you're in town for LISA conference, why not spend a day getting free training from Sun Microsystems? btw, you don't have to be attending the LISA conference to go to this summit - so if you just live nearby, you should take advantage of this opportunity!

Oh, and it comes with lunch and a chance to win free prizes, too, FTW!

Friday August 28, 2009

OSCON, Women in FLOSS, me and a puppet named Jack Adams

A month ago, I was lucky enough to go to a few bits & pieces of OSCON in San Jose with my exhibit pass.

While there I got to meet a TON of really cool, really clued in folks at the OpenSolaris booth. This was a different experience than I've had at other conferences doing booth duty. First of all, our booth was right by the front door, was large with couches for lounging, and we had a lot of cool stuff to give a way.  Anyone that installed OpenSolaris (even just in a virtual box) on their laptop got a free t-shirt. We were also giving away install media and getting started guides, of course, as well as cool stickers for your laptop that said "Powered by OpenSolaris" (I got one myself!).  The people that approached the booth not only knew what Sun did already, but were at least relatively aware of Solaris. Some hadn't used the OS in awhile, some wanted to know the big differences between OpenSolaris and Solaris, others just had questions about very specific technologies.

I got to show my lack of skills at Guitar Hero as I was pitted against Microsoft's Sara Ford in a battle of the operating systems.  To be fair, I'd only played the game once before, and that was more than 18 months before. If it had been Tekken or even Wii Bowling, it would've been a different story, I tell ya!

(Photo by Pınar Özger)

I attended the Women of Free/Libre Open Source Software BoF (Birds of a Feather) session run by Kirrily Robert, which had an impressively large turnout - around 25 people, mostly women (the rest were "advocates" :). It was good to meet a lot of other women working in Open Source and just in technology in general. Like a sneak preview of the Grace Hopper Celebration of Women in Computing conference, though surprisingly few of these women were familiar with that conference.  We tried to keep it from turning into a venting session about some clueless and/or rude men we've all worked with in the past, and tried to give each other suggestions for things we've found has worked.  Kirrily then had us all go around the room to discuss our favorite woman themed book. Mine, of course, was Women Don't Ask: Negotiation and the Gender Divide. I'm hoping she'll post the complete list soon, as I heard some very interesting titles come by!

Our Solaris Security BoF was just after that, so I couldn't stay for the entire Women in FLOSS BoF.  When I got to our BoF room, I was dismayed at discovering the facilities team had taken away our projector! I had checked everything out the night before, to make sure our OpenSolaris laptops would work with their projectors and even confirmed with the A/V guy that we would have the same equipment for our BoF on Friday. Everyone I asked that was working for the site said we'd have the equipment, but apparently not.  This started us off on a bad foot - but fortunately, many of us had brought laptops with the presentation on it that we were able to distribute through the small crowd so they could follow along.

I will admit, I was very disappointed by our small turnout we had at our BoF. The guys that were there (sorry, except for Sun staff, it was only male attendees) were very interested in our topics of discussion and asked a lot of great in depth questions. It was taped, so hopefully we'll have the video soon!

Speaking of videos, I was also able to help Jack Adams, a puppet, with his OpenSolaris security concerns and problems.  This came out well, considering the lack of script. All that improv training at the Gaslighter Theatre comes in handy, even for technical talks. Good job, Deirdre, for putting this together! Enjoy!

(though I really should've taken off my badge, so you could see my "I HEART OpenSolaris" shirt better :-)

Thursday August 13, 2009

Managing Your ON Mercurial Gate

Working on my recent projects, I became frustrated with a lack of one-stop-shop for Mercurial for use with OpenSolaris development. My focus is on the ON (Operating System and Networking) Consolidation, of course.  As an internal developer, my steps assume access to things like usr/closed.  If you are external, you will need to get your closed binaries from the closed binary tarballs.

I did find the HG Workflow document helpful, but not complete for my every day tasks. You should read that as a starting point, as it has lots of good tidbits on backing up your changes and managing project gates.

Please send any corrections or additional tips you might have this way, and I'll update this post.

Setting Up Yourself

First and foremost, make sure you have set up for cadmium and have your .hgrc set up as follows:

$ hgsetup

[...]

$ more .hgrc

[extensions]
hgext.cdm=/ws/onnv-tools/onbld/lib/python/onbld/hgext/cdm.py

[email]
from=First.Lastname@Sun.COM

[paths]
onnv-gate=ssh://onnv.sfbay.sun.com//export/onnv-gate
onnv-clone=ssh://onnv.sfbay.sun.com//export/onnv-clone
onnv-closed=ssh://onnv.sfbay.sun.com//export/onnv-gate/usr/closed
onnv-closed-clone=ssh://onnv.sfbay.sun.com//export/onnv-clone/usr/closed


[merge-tools]
filemerge.gui=True
filemerge.args=-a $base $local $other $output
filemerge.priority=1
filemerge.executable = filemerge
filemerge.checkchanged = true
filemerge.premerge = false

meld.gui=True
meld.priority=0

gpyfm.gui=True
gpyfm.priority=0

[ui]
username=Valerie Bubb Fenwick <First.Lastname@Sun.COM>
style=/ws/onnv-tools/onbld/etc/hgstyle

without the style settings, your Change Request Team Advocates will have difficulty reading your "hg outgoing -v" output and will likely put your RTI (Request to Integrate) on hold.  I have customized my filemerge utility to be TeamWare's familiar filemerge.

Note: Email addresses used in here need to be real, routable addresses!

Setting Up Your Gate

Our build server leverages ZFS, which I highly recommend, as it gives you the quick ability to create snapshots before doing a major rewhack of your code. Here's what I do on the build server with ZFS:

$ zfs create builds/bubbva/<workspace>
$ cd <workspace>
$ hg init
$ hg pull -u ssh://onnv.sfbay//export/onnv-clone/
$ hg update
$ hg reparent ssh://onnv.sfbay//export/onnv-clone/
$ hg clone ssh://onnv.sfbay.sun.com//export/onnv-clone/usr/closed <workspace>/usr/closed

Now, if you're not using a ZFS pool for doing your development, it's a little easier to setup:

$ hg clone ssh://onnv.sfbay//export/onnv-clone/ <workspace>
$ hg clone ssh://onnv.sfbay.sun.com//export/onnv-clone/usr/closed <workspace>/usr/closed

Note that the seemingly extraneous slash is not so, it is part of the communication with ssh and is indeed required.  I don't know why hg clone won't work with an otherwise empty directory as its target, which would make dong this with a ZFS pool much simpler, but it doesn't.

On the ZFS snapshots, I recommend coding the date into the snapshot name, as the default listing of snapshots does not include that information, which makes it very tricky to figure out "what did I call that snapshot yesterday!?".

$ zfs snapshot builds/bubbva/<workspace>@Aug14_01
$ zfs snapshot builds/bubbva/<workspace>@Aug14_02

Finding Files in the Source

I often find that I know the name of the file I want to modify, but really have no idea of where it resides in the source - or perhaps I just know a partial name, like "softtoken".  In teamware, I would always just grep the nametable, but since Mercurial has no equivalent concept, there is nothing quite that fast. Here's what I do now instead:

$ hg manifest | grep <filename>

Editing Files

Unlike with SCCS, there is no need to checkout files - just use vi/vim/ed/emacs/xemacs/etc and have at it.  If you don't like your changes, simply revert.

$ hg revert <filename>

I've had mixed results with this, so find out what the previous revision to your changes was with:

$ hg log <filename> | more

If you need to create a new file:

$ hg add <filename>

To remove:

$ hg rm <filename>

To move (this works on entire directories, as well):

$ hg mv <filename>

When you are satisified with your changes:
$ hg commit

Managing Children to Build

It's always a good idea to do builds on both SPARC and x86, even if your changes seem like they're architecturally neutral. In fact, many members of the Change Review Team will require it.  Some folks will even recommend you don't build in your "change master" to ensure you haven't forgotten to commit a file or "hg add" a new one. That's not strictly necessary, as long as you've done a build from a child of your main gate on another architecture, though, if you've done a lot of moving things around or creation of new files, you really should do it.

The problem comes from if you have done multiple "recommits" in your build master, this confuses your children. One way you can manage this is to always bring over a fresh build child. That's cumbersome though, at best.

Here's what I do (IN THE BUILD CHILD ONLY! NOT FROM A GATE YOU WANT TO PUSH FROM!):

$ hg pull
$ hg update -C

Preparing for Review

First, commit your changes. This will give you a chance to put all the relevant CR IDs into your comments. Unfortunately, every CR will be associated with EVERY file in your changeset. That's just how mercurial works.

(note: for advanced users, you can try Mercurial Queues, which does a much better job of managing this. I haven't tried it yet, but maybe I will on my next gate with multiple bug fixes. Mark Phalan has a nice blog on it.)

$ hg commit

If you're working with simply open source, this convenient option has been provided to prepare and publicly post your webrevs to http://cr.opensolaris.org/~<username> [1]:

$ hg webrev -O -U

Vladimir Kotal has a good blog entry on automatic webrev uploads.

I've been using a wrapper (hgwr, formerly wxwr), originally from Bill Sommerfeld, for webrev for a long time that keeps revisions of reviews available. This is handy so that you can incorporate changes from one code reviewer & post the updated webrev for that reviewer to verify you understood their comments, while not changing the code under another reviewer.

This is great for me, as a developer, as well, because as reviews trickle in, they all refer to a specific line number. If I've already incorporated changes, then the line numbers may have changed significantly. Having the original review source available is invaluable.

If you use this script, or something like it, the -U option to webrev is not useful. Instead you can use scp (MAKE SURE YOU STILL SPECIFIED -O for OpenSource to the wrapper, or your bug links will all be to the internal site):

$ scp -r <workspacename>.<reviewnum> bubbva@cr.opensolaris.org:

(Note: that trailing ":" is not a typo, but required scp syntax.)

If you're additionally working in closed source, you'll need to utter the following:

$ cd <workspace>/usr/closed/
$ hg webrev

In case it's not obvious, do not load this webrev to opensolaris.org ;)

Resynching With The Clone

This starts with a simple:

$ hg pull -u

but you will always have to merge, even if nobody changed the same files you did. One thing I've learned the hard way about Mercurial is that if it can't open a tool to do a merge (in the case that someone has updated the same file you did) it will simply do the merge for you and do nice things like add a blank line in the middle of an enumerated list...)

So, if like most of us you don't have your workspace on your desktop, but rather on a build machine, you'll want to start this process like this:

$ ssh -X <buildmachine>

Which will allow the graphical mergetools, like filemerge, to open when you get to the next step:

$ hg merge

and you'll need to commit again:

$ hg commit

More Unusual Tasks

Finding what changeset changed which lines:

$ hg annotate <filename>

Finding out which changesets impacted a file (useful for backing out individiual changes):

$ hg log <filename>

Finding History of a File if It's Been Moved

Because Mercurial isn't really a file based source code management system,
when you move a file the history does not move with it. That is, it appears as if it's a new file. You can still pull some of this history (like which changes were introduced under what name):

$ hg history -f <filename>
$ hg log -f <filename>
$ hg annotate -f <filename>

I Made Changes to a File Then Moved It and Want To Back Out the Changes (but not the move)!

Oops - I did this. Once.  Because of how poorly mercurial handles file level operations, this is difficult to correct. For example, I made some minor edits to a file, including updating the copyright date, then I moved it. hg revert no longer worked!  I was able to manually revert the changes, the file still showed up as changed in my workspace and 'hg outgoing -v'.

While I was told that it would have been acceptable to push this junk, it seemed sloppy to me.  Due to the lack of per file controls, it is actually pretty easy to apply your changes to a new workspace using patch(1) and the "patches" provided by webrev, then redoing the moves, as needed.

Ready to Integrate!

Of course, you've read all of the RTI Nits, done all your testing, filed any documentation and test bugs and made sure they can be fixed at the same time as your integration and gotten your RTI (Request to Integrate) approved by a member of the Change Review Team... then you're ready to go!  The problem is, so are lots of other people...

This is what I call the Mercurial Push Dance.  All it takes is one more implementor heading for the gate at the same time, to begin this nasty tango...

$ ssh -X <buildmachine>
(because you will have to merge...)
$ hg commit
$ hg pull -u
$ hg merge
$ hg commit
$ hg recommit

If you had actual conflicts (ie same files changed), CHECK THE MERGES. Run webrev again and make sure only your changes are there. Because the mergetools hooked into Mercurial grab focus when they come up, they are known to grab spare characters and insert them into your code.  I've found stray "$" and other things that just wouldn't be a good thing to push.

Rinse & Repeat, until other folks stop beating you to the gate. When you're ready:

$ cd usr/closed
$ hg path default > /tmp/closed-mommy
$ hg reparent ssh://onhg@onnv.sfbay.sun.com//export/onnv-gate/usr/closed
$ hg push

[Closed gate changes always need to be done first, because once you push to the open gate, the incremental build will start.]

$ cd ../..
$ hg path default > /tmp/open-mommy
$ hg reparent ssh://onhg@onnv.sfbay.sun.com//export/onnv-gate
$ hg push

[...]

After you finish the Tango de la Muerte... I mean, the Mercurial Push Dance and have successfully gotten your bits into the gate, don't forget to:

$ hg reparent `cat /tmp/open-mommy`
$ cd usr/closed
$ hg reparent `cat /tmp/closed-mommy`

[1] These all assume you've set up your SSH key on the opensolaris.org site. This is required for posting webrevs and doing integrations into the main gate.

Many thanks to the other developers who hang out on irc.sfbay/#hg-help and freenode.net/#onnv-scm, particularly Rich Lowe, Mark J Nelson and David Powell.

Thursday July 30, 2009

SSH with aes256ctr support not working on some S10 systems

I've been getting emails today about SSH aes256ctr being broken on some Solaris 10 machines.

This goes back to my work earlier to get strong crypto included by default on all Solaris 10 systems.  This started in Solaris 10 Update 4, and I guess I figured everyone would read my blog, jump for joy and upgrade their systems. ;-)

It seems some of you haven't and are now seeing errors like:

sshd[8975]: [ID 800047 auth.crit] fatal: matching cipher is not supported: aes256-ctr

Which is a direct result of Sun's SSH now taking advantage of  the presumed availability of strong crypto on the systems.  This works fantastically well on newer Solaris 10 systems.

This issue is now covered by a bug, and you can see one workaround there.

Let's assume you *do* want strong crypto, though, and you want to stay on an older release of Solaris 10. In which case, you need to install SUNWcry and SUNWcryr onto these older Solaris 10 systems and reapply all cryptographic framework patches.  You need to reapply the patches, because when you installed them before SUNWcry & SUNWcryr were not on the system, so would've missed all the patching goodness for their bits.  It's important that you do this, or you will end up with mismatched bits for the cryptographic framework, which will have undefined (ie probably not good) results.

Tuesday July 21, 2009

OpenSolaris Security BoF on 23 July 2009 8PM! 8:00pm  Thursday, 07/23/2009

OSOSOS - Offering Security in OpenSource Operating Systems
Location: Ballroom A3/A6

Moderated by: Christoph Schuba

  Many operating system security mechanisms are necessary for
  developers to build secure software. While this session presents a few
  such mechanisms available and under development in OpenSolaris, it
  primarily seeks the dialogue and discussion how important these features
  are and how they compare to those of other OSes.

Speakers will do short talks on the Cryptographic Framework (Valerie Fenwick - that's me!), Priveleges (Scott Rotondo) and Zones/TX (Glenn Faden), followed by a panel from all presenters, plus Christoph Schuba and Glenn Barry (Kerberos guru).

BoFs are free, you just need to register for the expo pass (also free!)

(2009-07-21 18:24:49.0) Permalink Comments [0]

Thursday June 25, 2009

Up to my eyeballs in tests

As a Change Request Team advocate, I am stringent about asking for test results and always very annoyed when an implementor complains about how complicated the tests are to run.

Now after having spent the last several days finding working test hardware from our pool of test machines, and fighting with test installations and executions... I'm still waiting for my baseline results. I haven't even run the full tests on my own bits yet.

Which is another story.... while my builds were successful and my changes to libelfsign seemed to be kosher, I found that after doing a bfu that my test machines wouldn't even boot.  No, I didn't change libc... so I was very surprised that such behaviour was seen. Yes, I knew things like kerberos and IPsec would not work correctly if libelfsign (a core component of the Cryptographic Framework) wasn't working - but inability to boot? I was shocked.  With some help from pwernau and meem, I finally got one of the systems up in single user mode to discover the linker was doing something... unusual.

Fortunately, a very responsive Rod Evans came and looked at my limping test system and figured out what the linker was doing wrong (and also something one of the libraries in my calling path was doing wrong), and now I've got systems I can play with.

Except when I forget to sync my x86 build workspace with my sparc workspace and I build archives without Rod's fix... and then wedge another test machine.

Hopefully the code will be up for review soon, when I will add another blog entry detailing what it is exactly I'm trying to do and why.

Sunday June 14, 2009

OpenSolaris Turns 4!

Wow, it's been four years now since Sun launched OpenSolaris.  We've come a long way since then - built up a budding community, taken lots of contributions from outside, and we're even turning out a pretty decent OS based on this now! It's on my desktop, laptop and home machine.  There's still a lot to do, but overall I'm very impressed.

It's been very cool doing code reviews openly and getting design feedback directly from the real world before any code is even written. This has greatly changed the way I do my job, for the better!

Tuesday June 02, 2009

Goodbye CDE...hello OpenSolaris!

I've been using OpenSolaris at home and on my laptop for months, but was still running Nevada builds on my SPARC desktop in the office... with CDE (you know, that super old, yet super fast, Common Desktop Environment).  Well, I got a new desktop recently which is Intel based, and with the brand spanking new release of OpenSolaris 2009.06, it seemed like I needed to move into the 21st century.

While I would still like an "advanced" installer, I do think the installation went amazingly quickly and very smoothly. I had to make several adjustments to the system after installation to get it running with NIS on the Sun internal network with a static IP address, and download some of my favorite software - which was so easy with the "pkg" command! This release is much faster and smoother than what I have been running on my laptop - clearly time to upgrade that as well. I was pleased to see how easy it was to install flash and acroread as well.

The problems didn't really start until I logged in with my Sun internal home directory mounted - when I found I had some horribly ancient and mostly broken GNOME configurations (probably from the last time I seriously played around with it, back in S9 or early S10 days). gnome-cleanup took care of that and got me to a nice clean GNOME login. A few minor adjustments so that things like mouse-over to make active for windows, and a change of my default gnome-terminal preferences and I'm mostly off and running.

I also hit problems with my .xmodmaprc file, as it apparently used "keycodes" which do not translate between Xsun on SPARC and Xorg on x86. Thanks to one of the desktop team members, Michael, he told me about "xev" and that it would be the keycode lines in the file I needed to fix. With a few tweaks, my ergonomic keyboard is now behaving just the way I like it.

I did try a modern mail reader, ThunderBird, but after being annoyed it didn't believe most of my mail folders were actually mail folders (due to missing IMAP leading message), and how annoying it was to save to the folders it did recognize, I switched back to pine after about 10 minutes. (yes, I know there is a newer version of that software, alpine, but I don't like that one either ;).

Now I just gotta figure out what to do about my network calendar being stored in a format for dtcm ;)

Monday June 01, 2009

OGB Town Hall tonight! I've made it to CommunityOne West and am enjoying the first set of sessions and just wanted to remind you all that the OpenSolaris Governing Board is doing our first Town Hall in room 305 of the Moscone Center as part of CommunityOne West. (2009-06-01 11:05:18.0) Permalink Comments [0]

Thursday May 28, 2009

OGB TownHall June 1 6PM! The current OpenSolaris Governing Board will be holding our first Town Hall open forum on June 1 at 6PM.  This event is part of the CommunityOne West events and will be in Room 305 of the Moscone Center.  Hope you can make it! (2009-05-28 11:58:56.0) Permalink Comments [0]

Wednesday May 20, 2009

Free Solaris and Java training

This is pretty cool - Sun Learning is offering a Career Stimulus Learning Package for free - there are some Solaris system administration classes, some Java and JavaFX classes and resume templates on the site. If you're a Sun Alum who has left the company within FY09, there are even more classes and some discounts offered.

Thursday May 14, 2009

Community One West

I've just registered for CommunityOne West on June 1-3 in San Francisco's Moscone Center! I am actually excited about my first CommunityOne event, and my first chance to meet directly with community members since beginning my term on the OGB (OpenSolaris Governing Board).

The OGB will be doing a Town Hall on Monday at 6PM, so please come over, get to know us, and tell us what's on your mind!  You can see this event and many others on the detailed schedule wiki. I am happy to see some security related talks! There is one by Dr. Christoph Schuba, a fellow Boilermaker, on Role-Based Access Control and the Cryptographic Framework, and another by Scott Rotondo and Secure Programming.

I'm not sure when registration closes, so if you want to go, sign up soon!

Tuesday April 28, 2009

Storing ZFS backups in the cloud... I'm still coming up fully to speed on Cloud computing and was just reading about this new tool from Glenn Brunette that uses existing OpenSolaris features, like ZFS and the Solaris Cryptographic Framework, to back up data securely to the Cloud. Pretty neat! (2009-04-28 14:12:03.0) Permalink Comments [0]

Friday April 24, 2009

SEED panel, RSA booth duty and Take Your Kids to Work Day

On Tuesday I had the pleasure of sitting on a panel of folks talking about open projects started from within Sun, at a conference for SEED Mentoring participants.  I enjoyed talking with people from all over the company and really enjoyed learning more about the other open projects. Of particular interest to me was the speaker, Durgam Vahia, from the OpenSPARC project. He mentioned a recently released FPGA that was made possible because of OpenSPARC and was released by Xilinx.  The board, based on the T1 microprocessor can be up and running with Solaris in about 30-45 minutes and is a great way for students or any interested parties to learn about the SPARC architecture.

That afternoon, I worked in the Sun booth at the RSA conference.  Even though our booth was a bit off the beaten path, I still saw a lot of traffic coming our way. People were really interested in the OpenSolaris LiveCDs and hearing about the latest things we have been up to in Solaris Security.  We also had an SCA 6000 card and a UltraSPARC T2 motherboard. The mother board was an odd "demo" - considering it had no power supply/memory/etc, and I really wished I had the OpenSPARC FPGA I had seen earlier that day :-)

Of course, I got several of the inevitable  questions that I couldn't answer (really because I don't know anything, but even if I did, I still couldn't say).

Yesterday was a fun and crazy day, starting off bright and early with an OGB meeting, followed by a few Take Your Daughters and Sons to Work Day activities. I got to see the Project Wonderland demo, which was very cool - and the kids enjoyed playing the games, then I was lucky enough to have lunch with the girls from TechBridge.  Their passion for science and math was truly inspiring!

Tuesday March 24, 2009

Yay! Elected to the OpenSolaris Governing Board!

Well, the results are in! I was elected to the 2009-2010 OpenSolaris Governing Board, along with several other fine candidates.  There were so many fine people running, I really had no idea if I would be elected. Now that I have been, I have a lot of work ahead of me, I think!

Unfortunately, the new constitution did not pass, due to lack of voter turnout, essentially. Which is exactly one of the major things the new constitution was trying to fix.  Basically, in the existing constitution, in order to get voting rights in communities and recognized for your efforts, you need to become a core contributor. The elections depend on all core contributors turning out and voting, but it turns out that many of them are not interested in general governance, but rather just their community.  The proposed constitution separated the rolls of electorate and contributor, so only those interested in governance would be required to vote.  Alas, it did not pass.

I am looking forward to this extra challenge and I am now definitely inspired to make sure I leave a positive mark on the community!