The Kitchen Sink Weblog: Tales from Sun's internet facing edge

As many of you know, Sun recently bought SeeBeyond . SeeBeyond is in the Integration space in the Application Software world. It seems to have been a good move from the Analysts' perspective, in the sessions I attended at the Gartner IT/XPO Symposium, the acquistion was mentioned not less than four times, usually accompanied by the comment "Sun is now a player in the SOA space" For those of you that are SeeBeyond customers, Gartner says the acquistion is a good thing for you, because instead of buying SeeBeyond for the customer base, like a recent Oracle acquistion, the SeeBeyond acquisition plugs a gap in our offerings, so there won't be conflicting, competing product lines.

SOA, which translates to "Service Oriented Architecture", is the hyped thing of the year, according to Garner. Used to be portals, now it is SOA. There were far too many sessions to pick from on SOA or it's kissing cousin, Web Services. It depends on who you talk to, but from what I can gather, SOA is a paradigm where you have loosely coupled (or completely uncoupled) application services that can talk to each and clients. Didn't seem like a new concept to me, but what seems to be new is the supporting technology that is coming out from the industry to support keeping the services uncoupled from the clients that use them, not to mention the BPM stuff that runs on top that coordinates workflow through all these services.

While you can have SOA after a fashion without using the WebServices standards, no one seriously considers doing this. The interoperability of your services would suffer greatly if you are not using the standards.

If you read my earlier post on the death of the database, the point underlying that session is that the use of SOA leads to an universal connectivity that makes persisting certain types of information unnecessary.

So is the hype deserved? I don't know. Depends on how much interconnectivity we see across companies and organizations.

This is my last day here. I was originally going to stay for the half day tomorrow and fly out at night, but with hurricane Wilma looming, I decided to get the heck out of dodge and changed my flight to leave at the ridiculous hour of 6AM EST tomorrow.

I visited some Sun folks at the ITxpo floor and chatted with the openSolaris folks. They have been blogging up a storm.

Despite the conference winding down, I've got a couple of topics to blog about, just haven't had the time yet.

For example, instead of blogging about SOA, I went to a party at Pleasure Island, known as Disney for adults. I think I have finally figured how to navigate this place (don't get me started about my commute), and found it pretty easily. Most of the bars are the standard disco type places, the 70s/80s place was fun, but the oddest place had a stand-up comedian dressed as "club president" interacting with the crowd.

This morning I attended a session that had the rather contraversial title "Death of the Database". The speaker asked all the media bloggers to stay for all of the session instead of rushing out and blog "So and so from Gartner says ..."

A better, although not perhaps not as big of draw of a session name, would have been "Evolution of Persistence". The speaker's point was that the role of a the database will change in the evolving tech landscape where data is everywhere in the network, real time, and doesn't need to be persisted into a database.

For example, why do you need an inventory database, if your system can get all the info needed from the RFID tags on your widgets in your warehouse? If the data you need only has to live for a transient time, ie. for the duration of a transaction, why persist it?

I found his assertions thought-provoking, but it raised more questions that it answered. Think of the security issues, will your financial institution really allow your application access to a transaction while it executes, so you can confirm that the order has been paid for? Can we really get away with not storing the state of an order in an eCommerce scenario? What happens if the service goes down and we lose the state and it isn't persisted? You get the picture.

As far as the question, will DBAs have a job in the future? He advised DBAs to start learning about middleware since the persistence will move up the stack in the form of queues, actual objects (ie. RFID tags), and in the services themselves. Your mileage on this advice may vary.

I haven't blogged here for awhile, but this week I'm at the Gartner IT/XPO conference and I thought I would capture some of what I'm hearing. The theme is "rapid results, faster ROI". Gartner seems to think that we are poised on a revolution of IT (I think it's been going on for a while), where projects can't take a year and we need to transition to more SOA.

So I agree that projects can't take a year. Most of our projects don't, they are more on the order of weeks, a lot of tweaking of what is already in place. In the web world the requirements are ever changing and our stakeholders often can't completely visualize what they want, not to mention that they change their mind. In all fairness they are just reacting to a changing marketplace. It's way better to prototype something, get the stakeholders to validate what you are doing and schedule the additional features for quarterly rollout. Upper management is way happier when they see tangible results.

One of the interesting things about attending one of these is finding out what your own company is doing. If you are doing SOA, you might need to abstract out the communications between your clients and services using an ESB (enterprise service bus) layer. Turns out Sun has open-ESB project on java.net, check it out

Capturing some images of summer before I completely lose myself in work again.

The fourth of July was beautiful in the City, I never saw any fog which is unusual for the summer, here are the city denizens sunning themselves in Dolores Park.

Our baby birds grew up and left the nest. First mama and papa bird left the nest and watched from nearby. Then the babies spent a day working out how to fly on the patio floor. Then they were gone.

We visited the alameda county fair where we saw both horse and pig racing as well as hobby collections. Waching piglets jumping hurdles on a track was very amusing. The collections had everything from barbies, to carved animals, to beer coasters.

Kathy

So mama bird is definitely a dove. June must be their nesting time, check out this story of a dove family The blue ring around the eye was the identifying factor that confirmed it for me. However our home decorating friends had it wrong, baby doves don't just hatch into little birds that sit upright. They hatch into little tiny birds that barely move, let alone hold up their heads. I was surprised that they were that much like humans.

We know this, because our dove family had the misfortune to encounter the painters that the association had hired to repaint the patio walls. The painters didn't get at first what I was trying to warn them about. But when mama bird flew away, disturbed by the racket, we all could see the little birds lying helplessly in the nest. During the day she kept trying to stick it out on the nest but many times the commotion the painters were making with moving trelises and ladders was too much for her. I thought about moving the hanging pot but was afraid that could be a bad idea. For all I know she wouldn't recognize it once moved.

After the painters left, it was at least a week before we saw the baby birds poke their heads up. I was worried as I thought we had disturbed them too much. But nature perserved.

In a time of death, a story of life.

Sometime ago I noticed a bird in my hanging redwood tree. It was very still, so much so that I thought it was dead. It's been there for a least a week, I see it as I go in between the garage and the house where I live. Once I caught mama bird away from her nest and I was able to peer into the nest to see two little white eggs.

Funny that she chose that place to nest. Although it's a patio surrounded on four sides, the nest is a hanging pot with a 8 inch redwood tree in it and separate from the other greenery and very visible to us humans... although not to the cat as it turns out. Whitman, our orange and white feline, strolls around the patio, completely oblivious to the natural incident occuring above him.

The redwood tree is from a wedding I attended several years ago. Each guest got a redwood tree spout in a plastic tube as a gift. One motivation of writing this, is to share with the still happy couple what became of their tree and it's role in sheltering new life.

I don't know my birds very well but someone thought it might be a dove

Next entry: baby birds

I'm sure you all have heard of Sarbannes Oxley. The legislation Congress passed in the wake of CEO wrongdoing (digression: if you really want to read up on some wrongdoing go here). It was supposed to add additional auditing controls to keep those shifty CEOs honest.

Somehow, don't ask me how, this has translated into additional hardening requirements for our servers. Which means I get to do my least favorite task, coordinating downtime for servers among multiple groups at Sun.

It got me thinking, though, about all those articles I've read. About how so and so company had to spend thousands, even millions on upgrades to software and even hardware to comply with the Sarbannes Oxley reporting requirements. It was a flashback for me to the Y2K days. Where if you didn't have a later model PC or server, patched up the wazoo, doom and despair would surely occur on New Years Day.

We've already talked about my marvellous sense of timing. I was, you got it, in QA for the year 1999. Fortunately my product was a combo software development environment plus appserver, not much to worry about, or so you would think. The problem was that it ran on all sorts of platforms, Unix, Windows, VMS, six different databases. And we had to certify our product against the Y2K patches all of these vendors produced. Certification meant running a whole slew of automated test suites against the product on various platform combos (ie. WindowsNT / Oracle). The other problem is the vendors kept generating patches.. whoops, we found another obscure edge case that might be a problem for Y2K, better patch it so we can't be sued. At some point we had to say enough was enough, put our stake in the ground, just so we could get our "Y2K" certified product out the door.

Lots of money spent on Y2K. Seems like lot of money being spent on Sarbannes-Oxley. Maybe I'm stretching it a bit, but could SOX be the reason that the tech economy is lifting out of the doldrums? I mean, might as well buy some hardware for that new accounting system as well. Food for thought.

Kathy

`` Despite the tons of examples and docs, mod_rewrite is voodoo. Damned cool voodoo, but still voodoo. ''
-- Brian Moore, bem@news.cmc.net

It is often said that it is a lot easier to launch a site than to EOL it. I have to say I haven't seen an exception to this statement yet. forte.sun.com was the first site I worked on. In the late fall of 2000, after tiring of QA'ing products, I took a leap into the unknown and signed up to be the engineering manager for the "Forte for Java Portal". What a marvellous sense of timing I had, to be working on something new in the web space as the dotcoms started imploding all around me. My promised staff of six turned into three contractors. My budget diminished every quarter. But we launched it and it was a success. We had a tight team and had a lot of fun.

It wasn't just an HTML site, it had several applications. For example, the Update Center. This feature was a non-HTML service that provided XML catalogs directly to the product (Sun Java Studio ... formerly known as Forte for Java). It was kind of a non-standard webservice, before such a term was coined.

So I guess it was only fitting, that I ended up working on it's demise. The Update Center has been replaced by another update system. Most of the forte.sun.com content had moved over a year ago to developers.sun.com, it took that long to get the new update system functional and the catalogs moved over. My piece of this should have been small, the machine serving forte.sun.com had been turned over to IT management, so all I needed to do was just log a ticket to get the redirects put in place and then check that everything worked. Or so I thought.

The problem was, that we needed to update the redirects in an Apache server. You see, we don't have very many instances of Apache here at Sun. About 2 years ago, we really started to eat our own dogfood. So Apache is not well known here. And so they asked me to help.

Ok, I thought no problem, I can figure this out, the mod_rewrite module is already being loaded (all those content redirects) so how hard can it be? So I googled away and the first thing I saw was the voodoo quote. Hmmm.

You see I didn't just want to do a redirect, I wanted the redirect to tack the original query string on the end of the target's URL query string. So I came up with (I've shortened the URLs for brevity):

RewriteRule ^/ffjcatalog(.*) http://www.sun.com/applink?sundispatcher=1.0.2&REDIR$1 [R,NE,L]

Problem was that the redirect worked, but it lost the query string. The next thing I tried was:

RewriteRule ^/ffjcatalog\?(.*) http://www.sun.com/applink?sundispatcher=1.0.2&REDIR&$1 [R,NE,L]

The problem with this is that the rewrite rule didn't get triggered. That should have tipped me off there, that mod_rewrite was doing something special with the query string... because it should have matched, but I didn't have a lot of confidence in my knowledge of regular expressions so I went on trying all sorts of permutations. It wasn't until I read through the RewriteCond doc that it finally dawned on me that the query string was being put into a variable I could use. Here's what worked.

RewriteRule ^/ffjcatalog$ http://www.sun.com/applink?sundispatcher=1.0.2&REDIR&%{QUERY_STRING} [R,NE,L]

These old sites, they don't take EOL'ing too kindly, they fight it until their last dying breath.

We've all seen the election maps published by the major media. Here's some maps that take the population densities into account.

I admit it. I'm a wannabe techie. I used to do hands on technical work, but that was back in another generation, as in before the internet. Now, it seems that most of my time is spent poking holes in requirements (and tell me again, why do you want to do this?) and cleaning them up so my engineers know what to code to. If I get technical, it's with the ops stuff, not into any code.

Speaking of a blast from the past. Remember VMS? I do. I'm glad Unix (and it's grown up rebellious spawn Linux) out-lasted it. I really didn't care for VMS, with all the privileges and quotas. But it used to be popular, so I had to learn it. I used to call VMS the Klingon of operating systems, because both were ponderous and stuck on ceremony. Also both have redundant systems. I can't quite remember but a second something (stomach?) saved Warf from certain death. Guess which species was MS-DOS? I leave that as an exercise to the reader.

So as a wannabe techie, it is with some trepidation I plunge into a description of a technical problem we are having. But it's an interesting problem, so I wanted to put it out there. We recently built a webservice with the thought of making it public for our users to use. Right now the only client is the web frontend that runs on our sites. The service mostly provides read-only data, *but* a client can do some writes. As one would expect we require a login-in (user id and password) for the writes, because we are keeping track of the data for each user. This of course means that the client has to pass the userid and password to the webservice. And here is the problem. We don't want the userid and password to be passed to the server in the clear.

The XWS security model seems to assume a peer to peer model rather than client/server. So you can secure the conversation, but only if each side has installed a certificate. This is fine for two servers. But what about a webservice that might have lots of clients ... prebuilt clients that might be used by non-technical users? Getting each user of a client to download the certificate and compile it in is not really an option.

It is interesting to look at how other webservices with clients have tackled this problem, ie. Amazon. They don't use a user id and password. Instead they generate a token for each client. The token is passed in the clear, but at least it is not a password. However not very secure.

The only solution we have come up with is to put the entire webservice under https. There seems to be no way to secure just some of the conversation to a webservice. But having everything under https is way expensive. Much more handshaking has to be done for each connection. If anyone has any bright ideas we are all ears.

Kathy

P.S. I know that some people have been asking about Alameda. Truth is I don't know much about it's status, as I have been reorged away and taken on new work. If I hear anything that I will pass it along.

Check out this site . It's a journal of a woman's motorcycle ride through the dead zone of Chernobyl. Click on the chapter links at the bottom of the page to read through it.

The black Russian humour adds to the writing, but the photographs of a moment frozen in time speak for themselves.

kathy

Kind of scary. Are we on the path to living in 10x12 cubicles like the ones shown in the beginning of the movie "The Fifth Element"? Stay And Stay And Stay What Extended StayAmerica tells us about our lives

Saw an email from one of the folks that works on java.net. He got an inquiry about open-sourcing Alameda and was asking about the idea. The powers to be are pondering that one, and I don't have much insight into their decision-making, but it is an interesting idea.

I guess the key question for me, is whether enough of audience for Watson are developers that are willing to contribute to it, or are they mostly consumers? Ie. is there enough grassroots effort to support it?

Despite having a small part in the java.net launch, I don't follow closely the open source world. Yeah, I of course I use software from open source projects (bugzilla rules!) and I follow some of the news. I even started reading the book "The Bazaar". But I'm not deep into it. So much of my job is worrying about keeping servers stable and interpreting marketing requests rather than doing cool technical work. I started wondering about how opensourcing Alameda would change the business model. The answer I came to, is not very much. Here is how I think it would work, you give the client away for free, although maybe at some point a distributor will charge for packaging it with doc & support. It comes with some channels for free, but others have to be paid for (especially if the matching backend service charges), probably on a subscription basis. Often there is a choice between HTML scraping for free (breaks easily) and accessing feeds/webservices for a price.

Word is that, glow (cool calendar Swing app) might be open sourced too.

Kathy

A few weeks ago I browsed through blogs.sun.com, and saw a number of cute stories about people's kids. Little kids, ones that need strollers, put their foot in their mouth (awww, how cute..).

So what is up with this baby boom? No less than 12 couples I know personally, either just had or are having babies. I've been to five baby showers this year. The checkers at Costco must be wondering why I keep showing up with armfuls of baby clothes.

Being the contrarian I am, I'm not having a kid as a 30ish-40ish woman. Nope, no more strollers for me. Just nice uninterrupted sleep. My contribution to the propagation of the human species happenned in my 20s. I guess I'm just an impatient person.

It's almost scary how the marketeers have zeroed in on my son. He just graduated from high school, and we've gotten six!! credit card offers for him. We have gotten several catalogs geared towards the furnishing of his hypothetical dorm room (actually some of it is kind of cool). And let's not forget the military, they are recruiting kids to "be all you can be". When they are 18, your kid no longer needs your permission to join up.

My son is reasonably computer saavy and aware of the privacy issues. So he adopted a alias online. Problem is that he didn't do a very good job of it and so we get two of everything (except the credit card offers), one with his name and one for Mike Rotch ... get it, Mi KRotch, ha ha. Ok, so it is only funny to a 16 year old boy.

Makes you wonder, how much more sophisticated the marketeers will be, when the current baby crop reaches 18. Just think about it. The schools, which surely will continue to be underfunded, will sell information about the students to corporations. Little Jimmy will be categorized on his strengths (not good at math, but very good at art), and activities (likes basketball, let's send him an ad for Nikes). Heck they aren't going to wait until he's 18. Kids are sophisticated consumers in their own right.. right now.

As Scott once said, you have no privacy, get over it .... Seems like he is right.

- Kathy