Friday June 13, 2008

FTP client API updated

After the first round of feedback, I've updated the draft of the API to address most of the issues raised.

Among the more significant changes are:

• Use of a static create() method instead of a constructor to allow for future factories.
• Most setXXX() methods now return 'this' to allow for chaining. e.g.:
  ftpclient.setPassiveMode().setBinaryType();
• listFiles() now returns an Iterator, which is also a Closeable, instead of a List. This will make the handling of large directory more efficient.
  
• Passwords are now char arrays instead of Strings to allow memory wiping after usage.
• FtpFile has been extended to cover data returned by RFC 3659 MLSD command.

The javadoc has been updated and can be found at the same place than last time.

Keep the feedback coming.

Friday May 23, 2008

FTP client API for JDK7

Introduction

For quite a while now I have been working on RFE 4650689 (RFE: Java needs public API for FTP) which has finally been approved for JDK 7 (don't ask).

I figured it's about time I post a draft of the API we've been working on and get some feedback from the community.

First and foremost, you have to realize that we already have an FTP API (see my earlier blog entry on that front) and that the new one has to be compatible with it. We do not want two implementations of the FTP protocol (bloat is bad, Mmmmkay?), so the proposed FTP client has to be usable by the existing protocol handler sun.net.www.protocol.ftp.FtpURLConnection. This had an impact on the design. Basically, we started from the existing code (residing in sun.net.ftp.FtpClient) and extended that.

Supported features

The goal was to support as much as possible of the existing RFCs: 959, 2228, 2389, 2428, and 4217. Some of the features in these RFCs are, however, rather obsolete so we used our best judgment as to what to actually implement. Feel free to correct us if you are convinced we are missing an important part.

The most notable features gained from the new API are:

• Multiple transfers per session which was not possible with the URL, URLConnection model.
• Resume interrupted downloads.
• 'Smart' directory listing with pluggable parser.
  
• Directory manipulation: mkdir, rmdir, cd, cdup, pwd.
• SSL/ftps support, including an FTPS protocol handler to work with URL/URLConnection.
  

API

If you want more details, check the current draft of the javadoc. Be aware that there is no guarantee that this will be the final API, or even that this feature will make the final JDK 7 cut. It still has to go through the approval process.

Feedback is not only welcome, it is very encouraged. So if you have any remarks, or requests, please drop me a note or, even better, use the openjdk networking email alias.

Wednesday May 21, 2008

Indiana Jones and the kingdom of the crystal skull

I was not at Cannes for the world premiere, but I was able to attend the first show of the new Indiana Jones movie here in Geneva. This was last night at 0:01 am. I love going to these special shows. The atmosphere is so different, mostly because the audience is entirely made of fellow movie lovers (you have to be to go to a movie theater between midnight and 2am). I'll grant you that Geneva is probably the wrong place for that. No offense, but people here are certainly not as exuberant as they can be in Paris or California for instance. And that's putting it mildly. A good proof of that was that the theater was hardly full. Still, that was a good natured audience and one guy even brought his trumpet and played the Indiana Jone themes a couple of time. He was pretty good at it and was rewarded by a big cheer.

But enough about all that, let's talk about the movie now.

 Of course, as it has been said repeatedly, there is basically no way a 4th Indiana Jones movie can meet expectations after all these years. And, unfortunately, it, indeed, doesn't. If I had to compare it to the other 3, I'd say it's not nearly as good as the first and third, but probably on par with number 2 (The temple of Doom) which I always considered to be the weakest, by a large margin, of the 3. Actually, the failings of the new movie have a lot in common with those of Temple of Doom. More on that later.

 I will not give you a summary of the plot, except to say that the action takes place in the 50s, that the Russians have replaced the Nazis, and that the Amazon has replaced Egypt. There is plenty of action and a lot of fun, but, it's rather nonsensical most of the time. Indiana Jones movies have always been a bit over the top and that is actually a major part of the fun, but this time it's way, way over the top. Surviving a nuclear blast by hiding in a fridge kind of over the top (no, I'm not making that up). One of the results is that the chase sequences are overdrawn and a lot cheesier than they should be.

The script also has a few problems. For instance, at the beginning it looks like it's trying to make a political point (against the communist-witch-hunt of the time) even if it seemed a bit forced. But then, abruptly and for no apparent reason, that plot is just dropped. Plain and simple. Gone, forgotten. I did find that annoying.

An other, and much more important, problem is the the 'quest' itself. We just don't really care about it. Looking for the Lost Ark or the Holy Grail was intriguing, exciting. It was rooted deep in our psyche. Searching for clues as to where these precious artifacts might be was a major part of the fun in the previous movies. Here, not only the plot has no echo in our minds, except for an unconvincing link to ElDorado, but there is no archeology (even fake one) going on. That's where, I think, it fails in a similar way as Temple of Doom. The historical (or pseudo historical) aspect of Indiana Jones is just missing. To make matter worse there is no 'treasure hunt' with hidden clues and so on, to speak of. All they have to do is go back to a place where one of them has already been. Which is easy, since that person is with them all along. Not very exciting.

All that aside it's still a mostly fun movie to watch. And more importantly, the movie stays true to the spirit and mythos of Indiana Jones. To me that was like going to visit an old, and dear, friend I hadn't seen in years. Sure enough the party was not as fun as before and there were a couple of awkward moments. But I was glad to see him again, in a good shape. We had fun together and I, as many, left the theater with a smile on my face.

Monday March 03, 2008

Webrev script updated

A very quick update. 

I've updated the webrev script to include the following changes:

• Fixed a bug when a whole directory was removed.
• Added automatic branch detection for mercurial.
  

It is now up to version 23.10.

Keep the feedback coming.

 

Monday February 11, 2008

webrev for OpenJDK: A code-review generator

Introduction

Webrev is a tool that has been developed internally at Sun over the years to help in the process of code-reviewing. What it does is extract information from a workspace and generate HTML pages highlighting all the changes. In short: it lists all the files that have been changed and provides several views of the changes.

So, when a developer wants to have his changes reviewed, he runs webrev, checks the output, publish it on a website then sends the link to potential reviewers. This has the advantage of providing a simple and common format for such reviews, which optimizes the time spent on them.

However it is tightly dependent on the Source Control Management System used, in our case, teamware (also known as codemanager).

With the advent of OpenJDK and the switch to Mercurial (hg) as a SCMS, it became necessary to update webrev to support the new environment. I foolishly (very foolishly) volunteered for the task.

To make a long story short, I looked at what the OpenSolaris folks were doing with webrev, since they also were on the process of switching to mercurial, but it turned out we had some divergent needs and constraints. The critical list being: OpenJDK needs a multi-platform tool and support for the mercurial forest extension. So, I forked the project and started 'fixing' webrev according to our needs. I thought it wouldn't take me too long (did I mention I was a bit foolish).

Anyway, it, of course, took much longer than any of us expected (partially thanks to a moving target), but with the help of all the hapless Sun's engineers who (sometimes unknowingly) tested the various versions for me, we now have a reasonable version of webrev. By reasonable I mean:

• Continued support for teamware (we still use it, you know)
• Support for mercurial
• Support for the forest extension
• Runs on Solaris, Linux, MacOS X and Windows (under the Cygwin environment)

So, how does it work?

The simplest way is to make changes in your workspace, cd to the top level directory and type 'webrev':

$ webrev
   SCM detected: mercurial

 No outgoing, perhaps you haven't commited.
      Workspace: /home/guest/MyWS
      Output to: /home/guest/MyWS/webrev
   Output Files:
        File1
                 patch cdiffs udiffs sdiffs frames old new
        File2
                 patch old
     index.html: Done.
Output to: /home/guest/MyWS/webrev
$

And voila! You have a subdirectory named webrev (as well as a zip file of it for easy transfer) which contains all the HTML pages needed for a code review.

Of course there are a bunch of options that let you specify, for instance, where to put the webrev, where the workspace is, which version to compare against. I've put together a small documentation page where you'll find the details on that.

Requirements

You will need a few other (rather standard) tools in your path for webrev to run:

• Korn Shell (aka ksh): this is a shell script that relies on a few Korn Shell idioms. It also expects to find it at /bin/ksh. Of course you can edit the first line of the script to change its location.
• awk (or gawk or nawk)
• perl
• jar
• zip

There is a bit of sanity checking done at the start, so it will most likely tell you if anything is missing

Grab it here

So now, just get your copy of webrev, give it a spin, and feel free to send me feedback and bug reports.

Thursday May 24, 2007

Pirates of the Caribbean: At World's End

Another day, another sequel (sequel of a sequel actually). Again it was released in Europe a few days before the US, which always make me snicker a bit (Hey, I do remember a time when I had to wait months after its US release for a movie to show up in theaters in Paris where I grew up. So I think it's fair enough).

Anyway, my expectations for that one where actually pretty low this time. The first "Pirates" took me totally by surprise (talk about low expectations when I went to see it!) and was a total blast. It was up there with Shrek when it comes to movies that surprised the heck out of me at the time. But the second installment widely reported flaws seriously dampened my enthusiasm. While there was some fun to have, most of the magic was gone. There were a number of reasons for that, and I'll come back to it later since it is relevant to the 3rd episode.

A funny thing happened while we were waiting to enter the theater (which was hardly full by the way): One of my friend who couldn't remember the previous episode asked me to refresh her memory. Turns out I failed miserably at it. The story was so confused and so uninteresting that it dragged down all my efforts to tell it (you try it! You'll see I'm not kidding). It also proved, to me at least, that I didn't really care about what would happen to the various characters of the movie. Low expectations, I'm telling you.

So, how bad is it? Let's see. The movie is about 2h45 long, and, if you ask me, that's about 1h too long. Seriously! The first 1h30 to 2h of it just drag forever. It's tedious, it's boring, it feels like it will never end! The good news is that it DOES end and the last 45' or so are fun to watch. The bad news is that it's not quite worth it.

And we have the usual suspects to blame for that misfire. First, and as way too often, the script. No real surprise since this 3rd part was shot back to back after the 2nd one, so there were very little chance the writing would have improved. The story is a confusing mess that doesn't make sense most of the time. I know it's fantasy, but come on! Is it too much to ask that the characters make sense within their world? That their motivation are believable? That their relationships seem to work? Because we have none of that here. Make us care for what what happens to them!

The second strike, for me, is Elizabeth, the character played by Kiera Knightley. It was obviously shoe-horned in there to give her a bigger part. In the first one, she was the archetypal "damsel in distress". She wasn't a very good one (missing charm and charisma if you ask me) but that didn't matter much. But now, she's been promoted to swashbuckling, ass-kicking, killing machine, and I'm sorry, but that doesn't make any sense at all. Plus she's not one bit convincing about it. To make matter worse, her relationship with Will (Orlando Bloom), which wasn't really hot stuff to begin with, is so flat, so unemotional, that when they do finally express their love to each other (they spend the first 3/4 of the movie ignoring each other) you're quite surprised. Everything about her is forced and feels fake. It doesn't help, of course, that she doesn't seem to be a very good actress either. At least not in these 3 movies, if you ask me.

Third strike, and I'll stop there, is that the humor that made the first one such a fun experience, is mostly gone. Oh, sure, Johnny Depp do have a few funny scenes and some good lines. His scene with Keith Richards, for instance, is genuinely funny and perfect for the character he created. But that doesn't save the movie.

A couple of quickies:

The special effects are worth mentioning for two reasons: they're perfect and they're probably why the movie falls apart. As is typical with Hollywood lately, too much focus on eye candy, not enough on story and characters.

There is an extra scene after the end credits. But considering they do last for a long time (8 to 9' I believe), it may not be worth waiting for it. If you don't have the patience, be assured you're not missing much.

Next sequel of a Sequel: Shrek the third, which hasn't been released in Europe yet (there is some karma for you). Somehow I suspect the trend of this summer will continue. We'll see soon enough. Not holding my breath though.

Wednesday May 09, 2007

Cross Platform Networking

So, you want to contribute to the openJDK networking library? Let me tell you: We (as in the current team working on it) are very glad you do, and I mean that. There is a lot of work to be done, and we definitely could use a little help. There are a few things you should know before heading down that road though. You see, as you probably already know, having a consistent behavior across all the supported platform is one of the main design goals of Java. But I need to let you in a little secret: cross-platform networking is hard, very hard, to achieve. Why is that so do, you ask? After all network protocols and APIs are well specified in RFCs and other standardization documents. After all, a socket is a socket, ain't it?

You wish!

And, actually, so do I! That would make my job so much easier! But there are a few problems with that assumption. First, RFCs are not always as ironclad as we would wish and can leave room to interpretation. One example of that is what happens with IPv4 multicast addresses in IPv6. The RFCs specify that an IPv6 socket has to support Ipv4 addresses for backward compatibility in the form of a special kind of IPv6 addresses (known as Ipv4-Compatible). This makes it possible to use a single IPv6 socket to talk to both world, the old Ipv4 one and the bold new IPv6 universe. Only problem with that is that the RFCs do not state clearly whether this should work for joining IPv4 multicast groups as well. So, Lo and Behold, guess what happens? On some systems it does work while on others it's not supported and, of course, they don't consider it a bug! I let you imagine the hoops we had to go through to get a consistent behavior here.

Then there are the unfortunate cases where the standard specifications are blatantly ignored for whatever reasons (don't ask, trust me on that one, you really don't want to know). An example of that would be the checksum field in the ICMP header which is supposed to be computed and set by the OS network layer before sending the packet. However some implementations of the TCP/IP stack leave that responsibility to the application layer. But, to be able to compute that checksum, the application needs to know what the source address will be in that header, which is not always readily available at that point, specially on machines with multiple network interfaces.

Another frequent issue is when we're asked to provide support for features that exist on some system and not others. A good example of these are the “Unix Domain” sockets (see RFE 4145756).

So, how do we deal with all that? We go through a succession of tough choices. For instance, the Unix Domain Sockets, while we think it does provide a somewhat valuable service, it is also a bit redundant with normal sockets, and, of course, it's not available everywhere. In that particular case, we decided that the cons were out-weighting the pros and voted against implementing it for the moment. But let's assume for a minute that we still wanted to do it at that point, we then would have to design an API that would not break if the feature was not available. And no, throwing an Exception when trying to create a Unix Domain socket on Windows, for instance, is not the right answer. So we would go to a “Service Provider Interface” (aka SPI), meaning the application would have to request the “service”, Unix Domain Sockets here, and, IF that service is available, get a reference to a “factory” in charge of creating these sockets (note: that's not the only approach to solve that kind of issue, but it's a rather common one). The end result is that it is now the responsibility of the application to deal with availability, or non-availability, of the feature, and, of course, it makes the code much more complex. What happens when the application absolutely rely on such a feature and it is not available? It has no option but to exit, stating something like “Sorry but this application can't run on your system...”. Which is a rather harsh blow against the “Run Anywhere” mantra. That's why we always think twice, or three, or four times, before going down that road. Specially when it comes to something as core as the Networking Stack.

In other, more minor,cases, like with certain socket options, we do actually word the documentation in a way that clearly states that this may be ignored, or re-interpreted, by the OS specific implementation. A perfect example of that is the documentation of Socket.setTrafficClass(int). Naturally, that approach also has its problems, and it's not rare that developers contact us, complaining about the fact that one of these methods seems to have no effect in their application.

I hope that by now you have a better idea of what kind of difficulties we have to deal with. But don't let that stop you working on your project, we'll be here to offer help and advice when you'll ask for them.

That's it for now. In a future post, I'll outline what we're currently working on.

Thursday May 03, 2007

Spider-man 3

And now for something absolutely non Java related....

Yesterday, with a few friends, we went to our local multiplex to watch the newly released Spider-Man 3 (Imagine that: it was released in Europe a few days before the US!). Quick background: As a teenager I grew up with Spidey and, like many at that time, I felt quite close to that nerdy Peter Parker. I also thoroughly, after years and years of waiting, enjoyed the first two movies. They succeeded in doing something pretty damn hard: updating a material from the 60s while staying faithful to its spirit and core appeal.

But that was then, let's see what we have now.

When the movie opens, everything is peachy in Peter Parker's life! Did I say peachy? Scratch that! He's pretty much on top of the world: He's sailing through his classes, the woman he's been dreaming about all his (admittedly short) life is absolutely crazy about him, and everybody, and I mean absolutely everybody, in town think his hooded alter-ego is the best thing that ever happened to the city that never sleeps. They're even going to give him the keys to the city, for crying out loud. The only shadow to that sunny picture is the fact that his best friend, Harry Osborne, is still mad at him because he believes Spider-Man killed his father. The nerves of some people! Meanwhile, Mary-Jane's life is going down the crapper because she got bad reviews to her latest Broadway production (yea, I know, don't get me started). At that point, if you have seen Spider-Man 2 you may think "Hey, this is the exact opposite situation as in the previous movie. How clever of them!". Well, not that clever if you ask me. You see, one of the major part of the character from the comics book and from the previous movies is that being Spider-Man is a curse to Peter. This is his cross to bear and what makes him so special. So, now, he's just another "man of steel". Booooring! The worst part is that it goes downhill from there.

There are many problems with that movie, the first one, as way too often, is the script. There is no focus whatsoever in that mess of a story. You have 3 villains, none of them remotely interesting or even developed, then you have a very poorly developed love triangle (actually 2 love triangles, a mess I'm telling you) and, to top it all, the return of the "dark side" (you know, the one that is so tempting, so easy to follow). And none of that works.

Second problem, the pace. That movie is way too slow and way too long. They could have cut 30 minutes of it without any problem. I'm quite surprised here, because until now Sam Raimi had a rather good track record on that front. Well, I guess there is a first time for everything. My 3 friends were as bored as I was during the whole movie. not good!

If all that wasn't bad enough, the acting is really sub-par here. I know, it's not a movie where you'd expect great performances, but the last two episodes provided some more than decent ones. Here, my best memory when it comes to acting, is Bruce Campbell doing an hilarious John Cleese imitation in the part of a French Maitre'D (I know it sounds ridiculous, but it works). All the main actors are going through the motions without delivering. The worst offender, to me, is Tobey Maguire during his "I'm bad!" stint. It was embarrassing to watch.

I could go on with a few minor things that annoyed the hell out of me, like the fact that the masks and hoods keep being removed and put back for no good reason and sometimes with a blatant disregard to continuity, not to mention any attempt at credibility. But I think you got the point. The 3 major flaws I mentioned above would kill any movie, so there is no real point punching more holes in that fast sinking boat.

Well, that was a disappointment. But I guess they had to adhere to the Hollywood law that says that in any trilogy, one of the episodes must be stinking bad. Mission accomplished, then.

Tuesday March 27, 2007

Spring cleaning and FTP

With OpenJDK upon us I figured it's about time I dusted my blog and start posting again since I intend for it to be one of the sources of information and discussion concerning the evolution of the networking stack (more on that later).

Anyway, since it's a question that comes up quite often these days, it occurred to me that the state of FTP support in J2SE would be a good subject for jump starting my writing efforts. So, without further ado:

FTP support in J2SE

Let's get something out of the way first: Yes I'm fully (dare I say painfully) aware that the current FTP API is somewhat lacking, to say the least. There is an RFE filed against it, 4650689 to be exact . It has rather high vote count (119 last time I checked and is on the top 25 RFEs), and, believe me, it hasn't gone unnoticed. I have been pushing for that RFE for a long time now and I have hopes that it will finally get approved soon. When that happens (fingers crossed) I'll post here the details of what we intend to do. And I'd like to personally thank all and everyone of you who voted for it and/or posted comments. It does help. A lot. So thanks again.

Meanwhile, let's have a look at we already have.

FTP is one of the protocol handlers mandated for the URL API. Which means that, for the time being, the only public FTP API in J2SE is through the URL and URLConnection API. In short, it means that to access a file on an FTP server you create a URL as specified in RFC 1738, get a URLConnection out of it, then use getInputStream() or getOutputStream() to get or put, respectively, the document. Did I say 'put'? Yep, I sure did! How many of you did know that you can do puts as well as gets? <counting...> Really, that many? I'm surprised. So, how many now knew that you can also provide a user name and password? An absolute file path? Or a type (binary for instance) of transfer?

Yes you can do all that. Let's look at a quick example:

URL url = new URL(“ftp://user:password@host.dom/%2Fetc/motd;type=a”);
URLConnection conn = url.openConnection();
PrintWriter out = new PrintWriter(conn.getOutputStream());
out.println(“Message of the day”);
out.close();

And voila! You just logged to an FTP server using a user name and password, then wrote to the /etc/motd file by doing a PUT in ASCII mode.

The important things to notice here are:

• Calling getOutputStream() instead of getInputStream() will trigger a PUT operation instead of a GET.

• If you need to authenticate when logging onto the server, prefix the hostname with user:password@

• If you need to use an absolute path use %2F (code for a slash) at the start of the file path.

• If you need to specify a type of transfer (default is binary) add ';type=<a,i,d>' at the end of the URL. 'a' means ASCII, 'i' Image (or binary) and 'd' means directory.

I'd like to add that the FTP protocol handler will also automatically try to use passive mode when possible, knows about IPv6 extensions and will, if the URL points to a directory, give you a listing of its content.

So, I hope you can see now that you can do many FTP operations through the URL API.

Like I said earlier, I know that many features are missing, like being able to do more than one operation per session, or rename a file, or use FTP over SSL and so on and so forth. But hopefully, not for much longer.

Wednesday July 21, 2004

There is no place like ::1

If you got that joke then you're part of a small club: the few (for now) who can read IPv6 addresses in literal form.

Yes, IPv6 again! Why do I come back to that topic? Because yesterday (July 20th 2004) ICANN officially announced support for IPv6 in their root servers.

What the hell does that mean, do you ask? It means that DNS, the mechanism that translates internet names like www.sun.com into IP addresses, can now return IPv6 addresses as well! The first 3 countries (or subdomains) to follow suite are Japan (.jp), Korea (.kr) and France (.fr).

Things are really moving fast on that front. After the Department of Defense mandate (see my blog entry dated June 28th) this is a very important step towards the large scale adoption.

What we need now is Linksys & Netgear providing IPv6 with 6to4 tunneling in their routers and we're all set!

P.S.: ::1 is the IPv6 equivalent for 127.0.0.1, the loopback address, also known as localhost or ... home.

Tuesday July 13, 2004

Firefox & Thunderbird: my setup

Another technical entry, but this time not Java related.

For quite a long time (years really) I have been using Mozilla a my main browser and mailer. About a year ago, I think, I switched to Firefox (aka Pheonix, aka Firebird) the standalone browser and Thunderbird, the standalone mailer. Both are from the Mozilla Foundation, so they inherited from the years of development and debugging that were put into the Mozilla suite. Why did I switch? Mostly because I wanted something a bit lighter, because I liked the look and feel better and also because I wanted to support an initiative that I thought was worthwhile (meaning: I thought it would be fun).

Besides all the security and privacy advantages, which are getting a lot of publicity these days, I like the extension mechanism and I wanted to share with you the few I use:

• Firefox Extensions:
  • AdBlock: This one is a must. I just can't live without it anymore. It let you filters out advertisements and other nasties (like web bugs, and other tracking meachanisms) based on pattern matching. A real life safer.
  • SwitchProxy: If like me you change your proxy on a regular basis (happens a lot on a laptop, or when switching between VPNs), this is a very useful toolbar. It let you pick a proxy setting from a drop-down menu. Very neat.
  • Dictionnary Search: Another must. Select a word, right-click to get the context menu and launch a dictionary search in a separate tab, and yes, you can configure which online dictionary to use. Very useful when blogging!
  • Noia 2.0 Extreme: Not an extension, but the theme I use. Very nice. On top of that, it does exist for Thunderbird (see below), so I have a unified look and feel for both apps.
• Thunderbird extensions:
  • Contact Sidebar: List your contacts from your address book in the sidebar, below your folders. Pretty neat.
  • Free Desktop Integration: Another must if you're using Linux or Solaris. What this extension does is use the notification area in the Gnome (or KDE) toolbar to let you know when a new mail arrived. That way your mailer can be iconified and you still get a visual notification. The Linux binary is available on the page, I had to port the sources to Solaris (Solaris 10 with Gnome 2.6 to be exact), but it works fine.
  • EnigMail: A very powerful extension that let you encrypt and decrypt your mail with PGP or GnuPG.
  • Noia 2.0: Again, not an extension but a very nice theme. See above.

That's it for now. Don't hesitate to let me know what your favorite extension is (yes, I know about mouse gestures, I just don't like them that much).

Sunday July 11, 2004

java.net API & proxies

I got some interesting feedback from my last entry, and I may post an update, but I'd rather sleep on it a few more days, at least. Thanks to all those who took the time to comment. I appreciate it.

It is my intent to have technical entries in my blog, not just rants or movie reviews. So I figure it's about time I get to it. One thing that bugs me is how badly documented the Java Networking API is. A lot of the bug reports, or requests for new features, we are getting prove that. Many times the answer is "You can already do that using this API" or "This is not a bug, it is supposed to do that". Of course we, the Networking Team, are to blame for that, but as all developers know, documentation is hard to do and very time consuming. The good news is that we've taken steps to remedy that situation. In particular you'll see that, in J2SE 5.0, we've improved the javadoc entries quite a bit. But we're also working on some other means, like white papers and articles that, hopefuly, will end up on the web site.

And that's precisely what I want to discuss here, I have written such a white paper, on proxy management in the Java networking API and I'd like you to review my most recent draft and let me know what you think of it, particularly whether it's clear or complete enough.

Here is a quick overview of its content:

• System Properties: We first discuss all the system properties related to proxy settings, there a quite a lot of them, and how they affect each other.
• New APIs in 1.5.0:
  • java.net.Proxy class: We present the newly introduced Proxy class and how to use it in new methods for Socket and URLConnection
  • java.net.ProxySelector: We go deep into the new mechanism to let applications dynamically assign proxies to the different protocols. The class is described, and using a practical example, we do show how to implement such a selector.

So, if you are interested you can get the PDF file, read it and, I hope, send me feedback, either through this blog or directly via Email.

Also, there are two other subjects I'd like to solicit feedback: What part of the java networking API would like to see described next? (some possible candidates are: Cookie management, IPv6 support, timeouts, etc.). And, what is the networking feature/bug that you think should be implemented/fixed in the next release, and why is it so important to you?

Thanks in advance. (2004-07-11 07:49:33.0) Permalink

Thursday July 08, 2004

Sun and OpenSource: Old but estranged friends?

Something that really ticks me off lately is the whole "Sun vs OpenSource community" thing!

Really, it's ridiculous! It's like watching two very old friends having bitter arguments, while forgetting how much they owe each other, and how much they need each other. Both party are guilty here, if you ask me (and if you didn't ask, or don't care,then stop reading now).

Let's start with full disclosure: I have been involved with the Open Source Community long before it was even called that, back when it was still called "Free Software". Heck, I was one of the main contributors to one of the most notorious, and most enduring (17 years and counting), project. I also co-founded a small not-for-profit organization dedicated to Free Software, which later, much later, became the French branch of the Free Software Foundation. Although I have to admit, my participation was minimal, specially compared to the main guy behind it, Loic Dachary (Hi Loic, how you doing?). But I still ended having that guy as a guest in my place for 10 days, which was quite an experience, as those of you who ever met him may imagine, but that is another story (maybe another blog entry, who knows). Ever since, I have been, and continue to be, a vocal proponent of Free Software and Open Source.

On the other hand, I joined Sun Microsystems almost 10 years ago and I have been invovled with Java, one way or the other, since the beginning (1995 and JDK 1.0.2 if you can believe it). And for the past 4 years, I have been integral part of the Java Engineering team, here at Sun.

So, why this rather long, and I apologize for that, disclosure, to the risk of looking like I'm bragging? Because it shows pretty clearly where I come from and where I stand: I have one foot firmly on each side of the fence (and we all know how uncomfortable it can be, specially when said fence is getting taller with each passing day. Ouch!). The being said, it doesn't mean I'm not bragging too.

Now, back to the main point: Sun and the Open Source community. You know, I remember a time when Sun was so completely synonymous of Free Software that almost all Free Software, including the GNU project, the X11 project, and hundreds of others, were developed primarly on Sun's platform. It used to be that the default configuation to these softwares was for SunOS. There were a number of reasons for that, but more importantly, it benefited both sides. And that special relationship is one of the reasons I joined this company in the first place. Then things started to change, and that relationship got looser. If I do remember well, the first release of Solaris (or SunOS 5) was the trigger, for 2 main reasons: First, it was based on a commercial Unix (System V.4 to be exact) instead of BSD. Second, Sun stopped providing compilers for free with the OS (which, to this day, I still think was a major mistake, but that's just me). Since then, both parties have drifted even more apart. And nowadays, most Open Source projects are developed on Linux, to the point where, more and more often, I actually do have to do dome porting (and I do mean the "messing around with the source code" kind of porting) when I want to run some OSS applications on my Solaris box. The horror, the horror! Specially to a guy who fondly remembers the days when he just had to type 'make' and wait for the build to finish.

Now, when Scott McNealy claims, on stage at JavaOne last week, that no other company contributed as much as Sun to the Open Source idea, he is absolutely right! And I wish the community would remember that. After all, aren't they the same people who are big on giving credit when credit is due?

Of course, Sun, as a company, should also remember that it did benefit quite a lot from all the contributions to Open Source. For instance, the main user interface of Sun's products is based on X11, right? And all these sales to academia, among many institutions, were without any doubt helped by the knowledge that there was this huge pool of free software just asking to be run on Sun's hardware! It was, and still is, a two way street. Both side should have realized that a long time ago, don't you think?

Let's talk about OpenSourcing Java, which seems to be the main point of dissension these days. I, for one, would love to get some of the benefits of the Open Source modus operandi when it comes to Java. Particularly as one the developers working on it. I have wished so many times that we could use the "release early, release often" model, and I would love to get direct, or indirect, help (in the form of code, suggestions or discussions) from all these talented developers out there. Also, considering my background, I would love that out of principle

But you know what? It's not going to happen. And it's almost certainly better that way! I won't go over all the arguments again, as you're probably tired of hearing, or reading, about them. Except for the main one: Write Once Run Anywhere is the obvious decisive factor here. I have turned this all over my head many, many times, and I can't figure a way to keep that without a tight control. It's that simple.

Let me also state again, as many have before me, that nothing stops you from creating an fully OpenSource version of Java. At least, nothing in theory, since all specifications are completely open. But I do know that it is incredibly hard! Almost impossibly hard, because there is more to binary compatibility than following the specifications. A lot more. And Sun should acknowledge that.

Which brings me to this question: What can we do to bridge the gap between Sun and the OSS community? How can we get these two, somewhat, estranged friends to be in good terms again?

To the Open Source Community, I'd say: lay off Sun's back a little! And remember to give credit when credit is due for crying out loud!

To Sun, I'd suggest to find ways to make life easier to the OSS developers and distributors. Why can't we have a license that makes it possible to easily redistribute the JDK with every Linux distribution? Why can't we find a way to make it easier for the developers of Kaffe or GNU Classpath, for example, to test the conformity of their implementation? And overall to play nice and fair with the community.

To me it doesn't seem like it would take a lot of efforts to make all this happen. But I have been known, at least to my friends and relatives, to be somewhat of an utopian. But, you know, sometime utopies do happen.

So what do you say? How about being friends again?

Wednesday June 30, 2004

JavaOne, J2SE 5.0 and Spiderman 2: Go get them Tiger!

Who needs sleep anyway? That's what I was telling myself while leaving Moscone center around 11:30pm after presenting at a late BOF and heading for the Metreon movie theatre, a ticket for the 12:01am showing of Spideman 2 in my pocket.

I love opening nights of long awaited block buster movies! Sure, the line can be a pain, but it can also be quite a bit of fun. After all, the people around you are probably as nut as you are otherwise they wouldn't be here, would they? And, once you are comfortably installed in your seat, nothing beats the atmosphere in the theatre. You can feel the eagerness of the crowd, and you can't help smiling when the audience bursts into applause for such silly reasons as the lights being turned off or the main title starting. There is also something to be said for a pristine copy of the film (of course, this will no longer be an issue once theatres all have switched entirely to digital projection, but it won't happen overnight).

Back to the movie. Peter Parker, and his super-hero alter ego, are back. It's 2 years later and Peter is confronted to a severe case of the "life sucks" syndrome. I won't go into too much details as I hate spoilers and some of the most enjoyable moments of the movie are a lot more fun if you don't know they are coming. I'll just say the opening sequence got the audience, yours truly included, laughing and cheering more than once. And sequences like that abound in this movie. It's refreshing to see a good action/super-hero flick that finds a right balance of action, character development and self derision.

Dang, I just blew it! I already told you the movie is good, while most reviewers would wait until the last paragraph before telling you what they really think, forcing you to go through their whole article in the process. Ho well! I guess you can leave now. Doesn't really matter, it still counts as a hit to this page

Still here? You know, you got way too much time to waste! OK, you asked for it.

It's a fun movie! I could tell you to go and see it, specially if you liked the first one, but it would be a waste, since I know you're going to go no matter what I say! One thing I appreciated is that, as a fan of Spidey when I was a kid, Sam Raimi was able to re-unite me with the feelings I experienced then. Particularly in the "it sucks to be Peter Parker" department. At first, I was a bit uncomfortable with it, thinking it was a bit overdone, but then I remembered that it is exactly what I was thinking back then, while reading the comics. Kudos, at least from me, to the director for sticking with it despite the temptation to tone it down a bit. The actions scenes are top notch, full of energy and, more importantly, they don't feel like you've seen them dozen of times in other movies. They do have a style, a rythm that make them standout.

There is a quite a bit of foreshadowing, specially for the upcoming 3rd installment, but it is rather well done and doesn't stick like a sore thumb. It does flow quite well with the story.

I have just a couple of minor gripes. The middle section could have been shortened a bit, say 10 minutes or so. Also I was not that impressed with Kirsten Dunst performance. She seemed a bit wooden to me. And, last, I felt that some of Aunt May's speeches (you know the "with great powers come great responsibility" kind) were a tad forced. They just didn't seem to come as natural. But, like I said, these are pretty minor and certainly didn't do any damage to the movie overall.

There you have it: Great sequel, can't wait for the DVD, and for once, I'm eagerly awaiting for the 3rd opus hoping they can keep the same level of quality and fun.

One last thing, if you were a JavaOne or an Apple WWDC attendee, as were half of the audience last night, you should have a good laugh at the last line of the movie.

Let me know what you think!

Monday June 28, 2004

Java: your magic ticket to IPv6

A couple of weeks ago I was in L.A. attending the IPv6 Summit where I was scheduled to make a presentation about Java's support of IPv6. while there, listening to the other presenters I realized something. There is a gap, a disconnect, between application developers and the network infrastructure people. You see, very few attendees were developers, or ISVs, so it felt a bit like most presentations were "preaching to choir" so to speak. Most people in the room were convinced that not only IPv6 was important, but it was also inevitable. It will happen, and, actually it is happening. Many of them were actively involved in making it happen. So how come, when I talk to application developers, they don't seem to care one bit about IPv6 support? Even when their application is heavily networked.

I've been pondering that question for the past 2 weeks and I've come up with a few elements of answer.

First, and foremost, most developers don't know what IPv6 is all about, and when they do know a bit about it they seem to believe that we don't really need it, what with NAT (that's Network Address Translation in case you didn't know) and similar technologies. Second, to most of them, supporting IPv6 means "porting your application", like supporting a new platform that almost nobody is using. In today's market, it's like a death sentence. But it's also a catch 22 situation. You see, many ISPs says they're not deploying IPv6 because there is no demand for it (at least in the US), and there is no demand for it because there are no application for it either. We got a bit of a deadlock here. I'll get back to that later. First I want to address the first point.

Why do we need IPv6? I won't bore you here with all the technical reasons, you can read about it from a number of sources, including at the IPv6 summit website, in particular in the tutorial section, I'll just mention the main 4 ones:

• Address space
• auto-configuration
• end to end security
• mobility

The first one, of course, is the real kicker: with address length of 128 bits instead of 32, IPv6 makes it possible to assign an address to about any device ever conceived and still not run out of addresses. Couple that with auto-configuration and you won't have to worry about IP addresses ever again.

Now, I know what you're thinking: "NAT solved that problem already". Not quite so, for one thing NAT creates a lot of problems when it comes to end to end communication (think P2P for example), and already some organizations are running out of "private addresses" as defined in RFC 1918. Also, it is has been estimated that if the developing countries (China, India, etc...) get to the point where a mere 20% of their population has Internet access, then we would need 4 times more addresses than are left available, and that's taking NAT into account! So, we don't know when we're going to run out of addresses, but it could happen almost overnight! That's why Asian countries and the European community are pushing IPv6 very hard. On top of that the U.S. Department of Defense (DoD) as announced that they will switch their whole network infrastructure to IPv6 by 2008 and they issued a mandate that, since Oct. 2003, all IT equipment they will purchase, be it hardware or software, has to be IPv6 enabled or have a clear roadmap proving it will be in time. So, if the DoD is one of your customers you probably should worry about IPv6. Same thing is true when it comes to mobile devices, like cellphones or PDA.

So IPv6 is not only needed, it is happening, big time! And, you, developers, should get on the ball!

But, what does it mean to port an application to IPv6? Well, herein lies the "bad news, good news" cliche. The bad news is for applications written in C or C++ (or even C#) the porting can range from simple, for client applications on Unix, to total nightmare, for server applications on Windows (you see Microsoft didn't deem necessary to provide a proper dual stack implementation therefore their IPv6 is the only one I know that is not compatible with IPv4). In any case it means source code changes, new binary relying on new libraries, and, of course, new bugs! Bottom line a lot of work. So what is the good news, you ask? The good news is if you were smart enough to write your application in Java, then you don't have to do anything! You're done. I mean, your code is already IPv6 enabled, you don't even have to recompile. That's right: that application you wrote, and compiled, with JDK 1.1 in 1996 will work just fine on an IPv6 system if you run it with the proper VM (1.4+ for Unixes, 1.5+ for Windows). Let me rephrase that: Any java application (as in the exact same bytecode) will run on any platform whether it's IPv4 or IPv6, Linux, Solaris or Windows! You got IPv6 for free! Remember that the next time you have to convince your management that you should use J2EE instead of .Net for instance!

If you'd like more details about IPv6 and Java, I'd point you to the presentation I made at the IPv6 summit.

In any case, I'll be at JavaOne, so don't hesitate to look me out and ask me any question.

You can catch me at the "ask the experts" booth (don't laugh I didn't pick the name) on Tuesday from 10:30am to 1pm, then at the Technical session TS-2476 on Tuesday from 2:45pm to 3:45pm, and finally we'll have a BOF for Java networking Tuesday at 10:30pm (Esplanade 304/306, Moscone center). I hope to see you there.