James Gosling: on the Java Road

« Moore's Law meets... | Main | Transcendental Medit... »

20050725 Monday July 25, 2005


Happily Subversive?

I've been spending the last few days helping figure out what we (Sun) should do about version control for all of our source files. We've been using a system called TeamWare that we developed in-house years ago. It's the father-of-BitKeeper. It's solid as a rock and scales well, but no one has worked on it for years and it's beginning to show its age (in particular, it has no web-based distributed development: it's based around NFS).

So I've been going through the alternatives. BitKeeper is "problematic" (mostly: incompatible with working with open source organizations). CVS has a huge raft of technical problems. We've thought about open-sourcing TeamWare, but there would be a lot of engineering effort required to bring it into the modern world and run on many different platforms. SubVersion+svk is looking interesting, but it's hard to tell how well it works under fire at scale.

I'd love to hear from folks who have used SubVersion (with or without svk) for multi-million-line code bases with thousands of versions.

(Thu Jul 21 12:22:25 PDT 2005)
Permalink Comments [20]

Comments:

I don't have precise numbers, but I know of at least three huge Subversion repositories : the gcc one, the KDE one (more than 400,000 revisions) and the Apache one (including all the CVS for Apache HTTPD server, the Jakarta projects and so on, with more than 225,000 revisions). There was a talk at ApacheCon 2005 about the migration of the Apache CVS repository to Subversion, unfortunately I cannot find any report of the talk.

Posted by Nicolas Lehuen on July 25, 2005 at 12:15 PM PDT #

I for one would love to see subversion used - as you say CVS has it's problems, even though we successfully continue to use in the JDS group, but it has become dated, and SVN does appear to be the direction that many projects are heading in. Though TeamWare is excellent in many respects, for true distributed development NFS simply doesn't cut it as even a simple check to see if we have the latest sources is painfully slow when compared to CVS or SVN. +1 from me - and I'm hoping it won't take too long to get it done so we can get developers contributing directly to OpenSolaris!

Posted by Darren Kenny on July 25, 2005 at 02:24 PM PDT #

<hr> <center>Subversion for Blastwave</center> <hr>
I am currently working on a subversion repository for Blastwave such that we can migrate ( gently ) towards a build system in which all dependencies and upper level dependants may be rebuilt nightly if desired. The objective here is to create a single large repository for all the software at Blastwave and to track the changes being made by the maintainers in order to get the software running well within the Solaris or OpenSolaris world.

My hope is that everyone will benefit from having the work of the Blastwave maintainers opened up to anyone that will want to know what we did to get software XYZ working well and then packaged for Solaris users.

Dennis Clarke
Director Blastwave.org

Posted by Dennis Clarke on July 25, 2005 at 03:40 PM PDT #

KDE has switched to Subversion this year. there are approximately 5.2 million lines of code in KDE, not counting the extras which are not part of the mainline distro. The only complaint so far has been that svn can be slower than cvs when doing large updates, but this seems to be a minor complaint. the advantages of being able to do atomic commits and merges far outweigh the slowness disadvantage. My 0.02. --Stefan

Posted by Stefan Teleman on July 25, 2005 at 04:25 PM PDT #

Hello

In my previous company, I led ( and actually performed - we were small company ) the activity to migrate from cvs to svn. It was medium size repository ~10k commits, some branches, ~300MB. I am also havnig my whole home on swan (home mounet from network) managed by subversion, whith repository on local drive (yes, I'm that crazy :) ).

My own experience with svn tells me several things

  • never ever use bdb backend
  • do daily backups of the repository ( I had to fix the repository several times, by combining two or more backup dumps into one )
  • your work directory may broke, and you may be unable to fix it
  • ( to be fair ) if you have broken working directory, people on #svn are usually able to help you

That said, svn is great tool, but I doubt that it the quality is high enough for critical mission work. And svn is not distributed, while teamware is.

Now svk. I have never used svk, but I am definetely going to use it. I read the documentation, and I really like that project. On the other hand, I'm afraid that it's yet too young project. Looking into their tracker gives you the idea of the state of the project.

That said, I do not want to criticize those projects, they are definetely great, and I'm using svn for some time without any problems. Compared to for example perfoce, their usage is really simple.

I'm not with sun for a long time, and I'm starting to like teamware more and more. I think that teamware and sv[nk] just have slightly different goals.

Do you miss web based access for teamware ? I can imagine read only acces done by cgi. The repository even does not need to know that it's accessible by http, no changes to it needed. It's bound to NFS ? I don't think so, any filesystem will do. Your local filesystem too. ZFS in ( hopefully near ) future will do.

Anyway, that's my 0.02 [ local currency ]

-- Vladimir

PS: Why oh why is the editing window here of the size of my invitation card ? :)

Posted by Vladimir Marek on July 26, 2005 at 01:38 AM PDT #

I think that there is already cgi for browsing sources in teamware. It's name is sccs*.cgi and its used e.g. in http://aggregate.eng/.

- Vita

Posted by Vita Batrla on July 26, 2005 at 02:13 AM PDT #

Have you considered more radical steps beyond CVS like DARCS or Superversion?

Also, I know there is substantial cost in open-sourcing TeamWare, but why not just scrub the code, dump it, and let the community decide if it warrants further work. The community will decide in the end anyway.

I know that Sun opens a fair bit of the software it EOLs. That is far more than just about any other company does. If Sun really wants to embrace openness, you should have a policy of opening software as part of the EOL process. As a customer of a EOLed software would you want any less?

Posted by Curt Cox on July 26, 2005 at 06:04 AM PDT #

The background on this page is cool, but makes it very difficult to read the actual text...

Posted by Paul Rivers on July 26, 2005 at 09:11 AM PDT #

If you're going to put a significant effort into something, put it into Subversion. That's the future of version control for open source projects. SVN support in all the major IDEs is maturing quickly, so if you were to stay with TeamWare you'd probably want to implement the SVN protocol on top of it anyway. Maybe you use SVN "out of the box", or maybe you make a new SVN backend based on TeamWare. Either way, it benefits you, and it benefits the entire community. Go for it.

Posted by Jason Sando on July 26, 2005 at 10:01 AM PDT #

I've used perforce (http://www.perforce.com) for about 8 years now, and it's operation has been flawless. The flexibility and dynamic capabilities of perforce for remote development have been very powerful for me. I have been considering subversion, but the version 1.0 nature of it and the reported problems tell me that it's not quite ready for me.

Posted by Gregg Wonderly on July 26, 2005 at 10:46 AM PDT #

Are you in a hurry?
Because http://www.bazaar-ng.org/ looks promising...
Cheers
JD

Posted by JD Evora on July 26, 2005 at 11:18 AM PDT #

My company has been using subversion for quite some time (way before 1.0), we've never had problems with broken work directories, and we have at least several thousand revisions as well as several branches. We found that using the 'fsfs' backend along with 'svnserve' instead of svn+apache gave us a huge performance boost and plays much nicer with our automated backup systems. We have no complaints at this poitn, and I use it in other projects as well. Out of the practical alternative *free* systems available, svn is probably the most mature. There are many organisations commercial and otherwise using it for internal and external projects. As far as svk, I've been told by a claimed subversion project member that svk is not mature yet, and they don't consider it a viable alternative to svn. So, my vote would be for svn. I feel like any serious issues that would arise could easily be resolved, and that it's licensing is compatible with the goals of the OpenSolaris community and other communities that SUN centers around the CDDL license.

Posted by Shawn Walker on July 26, 2005 at 12:24 PM PDT #

We migrated our code base from CVS last year. Our code base is likely between 3 and 5 million lines of code (not counting branches and forks) with just over 70,000 revisions. After the migration, we have found that load on our servers to be tramatically less, as well as disk requirements were slightly smaller (from 13Gb to 11Gb). SVN integrates nicely with our issue tracking software and our corportate intranet. We have also developed a home grown web based svn browser (written in Java, soon to be OSS) that also exposes the repository via RSS. Over all, we have had no regrets about the transition and are very happy with SVN. /colin

Posted by Colin Bendell on July 26, 2005 at 12:58 PM PDT #

We just (2 weeks ago) started trying subversion on a new project as a pilot, so I can't really talk about performance yet. One thing I love about svn is that you can get a little tool called trac for it. Trac integrates svn with issue tracking, wiki and RSS feeds plus simple release planning capabilities. And it hits the sweet spot for me: just enough, not fuss around it. That even includes the documentation, which had everything I needed without boring me.
I highly recommend having a look at it, it was pretty much the only piece of software that managed to impress me in the last years. Ok -- maybe I should count Google Maps, too :-) Of course I can't tell how far trac scales either, but it is worth looking at even if you know you won't use it.
Cheers, Peter.

Posted by Peter Becker on July 26, 2005 at 02:04 PM PDT #

James, thanks for talking about that publicly :) I'm fairly new to Sun and I don't want to complain but TeamWare is somehow... complicated sometimes :)

Posted by Romain Guy on July 26, 2005 at 02:37 PM PDT #

Perforce is simply the best tool I've ever used. Though I don't know if it's checkout model will scale. It has an OSS friendly policy as well. In a recent project I've had to use Subversion and CVS quite extensively.. after Perforce it is a very painful experience.

Posted by am on July 26, 2005 at 03:02 PM PDT #

Whilst Subversion seems to be getting some use, it seems rather heavyweight, and I do not think it's a good idea to have to use some shim layer on top to get such basic features as distributed repositories.

BitKeeper is obviously a non-option now. git, its replacement in Linux, has quickly become an in-joke, and not in a good way.

Fixing up Teamware seems silly to me: it's essentially a dead project (though one with heavy use in a certain company).

The Xen project has started using Mercurial (hg), and I've been using it myself for some time. It's super-simple and works very well in my experience. And it's <em>fast</em>

Mercurial

Don't be surprised if Linus drops the git train-wreck in favour of hg soon...

Posted by 192.18.1.4 on July 27, 2005 at 07:40 AM PDT #

FWIW, here's my experience. Not that I'm against Subversion or anything, but it's certainly annoying sometimes.

Posted by Kohsuke Kawaguchi on July 27, 2005 at 08:35 AM PDT #

http://svn.genunix.org/repos/ (or svn://svn.genunix.org) - should I say more ?

Posted by Cyril Plisko on July 27, 2005 at 11:47 PM PDT #

Another vote for Mercurial here. I work with Matt, who wrote it. It's in Python, so I imagine it could be ported to run on Jython. At least, I'm going to try to do that for the Kaffe project. :-)

Posted by Jim Pick on July 28, 2005 at 11:29 AM PDT #

Post a Comment:

Comments are closed for this entry.