Kelly O'Hair's Weblog (blogs.sun.com)
Friday Nov 16, 2007
Transitioning from TeamWare to Mercurial
Just thought I would pull together some basic guidelines for anyone transitioning from TeamWare workspaces to Mercurial repositories.
I'm assuming that multiple branches will not be maintained in the Mercurial repositories in this information. Mercurial will allow you to maintain multiple branches of development in a single repository. It's my opinion that multiple branches of development can be done safer and more reliably by just have a separate clone for the separate branches. Anyone experiencing this feature in SCCS files where separate revision trees can be maintained on a file will probably agree.
TeamWare Basics
Skip this section if you are a regular TeamWare user.
A TeamWare workspace consists of a directory of files under SCCS control, each file is managed individually. Throughout the TeamWare workspace are directories called SCCS contain s.filename files which contain the original file as it was first entered into the SCCS directory, plus deltas to convert that original file to the various increasing numeric revisions of the file. A read-only file is kept in the parent directory of the SCCS directory, edits to the file requires you to use the 'sccs edit filename' command. A 'sccs delget filename' command is used to define a new revision of the file. Each revision of a file can contain a comment. SCCS manages source files and revisions to source files. TeamWare manages batches of SCCS files. TeamWare features and SCCS features get blurred sometimes, but SCCS can and is often used independent from TeamWare.
TeamWare allows for any number of workspaces with a child/parent relationship and also has a very nice code merging tool called filemerge. TeamWare allowed you to have partial workspaces. The top of the workspace contains a Codemgr_wsdata directory that holds various TeamWare book-keeping files. It's drawbacks besides not being open source are around performance and the lack of features like changesets and revision markings (tags). Over the years the short-comings have been somewhat corrected with various scripts and tools written by various teams.
Mercurial For TeamWare Users
Mercurial (like TeamWare) allows you to have any number of repositories (assume repository==workspace) and allows you to access repositories via NFS paths or with ssh:// or even http:// paths.
In many ways, at a high functional level, your Mercurial experience will be similar to the experiences you have had with TeamWare, but the details are vastly different, especially if you have become dependent on the specific format of a TeamWare workspace or the contents of the SCCS files.
At the very top of the Mercurial repository is a hidden directory called .hg which holds the Mercurial book-keeping files plus a secure set of all "commited" sources and changes to those sources.
Unlike TeamWare, where the visible source files were read-only until you explicitly used 'sccs edit' to explicitly edit them, the Mercurial "working set" sources are all read-write, and you are free to edit these files at any time. So by default you will have a working set of read-write sources and the more permanent committed files that are saved in your .hg directory.
With Mercurial, all changes to a repository are done with via "changesets", which are originally created with an 'hg commit' somewhere along the line. An ideal changeset would be all the file changes/renames/deletes/adds for one particular bug, but a changeset can be small or very large. New files, deleted files, and renamed files must all be done via a changeset. You use 'hg commit' to commit file changes into a "changeset" in your own repository, it doesn't go anywhere unless someone pulls it from your repository, or you push the changeset somewhere. The 'hg pull' is like the TeamWare bringover command, and 'hg push' is like the TeamWare putback command, well sort of. Both the 'hg push' and 'hg pull' push or pull "changesets" or committed changes to and from the .hg directories of a repositories. So your working set files are NOT automatically updated when the files in the .hg directory changes (where changesets are kept), you must explicitly run 'hg update' to update your working set files. And it's important to note that with Mercurial you do not "pull or push files" but the changesets or changes to the entire repository. This is very different from TeamWare which manages SCCS files, where you could bringover or putback individual files.
The changeset concept is like a repository wide SCCS revision number, one changeset id defines the state of the entire repository. A changeset that has no children changesets is called a "head", and there should only be one head, which is also called the tip. But when you do a pull, you often end up with multiple "head" changesets, and the gola is to perform an 'hg merge' and 'hg commit' a new "merge" changeset that will become the single "head" or "tip". Regardless of any specific file changes that might be conflicting, a merge changeset will always be needed to get back to one "head".
Roughly Equivalent Command Mappings
NOTE: Optimally, the use of 'hg commit' should be done after all the file adds, deletes, renames, and edits are done. An ideal changeset is one that contains all the changes for a particular feature or bug fix.
| Action | TeamWare | Mercurial |
|---|---|---|
| Create new workspace/repository | workspace create | hg init |
| Create a child workspace/repository | bringover -p parent -w child . | hg clone parent child |
| Add a file | sccs create filename | hg add filename && hg commit |
| Delete a file | workspace filerm filename | hg remove filename && hg commit |
| Rename a file | workspace filemv filename1 filename2 | hg rename filename1 filename2 && hg commit |
| Change a file | sccs edit filename && vi filename && sccs delget filename | vi filename && hg commit |
| Rename a workspace/repository | workspace move oldpath newpath | mv oldpath newpath |
| Delete a workspace/repository | workspace delete -f path | rm -f -r path |
| Verify workspace/repository | workspace check | hg verify |
| Pull changes from another workspace/repository | bringover -p parentpath . && resolve | hg pull parentpath && hg update && hg merge && hg commit |
| Push changes to another workspace/repository | putback -p parentpath . | hg push parentpath |
| Get the name of the workspace/repository | workspace name | hg root |
| Get the name of the default parent workspace/repository | workspace parent | hg paths |
| Resolve merge conflicts | resolve | hg merge |
| Freshen the working set files | sccs get filename | hg update filename |
| Check for incoming changes | bringover -n -p parentpath . | hg incoming parentpath |
| Check on outgoing changes | putback -n -p parentpath . | hg outgoing parentpath |
| Details on changes to a single file | sccs prs filename || sccs prt filename | hg log -v filename && hg annotate filename |
| Viewing file changes | sccs diff filename | hg diff filename |
| Undo file changes | sccs unedit filename | hg revert filename |
| Listing uncommited edited files | workspace find -c -OR- sccs tell | hg status |
| Listing all managed files | workspace updatenames && cat Codemgr_wsdata/nametable | sort | cut -d' ' -f1 | hg locate |
| List a source file annotated with the revisions | Various custom scripts | hg annotate filename |
Using Mercurial Example
Here is a simple example of pulling a repository and making a changeset.
In this example /export2/build_integration/repos/control is a path to a amsll Mercurial repository.
-
# # Step 1: Check hg version
# Make sure you have access to the right Mercurial.
<1> rm -f -r ${HOME}/MercurialExercises/Exercise1
<2> mkdir -p ${HOME}/MercurialExercises/Exercise1/temp
<3> hg version
Mercurial Distributed SCM (version 0.9.3)
Copyright (C) 2005, 2006 Matt Mackall
Using TeamWare and webrev to import a changeset
The tool webrev creates a set of web pages than can be used to browse code changes, but more recent versions create patch files (looks like "diff -r -u") that can be fed into gpatch (GNU patch) or a similar tool to apply the changes.
In this example /export2/build_integration/ws7/control is a path to a TeamWare integration workspace and /export2/build_integration/repos/control is a path to an equivalent Mercurial repository.
# # Step 1: Bringover a test control workspace
# Creates your own private workspace (may take a few minutes)
<1> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_ws
<2> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_ws
<3> cd ${HOME}/MercurialExercises/Exercise2/your_ws
<4> bringover -q -p /export2/build_integration/ws7/control -w control .
Parent workspace: /export2/build_integration/ws7/control
Child workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
Examined files: 55
Bringing over contents changes: 55
Examined files: 55
Contents Summary:
55 create
# # Step 2: Private child workspace of the above workspace to hold changes.
<5> cd ${HOME}/MercurialExercises/Exercise2/your_ws
<6> bringover -q -p control -w control-work .
Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
Child workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
Examined files: 55
Bringing over contents changes: 55
Examined files: 55
Contents Summary:
55 create
# # Step 3: Make changes in the child workspace
<7> cd ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/make
<8> sccs edit Makefile
1.313
new delta 1.314
444 lines
<9> echo '#harmless' >> Makefile
<10> sccs delget -y'9999999: Fixed world peace' Makefile
No id keywords (cm7)
1.314
1 inserted
0 deleted
444 unchanged
1.314
No id keywords (cm7)
445 lines
<11> cd ..
<12> putback -n .
Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
Child workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
Examined files: 55
Would put back contents changes: 1
update: make/Makefile
Examined files: 55
Contents Summary:
1 update
54 no action (unchanged)
No changes were put back
<13> webrev -l .
SCM detected: teamware
File list from: 'putback -n .' ... Done.
Workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
Compare against: ${HOME}/MercurialExercises/Exercise2/your_ws/control
Output to: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev
Output Files:
make/Makefile
patch cdiffs udiffs sdiffs frames old new
Generating PDF: Skipped: no output available
index.html: Done.
<14> cat webrev/control-work.patch
--- old/make/Makefile Thu Oct 4 19:09:19 2007
+++ new/make/Makefile Thu Oct 4 19:09:19 2007
@@ -442,3 +442,4 @@
.PHONY: all build what clobber insane freshen \
fastdebug_build debug_build product_build setup
+#harmless
# # Step 4: Clone a test control repository
# Creates your own private repository to play with (may take a few minutes)
<15> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_repos
<16> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_repos
<17> cd ${HOME}/MercurialExercises/Exercise2/your_repos
<18> hg clone /export2/build_integration/repos/control control
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 24 changes to 24 files
24 files updated, 0 files merged, 0 files removed, 0 files unresolved
<19> hg clone control control-work
24 files updated, 0 files merged, 0 files removed, 0 files unresolved
# # Step 5: Import the patch into the repository
<20> cd ${HOME}/MercurialExercises/Exercise2/your_repos/control-work
<21> gpatch -u -p1 < ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev/control-work.patch
patching file make/Makefile
<22> hg status
M make/Makefile
<23> hg diff
diff -r 48e79d6618ee make/Makefile
--- a/make/Makefile Sun Sep 30 17:55:14 2007 -0700
+++ b/make/Makefile Thu Oct 04 19:09:32 2007 -0700
@@ -442,3 +442,4 @@ sponsors-bringover: sponsors-freshen
.PHONY: all build what clobber insane freshen \
fastdebug_build debug_build product_build setup
+#harmless
<24> hg commit -m "9999999: Fixed world peace"
<25> hg outgoing
searching for changes
changeset: 1:23e0962ced6d
tag: tip
user: ${USER}
date: Thu Oct 04 19:09:36 2007 -0700
summary: 9999999: Fixed world peace
<26> hg push
pushing to ${HOME}/MercurialExercises/Exercise2/your_repos/control
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
Beginner Gotchas for TeamWare Users
| Gotcha |
Why? |
|---|---|
| Not setting up your ~/.hgrc file. | The name you define in ~/.hgrc with "[ui]" and "username=" is the name that will be permanently recorded in the changesets you create with 'hg commit'. I don't recommend adding your email address in username, but that's up to you, just keep in mind it will be public information when your changesets reach a public repository. TeamWare/SCCS used your system username, but very few TeamWare workspaces were ever made public. |
| Forgot the 'hg update' | After an 'hg pull' (aka bringover), don't forget the 'hg update', or use 'hg pull -u'. The default pull and push just updates the changesets and doesn't update your read-write working set of files. You need to be careful about updating the working set files on shared repositories they could get updated while others are viewing them. |
| Forgot to merge | After an 'hg pull', you need to 'hg update', and if you have changesets that you have not committed you will also need to 'hg merge' and 'hg commit'. If you forget you will end up with multiple heads and a more difficult time merging later. |
| Forgot to commit after a merge (multiple heads) | After 'hg merge' you need to 'hg commit'. The merge just prepares you for the 'hg commit' of a merge changeset. If you forget you will end up with multiple heads and a more difficult time merging later. |
| Making accidental edits | Mercurial working set files are always read-write and ready to edit, no 'sccs edit' action is necessary. Use 'hg status' to monitor what files you have changed. |
| Using the wrong relative path | File paths supplied to 'hg' commands are relative to the current directory, the TeamWare bringover and putback commands want paths relative to the root of the workspace, regardless of the current directory. |
| Not defining the file .hgignore for 'hg status' | The 'hg status' command tells you what outstanding changes you have in your working set, by default it looks in '.' or the entire directory, but if there are files created during a build, you want 'hg status' to ignore those files. Make sure you define the .hgignore file so that 'hg status' will only find files in the directories you want managed by the repository. TeamWare never really helped with the problem of forgetting to 'sccs create' your files, 'hg status' solves this common problem. |
| Using NFS/UFS for team integration areas | TeamWare for the most part was designed around sharing data via NFS or UFS file systems. Mercurial can work the same way, but when using it for team integration areas we recommend the use of the ssh:// parent path mechanisms described in the Mercurial Book . Unless everyone in the team or group is in the same Unix group, have the same default group, and all use 'umask 2', using NFS/UFS will be problematic. Mercurial obeys the strict Unix rules of file creation and permissions, and over time TeamWare has adjusted itself (perhaps improperly) to avoid the file permission issues you can see with Mercurial. |
| Too quick on the 'hg commit' | Once a changeset is created (the 'hg commit'), and pushed, it's pretty permanent. Make sure that before the 'hg commit' happens that the changes are correct, reviewed, the right ones, and complete, otherwise you'll be creating yet another one to correct your mistakes. |
| Doing a push with outstanding working set changes | The 'hg push' will not detect any outstanding changes to your working set, it just pushes the existing changesets. ALWAYS use 'hg status' before an 'hg push' to make sure you have created all your changesets with 'hg commit', unless of course you have changes you don't want to push. |
| Committing a sensitive file | Accidental additions of sensitive source files can be a big problem. Completely removing a sensitive file that has been accidently added to a repository can be a real problem. be very careful what files you add to a repository! Adding non-open source files to an 'open source' repository will inflict major pain on many people. |
| Doing anything to the .hg files | Don't mess with the .hg data files, if you do you are INSANE, leave that to the Mercurial professionals. If you suspect they have been corrupted, use 'hg verify' to check. Backups are always important, so make sure you keep a relatively recent backup repository. If you can't 'hg rollback', save the repository somewhere, clone a fresh copy from your parent, remove the working set files completely from the clone, and copy in the working set from your corrupted repository (but not the .hg files). Now you can use the standard 'hg status' and 'hg diff' to see what file changes you may have lost and adjust. |
| Using SCCS keywords | Mercurial by default does not support anything like SCCS keywords in files. You should remove these or find another solution. |
| Looking for putback comments or history files | Changeset comments represent BOTH the SCCS comments and the effective TeamWare putback comment. |
| Using problematic filenames | Watch out for directory and filenames that only differ in case (e.g. test and Test), at least on the Mac and Windows these can be troublesome. Long pathnames (>255 characters) can also be a problem. |
Converting a TeamWare Workspace to a Mercurial Repository
Converting a TeamWare workspace to a Mercurial repository (without history) is pretty trivial:
bringover -p your_workspace -w /tmp/repo .
cd /tmp/repo
workspace parent -u
rm -f -r Codemgr_wsdata
rm -f -r deleted_files
foreach i ( `find . -name SCCS` )
( cd $i/.. && sccs edit SCCS )
rm -f -r $i
done
hg init
hg add
A simple source tree can be turned into a Mercurial repository with just hg init; hg add. Turning a TeamWare workspace into a plain source tree is relatively simple too, I just create a separate workspace, purge a few files, make sure all the sources are in 'edit' mode, and remove the SCCS directories.
Performance Comparisons and Data
Nothing but good news in this area, for both time and space.
Many of the past tricks used to speed up TeamWare bringovers and putbacks, especially over slow connections should not be necessary with Mercurial, it is very fast. The initial 'hg clone' of a repository should be considerably faster, but the most important actions of 'hg pull' or 'hg push' will be so much faster you may question if the action actually happened. Unlike TeamWare, only the changesets are transported, and many fewer files are accessed and in a more efficient manner.
The size of the repositories should also be smaller (at least 50% smaller) than the equivalent TeamWare workspace, this isn't surprising due to the lack of compression and age of SCCS file.
Posted at 11:04AM Nov 16, 2007 by kto in General |
Subscribe!
