Mercurial TOI slides

Monday Dec 10, 2007

The slides for Mercurial TOI can be found at http://blogs.sun.com/simford/resource/Mercurial-simford.pdf

Mercurial Best Practices

Friday Aug 17, 2007

These are some good practices for those who are not familiar with Mercurial Source Code Management (SCM) tool. Assuming that you are working on many Mercurial based repositories on OpenSolaris.org community, and your local workspaces are inside SWAN.

1. understand the differences between Mercurial and Subversion
   check my other blog Differences between Subversion and Mercurial for more details
   
   
2. register an account in OpenSolaris.org community,
   and upload your ssh pub key in your profile page
   
   
3. subscribe to the mailing list that repository will send commit notifications to,
   otherwise the mailing list will reject the notification email for your putbacks
   
   
4. create proper $HOME/.hgrc file
   [ui]
   username = [your account on OpenSolaris community]
   
   [extensions]
   hgext.fetch =
   
   
5. use ssh proxy if you cannot connect to hg.opensolaris.org server directly
   vi $HOME/.ssh/config
   

   Host *.opensolaris.org
            ProxyCommand /usr/lib/ssh/ssh-socks5-proxy-connect -h [socks proxy address] %h %p
   

   you can get available sock5 proxy list from internal web page
   
   
6. clone from SWAN parent to save time
   clone from hg.opensolaris.org directly may be very slow some times. if true,
   you can clone from SWAN parent, then re-direct the parent to hg.opensolaris.org one.
   the parent info is kept in 'default' entry of [WS]/.hg/hgrc file
   
   
7. update local workspace before making local modifications
   this will avoid potential multiple heads mess
   note: 'hg fetch' will do 'hg pull && hg up && hg merge && hg commit' for you
             'hg fetch' cmd only available when you enable hgext.fetch extension
             in $HOME/.hgrc (tips #4)
   
   
8. if multiple heads encountered in local workspace
   please use 'hg merge && hg commit' to eliminate it locally.
   DO NOT use 'hg push -f' to force the putback,
   otherwise it will propagate the heads to the parent workspace!
   
   you can also use 'hg rollback' to undo last commit
   

   scenario:
   you make local modifications, then 'hg commit'.
   after that, you also 'hg pull && hg up', then two heads encountered.
   
   solution:
   --  'hg rollback' to undo last commit
   --  'hg pull && hg up' to update workspace
   --  'hg ci' again to commit local modifications
   --  'hg push' to put back your commits
   
   reference:
   check 'hg help rollback' for more details
   

Converting from Subversion to Mercurial

Thursday Jun 14, 2007

As Mercurial is more and more in vogue inside Sun, many workspaces need to be converted from other Source Code Management (SCM) into Mercurial. hgsvn is a good tool to convert SVN based workspaces into Mercurial (HG) based.

1. Download latest version of hgsvn (current version is 0.1.3):
   http://cheeseshop.python.org/pypi/hgsvn
   
   
2. Untar hgsvn, then install (setup.py will download setuptools during installation):
   $ cd hgsvn-0.1.3
   $ su -
   {passwd}
   # export http_proxy="http://webcache.japan.sun.com:8080/"
   # python setup.py install
   
   
3. Create and pull svn workspace:
   $ mkdir g11n-ws
   $ hgimportsvn http://agc163.prc.sun.com/svn/cws/g11n-ws/trunk g11n-ws
   $ cd g11n-ws
   $ hgpullsvn
   
   
4. voila

 

References:

• http://cheeseshop.python.org/pypi/hgsvn
  
• Converting from Subversion to Mercurial
  

Differences between Subversion and Mercurial

Thursday May 17, 2007

I'm using both Subversion (SVN) and Mercurial (HG) for source codes management (SCM). There are some differences between them for daily using. I document them in this article and update it when I find new differences.

1. commit : push
   'svn ci' will commit the server directly
   'hg ci' will commit to local history
   'hg push' will push the changes to server
   
   
2.  locally : globally
   all svn cmds will check current and subdirs only, eg commit, status
   but hg cmds will check globally, including the whole workspace
   
   if you want to check current and subdirs only, append '.' to hg cmds
   eg, hg status .
   
   
   
3. partially checkout : fully checkout
   svn can checkout a portion of the workspace (a module or subdir)
   hg can't, it will clone the whole workspace
   
   
4. empty dir
   mercurial will ignore empty dirs when invoking 'hg add', more details
   if you want to add those dirs into mercurial, add a hidden file under it
   $ touch subdir/.hidden
   $ hg add subdir
   
   
5. symbolic link
   mercurial 0.9.3 doesn't support symbolic link file type:
   
   [qd139970@agc141 rtc]$ hg --version
   Mercurial Distributed SCM (version 0.9.3)
   
   Copyright (C) 2005, 2006 Matt Mackall <mpm@selenic.com>
   This is free software; see the source for copying conditions. There is NO
   warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
   
   [qd139970@agc141 rtc]$ hg add RTC_Template_O5U1
   RTC_Template_O5U1: unsupported file type (type is symbolic link)
   [qd139970@agc141 rtc]$
   
   PS: link1 and link2 mentioned that this feature is supported in trunk.
   PS2: Mercurial 0.9.4 supports symbolic links now(Jun 25, 2007).
   
   
6. account name
   Mercurial will look at $HOME/.hgrc to get the account info when 'hg commit',
   if there's not 'username' entry in $HOME/.hgrc, Mercurial will use
   $USER@$HOSTNAME as the username in log message.
   if this is not what you want, please specify your username in $HOME/.hgrc
       [ui]
       username = [your prefer username]
    
   
7. multiple heads
   it's quite easy to create multiple heads in Mercurial workspace.
   you commit new changeset locally, and 'hg pull' from parent workspace
   to get new changesets from team members, then two heads created.
   'hg heads' will show the heads information.
   if multiple heads exist, you need to 'hg up && hg merge && hg commit'
   before you can 'hg push' your changeset to parent workspace.
   
   one possible way to avoid the mess is running 'hg pull && hg up' before
   you commit any modifications.
   
   if you enable hgext.fetch extension, 'hg fetch' will do this for you:
   'hg pull && hg up && hg merge && hg commit'
   
   add following two lines in $HOME/.hgrc can enable hgext.fetch extension
       [extensions]
       hgext.fetch =
   
   

Enable push notification on Mercurial repository

Saturday Apr 28, 2007

OS: Solaris Nevada (build 59, sparc)
Mercurial: 0.9.3

Due to the lack of 'diffstat' tool on Solaris, the notification extension (hgext.notify) doesn't work well on Solaris. After several trying, I finally made it work.

Here is the setup steps:

 

1. vi .hg/hgrc
   
   [web]
   contact = {your name}
   description = Docking Workspace for Solaris WOS, SunSolve (Patch) and JES
   style = gitweb
   allow_archive = gz zip bz2
   
   [extensions]
   hgext.patchbomb =
   hgext.notify =
   
   [hooks]
   # send one email per change
   #incoming.notify = python:hgext.notify.hook
   # send one email per group of changes
   changegroup.notify = python:hgext.notify.hook
   
   [email]
   from = svnroot@agc163.prc.sun.com
   method = /usr/sbin/sendmail
   
   [web]
   baseurl = http://agc163.prc.sun.com/hg/
   
   [notify]
   # multiple sources can be specified as a whitespace separated list
   sources = serve push pull bundle
   # set this to False when you're ready for mail to start sending
   test = false
   config =
   # repos live in /workspace/scm/hg/hgroot on server, so strip 5 "/" chars
   strip = 5
   
   # you can override the changeset template here, if you want.
   # If it doesn't start with \n it may confuse the email parser.
   # here's an example that makes the changeset template look more like hg log:
   template = \ndetails:   {baseurl}{webroot}/rev/{node|short}\nchangeset: {rev}:{node|short}\nuser:      {author}\ndate:      {date|date}\ndescription:\n{desc}\n
   
   [reposubs]
   # key is glob pattern, value is comma-separated list of subscriber emails
   * = {email address}
   
   
2. When tried to push back, it failed
   
   [gbuild@agc141 s11]$ hg push
   Enter passphrase for key '/export/home/gbuild/.ssh/id_dsa':
   pushing to ssh://hgroot@agc163.prc.sun.com/docking
   searching for changes
   remote: adding changesets
   remote: adding manifests
   remote: adding file changes
   remote: added 1 changesets with 76 changes to 76 files
   remote: /bin/sh: diffstat: not found
   remote: error: changegroup.notify hook raised an exception: [Errno 32] Broken pipe
   [gbuild@agc141 s11]$
   
   
3. After searching on Internet, I found the patch for Solaris
   
   http://www.selenic.com/mercurial/bts/file270/diffstat.patch
   
   

   diff -r 730cbd26552c -r 79639a44dd23 mercurial/patch.py
   --- a/mercurial/patch.py	Wed Apr 04 02:28:29 2007 -0300
   +++ b/mercurial/patch.py	Wed Apr 04 03:09:26 2007 -0300
   @@ -635,6 +635,8 @@ def export(repo, revs, template='hg-%h.p
            single(rev, seqno+1, fp)
    
    def diffstat(patchlines):
   +    if not util.find_in_path('diffstat', os.environ.get('PATH', '')):
   +        return
        fd, name = tempfile.mkstemp(prefix="hg-patchbomb-", suffix=".txt")
        try:
            p = popen2.Popen3('diffstat -p1 -w79 2>/dev/null > ' + name)

4. But it still failed when push back
   
   [gbuild@agc141 s10u3]$ hg --traceback push
   Enter passphrase for key '/export/home/gbuild/.ssh/id_dsa':
   pushing to ssh://hgroot@agc163.prc.sun.com/docking
   searching for changes
   remote: adding changesets
   remote: adding manifests
   remote: adding file changes
   remote: added 1 changesets with 8 changes to 8 files
   remote: # /usr/lib/python2.4/vendor-packages/mercurial/patch.pyc has bad mtime 1177746898 vs 1172046709
   remote: error: changegroup.notify hook raised an exception: cannot concatenate 'str' and 'NoneType' objects
   [gbuild@agc141 s10u3]$
   
   
5. After rename /usr/lib/python2.4/vendor-packages/mercurial/patch.pyc, the error still existed.
6. I finally found following patch after several trying:
   
   --- /usr/lib/python2.4/vendor-packages/mercurial/patch.py.old   Sat Apr 28 19:02:05 2007
   +++ /usr/lib/python2.4/vendor-packages/mercurial/patch.py       Sat Apr 28 16:29:14 2007
   @@ -659,6 +659,8 @@
            single(repo.lookup(rev), seqno+1, fp)
    
    def diffstat(patchlines):
   +    if not util.find_in_path('diffstat', os.environ.get('PATH', '')):
   +        return ""
        fd, name = tempfile.mkstemp(prefix="hg-patchbomb-", suffix=".txt")
        try:
            p = popen2.Popen3('diffstat -p1 -w79 2>/dev/null > ' + name)
   
   

 

access Mercurial over ssh tunnel

Monday Apr 09, 2007

Accessing Mercurial through http/apache is handy, but it requests the workspace is owned by the same user with httpd (webservd:webservd on solaris). So accessing Mercurial over ssh tunnel is a better choice (this is also the way http://www.opensolaris.org/ chose).

Client side:

1. $ ssh-keygen -b 1024 -t dsa
[ create id_dsa.pub for server side ]

2. $ cat <<EOF > $HOME/.hgrc
   [ui]
   username = User Name <user.name@example.com>
   

 
Server side:

1. $ cat <<EOF > $HOME/.ssh/authorized_keys

   command="cd /workspace/scm/hg/hgroot; /usr/demo/mercurial/hg-ssh g11n",\ no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-[type] [key] 

   EOF

 

Client side:

1. $ hg clone ssh://agc163.prc.sun.com/g11n
2. [play with it]
   
3. $ hg ci
4. $ hg push

Setup Mercurial on Solaris Nevada build 59

Monday Apr 09, 2007

Solaris Nevada build 59 shipped with Mercurial 0.9.3:

 $ hg --version
Mercurial Distributed SCM (version 0.9.3)

Setup Mercurial server on it is pretty easy.

1. $ mkdir -p /workspace/scm/hg/hgroot/g11n
2. $ cd /workspace/scm/hg/hgroot/g11n
3. $ hg init
4. $ cat <<EOF > .hg/hgrc
   [web]
   contact = Simford Dong
   description = G11N Internal CWS
   
   style = gitweb
   allow_archive = gz zip bz2
   allow_push = *
   
   
5. $ cp /usr/demo/mercurial/hgwebdir.cgi /workspace/scm/hg/hgroot
6. $ chmod a+x /workspace/scm/hg/hgroot/hgwebdir.cgi
7. $ cat <<EOF > /workspace/scm/hg/hgroot/hgweb.config
   [paths]
   g11n = g11n
   
8. Append following lines into  /etc/apache2/httpd.conf
   ScriptAliasMatch  ^/hg(.*)  /workspace/scm/hg/hgroot/hgwebdir.cgi$1
   <Directory "/workspace/scm/hg/hgroot">
       Order allow,deny
       Allow from all
       AllowOverride All
       Options ExecCGI
       AddHandler cgi-script .cgi
   
       <Limit POST>
       AuthType Basic
       AuthName "Mercurial Repository"
       AuthUserFile /workspace/scm/hg/auth/hgpasswd
       Require valid-user
       </Limit>
   </Directory>
   
9. # svcadm refresh svc:/network/http:apache2
10. # chown -R webservd:webservd /workspace/scm/hg/hgroot/g11n
11. 
    
12. Done.

NOTE: following part will enable push back over http, if you don't like this feature, you can skip them.
             - 'allow_push = *' part of step 4
             - '<Limit POST> ... </Limit>' part of step 8
             - step 10
 

access Mercurial workspace through ssh tunnel

Monday Nov 27, 2006

It's quite easy to setup ssh tunnel for Mercurial workspace:

$ mkdir -p .ssh

$ cat <<EOF > .ssh/authorized_keys

command="/usr/demo/mercurial/hg-ssh /export/scm/hgroot/repos/g11n-cws",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-[type] [key] 

EOF

 

Please be sure to put above contents into one line (though very long).

More details can be found at selenic.com

Tried to enable hg push over http, but failed

Wednesday Oct 18, 2006

For a centralized workspace with multiple committers, it's important to make sure the write access to the workspace works fine. At the beginning we try to use push back over http, described in selenic.com.
- create .htaccess
- create htpasswd
- put 'allow_push = *' in .hg/hgrc
- put 'push_ssl = false' in .hg/hgrc
...

But it just don't work.
:(

I found the same problem as Georg encountered:
http://marc.theaimsgroup.com/?l=mercurial&m=115438570707603&w=2

Any one who has the workaround for it please feel free to let me know.
:)

Setup centralized Mercurial (hg) workspace with http support

Wednesday Oct 18, 2006

The server hosting the centralized workspace is Solaris Nevada snv_45 SPARC.

• download hg tool from OS.o SCM download center, and install it.

• enable apache2 server in Nevada:

-bash-3.00# cd /etc/apache2
-bash-3.00# cp httpd.conf-example httpd.conf

Add following settings in /etc/apache2/httpd.conf:

Alias /hg /var/apache2/cgi-hg
<Directory "/var/apache2/cgi-hg">
    DirectoryIndex index.cgi
    AddHandler cgi-script .cgi
    Options ExecCGI
    Order allow,deny
    Allow from all
</Directory>

-bash-3.00# svcs -a |grep apache2
online          9:41:50 svc:/network/http:apache2
-bash-3.00# svcadm enable svc:/network/http:apache2
-bash-3.00#

• create an account hg, with home at /export/home/hg, as user hg

[hg@agc108 ~]$ mkdir hgroot
[hg@agc108 ~]$ cd hgroot
[hg@agc108 ~]$ hg init g11n
[hg@agc108 ~]$ cd g11n
[hg@agc108 ~]$ chmod g+w .hg .hg/*
[hg@agc108 ~]$ chmod g+s .hg .hg/data
[hg@agc108 ~]$ cat << EOF > .hg/hgrc
[web]
contact = Simford Dong
description = G11N Common Workspace (g11n-cws)
style = gitweb
allow_archive = gz zip bz2

[hooks]
commmit = /net/agc108.prc.sun.com/export/home/hg/bin/commithook.sh
incoming = /net/agc108.prc.sun.com/export/home/hg/bin/commithook.sh
EOF
[hg@agc108 ~]$ cat <<EOF > /net/agc108.prc.sun.com/export/home/hg/bin/commithook.sh
#!/bin/sh

SUBJECT=`hg log -r $NODE | grep "^summary:" | cut -b 14-`
hg log -vpr $NODE | mail -s "commit: $SUBJECT" [notification email]
EOF
[hg@agc108 ~]$ chmod a+x /net/agc108.prc.sun.com/export/home/hg/bin/commithook.sh
[hg@agc108 ~]$

• as user root

-bash-3.00# cd /var/apache2/
-bash-3.00# mkdir cgi-hg
-bash-3.00# cd cgi-hg
-bash-3.00# cp [hg install path]/hgwebdir.cgi index.cgi
-bash-3.00# chmod a+x index.cgi
-bash-3.00# cat <<EOF >hgweb.config
[paths]
/g11n = /export/home/hg/hgroot/g11n
EOF
-bash-3.00# mkdir g11n
-bash-3.00# cd g11n
-bash-3.00# cp [hg install path]/hgweb.cgi index.cgi
-bash-3.00# chmod a+x index.cgi

change 'make_web_app' function in index.cgi as below

def make_web_app():
    return hgweb("/export/home/hg/hgroot/g11n", "g11n cws")

• restart apache2

-bash-3.00# /usr/apache2/bin/apachectl restart

• that's it!

you should be able to browse the workspace in firefox: http://agc108.prc.sun.com/hg/g11n.
you can also clone it and push back freely:
[simford@agc130 tmp]$ hg clone /net/agc108.prc.sun.com/export/home/hg/hgroot/g11n
...
play with it
...
[simford@agc130 tmp]$ umask 0002
[simford@agc130 tmp]$ hg push

• Last but not the least, 'umask 0002' is very important, especially in NFS environment.
  

All users should be able to push back to the centranlized workspace, so make sure the users in the same group have the +w rights to all files in the workspace.