Sometimes when you do an incremental build, you run into some fluff that kills your build:
==== cpio archives build errors (DEBUG) ==== Failed to create generic kernel archive: 200550 blocks cpiotranslate: kernel/misc/amd64/sysinit: no packaging info cpiotranslate: kernel/misc/sysinit: no packaging info
And everyone is supposed to know how to handle these:
[th199096@aus-build-x86 mms]> ls -al proto/root_i386/kernel/misc/amd64/sysinit -rwxr-xr-x 1 th199096 staff 4200 Sep 25 21:22 proto/root_i386/kernel/misc/amd64/sysinit [th199096@aus-build-x86 mms]> rm proto/root_i386/kernel/misc/amd64/sysinit proto/root_i386/kernel/misc/sysinit [th199096@aus-build-x86 mms]> `which nightly` -in nightly.env
Hey, what do you know, Ken Erickson did have a Flag Day for those who maintain private copies of bfu; Heads up for everyone else], but it still does not mention cleaning up the turd on your own.
After all, everyone knows how to deal with these turds.
So I think I have found a bug in our implementation of Mercurial. I'm all excited and I'm about to send the following email off to the ON gatekeepers:
Take a zfs clone of a workspace which has ssh://aus1500-home//pool/ws/th199096/spe-gate as a parent.
[th199096@jhereg ~]> zfs clone pool/builds/th199096/spe-gate@postjimw pool/builds/th199096/spe-fix [th199096@jhereg ~]> ws /builds/th199096/spe-fix Workspace : /builds/th199096/spe-fix Workspace Parent : ssh://aus1500-home//pool/ws/th199096/spe-gate Proto area ($ROOT) : /builds/th199096/spe-fix/proto/root_i386 Root of source ($SRC) : /builds/th199096/spe-fix/usr/src Root of test source ($TSRC) : /builds/th199096/spe-fix/usr/ontest Current directory ($PWD) : /builds/th199096/spe-fix
Reparent to ssh://aus1500-home//pool/ws/nfs41-clone
[th199096@jhereg spe-fix]> hg reparent ssh://aus1500-home//pool/ws/nfs41-clone
[th199096@jhereg spe-fix]> ws usr/closed/
Workspace : /builds/th199096/spe-fix/usr/closed
Workspace Parent : ssh://aus1500-home//pool/ws/th199096/spe-gate/usr/closed
Proto area ($ROOT) : /builds/th199096/spe-fix/usr/closed/proto/root_i386
Root of source ($SRC) : /builds/th199096/spe-fix/usr/closed/usr/src
Root of test source ($TSRC) : /builds/th199096/spe-fix/usr/closed/usr/ontest
Current directory ($PWD) : /builds/th199096/spe-fix/usr/closed
[th199096@jhereg closed]> hg reparent ssh://aus1500-home//pool/ws/nfs41-clone/usr/closed
[th199096@jhereg closed]> exit
exit
[th199096@jhereg spe-fix]> hg list
added:
usr/src/cmd/fs.d/nfs/sped/Makefile
usr/src/cmd/fs.d/nfs/sped/sped.c
usr/src/cmd/fs.d/nfs/sped/sped_dt.d
usr/src/cmd/fs.d/nfs/sped/sped_server.c
usr/src/cmd/fs.d/nfs/sped/spedaemon.c
usr/src/cmd/fs.d/nfs/sped/spedaemon.h
usr/src/cmd/fs.d/nfs/svc/spe.xml
usr/src/head/rpcsvc/spe_prot.x
usr/src/uts/common/fs/nfs/spe.c
usr/src/uts/common/nfs/spe.h
usr/src/uts/common/nfs/spe_attr.h
usr/src/uts/common/nfs/spe_impl.h
modified:
usr/src/cmd/fs.d/nfs/Makefile
usr/src/cmd/fs.d/nfs/svc/Makefile
usr/src/head/Makefile
usr/src/head/rpcsvc/daemon_utils.h
usr/src/lib/libshare/nfs/libshare_nfs.h
usr/src/pkgdefs/SUNWhea/prototype_com
usr/src/pkgdefs/SUNWhea/prototype_com
usr/src/pkgdefs/SUNWnfscr/prototype_com
usr/src/pkgdefs/SUNWnfscu/prototype_com
usr/src/uts/common/Makefile
usr/src/uts/common/Makefile.files
usr/src/uts/common/dserv/dserv_mds.c
usr/src/uts/common/fs/Makefile
usr/src/uts/common/fs/nfs/ds_srv.c
usr/src/uts/common/fs/nfs/nfs41_srv.c
usr/src/uts/common/fs/nfs/nfs41_state.c
usr/src/uts/common/fs/nfs/nfs_sys.c
usr/src/uts/common/nfs/Makefile
usr/src/uts/common/nfs/mds_state.h
usr/src/uts/common/nfs/nfs4.h
usr/src/uts/common/nfs/nfssys.h
usr/src/uts/intel/nfs/Makefile
usr/src/uts/sparc/nfs/Makefile
Time spent in user mode (CPU seconds) : 3.61s
Time spent in kernel mode (CPU seconds) : 5.44s
Total time : 0:27.46s
CPU utilisation (percentage) : 32.9%
End up making changes to usr/src/head/rpcsvc/ds_prot.x, usr/src/uts/common/fs/nfs/ds_srv.c, usr/src/uts/common/fs/nfs/nfs41_state.c, and usr/src/uts/common/nfs/nfs_serv_inst.h
[th199096@jhereg spe-fix]> vi usr/src/head/rpcsvc/ds_prot.x [th199096@jhereg spe-fix]> vi usr/src/uts/common/dserv/dserv_mds.c [th199096@jhereg spe-fix]> vi usr/src/uts/common/fs/nfs/ds_srv.c [th199096@jhereg spe-fix]> vi usr/src/uts/common/fs/nfs/nfs41_srv.c [th199096@jhereg spe-fix]> vi usr/src/uts/common/fs/nfs/nfs41_state.c [th199096@jhereg spe-fix]> vi usr/src/uts/common/fs/nfs/nfs41_state.c [th199096@jhereg spe-fix]> pushd usr/src/uts/common/fs/nfs /builds/th199096/spe-fix/usr/src/uts/common/fs/nfs /builds/th199096/spe-fix [th199096@jhereg nfs]> grep ds_guid_info_idx * ds_srv.c: mds_server->ds_guid_info_idx, nfs41_state.c:rfs4_index_t *ds_guid_info_idx; nfs41_state.c: instp->ds_guid_info_idx = rfs4_index_create(instp->ds_guid_info_tab, [th199096@jhereg nfs]> vi ds_srv.c [th199096@jhereg nfs]> popd /builds/th199096/spe-fix [th199096@jhereg spe-fix]> vi usr/src/uts/common/fs/nfs/nfs41_state.c [th199096@jhereg spe-fix]> vi usr/src/uts/common/fs/nfs/nfs41_state.c\ ? [th199096@jhereg spe-fix]> vi usr/src/uts/common/nfs/nfs_serv_inst.h
Reparent back to ssh://aus1500-home//pool/ws/th199096/spe-gate
[th199096@jhereg spe-fix]> hg reparent ssh://aus1500-home//pool/ws/th199096/spe-gate [th199096@jhereg spe-fix]> ws usr/closed/ Workspace : /builds/th199096/spe-fix/usr/closed Workspace Parent : ssh://aus1500-home//pool/ws/nfs41-clone/usr/closed Proto area ($ROOT) : /builds/th199096/spe-fix/usr/closed/proto/root_i386 Root of source ($SRC) : /builds/th199096/spe-fix/usr/closed/usr/src Root of test source ($TSRC) : /builds/th199096/spe-fix/usr/closed/usr/ontest Current directory ($PWD) : /builds/th199096/spe-fix/usr/closed [th199096@jhereg closed]> hg reparent ssh://aus1500-home//pool/ws/th199096/spe-gate/usr/closed [th199096@jhereg closed]> exit exit
And no changes ???
[th199096@jhereg spe-fix]> hg outgoing comparing with ssh://aus1500-home//pool/ws/th199096/spe-gate searching for changes no changes found [th199096@jhereg spe-fix]> hg push pushing to ssh://aus1500-home//pool/ws/th199096/spe-gate searching for changes no changes found [th199096@jhereg spe-fix]>
Up until I've just been cutting and pasting what had happened, now I start to debug to show that I'm not just emailing without trying anything:
Hmm, I happen to have a copy of that gate on this machine:
[th199096@jhereg spe-fix]> diff usr/src/head/rpcsvc/ds_prot.x ../spe-gate/usr/src/head/rpcsvc/ds_prot.x 255d254 < utf8string ds_path; [th199096@jhereg spe-fix]> diff usr/src/uts/common/dserv/dserv_mds.c ../spe-gate/usr/src/uts/common/dserv/dserv_mds.c [th199096@jhereg spe-fix]> diff usr/src/uts/common/fs/nfs/ds_srv.c ../spe-gate/usr/src/uts/common/fs/nfs/ds_srv.c 139,141d138 < UTF8STRING_FREE(res_ok->guid_map.guid_map_val[i]. < ds_path); < 711,713d707 < (void) utf8_copy(&pip->ds_path, < &guid_map[count].ds_path); < [th199096@jhereg spe-fix]> diff usr/src/uts/common/fs/nfs/nfs41_srv.c ../spe-gate/usr/src/uts/common/fs/nfs/nfs41_srv.c [th199096@jhereg spe-fix]> diff usr/src/uts/common/fs/nfs/nfs41_state.c ../spe-gate/usr/src/uts/common/fs/nfs/nfs41_state.c 1919c1919 < * this will populate the following MDS tables. --- > * this will populste the following MDS tables. 2016c2016 < ds_guid_info_t *pip; --- > mds_pool_info_t *pip; 2022c2022 < rw_enter(&ds_guid_info_lock, RW_READER); --- > rw_enter(&mds_pool_info_lock, RW_READER); 2024c2024 < pip = (ds_guid_info_t *)rfs4_dbsearch(ds_guid_info_path_idx, --- > pip = (mds_pool_info_t *)rfs4_dbsearch(mds_pool_info_path_idx, 2031c2031 < rw_exit(&ds_guid_info_lock); --- > rw_exit(&mds_pool_info_lock); 2119c2119 < ds_guid_info_path_compare(rfs4_entry_t entry, void *key) --- > mds_pinfo_path_compare(rfs4_entry_t entry, void *key) 2121c2121 < ds_guid_info_t *pip = (ds_guid_info_t *)entry; --- > mds_pool_info_t *pip = (mds_pool_info_t *)entry; 2127c2127 < ds_guid_info_path_mkkey(rfs4_entry_t entry) --- > mds_pinfo_path_mkkey(rfs4_entry_t entry) 2129c2129 < ds_guid_info_t *pip = (ds_guid_info_t *)entry; --- > mds_pool_info_t *pip = (mds_pool_info_t *)entry; 2433,2438d2432 < instp->ds_guid_info_path_idx = < rfs4_index_create(instp->ds_guid_info_tab, < "DS_guid_path-idx", ds_guid_info_hash, ds_guid_info_path_compare, < ds_guid_info_path_mkkey, < TRUE); < [th199096@jhereg spe-fix]> diff usr/src/uts/common/nfs/nfs_serv_inst.h ../spe-gate/usr/src/uts/common/nfs/nfs_serv_inst.h 193d192 < rfs4_index_t *ds_guid_info_path_idx; [th199096@jhereg spe-fix]>
So the files are different
[th199096@jhereg spe-fix]> diff usr/src/uts/common/nfs/nfs_serv_inst.h /net/aus1500-home/pool/ws/th199096/spe-gate/usr/src/uts/common/nfs/nfs_serv_inst.h 193d192< rfs4_index_t *ds_guid_info_path_idx;
Yes, really they are
Why doesn't Mercurial think so?
It knows that the files have changed
[th199096@jhereg spe-fix]> hg list
modified:
usr/src/head/rpcsvc/ds_prot.x
usr/src/uts/common/fs/nfs/ds_srv.c
usr/src/uts/common/fs/nfs/nfs41_state.c
usr/src/uts/common/nfs/nfs_serv_inst.h
[th199096@jhereg spe-fix]> hg outgoing
comparing with ssh://aus1500-home//pool/ws/th199096/spe-gate
searching for changes
no changes found
And here the email stops as I RTFM and realize I forgot a step:
[th199096@jhereg spe-fix]> hg commit [th199096@jhereg spe-fix]> hg outgoing comparing with ssh://aus1500-home//pool/ws/th199096/spe-gate searching for changes changeset: 7779:ced6eccb4366 tag: tip user: Thomas Haynesdate: Tue Sep 30 14:57:46 2008 -0500 summary: Fix up for new NFS instances
Without a hg commit, even with changes, there is nothing to integrate back to the parent.
So, instead of an email, I blog about it...
[th199096@jhereg spe-fix]> hg push pushing to ssh://aus1500-home//pool/ws/th199096/spe-gate searching for changes Are you sure you wish to push? [y/N]: y remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 4 changes to 4 files
So I maintain nfs41-gate which is a development branch of onnv-gate. With the introduction of Mercurial as our version control system, my life has changed a bit, but the basic tasks for gatekeeping stay the same:
I've noticed that I find merging to be easier with Mercurial, so it tends to happen more often.
I typically already have an existing workspace and I'll do an incremental build in it. Also, I do this for both sparc and i386. I don't have to worry about conflicts or merging since nothing changes in the child. A typical session would be:
[th199096@aus-build-x86 ~]> ws /builds/th199096/nfs41-gk Workspace : /builds/th199096/nfs41-gk Workspace Parent : ssh://aus1500-home//pool/ws/nfs41-clone Proto area ($ROOT) : /builds/th199096/nfs41-gk/proto/root_i386 Root of source ($SRC) : /builds/th199096/nfs41-gk/usr/src Root of test source ($TSRC) : /builds/th199096/nfs41-gk/usr/ontest Current directory ($PWD) : /builds/th199096/nfs41-gk [th199096@aus-build-x86 nfs41-gk]> hg pull -u pulling from ssh://aus1500-home//pool/ws/nfs41-clone searching for changes adding changesets adding manifests adding file changes added 1 changesets with 93 changes to 93 files 93 files updated, 0 files merged, 0 files removed, 0 files unresolved [th199096@aus-build-x86 nfs41-gk]> ws usr/closed/ Workspace : /builds/th199096/nfs41-gk/usr/closed Workspace Parent : ssh://aus1500-home//pool/ws/nfs41-clone/usr/closed Proto area ($ROOT) : /builds/th199096/nfs41-gk/usr/closed/proto/root_i386 Root of source ($SRC) : /builds/th199096/nfs41-gk/usr/closed/usr/src Root of test source ($TSRC) : /builds/th199096/nfs41-gk/usr/closed/usr/ontest Current directory ($PWD) : /builds/th199096/nfs41-gk/usr/closed [th199096@aus-build-x86 closed]> hg pull -u pulling from ssh://aus1500-home//pool/ws/nfs41-clone/usr/closed searching for changes adding changesets adding manifests adding file changes added 1 changesets with 3 changes to 3 files 3 files updated, 0 files merged, 0 files removed, 0 files unresolved [th199096@aus-build-x86 closed]> exit exit [th199096@aus-build-x86 nfs41-gk]> `which nightly` -in nightly.env
At which point I go do something else, like blog about what I am doing.
I said typical, except that this change set (which gets rid of auth records in the MDS as a byproduct) happens to have touched something in closed. We rarely seem to make changes there.
Okay, the build is done (remember to check the logs) and in this case it did not fail. So push the BFU bits out and then send out email telling people about the new reference bits.
[th199096@aus-build-x86 nfs41-gk]> ~/gk/hg-push.sh nfs41-gk i386 2008-09-30 ARCH=i386 BASE=/builds/th199096 AUS=/net/aus1500-home.central/pool/ws/nfs41-gate-hg-archives/i386 NIGHTLY=/builds/th199096/nfs41-gk/archives/i386/nightly NIGHTLY_ND=/builds/th199096/nfs41-gk/archives/i386/nightly-nd DATER=/builds/th199096/nfs41-gk/archives/i386/2008-09-30 DATER_ND=/builds/th199096/nfs41-gk/archives/i386/2008-09-30-nd + mv /builds/th199096/nfs41-gk/archives/i386/nightly /builds/th199096/nfs41-gk/archives/i386/2008-09-30 + mv /builds/th199096/nfs41-gk/archives/i386/nightly-nd /builds/th199096/nfs41-gk/archives/i386/2008-09-30-nd + cp -r /builds/th199096/nfs41-gk/archives/i386/2008-09-30 /net/aus1500-home.central/pool/ws/nfs41-gate-hg-archives/i386/2008-09-30 + cp -r /builds/th199096/nfs41-gk/archives/i386/2008-09-30-nd /net/aus1500-home.central/pool/ws/nfs41-gate-hg-archives/i386/2008-09-30-nd + rm /net/aus1500-home.central/pool/ws/nfs41-gate-hg-archives/i386/latest /net/aus1500-home.central/pool/ws/nfs41-gate-hg-archives/i386/latest-nd + cd /net/aus1500-home.central/pool/ws/nfs41-gate-hg-archives/i386 + ln -s 2008-09-30 latest + ln -s 2008-09-30-nd latest-nd
This case is a bit more complicated and can be summarized by:
By not changing the nfs41-gate until the final moment, I can throw everything away if needed. And believe me, as painful as that is, I've done it. Also, note that when I talk about a workspace above, I am also taling about working in parallel with the closed version of it.
But now onto a detailed example:
Get a backup snapshot of the gate:
[th199096@aus1500-home ~]> zfs snapshot pool/ws/nfs41-clone@sync99
Now grab your copy for merging
[th199096@aus1500-home ~]> cd /pool/ws/th199096/ [th199096@aus1500-home th199096]> ~/bin/hg-clone ssh://aus1500-home//pool/ws/nfs41-clone nfs41-syn c 397b36b5473d === clone open tree: ssh://aus1500-home//pool/ws/nfs41-clone === requesting all changes adding changesets adding manifests adding file changes added 7515 changesets with 101024 changes to 51313 files updating working directory 42507 files updated, 0 files merged, 0 files removed, 0 files unresolved 2a39f20bc20e === clone closed tree: ssh://aus1500-home//pool/ws/nfs41-clone/usr/closed === requesting all changes adding changesets adding manifests adding file changes added 968 changesets with 8269 changes to 4389 files updating working directory 2677 files updated, 0 files merged, 0 files removed, 0 files unresolved
~/bin/hg-clone is a simple script to get both the open and closed versions of the gate.< /p>
And reparent it to onnv-clone
[th199096@aus1500-home th199096]> ws nfs41-sync/ Workspace : /pool/ws/th199096/nfs41-sync Workspace Parent : ssh://aus1500-home//pool/ws/nfs41-clone Proto area ($ROOT) : /pool/ws/th199096/nfs41-sync/proto/root_i386 Root of source ($SRC) : /pool/ws/th199096/nfs41-sync/usr/src Root of test source ($TSRC) : /pool/ws/th199096/nfs41-sync/usr/ontest Current directory ($PWD) : /pool/ws/th199096/nfs41-sync [th199096@aus1500-home nfs41-sync]> hg reparent ssh://onnv.eng//export/onnv-clone
Pull and merge
[th199096@aus1500-home nfs41-sync]> hg pull -u pulling from ssh://onnv.eng//export/onnv-clone searching for changes adding changesets adding manifests adding file changes added 210 changesets with 2122 changes to 1808 files (+1 heads) not updating, since new heads added (run 'hg heads' to see heads, 'hg merge' to merge) [th199096@aus1500-home nfs41-sync]> hg merge merging usr/src/cmd/Makefile merging usr/src/cmd/zfs/zfs_iter.c merging usr/src/cmd/zfs/zfs_main.c merging usr/src/lib/Makefile merging usr/src/lib/libzfs/common/libzfs.h merging usr/src/lib/libzfs/common/libzfs_dataset.c merging usr/src/lib/libzfs/common/mapfile-vers merging usr/src/pkgdefs/Makefile merging usr/src/pkgdefs/SUNWcsu/prototype_com merging usr/src/pkgdefs/SUNWhea/prototype_com merging usr/src/pkgdefs/etc/exception_list_sparc merging usr/src/uts/common/Makefile.files merging usr/src/uts/common/Makefile.rules merging usr/src/uts/common/fs/zfs/dsl_dataset.c merging usr/src/uts/common/fs/zfs/zfs_ioctl.c merging usr/src/uts/common/sys/Makefile merging usr/src/uts/common/sys/fs/zfs.h merging usr/src/uts/intel/Makefile.intel.shared merging usr/src/uts/intel/os/minor_perm merging usr/src/uts/intel/os/name_to_major merging usr/src/uts/sparc/Makefile.sparc.shared merging usr/src/uts/sparc/os/minor_perm merging usr/src/uts/sparc/os/name_to_major 1785 files updated, 23 files merged, 62 files removed, 0 files unresolved (branch merge, don't forget to commit)
So, I used filemerge to do any manual editing in the above merge. It is invoked in my .hgrc:
# Merge tool [merge-patterns] ** = filemerge [merge-tools] filemerge.executable = /ws/onnv-tools/teamware/bin/filemerge filemerge.args = -a $base $local $other $output filemerge.checkchanged = true filemerge.gui = true
Then I would commit and repeat the cycle for the closed branch:
[th199096@aus1500-home nfs41-sync]> hg commit [th199096@aus1500-home nfs41-sync]> ws usr/closed Workspace : /pool/ws/th199096/nfs41-sync/usr/closed Workspace Parent : ssh://aus1500-home//pool/ws/nfs41-clone/usr/closed Proto area ($ROOT) : /pool/ws/th199096/nfs41-sync/usr/closed/proto/root_i386 Root of source ($SRC) : /pool/ws/th199096/nfs41-sync/usr/closed/usr/src Root of test source ($TSRC) : /pool/ws/th199096/nfs41-sync/usr/closed/usr/ontest Current directory ($PWD) : /pool/ws/th199096/nfs41-sync/usr/closed [th199096@aus1500-home closed]> hg reparent ssh://onnv.eng//export/onnv-clone/usr/closed [th199096@aus1500-home closed]> hg pull -u pulling from ssh://onnv.eng//export/onnv-clone/usr/closed searching for changes adding changesets adding manifests adding file changes added 15 changesets with 145 changes to 137 files (+1 heads) not updating, since new heads added (run 'hg heads' to see heads, 'hg merge' to merge) [th199096@aus1500-home closed]> hg merge 135 files updated, 0 files merged, 10 files removed, 0 files unresolved (branch merge, don't forget to commit) [th199096@aus1500-home closed]> hg commit
The next step is a clone/build on one of the build machines. As this looks a lot like the cloning in this section and the build from the prior, I'm going to leave it out.
After the build and verification is done, we prepare the nfs41-gate for the integration. Because of the branch merge comments not being in the approved RTI format, we need to turn off sanity checking for this operation. Note that it is okay to keep on for developers pushing to the gate:
[th199096@aus1500-home ~]> su - nfs4hg Password: Sun Microsystems Inc. SunOS 5.11 snv_92 January 2008 [nfs4hg@aus1500-home ~]> cd /pool/ws/nfs41-gate/usr/closed/.hg [nfs4hg@aus1500-home .hg]> cp hgrc hgrc.good [nfs4hg@aus1500-home .hg]> vi hgrc [nfs4hg@aus1500-home .hg]> diff hgrc hgrc.good 73c73 < #pretxnchangegroup.1 = python:hook.sanity.sanity --- > pretxnchangegroup.1 = python:hook.sanity.sanity
Reparent to the gate and push
[th199096@aus1500-home closed]> hg reparent ssh://nfs4hg@aus1500-home//pool/ws/nfs41-gate/usr/clos ed [th199096@aus1500-home closed]> hg push pushing to ssh://nfs4hg@aus1500-home//pool/ws/nfs41-gate/usr/closed searching for changes Are you sure you wish to push? [y/N]: y pushing to ssh://nfs4hg@aus1500-home//pool/ws/nfs41-gate/usr/closed ... remote: Preparing gk email... remote: ...gk email sent
Fix the .hgrc back to turn on sanity checking and repeat for the open bits.
So I filed two bugs with my experiences with VirtualBox 2.0.2 last week:
If you search the Bug Database for 6753564, it returns 6753569. If you try to go there directly with the URL I provided, then you get bug not found. Ahh, the first match happens because I mention 6753564 in other bug. Well, the link exists in 6753569 and I'm sure the bug will be opened up for public viewing sooner or later.