Doth quoth the DaveM

Fixing web page with tiny fonts

I finally got fed up with the small fonts of a web site I visit regularly, so I went searching for a solution. I'd read about Greasemonkey in Lifehacker, so I checked it out. It installed as a Firefox extension easily enough, and I restarted Firefox. I searched the Greasemonkey scripts repository for a solution to my font problem, with an idea that I could fix it using some additional CSS.

In searching for something to add some CSS to the offending site, I happened upon a reference to Stylish, another Firefox extension that specifically deals with style issues, so I installed it. After restarting Firefox again (why is it necessary to restart in order to load an extension?), I visited the offending web site, and then clicked the little Stylish icon at the bottom of the Firefox window. This brought up a menu where I could write a style for the particular page or the website as a whole. That brought up an editor where I could add the CSS I wanted. I browsed the CSS from the offending web site and just cut and pasted all the parts that specified font size, and changed the font sizes from x-small and xx-small to small ! important. The ! important part overrides the style specified by the site.

I reloaded pages from the web site I was trying to modify, and voila, it was now rendering with larger, more readable fonts. Very cool!

Posted at 04:52PM Feb 07, 2007 by Dave Marquardt in Programming  | 

Tcl and dtrace

I'm really intrigued by the idea of adding a DTrace provider to Tcl. Only problem is, I don't know the Tcl internals all that well at this point. My big concern is that Tcl compiles procs (procedures) into byte code, and I'm not sure how that all works and will interact with the dtrace provider. So far I've seen some information on dtrace providers for SAX and Perl, and in particular the information on the Perl dtrace provider made things look a lot simpler than I thought. So, this is another idea on my "to do" list.

Posted at 12:51PM Oct 25, 2005 by Dave Marquardt in Programming  |  Comments[1]

Interesting problem with ifile, malloc and libumem

As I've mentioned in the past, I use ifile to sort my e-mail. I'm not pleased with the performance on Solaris Express, and I've done some work in the past on it and integrated the changes into Sourceforge. For some reason I got interested in using the libumem version of malloc, realloc and free, as the libumem page says they should work and may provide better performance. This is a bit bass-ackwards, in that I ought to do some sort of profiling to see where the performance problems are, but that's beside the point of this entry! :-) So, I tried linking ifile with -lumem, that worked okay.

But over time, I noticed that the behavior of ifile was different with libumem. ifile keeps a database of words in e-mail messages, with frequencies and an indication of the last time the word frequency in the database was updated. Infrequently used words are removed from the database. I particularly noticed this problem when I re-indexed my e-mail with ifile-gnus-initialize.sh. The version that did not use libumem trimmed words and the database grew slowly, which I expected based on past behavior. But the libumem version grew far larger.

So I ran a test. ifile doesn't have any sort of unit test suite, so I just did some small tests of my own. When I added the 9th file to the database, I saw that the libc-malloc ifile trimmed many old words, but the libumem-malloc ifile did not!

The funny thing is that the trim function is about arithmetic. Before the 9th file is added to the database, the databases are identical. So something in libumem is tickling a bug.

Now, the bug could be in libumem, but I'm far more suspicious of ifile. ifile calls malloc, free and realloc in many places without checking the return values! But if there was a problem there, I would think I would get core dumps in the offending place.

So, I actually have more than one course of action available. First I tried to use dbx, but for some reason I still don't understand, it hung at the time I tried it. Secondly, I could use libumem's debug facilities to see if I can find anything unusual going on. Third, I could use dtrace to see what's going on within the two different versions of the program. But since the libc version of ifile works okay, I'll have to get back to this when I have more time.

Posted at 12:38PM Oct 25, 2005 by Dave Marquardt in Programming  |  Comments[1]

Expect rocks!

I recently tested some code on one of our platforms that's still in its early stages, and the method for bootstrapping up to the OBP prompt involves a couple of console windows. In the first console, you have system controller prompts and the interaction is quite minimal. In the second console, you have no prompts, and some of the actions cause output in the first console. I used autoexpect to help me write an Expect script for each console. That was okay, but it required opening multiple xterms and starting the script in each, and didn't take advantage of the second console causing feedback in the first console. So, I remembered multixterm. While there was probably a way to get multixterm to open the xterms for me and start the script in each, that still didn't take advantage of the second console causing feedback in the first console. So, I borrowed the code in multixterm that creates xterms attached to applications and controls them, and wrote a single script that

• opens two xterms
• starts the consoles
• brings the system to the OBP prompt

Of course, the day after I finished this some newer software was installed that didn't have quite the same output, but it was easy enough to fix my script to handle that.

To me, Expect rocks for automating tasks like this. And with tools like autoexpect and multixterm, it's really quite easy to automated what at first looks like it might be too complex.

Posted at 01:56PM Nov 23, 2004 by Dave Marquardt in Programming  |  Comments[1]

ifile ideas

I use ifile to sort my e-mail. As the Savannah ifile project page says

ifile is a general purpose e-mail filter that has been adapted to a number of e-mail clients including MH, EXMH, Gnus, pine and procmail-compatible clients. It strives to be fast and efficient and to provide the best filtering possible. It can be used to filter out junk e-mail (aka spam).

ifile uses Naive Bayes to filter e-mail into folders, based on how the user wants to filter into folders. You have to train it a bit, but to me that's easier than trying to keep a bunch of rules updated in .procmailrc or some other place.

One problem I have with ifile is that the way it is invoked from Gnus, my e-mail reader, is one message at a time. So for each message, ifile

• reads the database
• reads the message
• sorts
• writes the database
• returns a folder name

That reading and writing of the database each time is really time consuming. It's not so much the I/O time, but the parsing that's slow.

It occurred to me recently that it would be nice to have some sort of daemon mode for ifile. Here are a couple of ideas:

• use IPC to pass the message data to the daemon ifile, and return the folder
• have the daemon create a shared memory segment, and be in charge of occasionally writing it out. Clients would attach to the shared memory segment and update it. Some sort of synchronization for access would be necessary, of course.

Maybe at some point I'll get a chance to implement these, but for now I don't have much bandwidth for it outside work.

Posted at 04:20PM Sep 16, 2004 by Dave Marquardt in Programming  | 

Self modifying Tcl code

I was playing with a Tcl proc (routine) that picks a random spot in a list and replaces it with another element. At first I did something like

# Replace a random spot in global list "matches" with a new list.
proc replace_match {pattern type instances} {

    global matches
    global srand_done; # initialized to 0 in init

    set l [list $pattern $type $instances]

    if {$srand_done} {
        set r [expr rand()]
    } else {
        set r [expr srand([clock clicks] + [clock seconds])]
    }

    set idx [expr int(floor($r * [llength $matches]))]

    set matches [lreplace $matches $idx $idx $l]
}

That works, and is pretty obvious, but from a performance standpoint, you have to check a global variable every time you call the routine.

So, something made me think of the idea of just hot patching the code, i.e. replace it with something else. Aha, I remembered that Tcl has a rename command to rename a proc, and if you rename it to the empty string, the proc disappears. So, here's the new solution:

# Replace a random spot in global list "matches" with a new list.
proc replace_match {pattern type instances} {

    global matches

    set l [list $pattern $type $instances]

    set idx [expr int(floor(srand([clock clicks] + [clock seconds]) * \
				[llength $matches]))]

    set matches [lreplace $matches $idx $idx $l]


    # now redefine replace_match to use rand() instead
    rename replace_match ""

    proc replace_match {pattern type instances} {

	global matches

	set l [list $pattern $type $instances]

	set idx [expr int(floor(rand() * [llength $matches]))]

	set matches [lreplace $matches $idx $idx $l]
    }
}

It's nice because I no longer have to use srand_done at all.

Posted at 05:25PM Aug 27, 2004 by Dave Marquardt in Programming  | 

Using MDB to figure out mysterious program behavior

In working with Tcl and SWIG, I was trying to write a typemap to convert between a Tcl list and a C global char *foo[1024]. I kept having problems when I would try to append to the list foo, i.e. I'd try set foo bar lappend foo bletch set foo and instead of getting the list bar bletch as I expected, I'd see something like bletch bletch. Really strange.

I decided to use Solaris' mdb debugger to figure out what was going on by setting watchpoints. First I set a watchpoint on the first part of foo, since I knew I'd be changing that, i.e. I ran foo:w This was changing as I expected. Then I examined foo and set a watchpoint at the address at the first word in foo. At this point, I discovered this address being used in malloc() and free(). Ah ha, something was freeing this.

What I narrowed this down to was the fact that I was using the underlying Tcl_Obj string representation as my char *'s, but Tcl didn't know that and was freeing them. I realized I needed to allocate my own memory for copies of the strings and copy them over from the Tcl objects to my memory. Once I did that, things worked. I had scratched my head for a while, but mdb made it really easy to figure out what was going on.

Posted at 09:54AM Jun 23, 2004 by Dave Marquardt in Programming  | 

Scripting with Tcl and SWIG

Lately I've been on a mission to provide a scripting interface to a test program that has a curses-based configuration tool that for some reason I have a lot of trouble with. I guess the designer and I have different ideas of how to design a curses-based configuration tool. Alos, the tool produces configuration files that, while they are in ASCII and can be read, they're not easily modified, so difficult to use for scripting.

I'm a fan of Tcl/Tk and also [incr Tcl], which is an object oriented extension to Tcl. [incr Tcl] seems ideal for dealing with structures and things you might want to do to them, or, in object oriented speak, methods you want them to do to themselves, I suppose. Well, how do you connect an existing program with a scripting language like Tcl? Particularly since Tcl doesn't understand what to do with C structures? There's this really cool tool called SWIG, or Simplified Wrapper and Interface Generator. Not only can it generate Tcl and [incr Tcl] code to interface with a program or libraries data structures, but it can handle many other scripting language, including Perl, Python, Ruby, and PHP. Since the only one of these languages I really know is Tcl (I know, it's a hole in my personal knowledge base), I'm using SWIG to generate Tcl and [incr Tcl] code. SWIG is pretty good at understanding structures with relatively basic types, like int, double, char, etc., and simple pointers types. But when it gets to something like char *foo[1024], i.e. an array of 1024 pointers to char, it needs some help. That's where you start using typemaps to convert from C types to scripting language types and back. While Tcl has arrays, they're associative arrays, i.e. their indices can be any string. C's arrays have numeric indices starting at 0. While it seems to me conceivable that you could map those numeric indices to their string counterparts, i.e. 0 to "0", 1 to "1", etc., SWIG doesn't seem to have a way to do that. At any rate, I decided to convert my global char *foo[1024] to a Tcl list when foo is referenced, and convert from a Tcl list to the C array when setting. Works really well, when it works. I'll have another posting about using MDB to figure out why it wasn't working! So what would be the alternative to SWIG? Write your own interfaces to the various structures. SWIG makes this really easy and is a big productivity booster.

Posted at 09:42AM Jun 23, 2004 by Dave Marquardt in Programming  |