Weblog

Friday May 13, 2005

FEM: Let's get technical Back again. They say a picture is worth a thousand words, but I don't know how to make pictures so I'll just use words and code snippets. Remembering from my previous entry that FEM was designed for use by the NFS server and that there is no user side API, let me try to tell you how to use FEM. Hmm, I just realized that it would help to know about the Vnode Interfaces project, which introduced the concept of vnode/vfs operation registration, however, I'll leave that to someone else. So let's say you're inside the kernel and you want to be notified of certain vnode operations, how do you do it? Step 1: Define the template for the ops you're interested in monitoring static const fs_operation_def_t my_ops_tmpl[] = { VOPNAME_OPEN, my_open, VOPNAME_WRITE, my_write, VOPNAME_SETATTR, my_setattr, VOPNAME_RWLOCK, my_rwlock, VOPNAME_SPACE, my_space, VOPNAME_SETSECATTR, my_setsecattr, VOPNAME_VNEVENT, my_vnevent, /* I haven't explained this one yet, tune in later */ NULL, NULL }; Step 2: Create your fem_t (your monitor array of operations) fem_t *my_ops; e = fem_create("my operations", my_ops_tmpl, &my_ops); if (e != 0) operation FAILED; Now you're ready to roll, pick the vnodes you want to monitor and throw your ops on them. Step 3: Install the monitored ops onto a vnode fem_install(vp, my_ops, (void *)myarg, OPUNIQ, NULL, NULL); That's it, you're done with the easy part. Now is where the work begins. Why did you throw a monitor on this vnode? What do you want to do when one of the monitored operations occurs? Its time to write your monitors. Like I said earlier, FEM is just a mechanism to intercept vnode/vfs operations, all the work is up to you when you write the monitor. However, before I continue, let me explain a few of the arguments to the fem_install() function. The first 2 should be obvious, but the others may need a little explanation. "myarg" is whatever you need to pass to you're monitor, in the case of NFSv4, I pass a one of the server state structures known as an rfs4_file_t. By the way, this arg cannot be NULL. The next arg is a flag which specifies how to install the monitor: FORCE - means install this monitor, no matter what other monitors may be on this vnode. OPUNIQ - means install this monitor if the operation set is unique (don't install the same set of monitors twice) OPARGUNIQ - means install this monitor if the ops/arg combination is unique The last two args are function pointers to hold and release functions for the arg that you pass in. These are optional. Step 4: Define your monitors This could have been step 1, and may well be step one during your design phase. So let me pick an op that is in my_ops array, like OPEN, and define a monitor for it. int my_open( femarg_t *arg, int mode, cred_t *cr) /* note, looks a lot like vop_open only uses femarg_t instead of vnode */ { int rc; myarg_t *myarg; myarg = (myarg_t *)arg->fa_fnode->fn_available; /* this is the arg you passed on fem_install() */ /* * you have now intercepted a call to open on this vnode, do whatever you want here before passing control * on to the next level, which could be another monitor or the underlying file system. * for nfsv4, I check for conflict and recall the delegation. */ rc = vnext_open(arg, mode, cr); /* here is where you could do some post processing, if desired */ return (rc); } That's it. Be creative, have fun, but be careful. You can install and uninstall (fem_uninstall()) monitors as you wish, but be careful what you do inside the monitors. It is up to you to prevent deadlocks. Calling other VOPs from inside a monitor is dangerous unless you know what you're doing. Also note, you can deny service just by having your monitor return without calling the vnext_ routine. This is getting too long already. I will need to write again to explain the vnode lifecycle events (how do you monitor a vnode before it is created and how do you know when it is destroyed) and VOP_VNEVENT. I also need to talk about the lack of user level APIs, though I think you should see why now. Here's a hint: 1) the only consumer so far is in the kernel. 2) FEM is just a framework, to have a user API, default monitors would have to exist in the kernel already. More later. ( May 13 2005, 10:51:19 AM PDT ) Permalink Comments [13]

Thursday May 12, 2005

FEM: What and Why? I'm back already with my second entry. FEM, or File Event Monitoring, is a formalized mechanism to monitor events on a vnode or vfs by intercepting the vnode/vfs operations. The framework enables the consumer to request event notification for specified files and directories. The consumers, which intercepts the events, are responsible for delivering the event to the next intercepter or the terminal destination. Monitors so placed, inline, have the greatest flexibility as they are able to affect the control flow both before and after the event is delivered. In simple terms, it is a way to intercept vnode/vfs operations. It is only a framework and provides no real monitoring or notification. The monitor, which the consumer is responsible for writing, does all the work. FEM simply give you the ability to install a monitor on the operations you're interested in. I may write a "FEM: How?" entry which will go into the details of how it works and how to use it. The questions you should be asking now is why. Why did I putback a project that had a goal of having no overhead when it isn't used? And why isn't there a user API to use it? And why do we need a vnode/vfs intercept framework at all? All good questions and I'll partially answer them by explaining where it is currently being used. The project that wanted this, or at least file event notifications, was NFSv4. In version 4 of the NFS protocol, file delegation was introduced. Others have/will blog about delegation, so I won't get into details here. Anyway, the NFS server can grant file delegations to its clients, which allows the clients to feel safe using that file without others having conflicting access. When a conflicting access does occur, the server will recall the delegation. Now, if Sun servers were only accessed by NFS clients, there wouldn't have been a need for FEM. However, because we allow local access to the files, the NFS server needed to be notified when that happens. Hence the need for FEM. When the server grants a delegation, it installs a monitor on the vnode of the file it delegated. Then all open, read, write, setattr, rwlock, setsecattr, remove, rename operations are monitored for conflicts. If the monitor detects a conflict, it kicks off a recall and makes the operation wait for the delegation to be returned. When the delegation is returned (or revoked, if necessary), the monitor lets the operation that caused the conflict to continue. So there you have it. An introduction to FEM. I realize there is so much more to write about it, and I will. My next entry will include the kernel interfaces to FEM and an example of the monitor used by the NFS server. Hasta pronto. ( May 12 2005, 09:30:30 AM PDT ) Permalink Comments [2]

Wednesday May 11, 2005

Gotta start somewhere. Greetings and salutations. Since this is my first ever blog entry, I figure it is best to keep it short and keep you wanting. My name is Jim Wahlig and I've been with Sun for a little over 4 years. I work, mostly, out of my home, here in Austin, TX. I started my career at Sun in the File System group and have since migrated into the File Sharing group, which is most commonly called, the NFS group. Though relatively new to Sun, I've been programming professionally for almost 20 years, started back in June of '85. While working in the FS group, I was on a project called FSI (File System Infrastructure). Though it never completed, it did spin off a couple of things that lead me to move into the FSH group. One of those spin offs was called Caller Context, another one was something known as FEM, File Event Monitoring. In an upcoming blog, I will talk about each of these, how NFS uses both of them and how there is still more to do with each. Hasta luego! ( May 11 2005, 12:30:43 PM PDT ) Permalink Comments [5]