Wednesday March 11, 2009
|
Ramblings from the Mountains Michael Hunter's Weblog |
|
Wednesday March 11, 2009 I recently purchased a Asus 1000HE 10" netbook. My initial impression is that I really like it. The keyboard is a little cramped but compared to playing around with various 9" netbooks this is very usable. I've gotten OpenSolaris installed and almost working on it. When I get the last kinks worked out I'll blog the details. ( Mar 11 2009, 08:04:18 AM PDT ) Permalink I promised a friend new to the iPhone that I'd post a list of the applications I'd used, tried, found useful, or found useless. That was a long time ago. The list keeps getting longer and it is really hard to keep up with the list of applications (mostly free) that I've downloaded and deleted. Wins:
Purgatory:
Downloaded but havn't tried or don't remember:
I recently purchased a Kindle 2. I've read electronic books in the past including on a laptop (I just got an Asus 1000HE netbook; I could see using it where good PDF reading or a backlight was necessary), a RocketBook (how fast does history disappear?), my old Palm based Treo, and my iPhone. I liked the RocketBook if content wasn't so hard to get or to put on the device. The week or so I've spent with my Kindle 2 has made it clear that it is miles ahead of those options. The iPhone Kindle Reader is a strong play in combination with the Kindle 2 but not on its own. This review from CNET gets the various iPhone options right in my experience. The biggest missing part to me from the iPhone Kindle reader isn't that the buying experience is way more painful then it needs to be but that you can't read subscriptions. The most common thing for me to read on the run is a periodical. My biggest current gripes with the Kindle 2 are the inability to organize my documents (hierarchy would be nice, tagging with multiple views would be even better) and the lame PDF reading/conversion (caveat I've only used free.kindle.com). I believe the original Kindle shipped with a protective case. The Kindle 2 didn't. I purchased a Belkin Neoprene Sleeve. It seems like something that should have come with the Kindle 2. I feel better putting the Kindle 2 into my backpack with something between it and the rest of the random set of stuff I carry around. ( Mar 11 2009, 06:23:56 AM PDT ) Permalink
Thursday August 21, 2008 I dumped my Treo 650 the other day for an iPhone 3G. The Treo was way long in the tooth and the 3G and the GPS receiver on the iPhone 3G made it irresistible to me. A few years after I graduated from college I worked for Pactel (later Airtouch) Teletrac and the concept of location tagging data has since been of interest to me. A few things (none of them surprising) which are big hits are:
And there are some things which are not big hits:
for those that can't last untethered So American Airlines announced Internet for those that can't work offline while flying. Well, okay, they announce some form of expensive cripplenet:
"full Internet" 'cept for those services we might wish to charge extra for later. Not sure what "Cell phone" has to do with that but being able to use VOIP is well withing the definition of "full Internet". Sadly I doubt most will realize the duplicity in AA's statement. I wonder how many people really need to be connected for the flight time? I guess if it kept me from buying expensive drinks it might be worth the $12.95 an hour. Barely. I suspect it will be slow enough it might actually increase my tab. ( Aug 21 2008, 03:05:03 PM PDT ) Permalink
Monday December 10, 2007 quick My current project team conducted a two day intensive review. We have team members in Ireland, the east and west US coasts, and Beijing. Generally we use a combination of email, phone, and irc to make this all happen. For this review we used a morning component that including our developer in Ireland. I took notes during this part of the meeting using Sun's externally available wiki. We then spent the afternoon annotating those. While we slept in the pacific timezone we got comments from our engineers in China and our engineer in Ireland did some additional annotation. And then we repeated it all. I'm sure there are better integrated tools to achieve this type of collaboration but these simple ones worked well for us. ( Dec 10 2007, 02:31:38 PM PST ) Permalink
Saturday November 03, 2007 cool stuff inside only Kudos for driving the economies of scale and some neat concepts in software in producing a 200U$ computer. Thats a price point that people are comfortable buying game systems at. And even without MS there is an impressive amount of functionality in this box. But to really make this grab the consumer it is going to have to look a little better then a cheap whitebox. A PS3 or Mac Mini form factor would be orders of magnitude better. I'm sure the chicken and egg issues of creating a market and having the resources to do the miniaturization are part of the issue but given the already existing industry in mini-itx and like cases that the gamers and others like that seems like a weak decision. ( Nov 03 2007, 06:15:51 PM PDT ) Permalink
Wednesday October 11, 2006 Reading Driving to Domination in the September 2006 issue of Linux Journal by Doc Searls we are made to beleive that Linux will win the desktop war because "Linux likes its drivers in the kernel tree". I'm sceptical that this is a cause for domination and not an effect of open source religion. And in fact a negative effect. If a platform has a stable interface an engineer can write a driver to then you are able to leverage a wider cast of people to help with support. But then you have to accept that some companies might want to keep their source closed and you need to have the engineering ability to maintain a stable interface. That might fly in the face of your goal if its a certain philsophical bent, but not if its user support. If your philisophical goals are truely aligned with your user community then it won't matter. All the drivers your users accept will have their source code available. I'm also uncomfortable with the word Domination. The point of building a healthy user community shouldn't be one of domination but rather of cooperation. I shouldn't be trying to beat you into submission on only using hardware which has open source drivers nor should I try building silos where you are only allowed to use drivers anointed by some open source high priest. Instead I should be building a dialog with my user base helping to choose the best solution while also helping the community to grow. A sign that this is more religion then reason comes in the sentence after the one quoted above. "There's no need to annoy the hacker and confuse the user with a CD of binary code and promotional lockware." Huh? Yea, thats one tactic. But its not specific to binary only drivers and its not necessary for driver distribution. ( Oct 11 2006, 11:05:23 AM PDT ) Permalink
Tuesday August 15, 2006 bloggin' friend and IPv6 One of my workmates started a blog. I had promised an outline for a IPv6 transition blueprint/whitepaper some time ago. I guess this is a sign I should actually write it up. ( Aug 15 2006, 11:54:07 PM PDT ) Permalink
Monday April 17, 2006 Network Approachability Strategy I contributed to the recently published Network Approachability Strategy. If you have any comments please send them to the Network Approachabilty mailing list (also available as a webforum). ( Apr 17 2006, 02:38:24 PM PDT ) Permalink
Monday October 10, 2005 Approachability For the past few months I've been working on Solaris Approachability. Our group is focused on making Solaris easier to use. I realize that isn't a very well defined concept but instead of attempting to describe what it means in a blog entry I'm going to point you to the community we are building on opensolaris.org. Check out what is going on, read the mailing list, and contribute ideas. ( Oct 10 2005, 09:30:06 PM PDT ) Permalink Comments [0]
Wednesday June 15, 2005 On March 31, 2005 support for MLDv2 and IGMPv3 was put back to build 13 of Nevada. Additionally support for a an API to access the new features available in the update protocols was putback. A non blogging coworker did all of the engineering work involved in this project. A group of us did the design review and myself and another engineer did the code review. MLD (Multicast Listener Discovery) and IGMP (Internet Group Management Protocol) provide the capability to discover multicast listeners and their interests on directly attached links. MLD does this for IPv6 and IGMP does this for IPv4. The v2 and v3 aspects of these protocols is adding the support to filter based on source. Previously traffic was filtered based on its target (I'm getting a specific multicast stream). Now listeners can claim interest in only streams from a specific source (I'm interesting in getting a specific multicast stream from some source I know is good). Both MLDv2 and IGMPv3 interoperate with their earlier versions. The API changes for the project allowed the user to manipulate filters for the groups they joined. The API RFC specifies three APIs. First of all is a Delta-Based API specified in section 5.1 of the RFC. This API allows one to specify which groups and sources one is interested in via socket options. MCAST_{JOIN,BLOCK,UNBLOCK,LEAVE}_GROUP and MCAST_{JOIN,LEAVE}_SOURCE_GROUP are the operations in this group. A previous blog entry talks a little about socket option processing and will help you find the implementations of these socket options. The second API is specified in section 5.2 of the RFC and is called the "Full-state" API. In this API there are two functions, setsourcefilter and getsourcefilter. These functions take a series of arguments which specify the Whew. Thats a lot of APIs for something so straightforward. Its often a blessing that the IETF community tends to iterate a lot. When successful they converge to some pretty good specifications. Historically they have kept themselves mostly to bits on the wire so they were careful to maintain backwards compatibility. Users didn't normally see this churn. But as they have gotten into APIs sometimes the churn can generate a little bit of pain. In this case the APIs layer nicely.
The putback was made up of 44 files. Besides the kernel and library changes netstat, snoop, and truss had to be modified. The first to display the filters, the second to display the new bits on the wire, and the last to display the new ioctls. If you are considering this type of protocol change its important to remember to modify snoop. The same is true of adding ioctls and remembering to update truss.
As far as library changes you can find the code in sourcefilter.c. The less evident part of making a library change in solaris is the library spec file. The one for the socket library can be found in socket.spec. This needs to be modified if you make a library change.
The kernel changes are spread over the majority of the files in the putback. A few of them (ip.c, ip6.c) havn't been opensourced yet due to IPsec issues. The main file to implement the protocol is igmp.c. Note that the MLD and IGMP protocols share a lot and are both implemented in this file. You can following the train of socket options down into the kernel directed by the opdes_t structures in ip_opt_data.c, icmp_opt_data.c, and udp_opt_data.c.
Tuesday June 14, 2005 Socket option processingAs part of Solaris 10 (and my first project at Sun) I upgraded the IPv6 APIs to RFC 3493 (Basic Socket Interface Extensions for IPv6) and RFC 3542 (Advanced Sockets Application Program Interface (API) for IPv6). The main part of this project involved adding or changing available socket options. I thought a walk through socket option processing would provided some insight into the command side of the control plane of the Solaris TCP/IP stack. Given my interest in IPv6 I'll be tracking a call to setsockopt(sd, IPPROTO_IPV6, IPV6_HOPLIMIT, &option, sizeof(option)); I'll point out side trips to where other argument combinations might take you. Its easy to generate a dtrace script which will trace setsockopt(). dtrace: pid 100887 exited with status 1 CPU FUNCTION 0 -> setsockopt 0 -> getsonode 0 -> getf 0 -> set_active_fd 0 <- set_active_fd 0 <- getf 0 <- getsonode 0 -> mutex_owned 0 <- mutex_owned 0 -> sotpi_setsockopt [...] We find setsockopt() here. Don't confuse this with the routine which you link against. This is the routine which receives control in the kernel.
int
setsockopt(int sock, int level, int option_name, void *option_value,
socklen_t option_len, int version)
{
struct sonode *so;
intptr_t buffer[2];
void *optval = NULL;
int error;
dprint(1, ("setsockopt(%d, %d, %d, %p, %d)\n",
sock, level, option_name, option_value, option_len));
if ((so = getsonode(sock, &error, NULL)) == NULL)
return (set_errno(error));
[... input validation and retrieval ...]
error = SOP_SETSOCKOPT(so, level, option_name, optval,
(t_uscalar_t)option_len);
[... error handling and exit code ...]
return (0);
}
I've compressed the first few lines and removed some code to make the point of our investigation clearer. The first thing that this routine does is to print out some debugging. With the existence of dtrace we should be seeing fewer of these types of debugging statements exist in the future. By using the source code browser you can look at the various routines and data types being used here. Focus first on 'struct sonode'. If you follow the link to the data type you will find a long comment talking about the use of a sonode (it represents the socket) and how it is used. The first functional thing we do is to call getsonode() which uses the fd to retried the file structure which holds the vnode and thus leads to the sonode. I'm going to skip the code which validates and stores the options passed into setsockopt(). Instead we are going to focus in on SOP_SETSOCKOPT(). The definition of this macro is:
#define SOP_SETSOCKOPT(so, level, optionname, optval, optlen) \
((so)->so_ops->sop_setsockopt((so), (level), (optionname), \
(optval), (optlen)))
What we get from this is that we are using the jump table stored on the sonode to call the socket specific setsockopt function. Jump tables can make debugging fun. But dtrace makes figuring out what function is ultimately being called a little easier (at least in specific cases). If we look up at our dtrace output we find the obviously named culprit sotpi_setsockopt(). Before looking at that we should look around a little to see where this is used. Obviously its called via the jump table, but how does it get into that jump table? A search turns it up in socktpi.c and ncafs.c. At this point we want to know if this is the only function that processes socket options. A search on sonodeopts_t (the jump table type) turns up another table in socksctp.c. Question answered. In this case we can continue on down the common setsockopt() path and remember that there is another path followed by SCTP and NCA that we will have to investigate some other time. So back to sotpi_setsockopt(). This function is an interesting mash of history and standards driven need to demultiplex options processing. If we look at the structure of this code our eye is first drawn to the two large case statements. Note that the first of these handle (in some cases partially) socket or tcp level options in common cases. The second large case statement fixes up SOL_SOCKET level options which fail. Its the bit of code inbetween those two statements which packages up the socket option and its related data and sends them to the transport specific routines.
optmgmt_req.PRIM_type = T_SVR4_OPTMGMT_REQ;
optmgmt_req.MGMT_flags = T_NEGOTIATE;
optmgmt_req.OPT_length = (t_scalar_t)sizeof (oh) + optlen;
optmgmt_req.OPT_offset = (t_scalar_t)sizeof (optmgmt_req);
oh.level = level;
oh.name = option_name;
oh.len = optlen;
mp = soallocproto3(&optmgmt_req, sizeof (optmgmt_req),
&oh, sizeof (oh), optval, optlen, 0, _ALLOC_SLEEP);
/* Let option management work in the presence of data flow control */
error = kstrputmsg(SOTOV(so), mp, NULL, 0, 0,
MSG_BAND|MSG_HOLDSIG|MSG_IGNERROR|MSG_IGNFLOW, 0);
mp = NULL;
mutex_enter(&so->so_lock);
if (error) {
eprintsoline(so, error);
goto done;
}
error = sowaitprim(so, T_SVR4_OPTMGMT_REQ, T_OPTMGMT_ACK,
(t_uscalar_t)sizeof (struct T_optmgmt_ack), &mp, 0);
if (error) {
eprintsoline(so, error);
goto done;
}
The code before soallocproto3() marshalls the options and their data into a structure. soallocproto3() does as we would guess. It allocates some space and copies the data passed. SOTOV returns the vnode associated with a sonode. kstrputmsg() puts the resulting message onto the stream associated with the socket. We will ignore for a moment what happens with that message. sowaitprim() waits for a response to that message. This is a blocking operation (as one would expect given the blocking semantics of setsockopt()). You could figure out what transport specific routine is handling this socket option in various ways, but given we already have some dtrace output I would tend to use that. A ways down the dtrace output you will find: [...] 0 <- soallocproto3 0 -> kstrputmsg 0 -> strput 0 -> mutex_owned 0 <- mutex_owned 0 -> canputnext 0 <- canputnext 0 -> strmakedata 0 <- strmakedata 0 -> mblk_setcred 0 -> crhold 0 <- crhold 0 <- mblk_setcred 0 -> putnext 0 -> mutex_owned 0 <- mutex_owned 0 -> mutex_owned 0 <- mutex_owned 0 -> icmp_wput 0 <- icmp_wput 0 -> icmp_wput_other 0 -> snmpcom_req 0 -> mi_offset_param 0 <- mi_offset_param 0 <- snmpcom_req 0 <- icmp_wput_other 0 -> snmpcom_req 0 -> mi_offset_param 0 <- mi_offset_param 0 <- snmpcom_req 0 <- icmp_wput_other 0 -> svr4_optcom_req [...] So it appears that after a fair bit of work we end up at icmp_wput(). We have M_PROTO data so we fall into the default case of the first switch and call icmp_wput_other() (also confirmed by our trace). In icmp_wput_other() we fall into the first case and need to remember the type of the request we sent down. If we look back to where we built it in sotpit_sockopt() (code listing above) we notice its a T_SVR4_OPTMGMT_REQ. At this point we call svr4_optcom_req. We skip over the first block which is looking at M_CTLs. We have a T_NEGOTIATE (see earlier code block) so we skip past the parameter checking switch statement and the following T_DEFAULT case. Then we enter a for loop checking parameters. The interesting thing to look at is opt_chk_lookup(). What the heck is this doing?
/* Find the option in the opt_arr. */
if ((optd = opt_chk_lookup(opt->level, opt->name,
opt_arr, opt_arr_cnt)) == NULL) {
If we look back up at the top of svr4_optcom_req() we see opdes_t *opt_arr = dbobjp->odb_opt_des_arr; I'm going to jump to the opdes_t relevant to our current discussion. Its at icmp_opt_arr. It would take some time to discuss the various elements of opdes_t but suffice to say that it codifies each socket option and its attributes. So after checking the options against the information we have we end up in the switch statement for completing the action. After some setup we call "setfn". We could figure out this is icmp_opt_set() from out dtrace work, from code examination, or (after some experience) from realizing that is the likely name due to its function. At this point we use the information from the original setsockopt() to determine which code block handles IPV6_HOPLIMIT. We end up in the block which looks like:
case IPV6_HOPLIMIT:
if (inlen != 0 && inlen != sizeof (int))
return (EINVAL);
if (checkonly)
break;
if (inlen == 0) {
ipp->ipp_fields &= ~IPPF_HOPLIMIT;
ipp->ipp_sticky_ignored |= IPPF_HOPLIMIT;
} else {
if (*i1 > 255 || *i1 < -1)
return (EINVAL);
if (*i1 == -1)
ipp->ipp_hoplimit = icmp_ipv6_hoplimit;
else
ipp->ipp_hoplimit = *i1;
ipp->ipp_fields |= IPPF_HOPLIMIT;
}
if (sticky) {
error = icmp_build_hdrs(q, icmp);
if (error != 0)
return (error);
}
break;
It wasn't my intention to talk too much about IPV6_HOPLIMIT specifically. The complexity of such a simple thing is created by the API specification which allows for one to set this via IPV6_PKTINFO, via IPV6_HOPLIMIT, or by default. I've run out of time to talk about the rest of the processing, walk you back up the chain or to explain what happens for options which are aimed at a layer above where they are handled. But the basic routines and data structures have been introduced. If you need to fix bugs in this area or add functionality I hope this helps you get started. Technorati Tag: OpenSolaris Technorati Tag: Solaris Technorati Tag: DTrace ( Jun 14 2005, 07:59:58 AM PDT ) Permalink Comments [0]
Monday April 18, 2005 SCTP support was integrated into Solaris 10. SCTP was originally designed for its use in telephony. Additionally work has been done to make SCTP work well in multi-homed environments. I'll spend a little time in a series of blog entries examining the SCTP API. The API is being developed by the tsvwg and a draft is currently available as draft-ietf-tsvwg-sctpsocket-10.txt1. Unix Network Programming Volume 1 is often referred to as the sockets bible, but the SCTP API has been moving fast enough that its not completely up to date. If you are using it expect that there will be some things which have changed. SCTP provides two different programming models: 1-1 and 1-many. Initially we will look at the 1-1 model as it closely maps to what people are use to when programming TCP socket applications. The main difference is that there is no half-close. If you are porting a TCP application that depends on the half-close you will need to recode it (probably using in band signaling). Other then things should be pretty familiar. We are going to open a socket and connect on the client side. On the server side we will open a socket, bind to an address we can find, listen on that socket, and then wait for connections. So the server code looks like this:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <netinet/sctp.h>
int main(int argc, char *argv[]) {
int sd, s, len, addrlen;
struct sockaddr_in servaddr, clientaddr;
char buffer[128];
if ((sd = socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP)) == -1) {
perror("socket");
exit(EXIT_FAILURE);
}
memset((void *)&servaddr, 0, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port = htons(30000);
bind(sd, (struct sockaddr *)&servaddr, sizeof(servaddr));
listen(sd, 1);
for(;;) {
addrlen = sizeof(clientaddr);
if ((s = accept(sd, (struct sockaddr *)&clientaddr,
&addrlen)) == -1) {
perror("accept");
exit(EXIT_FAILURE);
}
while((len = read(s, buffer, sizeof(buffer))) > 0)
write(s, buffer, len);
close(s);
}
}
and the client code looks like this: [edited 5/19/05 due to inserting the server code twice]
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <netinet/sctp.h>
int main(int argc, char *argv[]) {
int sd, s, len;
struct sockaddr_in servaddr;
char buffer[128];
const char msg[] = "Hello, World\n";
if ((sd = socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP)) == -1) {
perror("socket");
exit(EXIT_FAILURE);
}
memset((void *)&servaddr, 0, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port = htons(30000);
if (connect(sd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) { perror("connect");
exit(EXIT_FAILURE);
}
write(sd, msg, sizeof(msg));
if ((len = read(sd, buffer, sizeof(buffer))) == -1) {
perror("read");
exit(EXIT_FAILURE);
}
buffer[len] = 0;
printf("%s", buffer);
return (EXIT_SUCCESS);
}
I've gone light on configuration (the addresses are hardcoded as the local machine port 30000) and on error handling. That should make it stand out clearly that other then the third argument to socket() and an extra header that this code looks exactly like the TCP equivalent. So how do we know this is using SCTP? We can look at the output from 'netstat -an' when the server is running:
[...]
SCTP:
Local Address Remote Address Swind Send-Q Rwind Recv-Q StrsI/O State
------------------------------- ------------------------------- ------ ------ ------ ------ ------- -----------
0.0.0.0 0.0.0.0 0 0 102400 0 32/32 CLOSED
*.30000 0.0.0.0 0 0 102400 0 32/32 LISTEN
[...]
So far all that we have done is very straightforward to anybody who has written socket code in the past. But I think it will be easier to learn the unique features that SCTP offers from an existing knowledge base. In the future we will work on making these little code snippets do something interesting and learn about SCTP in the process. 1Guaranteed to become a dead link once the draft is rev'd. Do the obvious link manipulation if its no longer valid or search for the update. ( Apr 18 2005, 04:28:30 PM PDT ) Permalink Comments [1]
Thursday March 31, 2005 I have a fairly old portable mp3 player the "Creative Nomad Jukebox Zen". Its a 20G device which came out about the time of the original iPod. In the last year or so I havn't been using it much. I've either listened to content on my laptop or off of CD in the car1. Recently I've become interested in using my Zen again. Encouraged by Dan Price's blog entry about using his camera with gphoto on Solaris 10 I decided to do this on my S10 box. Initially I looked at gnomad which lead me to libnjb. Libnjb is a library which uses libusb to access a wide range of Creative devices2. Other then having to cover the u_int{8,16,32,64}_t types they used with uint{8,16,32,64}_t types from Now all I have to do is to get this into blastwave...
1I need a reasonable portable/car/stationary audio integration story but havn't had the energy to work it out.
2The FAQ/HACKING/README files supplied with libnjb are a treasure trove of information including some nice information about how to obtain more information about the on the wire protocol and which devices support which protocols. This feels like a really well done project.
( Mar 31 2005, 11:18:30 AM PST )
Permalink
Comments [0]
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
