Tuesday February 05, 2008 Matt Ingenthron's Stream of Consciousness
Tuesday February 05, 2008 OpenSolaris Web Stack: Setting Services on Apache
Solaris Express Developer Edition 01/08 (a.k.a. SXDE 01/08) has been released! One of the features of the new release is the integration of much of the work from the Web Stack project
So you're probably wondering, what is different or better about the Web Stack (including, but not limited to Apache, MySQL and PHP) integrated into OpenSolaris. Well, the code is the same stuff available at the various project sites. This, of course, is by design. Though it is Open Source of one license or another, there's no real value in deviating from what the upstream communities have released.
Still, there are some interesting places OpenSolaris can add value. OpenSolaris has things like DTrace, SMF, lots of security features, etc.
Jyri has already written about how easy it is to run PHP on OpenSolaris, including SXDE 01/08. Assume for a moment though that what you want is a 64-bit Apache with mod_proxy. Well, how do you set that up?
The team has done a great job of integrating the web stack components. One example is using svccfg to make changes to how a service is set up to run. So, let's look at modifying the service configuration of Apache. First, we start it:
# svcs http:apache22 STATE STIME FMRI disabled 19:35:01 svc:/network/http:apache22 # svcadm enable http:apache22
The process is up and running, but is it 32 or 64 bit? Hmmm. Let's find out. First we need to know what the process is. We could grep for http or something like that, but there's a better way to see which processes are associated with a service:
# svcs -p http:apache22
STATE STIME FMRI
online 19:54:30 svc:/network/http:apache22
19:54:30 1154 httpd
19:54:31 1155 httpd
Okay, now we see the processes. How do I find out if it's 32 or 64 bit? pargs(1) can tell tell us what the binary is, then we can look at the file:
# pargs -x 1154 1154: /usr/apache2/2.2/bin/httpd -k start AT_SUN_PLATFORM 0x08047fda i86pc AT_SUN_EXECNAME 0x08047fe0 /usr/apache2/2.2/bin/httpd AT_PHDR 0x08050034 AT_PHENT 0x00000020 AT_PHNUM 0x00000007 AT_ENTRY 0x0806cc10 AT_SUN_LDDATA 0xfeffa000 AT_BASE 0xfefc0000 AT_FLAGS 0x00000000 AT_PAGESZ 0x00001000 AT_SUN_AUXFLAGS 0x00000002 AT_SUN_HWCAP 0x0043dc6f SSSE3 | AHF | CX16 | MON | SSE3 | SSE2 | SSE | FXSR | MMX | CMOV | SEP | CX8 | TSC | FPU # file /usr/apache2/2.2/bin/httpd /usr/apache2/2.2/bin/httpd: ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, stripped
Okay, it's just 32-bit. Since it's so nicely integrated, perhaps there is a service property to switch it. Let's see what service properties we have to mess with:
# svcprop http:apache22 httpd/enable_64bit boolean false httpd/server_type astring prefork httpd/stability astring Evolving httpd/startup_options astring "" httpd/value_authorization astring solaris.smf.value.http/apache22 network/entities fmri svc:/milestone/network:default network/grouping astring require_all network/restart_on astring error network/type astring service filesystem-local/entities fmri svc:/system/filesystem/local:default filesystem-local/grouping astring require_all filesystem-local/restart_on astring none filesystem-local/type astring service autofs/entities fmri svc:/system/filesystem/autofs:default autofs/grouping astring optional_all autofs/restart_on astring error autofs/type astring service startd/ignore_error astring core,signal general/action_authorization astring solaris.smf.manage.http/apache22 general/enabled boolean false general/value_authorization astring solaris.smf.value.http/apache22 general/entity_stability astring Evolving start/exec astring /lib/svc/method/http-apache22\ start start/timeout_seconds count 60 start/type astring method stop/exec astring /lib/svc/method/http-apache22\ stop stop/timeout_seconds count 60 stop/type astring method refresh/exec astring /lib/svc/method/http-apache22\ refresh refresh/timeout_seconds count 60 refresh/type astring method tm_common_name/C ustring Apache\ 2.2\ HTTP\ server tm_man_httpd/manpath astring /usr/apache2/2.2/man tm_man_httpd/section astring 8 tm_man_httpd/title astring httpd tm_doc_apache_org/name astring apache.org tm_doc_apache_org/uri astring http://httpd.apache.org restarter/logfile astring /var/svc/log/network-http:apache22.log restarter/contract count 95 restarter/start_pid count 1139 restarter/start_method_timestamp time 1202270070.318463000 restarter/start_method_waitstatus integer 0 restarter/auxiliary_state astring none restarter/next_state astring none restarter/state astring online restarter/state_timestamp time 1202270070.320630000
There's a lot of stuff because of the various dependencies and other variables that SMF needs for some of the other magic. That one labeled "httpd/enable_64bit" looks like the one we need though. Let's change it using svccfg:
# svccfg -s http:apache22 svc:/network/http:apache22> listprop httpd/* httpd/server_type astring prefork httpd/stability astring Evolving httpd/startup_options astring httpd/value_authorization astring solaris.smf.value.http/apache22 httpd/enable_64bit boolean false svc:/network/http:apache22> setprop httpd/enable_64bit=true svc:/network/http:apache22> listprop httpd/* httpd/server_type astring prefork httpd/stability astring Evolving httpd/startup_options astring httpd/value_authorization astring solaris.smf.value.http/apache22 httpd/enable_64bit boolean true svc:/network/http:apache22> exit
Now, when we change these service properties, we need to apply the changes by refreshing the service and then restarting it. Then we can see if we got the desired effect:
# svcadm refresh http:apache22
# svcadm restart http:apache22
# svcs -p http:apache22
STATE STIME FMRI
online 20:44:40 svc:/network/http:apache22
20:44:40 1683 httpd
20:44:41 1684 httpd
20:44:41 1685 httpd
20:44:41 1686 httpd
20:44:41 1687 httpd
20:44:41 1688 httpd
20:44:41 1689 httpd
# pargs -x 1683
1683: /usr/apache2/2.2/bin/amd64/httpd -D 64bit -k start
AT_SUN_PLATFORM 0xfffffd7fffdfffcf i86pc
AT_SUN_EXECNAME 0xfffffd7fffdfffd5 /usr/apache2/2.2/bin/amd64/httpd
AT_PHDR 0x0000000000400040
AT_PHENT 0x0000000000000038
AT_PHNUM 0x0000000000000008
AT_ENTRY 0x000000000042d450
AT_SUN_LDDATA 0xfffffd7fff3fa000
AT_BASE 0xfffffd7fff394000
AT_FLAGS 0x0000000000000000
AT_PAGESZ 0x0000000000001000
AT_SUN_AUXFLAGS 0x0000000000000002
AT_SUN_HWCAP 0x000000000041dc77 SSSE3 | CX16 | MON | SSE3 | SSE2 | SSE | FXSR
| MMX | CMOV | AMD_SYSC | CX8 | TSC | FPU
# file /usr/apache2/2.2/bin/amd64/httpd
/usr/apache2/2.2/bin/amd64/httpd: ELF 64-bit LSB executable AMD64 Version 1 [SSE FXSR CMOV FPU], dynamically linked, stripped
Looks good. Apache, as you may know, has both prefork and worker models. I happened to spot in there that there is a property for that. Since all of the modules I'm using are thread safe, I think I'll turn it to worker...
# svccfg -s http:apache22
svc:/network/http:apache22> listprop httpd/*
httpd/server_type astring prefork
httpd/stability astring Evolving
httpd/startup_options astring
httpd/value_authorization astring solaris.smf.value.http/apache22
httpd/enable_64bit boolean true
svc:/network/http:apache22> setprop httpd/server_type=worker
svc:/network/http:apache22> exit
# svcadm refresh http:apache22
# svcadm restart http
svcadm: Pattern 'http' matches multiple instances:
svc:/network/http:squid
svc:/network/http:apache22
# svcadm restart http:apache22
Very nice, now I can go edit my httpd.conf and enable proxying!
I went from 32 to 64 bit, and changed MPMs all without having to go rebuild Apache, edit rc files, etc. Plus, the current configuration is always easy to find with svcprop.
Once other projects, like Visual Panels, come along we should then see a nice GUI way of understanding and editing our services. The other nice advantage is if I have a large number of systems I want to configure the service on in a similar way, SMF makes it simple to replicate that service configuration to other systems. Alternatively, I could just jumpstart them and apply a service config.
You may be asking, but what about that stuff in httpd.conf? Well, remember our goal here in the Web Stack project is to integrate nicely with those upstream projects. Admins know how to edit their httpd.conf files, and it wouldn't add much value to start making that many modifications to the Apache HTTP Server. The philosophy is integrate with the features in OpenSolaris, but stay true to the community that produced the code.
Download and play with SXDE 01/08 or one of the Project Indiana builds after it synchs up with the latest code, or join us over in the OpenSolaris Web Stack project.
( Feb 05 2008, 11:42:39 PM PST ) Permalink Comments [3]Post a Comment:
Comments are closed for this entry.
The svcprop 64-bit property is a hack. I'm really surprised to see such an implementation coming from Sun.
Perhaps I should shoot an e-mail to Jiry and ask him why they didn't use isaexec.
I mean, there are *clear* guidelines on how 32- and 64-bit binaries are to be delivered on a system. And explicitly selecting 32- or 64-bit binaries should never be a necessary action in Solaris, since there is isaexec exactly for that purpose.
Are they experimenting, or what? It just doesn't make any sense whatsoever.
Posted by UX-admin on February 07, 2008 at 02:51 AM PST #
I think the project team is certainly aware of isaexec. I certainly am. It certainly wasn't intended as a hack.
I think you may be forgetting the fact that Apache may be used in different ways, and some of those may be using modules which are not 64-bit clean. With the components in Web Stack, Sun isn't in a position to bring everything to 64-bit and make changes in the upstream... and even if Sun did, not all of the upstream folks may agree with that idea?
In other cases, 32-bit has been a conscious choice. For instance, I know of users of Apache who are using mod_perl. The challenge they had with mod_perl is that perl code is fond of hashmaps, and going to 64-bit meant doubling the pointers, which actually gave them less space (given the fact that memory for a system costs something) in the heap for the data they needed for the app. Obviously, there are other ways to solve this problem, but they had bigger fish to fry at the time.
The reason I talked about mod_proxy in the blog is that it is a clear case where someone may want to use 64-bit and worker MPMs. Running PHP and various extensions is a clear case where you may want to avoid 64-bit. This is partially because the extensions may not support it, and it's partially because it just may not add any value.
In any event, none of this is set in stone at this point and we try to discuss it prior to going through ARC and putback. We should certainly discuss it over on webstack-discuss if you still think there's a good case for modifying Apache to use isaexec(). I believe we'd still need a flag in Apache. Even Java doesn't always run 64-bit-- there's a flag there.
Posted by Matt Ingenthron on February 07, 2008 at 09:46 AM PST #
Yes, I believe Apache should also use isaexec. It's been a while since Apache's been in Solaris, so we're approaching a point where Apache should also go 64-bit.
For example, before the webstack came out, I was forced to compile, package, and test a complete PHP + PostgreSQL + Apache + Oracle stack.
The problem was, Oracle doesn't deliver 32-bit client libraries on sparc any more, so I had to compile *everything* to be 64-bit, just in order to be able to link against the 64-bit oracle client libraries.
Can you imagine the overhead of having to compile, package and test both 32- and 64-bit:
- PostgreSQL
- Net-SNMP
- OpenSSL
- OpenLDAP
- UNIX ODBC
- Dovecot IMAP server
- PHP5
Just to be able to produce a PHP binary and libraries capable of talking to an Oracle database?
It was violent. It was tiresome. It was blood, sweat, and tears...
Anyway, to come back to 64- versus 32-bit, yes, on sparc, a 32-bit binary will actually run faster than a 64-bit one. However, on i86pc, the 64-bit binaries are actually faster, because the compiler has at least one extra register to work with, and that makes for a big performance impact (in a good way).
So I at least am convinced, that we should have a complete 64-bit stack, delivered and using isaexec, the mechanism intended for exactly this particular scenario.
Posted by UX-admin on February 08, 2008 at 04:27 AM PST #