Dan Anderson's Weblog

Where's the Crypto Libraries?

One thing's that not immediately apparent when using OpenSolaris crypto functions is where are the crypto libraries? The OpenSolaris Cryptographic Framework API is based on the PKCS#11 industry standard from RSA Laboratories. However, the PKCS#11 crypto libraries aren't shown by ldd(1M), which lists dynamic libraries used in executables. For example, lets look at encrypt(1M):

$ ldd /usr/bin/encrypt
	libc.so.1 =>	 /lib/libc.so.1
	libm.so.2 =>	 /lib/libm.so.2

Ldd only lists the C and math library—crypto functions are missing. That's because /usr/bin/encrypt is just a hard link to /usr/lib/isaexec, which chooses the correct binary to execute (/usr/bin/{i86,amd64,sparcv7,sparcv9}/encrypt, depending on the isainfo -n value):

$ ls -li /usr/bin/encrypt /usr/lib/isaexec
9340 -r-xr-xr-x 94 root bin 8248 2008-11-19 16:13 /usr/bin/encrypt
9340 -r-xr-xr-x 94 root bin 8248 2008-11-19 16:13 /usr/lib/isaexec
$ isainfo -n
amd64

So, lets try ldd again with /usr/bin/amd64/encrypt, now that we're older and wiser :-)

$ ldd /usr/bin/amd64/encrypt
	libkmf.so.1 =>	 /lib/64/libkmf.so.1
	libpkcs11.so.1 =>	 /usr/lib/64/libpkcs11.so.1
	libcryptoutil.so.1 =>	 /lib/64/libcryptoutil.so.1
	libc.so.1 =>	 /lib/64/libc.so.1
	libkmfberder.so.1 =>	 /lib/64/libkmfberder.so.1
	libmd.so.1 =>	 /lib/64/libmd.so.1
	libnsl.so.1 =>	 /lib/64/libnsl.so.1
	libsocket.so.1 =>	 /lib/64/libsocket.so.1
	libxml2.so.2 =>	 /lib/64/libxml2.so.2
	libmp.so.2 =>	 /lib/64/libmp.so.2
	libscf.so.1 =>	 /lib/64/libscf.so.1
	libpthread.so.1 =>	 /lib/64/libpthread.so.1
	libz.so.1 =>	 /lib/64/libz.so.1
	libm.so.2 =>	 /lib/64/libm.so.2
	libuutil.so.1 =>	 /lib/64/libuutil.so.1
	libgen.so.1 =>	 /lib/64/libgen.so.1

Lots more libraries, but it's still missing functions loaded from dlopen(3C)-ed libraries, which are most of the PKCS#11 crypto functions.

Fortunately in a recent blog by Darryl Gove he mentions a technique he learned from Rod Evans of having the loader list what libraries are dynamically loaded. Lets try that with encrypt:

$ echo "Some cleartext test data" >testfile.aes
$ dd if=/dev/random of="keyfile.aes256 bs=32 count=1
$ LD_DEBUG=files /usr/bin/amd64/encrypt -a aes -k keyfile.aes256 -i outfile.aes -o outfile.clear >encrypt.lddebug 2>&1

Now we have a listing of files dynamically loaded in encrypt.lddebug along with a lot of other debug information.

00980: hardware capabilities - 0x25ff7  [ AHF SSE3 SSE2 SSE FXSR AMD_3DNowx AMD_3DNow AMD_MMX MMX CMOV AMD_SYSC CX8 TSC FPU ]
00980: configuration file=/var/ld/64/ld.config: unable to process file
00980: file=/usr/bin/amd64/encrypt  [ ELF ]; generating link map
00980:     addr:               0x400000  size:              0x1615c
00980:     lmid:                   BASE  lmco:                 0x20
00980: file=/usr/bin/amd64/encrypt;  analyzing  [ RTLD_LAZY RTLD_GLOBAL RTLD_WORLD RTLD_NODELETE ]
00980: file=libc.so.1;  needed by /usr/bin/amd64/encrypt
00980: file=/lib/64/libc.so.1  [ ELF ]; generating link map
00980:     addr:     0xfffffd7ffefc0000  size:             0x1b2938
00980:     lmid:                   BASE  lmco:                 0x20
. . . (snip)

As you can see, it's not very readable and interlationships are hard to see. Fortunately, by using Darryl's ld_dot script, this can be boiled down into a more-readable .dot format file that shows libraries loaded as a directed graph (digraph):

$ ld_dot encrypt.lddebug
$ mv dot.dot encrypt.dot 
$ cat encrypt.dot
digraph G {
encrypt -> libc_so_1 [style=bold];
ld_so_1 -> libc_so_1 [style=dotted];
libc_so_1 -> ld_so_1 [style=dotted];
libc_so_1 -> en_US_UTF_8_so_3 [style=dotted];
en_US_UTF_8_so_3 -> libc_so_1 [style=bold];
en_US_UTF_8_so_3 -> methods_unicode_so_3 [style=bold];
methods_unicode_so_3 -> libc_so_1 [style=bold];
libc_so_1 -> en_US_ISO8859_1_so_3 [style=dotted];
en_US_ISO8859_1_so_3 -> libc_so_1 [style=bold];
encrypt -> libcryptoutil_so_1;
libcryptoutil_so_1 -> libc_so_1 [style=bold];
encrypt -> libpkcs11_so_1;
libpkcs11_so_1 -> libc_so_1 [style=bold];
libpkcs11_so_1 -> libcryptoutil_so_1;
libpkcs11_so_1 -> pkcs11_kernel_so [style=dotted];
pkcs11_kernel_so -> libc_so_1 [style=bold];
pkcs11_kernel_so -> libcryptoutil_so_1;
pkcs11_kernel_so -> libmd_so_1;
libmd_so_1 -> libc_so_1 [style=bold];
libpkcs11_so_1 -> pkcs11_softtoken_so [style=dotted];
pkcs11_softtoken_so -> libc_so_1 [style=bold];
pkcs11_softtoken_so -> libsoftcrypto_so_1;
libsoftcrypto_so_1 -> libc_so_1 [style=bold];
pkcs11_softtoken_so -> libcryptoutil_so_1;
pkcs11_softtoken_so -> libmd_so_1;
libmd_so_1 -> libc_so_1 [style=bold];
}

This file can then be used generate a graphic file that visually shows the digraph. This can be done with either the "dot" command from http://graphviz.org/ or online with this webpage http://graph.gafol.net/ I prefer the graphs generated by the webpage over GraphViz, as the former draws dotted lines and the lines appear sharper. However, there's no privacy with the web-generated graphs, so don't use it for stuff you want to keep private.

GraphViz binaries compiled for Solaris are available from http://www.sunfreeware.com/, http://www.blastwave.org, and http://www.opencsw.org/

Here's the gafol website-produced version created from file encrypt.dot above. This shows the libraries used for OpenSolaris, but Solaris 10 is similar:

Legend:

• Bold solid lines shows libraries explicitly linked and visible with ldd(1M) and loaded with the dependent library or executible.
• Non-bold solid lines shows libraries that are lazy-loaded (dynamically loaded at run time),
• Dotted lines shows libraries that were loaded with dlopen(3C).

Here's a brief tour of the major crypto libraries shown in the digraph:

• The libpkcs11 library contains the PKCS#11 API (C_*() functions, such as C_Initialize()).
• That in turn calls library pkcs11_softtoken or pkcs11_kernel, for userland or kernel crypto providers. The latter is used mostly for hardware-assisted cryptography (such as n2cp for Niagara2 SPARC processors), as that is performed more efficiently in kernel space with the "kCF" module (Kernel Crypto Framework). Additionally, for Solaris 10, strong crypto algorithms were split off in separate libraries, pkcs11_softtoken_extra
• libcryptoutil contains low-level utility functions to help implement cryptography.
• libsoftcrypto (OpenSolaris and Solaris Nevada only) implements several symmetric-key crypto algorithms in software, such as AES, RC4, and DES3, and the bignum library (used for RSA).
• libmd implements MD5, SHA, and SHA2 message digest algorithms
• libkmf, not used in this example, implements Key Management Framework (see kmfcfg(1M)).

Here's a similar graph produced by the OpenSSL command
LD_DEBUG=files /usr/bin/amd64/openssl speed -engine pkcs11 >openssl.lddebug 2>&1
(click to enlarge):

This graph is similar to the encrypt(1M) command graph above, with the addition of libcrypto, a OpenSSL-specific library. By using the -engine pkcs11 option, this causes the Solaris crypto algorithms to be used in some cases instead of the default OpenSSL crypto algorithms.

Posted at 05:08PM May 21, 2009 by DanX in Solaris  |  Comments[1]