Troubleshooting Web Server crashes : enabling core dumps

Troubleshooting Web Server crashes : enabling core dumps

I get a lot of questions regarding what to do when Web Server instance crashes as shown by error logs and you do not find the core file.

The core file will be dumped in the Web Server instance's config directory. For example, if the Web Server is installed in /opt/SUNWwbsvr directory, core file will be /opt/SUNWwbsvr/https-hostname/config/core.

If you are using SSL, core dumps will be disabled by default. You can force them on by setting the SSL_DUMP environment variable before starting Web Server:

$ SSL_DUMP=1
$ export SSL_DUMP
$ ./start

Check coreadm(1M) and ulimit(1) to see if Solaris will allow processes to dump core. Use the pstack(1M) command to obtain a stack trace. You can use the same coreadm (core file administration) command to set appropriate values.
$coreadm

  global core file pattern:
  global core file content: default
  init core file pattern: core.%p
  initcore file content: default
  global core dumps: disabled
  per-process core dumps: enabled
  global setid core dumps: disabled
  per-process setid core dumps: disabled
  global core dump logging: disabled

Note that I have set core file pattern to core.%p instead of the usual core . If a process with pid lets say 1000 dumps core, it will generate a core file with name core.1000 to avoid overriding in case the server dumps multiple core files. But this is not necessary.

If your Operating System is Linux, make sure that you set ulimit to unlimited before starting the server.
$ulimit -c unlimited or

$ulimit -S -c unlimited or
or edit /etc/security/limits.conf and followed by editing /etc/profile

To get core dumps on Windows, first make drwatson as a default debugger:
C:\WINDOWS\system32>drwtsn32.exe -i

To change various settings of this drwatson,
C:\WINDOWS\system32>drwtsn32.exe
This opens a window where you can specify where to dump the core.

To set back the default debugger to MSVC, change registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug to "C:\Program Files\Microsoft Visual Studio\Common\MSDev98\Bin\msdev.exe" -p %ld -e %ld
Set Auto to 0.
Set UserDebuggerHotKey to 0.

Some more suggested reading

1. http://publib.boulder.ibm.com/httpserv/ihsdiag/coredumps.html
   
2. http://developers.sun.com/solaris/articles/DebugLibraries/DebugLibraries_content.html  - The article shows how to use elfdump -p and dbx to check corefiles.

Posted by meena ( Mar 14 2007, 12:53:11 PM IST ) Permalink Comments [0]