Alan Hargreaves' Weblog

The in.telnetd vulnerability/exploit (3rd update)

Before I get into the meat of this posting, let me acknowledge that, yes, this was an almighty cock up and should not have happened. It did happen. Let's move on.

Also, while I might not agree with the publication of zero day exploits. Again, It happened. There's really not much I can do about that. There's really no point in being upset about it.

The upside to the posted exploit was the fact that because the code was available, the poster included an analysis of what was going wrong, pointing at the code that was broken. This almost certainly saved us some time in troubleshooting the issue. For this part of the post, you have my thanks.

I would certainly be interested if the person who posted the exploit could tell us how he found the problem; for no other reason, than I'm simply interested.

Anyway, this blog is supposed to be about getting it fixed.

All the times below are Australia/NSW.

One of our National SSEs (Rodney) was on-site with a large customer yesterday. This customer had asked him about a telnet exploit and described the problem to him. Rodney gave me a call and asked me about it at about 1pm. I hadn't and on hearing the description (initially only described to him as a root exploit) Ttwo of us (thanks for your help Chris) dove into the code to start looking at how telnet -l-froot could behave as it did. At this point I did not know about the zero day exploit posting. Once we worked out what was going on, I called Rodney back and explained the full implications of the bug to him so he could explain it to the customer.

We told them that they could block the root vulnerability by uncommenting the CONSOLE= line in /etc/default/login. Note that this has been the default since Solaris 10 update 2 almost forever. However, I still see lots of customer configurations where it is commented out. The only other way to protect against the other implications (login as any user without a password) would be to disable the telnet service until we could fix the issue. eg

# svcadm disable svc:/network/telnet:default

We then started looking through the code to determine the best way to fix this. I logged an internal escalation and was in the process of logging a high priority bug when I saw the following in the SCCS history of usr/src/cmd/cmd-inet/usr.sbin/in.telnetd.c

D 1.67  07/02/11 19:46:41 danmcd        90 89   00009/00010/04896
6523815 LARGE vulnerability in telnetd

I immediately had a look at the bug and banged of an email to Dan stating that I could probably get IDR patches built for on10 pretty quickly. After a brief discussion of the bug and the fix, he pointed me at the manager of the group responsible for the backport and I started the backport (actually a very simple fix).

I got the IDRs built and basic testing done by about 5pm and started writing the Sun Alert.

The documentation for how to write a Sun Alert and specifically the actions that need to be taken to get interim fixes available were spot on and I sent off the initial draft at about 6:45 along with sending a request for getting the IDR patches turned into ISR patches (Interim Security Relief) and getting them published.

Just before 9pm, I started getting into discussions with UK based folk in Derrick Scholl's group about getting the Sun Alert out and what needed to happen for me to get a gate open to get the fix back into the patch gate.

Thanks to Angela, Paul, Brent and Bill for working hard to get to the point that I could log the RTI at 10:10, and start doing the minor nit type stuff that needed to be done before Bill could pass the RTI onto the on10-patch gatekeepers and I could go home (at about 11:15).

As an aside, I missed my train connection by four minutes due to a late running North Shore train and spent an hour sat on a blacked out Hornsby station, getting home at about 2:30am)

While I was traveling (and sleeping) the heads up went out to the gatekeepers and all folk who needed to know about this so that when I came online at 8 this morning, I had very little to do before doing the actual putback into the patch gate (which happened at about 8:30).

The gatekeepers immediately closed the gate and started work on a patch.

The reason that I've detailed what I've been through with this is to point out one thing.

The speed at which I was able to do this and get to the point that an ISR patch will be shortly available publicly, is nothing short of phenomenal. For Sun to respond to and address a vulnerability like this in around 24 hours would have been completely unheard of even two to three years ago.

But it's not just the processes here. What really made for speed here was an incredibly focussed and helpful who had an interested in rapidly getting this addressed. Without the help of folks like Dan, Rodney, Chris, Angela, Paul, Brent, Bill and Seth, and not forgetting the gatekeeping team for pulling out the stops to start building a formal patch, none of this would be possible. If I've missed anyone, please forgive me, I didn't get a lot of sleep last night :)

I love working for a company that has people like this.

update 1

The ISR patches are available for free download from http://sunsolve.sun.com/tpatches. The details of the patches are:

   IDR125457-01 SunOS i386_x86: in.telnetd can call login with an
                option given as a username

   IDR125456-01 in.telnetd can call login with an option given as a
                username

update 2

Sun Alert 102802 is publicly available talking about this issue. Section 4 should shortly be modified to add the following paragraph:

Interim Security Relief (ISRs) are available from http://sunsolve.sun.com/tpatches for the following releases:

SPARC Platform

Solaris 10 IDR125456-01

x86 Platform

Solaris 10 IDR125457-01

Note: This document refers to one or more Interim Security Relief (ISRs) which are designed to address the concerns identified herein. Sun has limited experience with these (ISRs) due to their interim nature. As such, you should only install the (ISRs) on systems meeting the configurations described above. Sun may release full patches at a later date, however, Sun is under no obligation whatsoever to create, release, or distribute any such patch.

Update 3

I've just been informed that the formal patches are release ready and should be released to sunsolve in the next few hours. Keep an eye out for:

120068-02 SunOS 5.10_sparc: in.telnetd Patch
120069-02 SunOS 5.10_x86: in.telnetd Patch

As these are security patches, they will be publicly available.

Technorati Tags: Solaris, OpenSolaris, Sun, Security

Posted at 11:25AM Feb 13, 2007 by Alan Hargreaves in OpenSolaris  |  Comments[16]