Paul Roberts' Musings

Monday February 05, 2007

File descriptors and setuid applications

A recent (-ish) post on BugTraq by the XFOCUS team spawned some discussion internally. Prior to publication of this issue, XFOCUS contacted us folks in the security vulnerability team for a response, which we provided.

It seems to us that this report is highlighting a general warning about a programming error which can lead to security vulnerabilities. When a Solaris process (and those within other Unix's) launches another application (e.g. via exec(2) or one of its relations) the new process image inherits the previous image's open file descriptors. In Solaris, this fact does not change when a setuid application is launched. Because a user has control of the process from which he or she launches setuid applications, that user can therefore control the file descriptors that are inherited by the setuid application. As a result, this puts a burden on the setuid program, demanding that it use its inherited file descriptors in a safe fashion so as not to allow this situation to be exploited (and this issue is mentioned in a set of secure programming guidelines distributed within Sun that is hopefully read by engineers who are creating or fixing setuid binaries).

The exploit contained within the SecurityFocus post takes the form of a setuid binary which is a manufactured example of this kind of programming error. It writes directly to file descriptor 2, assuming that it's the usual stderr, but of course this assumption is not safe, and because the calling process closes file descriptor 2 before exec-ing the setuid program, the open(2) that occurs earlier within the setuid binary will result in that file being opened as file descriptor 2, causing the setuid application to write to that file incorrectly, at the behest of the user controlled exec-ing process.

Hopefully these programming errors are not prevalent in Sun's code (but of course if there were no vulnerabilities we wouldn't need a vulnerability team), and searching for these kinds of errors is part of the general code sweeping and review processes that go on at Sun. Certainly if a specific instance of this problem is found in an app shipped by Sun, then that is an issue that Sun would work on fixing (by patching the setuid application) and publishing in the form of a Sun Alert as soon as possible (and our team always welcomes disclosure of such issues via our normal mail address), but the security vulnerability team does not currently have plans for a direct response to the general warning published by XFOCUS.