Dave's Bit Bucket

Thursday July 26, 2007

Automated Lip Reading

I watched a fascinating documentary on television the other night, which not only revealed some interesting information on an infamous historical figure, but also raised the curtain on a technology I'd expect to see much greater deployment of in the near future.

Please bear with me, and I'll describe the whole thing in context.

Between 1941 and 1944, Eva Braun - who had received some training in cine camera work and cinematography - shot a whole bunch of silent colour footage of the life that she and Adolf Hitler shared at the Berghof, the near-equally infamous guests they entertained, etc. This footage was discovered in 1945 by the OSS, who were looking for any evidence that would be useful to the prosecution in the Nuremburg trials. As the films were silent, they were considered to be of no evidential value and have remained, for decades, a simple historical curiosity.

A profoundly deaf computer scientist - who is, coincidentally, German - determined some years ago that computers could potentially use image analysis to lip-read, using the techniques that he employs day to day. He developed image analysis software which can, with a high degree of accuracy, map mouth, jaw and throat movements from captured video onto a computer-modelled head, and thus to phonemes. The software - which is currently optimised for German - is able to not only make such a mapping when the captured footage presents a subject face-on to the camera, but also when the subject has their face at anything up to 120 degrees from face-on.

Thus, with a little training (the details of which were unfortunately glossed-over, but which appeared to mostly involve identifying which area of the captured video represented a speaker's mouth), the software was able to translate the lip movements of the filmed subjects into speech components. Using the vocal talent of a number of German actors - one of whom was judged to impersonate Hitler particularly closely, based on the one covert audio recording of Hitler ever made, and the only recording of him in conversation, rather than giving a public speech - a vocal track was made to accompany the Berghof footage, and the two were dubbed together.

This was shown in the documentary, with English subtitles.

Now, consider the ramifications of automated lip reading (ALR) as applied in other contexts. If facial recognition software is able to not only identify a face but its components, such that it could pass details of mouth location to ALR software, if could become possible to reconstruct speech (and potentially and eventually, text) from high-resolution CCTV footage.

Given enough computing power, this could potentially be done for every face in a crowd.

Of course, caveats apply. German is a particularly clearly-enunciated language where every syllable is sounded, has a relatively small number of consistent pronounciation rules, and does not have variations in meaning associated with tonality. Tonal languages such as Thai, where a given small phonemal sequence enunciated in a tenor voice can mean something entirely different to the same phonemal sequence enunciated baritone, would most likely still require a skilled human interpreter to give meaning to the sound that ALR output would have, unless ALR is expected to eventually be able to interpret such things based on analysis of apparent constrictions in the footage of the speaker's throat.

Nonetheless, it's interesting...

Update: The technology on show was developed by Frank Hubner; I've also found a paper on automated head-recognition algorithms, designed specifically to facilitate mouth area identification for automated lip reading, here. Seems there's more folk working on this than initially meets the eye - it's a space to watch.

(2007-07-26 07:53:19.0) Permalink Comments [1]

"Integrity Checking in Depth"

Over the last few weeks, a number of folk have asked me about various elements of integrity checking and intrusion detection - so I figured it would be useful to aggregate my thoughts here.

First, I distinguish between integrity checking and intrusion detection, on the grounds that intrusion detection needs to be realtime whereas integrity checking does not; you can run BART or Tripwire a couple of times a day to gather reports of changes (this being the maximum recommended frequency, incidentally, as doing a full sweep of a system consumes a reasonable amount of system resource), whereas you'd want Prelude (or whatever IDS you're running) to scream at you in the form of an SNMP trap the moment it sees some activity it's not happy with.

With Solaris 10, you actually get the luxury of multiple methods of integrity checking; the title of this article is an allusion to the popular and valuable concept of "Defence in Depth", which is reflected in the ways in which you can make integrity checking work. The ways in which these integrity checking mechanisms relate to and complement eachother is subtle, and thus worth documenting. Out of the box, you get elfsign and BART, and with a little Internet connectivity, you can also call upon the services of the Solaris Fingerprint Database (SFpDB).

Today, elfsign verify <pathname> can tell you whether a given binary was genuinely released by Solaris Release Engineering as part of a production build of Solaris 10, or a subsequent Sun-issued patch to it. What it won't tell you, is if the binary is the current version, patch-wise; it also won't tell you anything about the config or change status of files which aren't ELF binaries, nor will it tell you about binaries bundled with Solaris which are part of specific third-party contributions (these don't get elfsigned by us, as from a legal perspective, signing a binary provided by a third party involves modifying that binary by changing its ELF header, thus creating a derivative work).

It is also worth noting that elfsign is a point of circular dependency. While anyone trying to trojan a Solaris 10 environment cannot get an elfsign-verifiable signature for their trojan binaries, if they were able - as a result of their attack mechanism - to trojan elfsign too, then it's Game Over. This is yet another good reason to deploy network-listening applications in sparse-root zones wherever possible, so that the elfsign binary remains usable but immutable from the application environment's perspective.

BART is a tool I typically describe - with tongue half in cheek - as "a poor man's Tripwire". While Glenn has produced a useful Blueprint on how to scale BART management, I still honestly think that, if you want to integrity-check a datacentre's-worth of installed systems, Tripwire is the superior product - this is owing to its cross-platform nature and the elegance of its key management. What both BART and Tripwire will tell you is, whether the file you had yesterday is the same as the file you have today. Both BART and Tripwire are agnostic regarding the file type, therefore they are applicable to scripts and config files as well as ELF binaries. However, they won't tell you where a file came from, nor what version it is. However, if someone manages to install a Trojan - or downgrade a Sun-issued and signed binary to a version which has an exploitable vulnerability which has subsequently been patched - either BART or Tripwire can potentially catch things which elfsign won't. The database of file digests which Tripwire holds on a system is also more resistant to attack than the equivalent BART database - the Tripwire db being signed with a key which is held on the Tripwire management infrastructure, rather than the host itself. Again, deploying network-accessible applications in sparse-root zones mitigates risk - most system binaries are loopback-mounted readonly in such environments, making them immutable even in the event of application compromise.

Last but not least, we have the Solaris Fingerprint Database. This contains comprehensive mappings between pathnames, MD5 digests and release / patch version numbers for scripts and binaries back to very early versions of Solaris; if you present a pathname and digest to it, it can tell you not only whether the associated file matches something produced by Solaris Release Engineering, but what release or patch version it is associated with. This maps to the functionality of elfsign plus the ability (when correlated with current patch levels) to spot binary version downgrade attacks, however Internet connectivity (direct or indirect) is required, and while it would be difficult to Trojan the MD5 digest generator to produce output which would match what the fingerprint database considered acceptable, it is not impossible (although sparse-root zones can help here, again).

So, there you have it - while any of the integrity verification tools above could potentially be compromised when used in isolation, owing to points of circular dependency (although to do so would be hard in all cases), a reinforcing combination of them plus the use of sparse-root zones results in an integrity enforcing and verification capability which would cause even a very capable attacker to have nightmares.

Glenn also has a Blueprint on integrating SFpDB and BART, which makes for good reading.

(2007-07-26 07:13:01.0) Permalink Comments [0]

Blindside

If you're not yet familiar with http://www.blindside.org.uk/, it's a blog / wiki site on which some interesting folk are recording some very forward thinking. The remit of Blindside is basically "take UK society in general, add technology (taking emerging technologies into consideration), and figure out what issues Government will face as a result, in 5 years' time or so".

Blindside is sponsored by the CSIA, and the Blindside folk are required to report their findings to CSIA on a regular basis. While the Blindside folk have a very large amount of experience, they acknowledge that they can't be experts in everything - so, particularly for the Wiki items, they encourage experts in the relevant fields to contribute to, and correct errors in, their records.

So, the bottom line is: if you care about UK Government's approach to technology, particularly technologies you are well-versed in, go to Blindside, read what's there, be intellectually stimulated and contribute your own wisdom, when you're not busy with your day job.

Thank you.

(2007-07-26 02:33:30.0) Permalink Comments [0]

Wednesday July 11, 2007

"An Enterprise Needs Only Four Computers"

With a tip of the hat to Greg P's posting here (and I thought the quote was attributed to Ken Olsen rather than Thomas J Watson...), I'd like to consider the individual enterprise, and propose that an enterprise only needs four computers.

The four computers in question would comprise two clusterable-or-otherwise-resilient systems in a primary datacentre, and two clusterable-or-otherwise-resilient systems in a business continuity or disaster recovery location. Each of these systems would most likely be populated with some x86 / x64-based boards and some SPARC-based boards. The boards would be grouped into physical domains (where provable, rather than merely certifiable, data segregation is required and covert channels are perceived to be a potential issue if other technologies are used), and physical domains would be sliced up into LDOMs on the SPARC side and (most likely) Xen domains on the x86 / x64 side, where either certifiable data segregation is required or OS-level admins need full independent control of their OS instance (eg, for running different OS versions or different patch levels, etc). Applications would live in zones.

User home directories would need their own OS instance, up until the point where a non-global zone is able to function as an NFS server. Doubling the home directory servers up as Sun Ray servers may also be sensible.

There's enough flexibility and granularity here, given sufficient attached storage, to run wellnigh any enterprise, if the computers are big enough. In fact, I suspect that a great many medium-sized enterprises could probably get by on four fully-loaded Sun Blade 6000 systems, at a pinch.

Now, I love reductio ad absurdum as a device of reasoning - I find that, sometimes, what pops out at the end of such a chain of reasoning may not be as absurd as first expected :-).

The reasons why you don't see enterprises running this way yet are, for the most part, human rather than technological. In particular, admins often feel uncomfortable about the fact that their envronment can potentially be affected by a "higher authority".

To give some concrete examples, if systems belonging to departments A and B in an organisation are consolidated into a pair of zones on a single host, each zone admin would have initial oncerns about who owned root in the global zone. Even if the systems were consolidated into logical or physical domains on a sizeable system, each domain admin would have initial concerns about who had root on the domain controller. Granted, such issues would very likely be alleviated in policy, but at the end of the day you'll still see some circumstances where admins, accreditors and auditors have issues about not being the ultimate authority with final control over a system.

This is why you're not likely to see an enterprise running on just four computers, just yet.

(2007-07-11 00:52:03.0) Permalink Comments [0]