Random ramblings

Friday October 12, 2007

Email regex address validation (aka "Web Developers and Clue, the Empty Set?")

Why, oh why, must I regularly be told by online registration forms that, sorry, but the email address I entered is not valid and I should enter a valid one before allowing registration? There's so many possible problems here:

1. The web developer just assumed they know what an email address is
2. The web developer knows what an email address is, but got the regex wrong
3. The web developer knows about regexes, but tried to be too clever
4. Syntactic validation still says nothing about functional validity

The most common cause is the first. To describe them as clueless muppets who should never be allowed to code up anything that remotely stands a chance of being something anyone else must ever interact with, would probably be too strong and not generally true, but it's close enough for me for now! . Really, if you're writing some web form that requires you to sanity-check an email address, how hard could it be to google for "email validation regex"? If they really were keen on doing it themselves, surely reading the addr-spec BNF would be a pre-requisite? A quick glance at least, surely? Sadly, it appears many (most?) web developers are incapable of such lofty levels of rigour. Worse, it appears some of these idiots have webpages with their incorrect, broken regexes ranked high in the google results.

The next problems are that it's easy to get regexes wrong, semantically at least, and it's also possible to get too clever. A good example is Stephen Shirley's email regex effort (javascript compatible). It's slightly wrong in not allowing foo@host addresses, a small oversight. It's also locale dependent in two ways. Firstly, 'a-z' can match non-ASCII characters in some locales (ie chars Stephen didn't intend to match). Secondly, Stephen's gotten too clever: the range match very sneakily relies on ASCII ordering - cute, but locale-fragile.

Before addressing the last possible problem, here's my simpler rendition of Stephen's regex, fixed to be locale-independent, split over multiple lines for readability, JavaScript compatible:

(")?([[:alnum:]!#$%&'*+/=?^_`{|}~-])+
(\.[[:alnum:]!#$%&'*+/=?^_`{|}~-]+)*(")?
@[[:alnum:]-]+(\.[[:alnum:]-])*\.?

I've checked this against RFC2822 and the ECMAScript specification for validity. Stephen's also had a look at it. For PHP, the following should work, using ereg:

  $regex  = "(\")?[[:alnum:]!#$%&'*+/=?^_`{|}~-]+";
  $regex .= "(\.[[:alnum:]!#$%&'*+/=?^_`{|}~-]+)*(\")?";
  $regex .= "@[[:alnum:]-]+(\.[[:alnum:]-])*\.?";
  
  if (ereg ($regex, $argv[1]))
    ....
  else
    ....

However, neither of the above are guaranteed to be correct. It's easy, in trying to fully validate the syntax, to miss some subtleties of syntax or meaning (in what you're trying to validate, or in the form you're describing that syntax in). This suggests it's a bad idea to try..

Which brings us to the next problem: Even if one manages to correctly positively validate the syntax, the address need not be functional. The system must still validate the email address, typically with a probe-email from which the registrant must retrieve a URL to complete the registration. Only then can the system know the email really is valid. Further, the developer can not rely on the client-side Javascript to have been run, or not have been subverted - they must sanitise the address server side too. So the syntax checking really only is for the convenience of the (registrant), to prevent them aimlessly waiting for an email that might never come if they've typo-ed their address (and typo-ed it in such a way as to be syntactically invalid!). So really, given the difficulty of getting it right, given that it's for the user's convenience and given the system likely will functionally test the address anyway, the syntax check should not be mandatory!

In conclusion:

• Less is more. The fewer restrictions on form, the less likely you'll get it wrong.
• If you're going to test for functional validity anyway, consider letting that count for syntactic validity (you're unlikely to crash your MTA with a bad address).
• Sanity check the address by all means, and give the user a hint if you think they may have made a mistake, but don't try to enforce your own (potentially incorrect) syntax check on user-supplied email addresses!
• Obviously, don't try to syntax check addresses server side either (you still have to take care about properly quoting, encapsulating and/or escaping addresses, particularly for a string that will be handed to some SQL or shell parser..)
• Don't try to be clever with regexes, you might get it wrong
• If, despite my rant above, you absolutely must syntactically validate email addresses on web registration forms, please use the regex above. It's probably still wrong, but it's less wrong than any other email address regular expressions I've found.

Update: Removed the begin and end anchor matches - whether they're appropriate depends on context of input - and added PHP example.

Update2: Add support for quoted-string, and add hyphen to domain parts, as per comments

Update3: See discussion for further corrections (which should re-inforce how bad an idea it is to try enforce syntax checks arbitrarily..).