swift234

Fix a few issues with parsing bulid configuration directive

* replaced yet another variant of isWhitespace with the version from
  clang/Basic/CharInfo.h.  The major difference is that our variant used to
  consider '\0' whitespace.

* made sure that we don't construct StringRefs that point after the end of the
  buffer.  If the buffer ends with "#", then MemoryBuffer will only guarantee
  that there is one additional NUL character.  memcmp(), OTOH, is allowed to
  access the complete span of the provided memory.  I colud not actually get
  this to crash on OSX 10.10, but I do remember similar crashes we fixed in Clang.

* added checks to reject extra tokens at the end of the build configuration
  directive -- see tests, that code used to compile without diagnostics.  The
  lexer tried to do this, but in a self-referential way -- by checking the
  NextToken variable (which is actually the previous token, when viewed from
  the point of lexImpl()).  The checks I added are a little too strict, they
  reject comments at the end of the directive, but at least we don't accept
  strange constructs.  Allowing comments would not be hard, just requires
  factoring out lexer's routines to skip comments so that they accept a pointer
  to the buffer and return the comment end point.  Filed
    <rdar://problem/16301704> Allow comments at the end of bulid configuration directives
  for that.

Found by inspection... I was grepping the codebase for 'isWhitespace'.


Swift SVN r14959