Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:36874
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain realplain.com from 209.235.148.23 cause and error)
Message-ID: <200804081558.m38FwmKT028086@mail13c35.nsolutionszone.com>
Message-ID: <016901c89991$6f86c4f0$0201a8c0@pc1>
To: <internals@lists.php.net>
Date: Tue, 8 Apr 2008 10:58:48 -0500
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Subject: [PATCH] Scanner fixes and tweaks
From: php_lists@realplain.com ("Matt Wilmas")

Hi all,

First, I just realized while going through the scanner code after the re2c
changes that there's a bug with heredoc handling (from my code changes last
year, oops) on something like:

<<<EOT
text
fooEOT
text
EOT;

Patch for 5.2 with just that fix:
http://realplain.com/php/heredoc_fix_5_2.diff

For 5.3 and HEAD, I fixed problems with yyleng (some would only affect
highlighting, tokenizer ext, etc.)  Double quoted heredocs with tabs/spaces
wouldn't work right (ex: <<<   "EOT").  Updated some comments.  And this
rule that was added...

<ST_LOOKING_FOR_PROPERTY>{WHITESPACE}+ {
    /* do nothing */
    goto restart;
}

Would lose that whitespace in highlighter, etc. or miss counting line
numbers.  Speaking of line numbers, I also removed the
CG(comment_start_line) variable since I can't see that it's needed anywhere,
and CG(zend_lineno) works for the "Unterminated comment..." warning (which
isn't working right now anyway :-)).

In HEAD I also tidied up the LNUM/HNUM rules which lost their tabs and Z_*
macros a year ago (v1.164).

http://realplain.com/php/scanner_fixes.diff
http://realplain.com/php/scanner_fixes_5_3.diff


- Matt