Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:43811
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain googlemail.com designates 209.85.218.161 as permitted sender)
DomainKey-Status: bad
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=mime-version:reply-to:in-reply-to:references:from:date:message-id
         :subject:to:cc:content-type:content-transfer-encoding;
        b=X9zbKnaE2+/c5yIPvD2P1DuzsPgtVAlWeiUSppkTa+BhzmyA3rRWuqArXy6EEvKPN9
         9ePIosBX7A9rNuyrT48N7ebTpvaKKsifhCpgb0Yo6MNAjfnKKGIH9S79a2dpmNeYlQbj
         Lig62/L3H9cgghsqgaPDSED5viiVZOmu9Ksnw=
MIME-Version: 1.0
Reply-To: RQuadling@googlemail.com
In-Reply-To: <49F94BC6.5060904@zend.com>
References: <6604D94D40FD465F992144110B075BB5@pc1> <49F94BC6.5060904@zend.com>
Date: Thu, 30 Apr 2009 08:48:01 +0100
Message-ID: <10845a340904300048w328591ccl62ebe069ba6f92b7@mail.gmail.com>
To: Dmitry Stogov <dmitry@zend.com>
Cc: Matt Wilmas <php_lists@realplain.com>, internals@lists.php.net, shire@php.net
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: [PHP-DEV] Re: [PATCH] Scanner "diet" with fixes, etc.
From: rquadling@googlemail.com (Richard Quadling)

2009/4/30 Dmitry Stogov <dmitry@zend.com>:
> Hi Matt,
>
> Does this patch fix EOF handling issues related to mmap()? (e.g. parsing =
of
> files with size 4096, 8192, ...). Now we have two dirty fixes to handle t=
hem
> correctly.
>
> The patch is quite big to understand it quickly. I'll probably take a loo=
k
> on weekend.
>
> -ANY_CHAR [^\x00]
> +ANY_CHAR [^]
>
> Is [^] a correct regular expression?
>
> Thanks. Dmitry.
>
> Matt Wilmas wrote:
>>
>> Hi Dmitry, Brian, all,
>>
>> Here's a scanner patch that I mentioned awhile ago, with a possible way =
to
>> work around the re2c EOF handling issues.
>>
>> The primary change is to do a "manual scan" like I talked about in areas
>> that match large amounts and can contain NULL bytes (strings/comments, w=
hich
>> are now scanned faster too), as is done for inline HTML. =C2=A0I called =
it a
>> "diet" :-) because it removes my complicated string regex patterns from =
a
>> couple years ago, which doesn't make the .l file much smaller after addi=
ng
>> the manual scan code (easier to understand...?), but it does result in a
>> ~34k reduction of 5.3's generated .c file...
>>
>> This fixes Bug #46817, as well as a better, more proper fix for the olde=
r
>> Bug #42767, both related to ending comments.
>>
>> Now inline HTML chunks aren't broken up when a tag starting with "s" is
>> encountered (<script> for JS, <span>, etc.), since it's unlikely to be a
>> long PHP <script> tag.
>>
>> If an opening PHP <SCRIPT> tag was used with a capital "S", it was misse=
d
>> if it wasn't the first thing scanned:
>>
>> var_dump(token_get_all("HTML... <SCRIPT language=3Dphp>"));
>>
>> Single-line comments with a Windows newline didn't include the full \r\n=
:
>>
>> var_dump(token_get_all("<?php // Comment\r\n?>"));
>>
>> Finally, part of the optimized scanning is that, for double quoted
>> strings, when the first variable is encountered (making it non-constant)=
,
>> the amount that's been scanned up to that point is remembered, which can
>> then be skipped over (up to the variable) after returning the quote toke=
n.
>> Previously that initial part of the string was rescanned -- the cost
>> dependent on how far "into" the string the first var is.
>>
>>
>> I think that's about all -- =C2=A0I'll send another message if I forgot =
to
>> mention anything... =C2=A0Just wanted to send this along quick for to yo=
u guys to
>> look at or whatever. =C2=A0It was basically done last week, I just had t=
o do a
>> couple finishing touches and verify that everything was OK.
>>
>> http://realplain.com/php/scanner_diet.diff (Merged changes, but didn't
>> test yet.)
>> http://realplain.com/php/scanner_diet_5_3.diff
>>
>>
>> Thanks,
>> Matt
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

Hmm. RegexBuddy explains that as ...

Match any character that is NOT a =E2=80=9C]=E2=80=9D

But it sure doesn't look valid without a closing ].



--=20
-----
Richard Quadling
Zend Certified Engineer : http://zend.com/zce.php?c=3DZEND002498&r=3D213474=
731
"Standing on the shoulders of some very clever giants!"