Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:55453 Return-Path: Mailing-List: contact internals-help@lists.php.net; run by ezmlm Delivered-To: mailing list internals@lists.php.net Received: (qmail 55285 invoked from network); 15 Sep 2011 14:51:31 -0000 Received: from unknown (HELO lists.php.net) (127.0.0.1) by localhost with SMTP; 15 Sep 2011 14:51:31 -0000 Authentication-Results: pb1.pair.com smtp.mail=nikita.ppv@googlemail.com; spf=pass; sender-id=pass Authentication-Results: pb1.pair.com header.from=nikita.ppv@googlemail.com; sender-id=pass Received-SPF: pass (pb1.pair.com: domain googlemail.com designates 209.85.215.54 as permitted sender) X-PHP-List-Original-Sender: nikita.ppv@googlemail.com X-Host-Fingerprint: 209.85.215.54 mail-ew0-f54.google.com Received: from [209.85.215.54] ([209.85.215.54:54425] helo=mail-ew0-f54.google.com) by pb1.pair.com (ecelerity 2.1.1.9-wez r(12769M)) with ESMTP id A8/D9-29375-2F0127E4 for ; Thu, 15 Sep 2011 10:51:30 -0400 Received: by ewy1 with SMTP id 1so1906858ewy.13 for ; Thu, 15 Sep 2011 07:51:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RZD6pnj48l/mz+BZ6Ai5TgMV+0bH/QVn9BBmQJ7mJYE=; b=FggHUzQQrjz9G9Ap428uL8EdviSpz3bduQk826qrGairgLTqzhmdT0g9KM7pc1tSGM jlCkZocms29DAPWcwlB/87l4hGdBbxmQAZhceYOsnrInerTwejeCivHymLrSRCsqcHZ8 e8Nsss/Tkwva5PuWgOxBSvh06USeuAeW43nVs= MIME-Version: 1.0 Received: by 10.14.18.69 with SMTP id k45mr421055eek.78.1316098286157; Thu, 15 Sep 2011 07:51:26 -0700 (PDT) Received: by 10.14.99.205 with HTTP; Thu, 15 Sep 2011 07:51:25 -0700 (PDT) In-Reply-To: References: <4E6FB55E.4060906@oracle.com> Date: Thu, 15 Sep 2011 16:51:25 +0200 Message-ID: To: Hannes Magnusson Cc: Ferenc Kovacs , Christopher Jones , internals@lists.php.net Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: [PHP-DEV] Revert Tokenizer behavior for 5.4 From: nikita.ppv@googlemail.com (Nikita Popov) Hi Hannes, thanks for your response! On Thu, Sep 15, 2011 at 9:22 AM, Hannes Magnusson wrote: > the change to ext/tokenizer/tests/token_get_all_variation16.phpt seems > really weird? I changed the test to not include a __halt_compiler statement anymore, as it didn't make sense in this context, and included a function call instead. The file would have been just a parse error, as __halt_compiler can only occur in the outermost scope or in namespace declaration score. Additionally the behavior of token_get_all on __halt_compiler is already tested in the other test (bug54089.phpt), so I though it wasn't necessary to test it there again. > And I'm unsure how this should actually work. > > A file containing: > > __halt_compiler() > > ?> > stuff > > Will return "stuff" after this patch.. that doesn't seem expected. First of all: The second patch (the _full one) isn't that important to me, as you can fetch the data after __halt_compiler(); quite easily in userland code, too (concat all tokens and calculate the offset from that, then substr it from the code). So if you think it doesn't make sense, I would be okay without it. The original reasoning was, that fetching the contents after __halt_compiler(); is the most probable thing you want to do if you are using token_get_all on a file with __halt_compiler. But I'm not sure that it is true and can imagine that in many cases you write scripts that don't really care about __halt_compiler statements and the additional T_INLINE_HTML would only hurt. Maybe Nicolas can provide further input on this. Thanks for looking at the patch. If you have further questions, please ask :) Nikita PS: I updated both patches to fix an issue with which tokens are considered dropped. I now explicitly state them in the if-statement, as the `destroy` variable wasn't quite the same (it also included the tokens that are translated into another token.)