Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:90585
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: softfail (pb1.pair.com: domain golemon.com does not designate 209.85.215.41 as permitted sender)
MIME-Version: 1.0
Sender: php@golemon.com
In-Reply-To: <56958BA2.1040504@gmail.com>
References: <CAESVnVoJBRF3VfKrnWGNCah3KAbxwjacPAwR1HZUC-T-vN=KDQ@mail.gmail.com>
	<77.F6.12097.81D2C865@pb1.pair.com>
	<B0800012-C341-468D-AE1C-BF6551AB356F@rouvenwessling.de>
	<CAESVnVobg-woJ2eHff_RSFkkMPZLyqOia1U+agSSTNxCjYc_tg@mail.gmail.com>
	<56958BA2.1040504@gmail.com>
Date: Tue, 12 Jan 2016 16:54:28 -0800
Message-ID: <CAESVnVoALYn73iFRAeyM0g74BJ=MjoiZn4TLMqGcKbfrr71Q+Q@mail.gmail.com>
To: Stanislav Malyshev <smalyshev@gmail.com>
Cc: =?UTF-8?Q?Rouven_We=C3=9Fling?= <me@rouvenwessling.de>, 
	Andrea Faulds <ajf@ajf.me>, PHP internals <internals@lists.php.net>
Content-Type: text/plain; charset=UTF-8
Subject: Re: [PHP-DEV] Re: [RFC] Normalize token_get_all() output (with flag)
From: pollita@php.net (Sara Golemon)

On Tue, Jan 12, 2016 at 3:26 PM, Stanislav Malyshev <smalyshev@gmail.com> wrote:
>> How does this sound?
>> 1. Keep the current RFC basically as is.  It's a minor addition to
>> token_get_all() which can be slotted into existing code to improve
>> readability, but offers little advantage beyond that.
>> 2. Make a new extension to prototype this PhpToken class outside of
>> the php-src tree, add all the extra goodies we want (array of token
>> return, iterator of token return, extra info, limited info, etc...)
>> 3. When this extension is good and solid, propose merging into core.
>
> Sounds great! I wonder if it'd be possible to make a real streaming
> iterator? That would be so nice.
>
I can't imagine why not.  The scanner lends itself perfectly to the
Iterator model and would end up making a much bigger dent in the
memory consumption than dithering between arrays and classes (though
"why not both" applies here). :)

Bouncing between several tasks atm, but I'll try to get something
started and invite some folks to the repo.

-Sara