Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:62871
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain zend.com designates 212.199.177.89 as permitted sender)
Message-ID: <5048C0AE.2060404@zend.com>
Date: Thu, 6 Sep 2012 19:26:38 +0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120828 Thunderbird/15.0
MIME-Version: 1.0
To: <ivan.enderlin@hoa-project.net>
CC: <internals@lists.php.net>
References: <CAF+90c-+Hgcv5B4+eGDS+jEj6oNoyefBWO-M=kvThaVdckz62g@mail.gmail.com> <504836A3.20904@zend.com> <50485647.1060102@hoa-project.net>
In-Reply-To: <50485647.1060102@hoa-project.net>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: quoted-printable
Subject: Re: [PHP-DEV] Moving to an AST-based parsing/compilation process
From: dmitry@zend.com (Dmitry Stogov)

On 09/06/2012 11:52 AM, Ivan Enderlin @ Hoa wrote:
> Hi Dmitry,
>
> On 06/09/12 07:37, Dmitry Stogov wrote:
>> Hi Nikita,
>>
>> Personally, I don't see any reason to build AST. As you mentioned
>> yourself, it will be slower and will require more memory. On the other
>> hand AST itself would allow to perform only very basic optimizations.
>> Most of them can be easily done on VM opcode level as well.
> The lexing and parsing processes will not be slower than actual, and th=
e
> construction of an AST is a new process. Well, as usual, new process
> requires new resources. But if we look further, it will certainly be a
> nice tool to perform better opcode caching, it will remove a lot of
> hacks, it will allow third-part tools working on safeness, security,
> quality etc. to go deeper at low-costs which is very important for PHP
> community, it will facilitate future works (implementations, features
> etc.) for PHP=85 Moreover, it may be possible that compiler compilers h=
ave
> a better lexing and parsing processes than actual?

Few years ago we replaced flex with re2c, and got some speedup because=20
re2c generated scanner used mmap() and didn't check for end of buffer,=20
but after a while we realized that in case the script size is=20
multiplication of PAGE_SIZE the scanner just crashes, so we had to add=20
hacks to workaround :)

> Someone said that AST is more =93academical=94, yes it is and that is w=
hy we
> can benefit from already done researches in this area to avoid memory
> overhead (one among others). We are not the first ones facing this
> problem. It requires some researches before starting to develop this.
> Let's try as a POC and we will quickly see if this is a wrong way or no=
t.

Of course you can try it and in case it's better, faster and/or more=20
clear it may be accepted. Currently, PHP allows overriding of=20
"zend_compile" routine, so as a first step you even may implement it as=20
a PHP extension without ZE modification.

Thanks. Dmitry.

> Cheers.
>
>
>>
>> Also, as it's not an easy task, the old "ugly hacks" will be replaced
>> with new mistakes, which would require new "hacks" in the future :)
>>
>> The only real advantage could be an ability to expose AST to PHP
>> scripts, but only few people may need it.
>>
>> Thanks. Dmitry.
>>
>> On 09/04/2012 11:57 PM, Nikita Popov wrote:
>>> Hey folks!
>>>
>>> Some people asked me what the advantages of using an AST-based
>>> parsing/compilation process are, so I put together a few quick notes
>>> in an RFC:
>>>
>>> https://wiki.php.net/rfc/ast_based_parsing_compilation_process
>>>
>>> It would be nice to get a few comments from other core devs on this.
>>>
>>> Nikita
>>>
>>
>>
>