Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:91822
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.161.174 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <0b5101d183a1$5b38fdd0$11aaf970$@belski.net>
References: <55B0CADD.8000807@gmx.de>
	<CAEZPtU7aPzPQ=DL5fn9fOEuU_hySm5WXUkbjFw+Sq0T5xNfb3A@mail.gmail.com>
	<55B136BE.8010006@gmx.de>
	<046401d0c644$9cf76880$d6e63980$@belski.net>
	<55B2B78A.4080202@gmx.de>
	<048001d0c665$ed19ec40$c74dc4c0$@belski.net>
	<55B2D779.9090409@gmx.de>
	<04d301d0c6e5$6b28f0c0$417ad240$@belski.net>
	<55B3B2E6.7080905@gmx.de>
	<056e01d0c7d9$b1edd990$15c98cb0$@belski.net>
	<55B559C7.9060304@gmx.de>
	<064301d0c85f$5260ac60$f7220520$@belski.net>
	<55C008B1.2050008@gmx.de>
	<05ec01d0ce97$4ee7c4e0$ecb74ea0$@belski.net>
	<55C09EE2.7010400@gmx.de>
	<063f01d0cec2$91b9baf0$b52d30d0$@belski.net>
	<55C0F908.2050708@gmx.de>
	<068e01d0cefb$6ffba3e0$4ff2eba0$@belski.net>
	<A428AF35-A176-404D-888C-7541100155B2@heroku.com>
	<016e01d17ef7$cd8a2660$689e7320$@belski.net>
	<25D67B35-88FF-4A5C-9421-BAF482B32A96@heroku.com>
	<036501d17fc3$d51357e0$7f3a07a0$@belski.net>
	<5E36AA30-831F-44BE-A148-8720B8E87E96@heroku.com>
	<94A89B0F-48BB-4DD8-A26B-3B6D2C8C2C9F@heroku.com>
	<7DF19331-9967-47C8-B21D-260C619BBD5E@heroku.com>
	<0b5101d183a1$5b38fdd0$11aaf970$@belski.net>
Date: Mon, 21 Mar 2016 19:54:17 +0100
Message-ID: <CAF+90c_Dt9Hacrt6364wiuxqJUpRr5=nMB_YWT8EQGiGrSE8FQ@mail.gmail.com>
To: Anatol Belski <anatol.php@belski.net>
Cc: David Zuelke <dz@heroku.com>, Christoph Becker <cmbecker69@gmx.de>, Pierre Joye <pierre.php@gmail.com>, 
	PHP internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary=001a114e5260ac58af052e939f2a
Subject: Re: [PHP-DEV] PCRE JIT stack size limit
From: nikita.ppv@gmail.com (Nikita Popov)

--001a114e5260ac58af052e939f2a
Content-Type: text/plain; charset=UTF-8

On Mon, Mar 21, 2016 at 7:41 PM, Anatol Belski <anatol.php@belski.net>
wrote:

> Hi,
>
> > -----Original Message-----
> > From: David Zuelke [mailto:dz@heroku.com]
> > Sent: Monday, March 21, 2016 12:30 AM
> > To: Anatol Belski <anatol.php@belski.net>
> > Cc: Christoph Becker <cmbecker69@gmx.de>; Pierre Joye
> > <pierre.php@gmail.com>; PHP internals <internals@lists.php.net>
> > Subject: Re: [PHP-DEV] PCRE JIT stack size limit
> >
> >
> > > On 20.03.2016, at 22:41, Anatol Belski <anatol.php@belski.net> wrote:
> > >
> > > Hi,
> > >
> > >> -----Original Message-----
> > >> From: David Zuelke [mailto:dz@heroku.com]
> > >> Sent: Sunday, March 20, 2016 10:10 PM
> > >> To: Anatol Belski <anatol.php@belski.net>
> > >> Cc: Christoph Becker <cmbecker69@gmx.de>; Pierre Joye
> > >> <pierre.php@gmail.com>; PHP internals <internals@lists.php.net>
> > >> Subject: Re: [PHP-DEV] PCRE JIT stack size limit
> > >>
> > >>> On 17.03.2016, at 05:22, David Zuelke <dz@heroku.com> wrote:
> > >>>
> > >>>> On 16.03.2016, at 21:38, Anatol Belski <anatol.php@belski.net>
> wrote:
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: David Zuelke [mailto:dz@heroku.com]
> > >>>>> Sent: Tuesday, March 15, 2016 11:58 PM
> > >>>>> To: Anatol Belski <anatol.php@belski.net>
> > >>>>> Cc: Christoph Becker <cmbecker69@gmx.de>; Pierre Joye
> > >>>>> <pierre.php@gmail.com>; PHP internals <internals@lists.php.net>
> > >>>>> Subject: Re: [PHP-DEV] PCRE JIT stack size limit
> > >>>>>
> > >>>>> Sure. So composer creates patterns to parse a file dynamically; I
> > >>>>> have not figured out why it only happens for some composer.json
> > >>>>> files (or maybe it happens for all of them and I just have not
> > >>>>> noticed), but this example here I extracted from a reproducibly
> > >>>>> failing "composer require" run; it runs into the JIT stack size
> > >>>>> limit
> > > on PHP 7
> > >> while it works fine on 5.x:
> > >>>>>
> > >>>>> https://gist.github.com/dzuelke/cc64a630c14416eda3e9
> > >>>>>
> > >>>> I've just tried this on Debian Jessie and on Windows with the
> > >>>> latest
> > > 7.0.x
> > >> builds, in both cases the error is PREG_BACKTRACK_LIMIT_ERROR. I'd be
> > >> next asking you to check the configuration. Runtime like system or
> > >> ini, or
> > > build config
> > >> can affect this. Anyway, I don't see an issue with PCRE JIT at the
> moment.
> > >>>
> > >>> Sorry, I simplified the example too far. Updated it again to throw a
> > >>> JIT
> > > stack
> > >> limit error again (but works fine with "php -dpcre.jit=0"):
> > >>>
> > >>> https://gist.github.com/dzuelke/cc64a630c14416eda3e9/de7b7293798ac57
> > >>> b4
> > >>> f75d822dc22a625be0fe9df
> > >>
> > >> Were you able to reproduce this? :)
> > >>
> > > Yes, it is reproducible. I was just investigating on another issue
> > > https://bugs.exim.org/show_bug.cgi?id=1803 which now seems unrelated.
> > > With the JIT stack - we can try to use a custom stack to increase its
> > > size by allocating a custom jit stack of an increased sizeas a
> > > workaround. If that works, it should be fine. I'll be working on a
> > > patch next week, then we can see this option makes sense. More on the
> > > topic is readable here
> > http://www.pcre.org/original/doc/html/pcrejit.html#stackcontrol .
> >
> > That sounds good. I suspect a significantly larger stack size than the
> PCRE
> > default of 32k will do the job for most cases. Arguably, what Composer
> uses
> > there is not a trivial pattern with all the nesting going on, but I was
> unable to
> > simplify it when I took a few minutes the other day, and while the
> > composer.json where the bug occurs is generated by code, I don't think it
> > qualifies as an extreme case.
> >
> I've just pushed a fix for this issue, could you please check? A global
> stack space
> Is used in a thread safe manner. I was first setting max to 256K, then
> decreased it
> To 64K for now. My tests on Linux and Windows show your snippet passing,
> Travis is good as well.
>
> > I guess though that some day, someone with a more complex pattern and/or
> > subject string will come with the same problem and say "this fails for
> me,
> the
> > 256k stack limit you set in PHP is not enough", so maybe an INI option
> should
> > exist? Although that's not terribly useful if one program needs it for
> one
> pattern.
> > Flags don't work well for this (unless it's e.g. a numeric one indicating
> multiples
> > of 128k or something, but that's also terrible).
> >
> The issue I have with more increasing the stack is that - while as we see
> till today
> the default stack size of 32K is enough in the common case, the max of it
> will be
> reserved. So the real usage will most likely stay under 32K, but the whole
> of max
> (say 64K now) will be unusable by anything else.
>

Are you sure about that? The documentation says "The arguments are a
starting size for the stack, and a maximum size to which it is allowed to
grow." Unless I'm misunderstanding something, it should not be allocating
the maximum size right away.

Nikita


> > Or change the pcre_match (et al) implementations to retry without JIT if
> the
> > error occurs?
> >
> Yes, this were thinkable and doable within the scope of 7.0. A new INI
> option were
> however something for 7.1. The common case, at least till now, shows we
> beat
> more than 99% of PCRE usage with no compatibility issue. I'd plea
> therefore,
> to
> keep this simple for now and see whether the real life brings more issues
> with
> common projects.
>
> Regards
>
> Anatol
>
>
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

--001a114e5260ac58af052e939f2a--