# PCRE jit bug with UTF-8 and lookbehind assertion - Thread: https://externals.io/message/91243 - Original archive: https://news-web.php.net/php.internals/91243 - Started by Christian Schneider on 2016-02-17T13:06:41+00:00 - Messages: 10 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #91243 — Christian Schneider — 2016-02-17T13:06:41+00:00 URL: https://externals.io/message/91243#91243 Hi there, we just ran into a version of the bug "JIT bug with lookbehind assertion": https://bugs.exim.org/show_bug.cgi?id=1189 To reproduce it you can use php -n -r 'ini_set("pcre.jit", 0); echo preg_replace("/\b(11|21|41)\b/u", "z", "x°11\n");' vs. php -n -r 'ini_set("pcre.jit", 1); echo preg_replace("/\b(11|21|41)\b/u", "z", "x°11\n");' Since the PCRE bug report dates from 2011-12-27 and is still marked NEW I wonder if it would be safer for PHP to disable pcre.jit in the recommended php.ini configuration files. Also: Does anyone know who might be able/willing to look at the upstream bug? Regards, - Chris --- ## #91252 — Rasmus Lerdorf — 2016-02-17T16:28:05+00:00 URL: https://externals.io/message/91243#91252 · In reply to: #91243 > Hi there, > we just ran into a version of the bug "JIT bug with lookbehind assertion": > https://bugs.exim.org/show_bug.cgi?id=1189 > > To reproduce it you can use > php -n -r 'ini_set("pcre.jit", 0); echo preg_replace("/\b(11|21|41)\b/u", "z", "x°11\n");' > vs. > php -n -r 'ini_set("pcre.jit", 1); echo preg_replace("/\b(11|21|41)\b/u", "z", "x°11\n");' > > Since the PCRE bug report dates from 2011-12-27 and is still marked NEW I wonder if it would be safer for PHP to disable pcre.jit in the recommended php.ini configuration files. > > Also: Does anyone know who might be able/willing to look at the upstream bug? > > Regards, > - Chris Just replying here since many people probably missed this since gmail classified it as spam. -Rasmus --- ## #91262 — Anatol Belski — 2016-02-18T08:16:10+00:00 URL: https://externals.io/message/91243#91262 · In reply to: #91243 Hi Christian, > -----Original Message----- > From: Christian Schneider [mailto:cschneid@cschneid.com] > Sent: Wednesday, February 17, 2016 2:07 PM > To: PHP internals > Subject: [PHP-DEV] PCRE jit bug with UTF-8 and lookbehind assertion > > Hi there, > we just ran into a version of the bug "JIT bug with lookbehind assertion": > https://bugs.exim.org/show_bug.cgi?id=1189 > > To reproduce it you can use > php -n -r 'ini_set("pcre.jit", 0); echo preg_replace("/\b(11|21|41)\b/u", > "z", "x°11\n");' > vs. > php -n -r 'ini_set("pcre.jit", 1); echo preg_replace("/\b(11|21|41)\b/u", > "z", "x°11\n");' > Seems valgrind doesn't detect any issues with this code. Using latest 7.0.4-dev code with PCRE 8.38, valgrind 3.10.0 on Jessie. Is there something else I could miss? Thanks Anatol --- ## #91313 — Anatol Belski — 2016-02-19T08:20:25+00:00 URL: https://externals.io/message/91243#91313 · In reply to: #91262 Hi Christian, > -----Original Message----- > From: Anatol Belski [mailto:anatol.php@belski.net] > Sent: Thursday, February 18, 2016 9:16 AM > To: 'Christian Schneider' ; 'PHP internals' > > Subject: RE: [PHP-DEV] PCRE jit bug with UTF-8 and lookbehind assertion > > Hi Christian, > > > […] > Seems valgrind doesn't detect any issues with this code. Using latest 7.0.4-dev > code with PCRE 8.38, valgrind 3.10.0 on Jessie. Is there something else I could > miss? > Could you please write back, what is the out difference between those two commands? Thanks. Anatol --- ## #91315 — Anatol Belski — 2016-02-19T09:18:02+00:00 URL: https://externals.io/message/91243#91315 · In reply to: #91313 Christian, > -----Original Message----- > From: Anatol Belski [mailto:anatol.php@belski.net] > Sent: Friday, February 19, 2016 9:20 AM > To: 'Christian Schneider' ; 'PHP internals' > > Subject: RE: [PHP-DEV] PCRE jit bug with UTF-8 and lookbehind assertion > > Hi Christian, > > > […] > Could you please write back, what is the out difference between those two > commands? > One more question - are you using (the bundled or external) PCRE 8.38? Thanks Anatol --- ## #91340 — keisial@gmail.com — 2016-02-21T00:26:58+00:00 URL: https://externals.io/message/91243#91340 · In reply to: #91313 > Could you please write back, what is the out difference between those > two commands? Thanks. Anatol In the first case, it correctly outputs «x°11» (78 c2 b0 7a). With jit enabled it produces «x�z» (78 c2 7a). That is, it is only outputting the lower byte of the utf-8 encoding of the U+00B0 character Tested on PHP 7.0.3 using the system libpcre 8.38 --- ## #91341 — Anatol Belski — 2016-02-21T10:42:35+00:00 URL: https://externals.io/message/91243#91341 · In reply to: #91340 > -----Original Message----- > From: Ángel González [mailto:keisial@gmail.com] > Sent: Sunday, February 21, 2016 1:27 AM > To: Anatol Belski > Cc: 'Christian Schneider' ; 'PHP internals' > > Subject: Re: [PHP-DEV] PCRE jit bug with UTF-8 and lookbehind assertion > > > > […] > In the first case, it correctly outputs «x°11» (78 c2 b0 7a). With jit enabled it > produces «x z» (78 c2 7a). That is, it is only outputting the lower byte of the utf-8 > encoding of the U+00B0 character Tested on PHP 7.0.3 using the system libpcre > 8.38 > Were you putting the snippets into a file or testing on the console? I had an issue while testing this on the console, that some chars was partially swallowed by terminal (which was a utf-8 terminal). When putting into a file, the output is same for both - "x°z". Please see also the continued discussion in the original ticket https://bugs.exim.org/show_bug.cgi?id=1189 . The offsets delivered by PCRE also seem to be correct, and valgrind doesn't find anything. It were great if you could confirm these insights. Thanks Anatol --- ## #91359 — Christian Schneider — 2016-02-23T14:34:04+00:00 URL: https://externals.io/message/91243#91359 · In reply to: #91341 Am 21.02.2016 um 11:42 schrieb Anatol Belski : > > […] > Were you putting the snippets into a file or testing on the console? I had an issue while testing this on the console, that some chars was partially swallowed by terminal (which was a utf-8 terminal). When putting into a file, the output is same for both - "x°z". Please see also the continued discussion in the original ticket https://bugs.exim.org/show_bug.cgi?id=1189 . The offsets delivered by PCRE also seem to be correct, and valgrind doesn't find anything. It were great if you could confirm these insights. I can reproduce it in a console and in a file. PCRE Library Version => 8.38 2015-11-23 I also reproduced it with a C program directly using the system PCRE library, no PHP involved. I attached the C source to https://bugs.exim.org/show_bug.cgi?id=1189 Regards, - Chris --- ## #91420 — keisial@gmail.com — 2016-02-25T20:11:53+00:00 URL: https://externals.io/message/91243#91420 · In reply to: #91341 > Were you putting the snippets into a file or testing on the console? I > had an issue while testing this on the console, that some chars was > partially swallowed by terminal (which was a utf-8 terminal). When > putting into a file, the output is same for both - "x°z". Please see > also the continued discussion in the original ticket > https://bugs.exim.org/show_bug.cgi?id=1189 . The offsets delivered by > PCRE also seem to be correct, and valgrind doesn't find anything. It > were great if you could confirm these insights. Thanks Anatol I was testing on a console and piping into hexdump. Using files showed -as expected- the same failure. The pcre-8.39 trunk (r1635), once you enable all the required options, does fix both the C test case and the php one (loading the new dynamic library instead of the system one). So yes, seems fixed in libpcre :) --- ## #91422 — Anatol Belski — 2016-02-25T20:29:14+00:00 URL: https://externals.io/message/91243#91422 · In reply to: #91420 Hi Ángel, > -----Original Message----- > From: Ángel González [mailto:keisial@gmail.com] > Sent: Thursday, February 25, 2016 9:12 PM > To: Anatol Belski > Cc: 'Christian Schneider' ; 'PHP internals' > > Subject: Re: [PHP-DEV] PCRE jit bug with UTF-8 and lookbehind assertion > > > > […] > I was testing on a console and piping into hexdump. Using files showed -as > expected- the same failure. > The pcre-8.39 trunk (r1635), once you enable all the required options, does fix > both the C test case and the php one (loading the new dynamic library instead of > the system one). > > So yes, seems fixed in libpcre :) Thanks for the further check. Yeah, 8.39 looks to be fixing the issue, just waiting for it to be released to upgrade in 7.0. Regards Anatol