Reporting bugs for phpng

Hi,

I tried the .deb package for phpng that Zend provides since yesterday
(thanks for that!) and I hit a memory consumption issue with the first
of my scripts I tried:

PHPNG:
Memory peak: 62914560 (55.97MB)
Elapsed time (s): 0.3383

PHP 5.5:
Memory peak: 1835008 (1.64MB)
Elapsed time (s): 0.0717

How/where can I make a bug report about it? Is https://bugs.php.net/ ok
for PHPNG bugs?

Thanks

Pascal

Hey:

Le 26/07/2014 04:42, Laruence a écrit :

Hey:

On Fri, Jul 25, 2014 at 9:14 PM, Pascal Chevrel pascal.chevrel@free.fr
wrote:

Hi,

I tried the .deb package for phpng that Zend provides since yesterday
(thanks for that!) and I hit a memory consumption issue with the first of
my
scripts I tried:

PHPNG:
Memory peak: 62914560 (55.97MB)
Elapsed time (s): 0.3383

PHP 5.5:
Memory peak: 1835008 (1.64MB)
Elapsed time (s): 0.0717

How/where can I make a bug report about it? Is https://bugs.php.net/ ok
for
PHPNG bugs?

PHPNG is not listed in bugs.php.net yet. but as it's a branch of PHP
, I think it's okey to report it there, and note it's PHPNG in the
comment. for now.

thanks

Hi,

I opened this bug:
https://bugs.php.net/bug.php?id=67725

I put a testcase triggering the memory leak in a github repo referenced in
the bug.
I am afraid this is not a bug.

it's due to our new immutable array implementation.

all these langs map array are constant array, such it will be treat as
immutable array, which means it will not be released untill the
request shutdown. (which will bring significant performance
improvement for saving array construct/destruct while use with
opcache)

I suggest you to increase the memory_limit as a workaround ..

Anyway, Dmitry, what do you think? maybe something like interned_string_size?

thanks

Regards

Pascal

--

--
Laruence Xinchen Hui
http://www.laruence.com/

The problem is that the more arrays I load, the more memory PHPNG consumes while current versions of PHP have that value mostly stable. I have with PHPNG a scalability issue I don't have with regular PHP. If tomorrow I increase the number of languages I support in my app from 70 to 200 (which is my long term goal btw), I would probably need to set memory_limit to 700MB which seems honestly crazy when PHP 5.x needs 8MB for the same script to run at the same speed. As an end user, I think it's a bug not a feature, I hope you understand my point of view :).

This isn’t so much a response to Pascal, more a question for the rest of you. My apologies if I’m misunderstanding this internally. As the array is in opcache, we can just use it directly without copying, right? In which case, shouldn’t Pascal’s issue not be happening, or is Pascal not using opcache? Or, are we copying each time we run the script? In which case, why aren’t we using opcache’s copy?

Andrea Faulds
http://ajf.me/

Le 31/07/2014 16:23, Laruence a écrit :

Hey:

On Thu, Jul 31, 2014 at 8:29 PM, Pascal Chevrel pascal.chevrel@free.fr
wrote:

Le 26/07/2014 04:42, Laruence a écrit :

Hey:

On Fri, Jul 25, 2014 at 9:14 PM, Pascal Chevrel pascal.chevrel@free.fr
wrote:

Hi,

I tried the .deb package for phpng that Zend provides since yesterday
(thanks for that!) and I hit a memory consumption issue with the first
of
my
scripts I tried:

PHPNG:
Memory peak: 62914560 (55.97MB)
Elapsed time (s): 0.3383

PHP 5.5:
Memory peak: 1835008 (1.64MB)
Elapsed time (s): 0.0717

How/where can I make a bug report about it? Is https://bugs.php.net/ ok
for
PHPNG bugs?

PHPNG is not listed in bugs.php.net yet. but as it's a branch of PHP
, I think it's okey to report it there, and note it's PHPNG in the
comment. for now.

thanks

Hi,

I opened this bug:
https://bugs.php.net/bug.php?id=67725

I put a testcase triggering the memory leak in a github repo referenced
in
the bug.

I am afraid this is not a bug.

it's due to our new immutable array implementation.

all these langs map array are constant array, such it will be treat as
immutable array, which means it will not be released untill the
request shutdown. (which will bring significant performance
improvement for saving array construct/destruct while use with
opcache)

I suggest you to increase the memory_limit as a workaround ..

Anyway, Dmitry, what do you think? maybe something like
interned_string_size?

Hi,

The problem is that the more arrays I load, the more memory PHPNG consumes
while current versions of PHP have that value mostly stable. I have with
PHPNG a scalability issue I don't have with regular PHP. If tomorrow I
increase the number of languages I support in my app from 70 to 200 (which
is my long term goal btw), I would probably need to set memory_limit to
700MB which seems honestly crazy when PHP 5.x needs 8MB for the same script
to run at the same speed. As an end user, I think it's a bug not a feature,
I hope you understand my point of view :).

I understand your point, and the problem, but the thing here is, do
you really needs loads all langs map arrays in a single request(as I
understand you only need one per request, right?)

Also, people on shared hosting can't change the memory_limit value, it
doesn't sound like a change that is good for PHP in shared hosting
environment which is a large part of PHP success.

You mention that it is beneficial when OPCache is activated, but it isn't
activated by default, so the theoretical gain would only work for people
that can activate it, right? Or does it mean that OPCache would be activated
by default in PHP 7?

I will suggest to enable it by default in NG.

Is there no way to release the memory programmatically before the request
ends? Shouldn't explicitely using unset() on the array at the end of each
iteration indicate to the engine that I don't just want to destroy the array
reference but also get the memory back? That sounds like a very significant
change in how PHP behaves today with arrays, doesn't it? PHPNG is promoted
as a big speed optimization that wouldn't impact userland code, well, in my
case it does and very significantly if parts of my app crash or use 20 or 30
times more memory than before for a 5% speed gain, it's not a win. If there
is a way to force PHP to unload that data from memory and basically behave
like PHP 5.x it could at least be in PHP 7 upgrade release notes for people
like me that build stuff on today's PHP strengths.

hmm, I will definitely ask dmitry 's idea..

thanks

Thanks,

Pascal

--

--
Laruence Xinchen Hui
http://www.laruence.com/

hi Pascal,

Laruence is right - immutable arrays are not destroyed until the end of
request and this causes a problem in your case.

I've committed a partial fix that destroys unnecessary copies when script
is cached by OPCache, but it doesn't work without opcache or when it
doesn't have enough shared memory.

Disabling immutable arrays for top-level code looks as a too pessimistic
decision. Most apps have their configuration arrays in top-level code and
only few might be affected by this problem...

I think, we may introduce new pragma that might enable/disable immutable
array creation during compilation.
So applications that use a lot of big arrays may modify the default
behaviour for some files.

e.g.

<?php
declare(immutable_arrays=>'off');
$a = array(...);
?>

Thoughts?

Thanks. Dmitry.

On Fri, Aug 1, 2014 at 12:39 PM, Pascal Chevrel pascal.chevrel@free.fr
wrote:

Le 01/08/2014 10:21, Laruence a écrit :

On Fri, Aug 1, 2014 at 12:34 AM, Pascal Chevrel pascal.chevrel@free.fr

wrote:

Le 31/07/2014 16:23, Laruence a écrit :

Hey:

On Thu, Jul 31, 2014 at 8:29 PM, Pascal Chevrel <pascal.chevrel@free.fr

wrote:

Le 26/07/2014 04:42, Laruence a écrit :

Hey:

On Fri, Jul 25, 2014 at 9:14 PM, Pascal Chevrel <
pascal.chevrel@free.fr>
wrote:

Hi,

I tried the .deb package for phpng that Zend provides since yesterday
(thanks for that!) and I hit a memory consumption issue with the
first
of
my
scripts I tried:

PHPNG:
Memory peak: 62914560 (55.97MB)
Elapsed time (s): 0.3383

PHP 5.5:
Memory peak: 1835008 (1.64MB)
Elapsed time (s): 0.0717

How/where can I make a bug report about it? Is https://bugs.php.net/
ok
for
PHPNG bugs?

PHPNG is not listed in bugs.php.net yet. but as it's a branch of

PHP
, I think it's okey to report it there, and note it's PHPNG in the
comment. for now.

thanks

Hi,

I opened this bug:
https://bugs.php.net/bug.php?id=67725

I put a testcase triggering the memory leak in a github repo referenced
in
the bug.

I am afraid this is not a bug.

it's due to our new immutable array implementation.

all these langs map array are constant array, such it will be treat as
immutable array, which means it will not be released untill the
request shutdown. (which will bring significant performance
improvement for saving array construct/destruct while use with
opcache)

I suggest you to increase the memory_limit as a workaround ..

Anyway, Dmitry, what do you think? maybe something like
interned_string_size?

Hi,

The problem is that the more arrays I load, the more memory PHPNG
consumes
while current versions of PHP have that value mostly stable. I have with
PHPNG a scalability issue I don't have with regular PHP. If tomorrow I
increase the number of languages I support in my app from 70 to 200
(which
is my long term goal btw), I would probably need to set memory_limit to
700MB which seems honestly crazy when PHP 5.x needs 8MB for the same
script
to run at the same speed. As an end user, I think it's a bug not a
feature,
I hope you understand my point of view :).

I understand your point, and the problem, but the thing here is, do
you really needs loads all langs map arrays in a single request(as I
understand you only need one per request, right?)

Hi,

No, this testcase is a cut down version of the real code I wrote to make
it easier to detect the problem I was facing, the real code is part of a
specific view that is also an API allowing to consult all translations
available for a specific string ID in our code base, examples:
http://transvision.mozfr.org/api/v1/entity/central/?id=
browser/chrome/browser/browser.dtd:bookmarkThisPageCmd.label

http://transvision.mozfr.org/string/?entity=browser/chrome/
browser/browser.dtd:bookmarkThisPageCmd.label&repo=central

Cheers

Pascal

On Tue, Aug 5, 2014 at 2:14 PM, Pascal Chevrel pascal.chevrel@free.fr
wrote:

Le 05/08/2014 14:01, Pascal Chevrel a écrit :

Le 05/08/2014 13:41, Dmitry Stogov a écrit :

Now it must be completely fixed in git.

Thanks. Dmitry.

Thanks Dmitry, will the .deb package be soon updated so as that I can
test that the issue is resolved on my side?

Actually, shouldn't 'php -i' expose somewhere the php-src revision that
was used for compiling to make QA easier? (PHP in general, not a PHPNG
specific question)

Regards

Pascal

--

compiling php from version control is for development purposes, so if
somebody decides to ship binaries based on an untagged php version, it is
his/her responsibility to provide a way for the end user to be able to
figure out the exact version which was used for the build.
the official php versions all use a separate version and even the
http://snaps.php.net/ tarballs have a prise date in the filename to make it
easier to tell which version was used as a basis (I think it would make
sense to also add the git verion to the filename).
I don't think that it would be a good idea to adjust our current build
system to check if the build is done from git and patch the php version
files on-the-fly so that the the resulting binary can tell which git
version was used.

--
Ferenc Kovács
@Tyr43l - http://tyrael.hu