Hi,
After looking through the new Memory Manager code, I assumed this would be
the case... (and just now got to test). Since heap->size is only updated
when a 256K or whatever block is actually allocated/freed, the number
returned by memory_get_usage() will only be a multiple of that. That
doesn't make it very useful anymore if one is using it to see how much
memory usage differs by changing, say, number of arrays/elements/variables,
strings, etc. as I was last month (small changes). And the example in the
manual definitely won't work as shown.
Does anybody else think that memory_get[peak_]usage() should work more like
before? I would imagine it can be "fixed" fairly easily, though it probably
needs another variable, and just a little extra overhead of inc./dec. that
variable more often (but I think that was done in the old MM anyway?).
Thanks,
Matt
Right now memory_get[peak_]usage() show the amount of REAL memory that PHP
(Zend Memory Manager) takes from system.
Previous memory manager showed size of emalloc()-ed memory without malloc()
overhead.
Also it didn't consider internal caches.
We need decide which behavior to use before 5.2.0 release.
Thanks. Dmitry.
-----Original Message-----
From: Matt W [mailto:php_lists@realplain.com]
Sent: Tuesday, July 25, 2006 7:53 AM
To: internals@lists.php.net
Subject: [PHP-DEV] memory_get_usage with new Memory ManagerHi,
After looking through the new Memory Manager code, I assumed
this would be the case... (and just now got to test). Since
heap->size is only updated when a 256K or whatever block is
actually allocated/freed, the number returned by
memory_get_usage()will only be a multiple of that. That
doesn't make it very useful anymore if one is using it to see
how much memory usage differs by changing, say, number of
arrays/elements/variables, strings, etc. as I was last month
(small changes). And the example in the manual definitely
won't work as shown.Does anybody else think that memory_get[peak_]usage() should
work more like before? I would imagine it can be "fixed"
fairly easily, though it probably needs another variable, and
just a little extra overhead of inc./dec. that variable more
often (but I think that was done in the old MM anyway?).Thanks,
Matt
Dmitry Stogov wrote:
Right now memory_get[peak_]usage() show the amount of REAL memory that PHP
(Zend Memory Manager) takes from system.Previous memory manager showed size of emalloc()-ed memory without malloc()
overhead.
Also it didn't consider internal caches.
Shouldn't we make the old behavior the default here?
regards,
Lukas
The idea of a memory limit is to as accurately as possible account
for the memory utilized by PHP. If our current calculation is more
accurate since it accounts for malloc overhead, great, it gives more
fine grained control to hosters utilizing this option.
Dmitry Stogov wrote:
Right now memory_get[peak_]usage() show the amount of REAL memory
that PHP
(Zend Memory Manager) takes from system.
Previous memory manager showed size of emalloc()-ed memory without
malloc()
overhead.
Also it didn't consider internal caches.Shouldn't we make the old behavior the default here?
regards,
Lukas--
Ilia Alshanetsky
Hi Dmitry, Ilia, et al.,
Ilia, the Memory Manager is checking the real size against memory_limit,
so it's still "more accurate" even after the functions were changed to
report more like the old way. I still wonder how much difference there may
be between size and real_size as it gets close to the limit...
After the changes, I see the usage difference after adding a variable, for
example, is slightly different than before, but I understand that's because
of counting overhead now or such. The old one just rounded the amount up to
a multiple of 8, I think...
Something else I'd like to see changed... Does anyone think
memory_get_[peak_]usage() should always be available, regardless of
whether memory_limit is actually enabled? Using the Windows binaries at
least, I was devastated :-D that they were unavailable for experimenting,
etc.
I don't see why not -- the function names IMO don't really suggest that
they're tied to memory_limit and all that's being saved by not having them
is a little overhead of updating the size/peak variables, right?
Thanks,
Matt
----- Original Message -----
From: "Ilia Alshanetsky"
Sent: Tuesday, July 25, 2006
The idea of a memory limit is to as accurately as possible account
for the memory utilized by PHP. If our current calculation is more
accurate since it accounts for malloc overhead, great, it gives more
fine grained control to hosters utilizing this option.Dmitry Stogov wrote:
Right now memory_get[peak_]usage() show the amount of REAL memory
that PHP
(Zend Memory Manager) takes from system.
Previous memory manager showed size of emalloc()-ed memory without
malloc()
overhead.
Also it didn't consider internal caches.Shouldn't we make the old behavior the default here?
regards,
Lukas
Matt W wrote:
Hi Dmitry, Ilia, et al.,
Ilia, the Memory Manager is checking the real size against memory_limit,
so it's still "more accurate" even after the functions were changed to
report more like the old way. I still wonder how much difference there may
be between size and real_size as it gets close to the limit...After the changes, I see the usage difference after adding a variable, for
example, is slightly different than before, but I understand that's because
of counting overhead now or such. The old one just rounded the amount up to
a multiple of 8, I think...Something else I'd like to see changed... Does anyone think
memory_get_[peak_]usage() should always be available, regardless of
whether memory_limit is actually enabled? Using the Windows binaries at
least, I was devastated :-D that they were unavailable for experimenting,
etc.I don't see why not -- the function names IMO don't really suggest that
they're tied to memory_limit and all that's being saved by not having them
is a little overhead of updating the size/peak variables, right?
The reason is performance. It takes some processing time to keep
accounting of memory usage and people don't using the feature do not
feel like its worth slowing PHP down.
Edin
Something else I'd like to see changed... Does anyone think
memory_get_[peak_]usage() should always be available, regardless of
whether memory_limit is actually enabled? Using the Windows
binaries at
least, I was devastated :-D that they were unavailable for
experimenting,
etc.
Counting memory all the time adds a fair amount of overhead to script
execution especially given how frequently PHP needs to allocate
memory. Therefore usage memory utilization functions are only
available when PHP is compiled with memory in which case it keeps
track of its mem usage.
Ilia Alshanetsky
I'm NOT a expert in this field. My POV may be quite dumb, but it may
point to a solution.
If memory is allocated, I would assume that somewhere there is a
mechanism to access it otherwise this is a memory leak.
So, rather than counting the memory in real time, why not have the
memory_get_usage() (not sure about peak usage) simply iterate the used
memory and calculate how much there is.
This would put the overhead in the function call and not throughout
the rest of the code.
Something else I'd like to see changed... Does anyone think
memory_get_[peak_]usage() should always be available, regardless of
whether memory_limit is actually enabled? Using the Windows
binaries at
least, I was devastated :-D that they were unavailable for
experimenting,
etc.Counting memory all the time adds a fair amount of overhead to script
execution especially given how frequently PHP needs to allocate
memory. Therefore usage memory utilization functions are only
available when PHP is compiled with memory in which case it keeps
track of its mem usage.
--
Richard Quadling
Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731
"Standing on the shoulders of some very clever giants!"
Ilia Alshanetsky wrote:
Something else I'd like to see changed... Does anyone think
memory_get_[peak_]usage() should always be available, regardless of
whether memory_limit is actually enabled? Using the Windows binaries at
least, I was devastated :-D that they were unavailable for
experimenting,
etc.Counting memory all the time adds a fair amount of overhead to script
execution especially given how frequently PHP needs to allocate memory.
Therefore usage memory utilization functions are only available when
PHP is compiled with memory in which case it keeps track of its mem usage.
That may hear off topic , but how you enable something else than 8M
without getting this counting overhead ?
It looks to me that --enable-memory-limit mixes 2 independant roles :
- setting the amount of available memory, and enabling eventually local
setting, - enabling the evaluation of the process consumption.
I would like to be wrong on this :)
toggg
Ilia Alshanetsky wrote:
Something else I'd like to see changed... Does anyone think
memory_get_[peak_]usage() should always be available,
regardless of
whether memory_limit is actually enabled? Using the Windows
binaries at
least, I was devastated :-D that they were unavailable for
experimenting,
etc.Counting memory all the time adds a fair amount of overhead to
script execution especially given how frequently PHP needs to
allocate memory. Therefore usage memory utilization functions are
only available when PHP is compiled with memory in which case it
keeps track of its mem usage.That may hear off topic , but how you enable something else than 8M
without getting this counting overhead ?
Eh? When you set a limit what are you trying to do? From my
experience most of the time this setting is used by hosting providers
to restrict memory utilization of various PHP scripts to avoid out-of-
memory situations and system abuse. As such you'd want as accurate
measurement as possible including any overhead and what not, since it
is possible to use the overhead to exceed or in some cases even to
bypass the memory limit entirely.
It looks to me that --enable-memory-limit mixes 2 independant roles :
- setting the amount of available memory, and enabling eventually
local setting,- enabling the evaluation of the process consumption.
That is correct. Memory limit by its nature requires PHP to count the
amount of memory consumed, since such tracking is performed it
enabled PHP to report internally via functions (or via Apache log)
about its memory utilization.
Ilia Alshanetsky
Hi Ilia,
"Ilia Alshanetsky" ilia@prohost.org wrote in message
news:70339B16-D50B-45B9-A55D-B58F9EEB679D@prohost.org...
Eh? When you set a limit what are you trying to do? From my
experience most of the time this setting is used by hosting providers
to restrict memory utilization of various PHP scripts to avoid out-of-
memory situations and system abuse. As such you'd want as accurate
measurement as possible including any overhead and what not, since it
is possible to use the overhead to exceed or in some cases even to
bypass the memory limit entirely.
Yes, hosting providers would enable the memory limit. But who wants to use
memory_get_[peak_]usage()? Not the hosting provider, but the application
developer.
- Ron
Yes, hosting providers would enable the memory limit. But who wants to use
memory_get_[peak_]usage()? Not the hosting provider, but the application
developer.
We have some process, that if cache has gotten stale and needs to be
recreated, the process can use 100MB or memory. We don't want to cap
it. However, we do want to know about it and use
apahce_child_terminate() to end that process. Right now, I have to
actually read /proc to figure out my memory usage. I would much rather
have a function. I guess I could enable a memory limit and set to like
1GB. Seems silly though.
--
Brian Moon
http://dealnews.com/
Its good to be cheap =)
Yes, hosting providers would enable the memory limit. But who wants
to use
memory_get_[peak_]usage()? Not the hosting provider, but the
application
developer.
The peak usage function was added with profiling in mind and keeping
track of scripts that spike memory, when I added it the intent was it
would help with profiling and debugging. Not something used everyday
in production environment. In dev environment speed is not an issue,
so you can enable memory limit and raise the limit to some very high
value such as 1Gb so it does not get in the way.
Ilia Alshanetsky
Ilia Alshanetsky wrote:
That may hear off topic , but how you enable something else than 8M
without getting this counting overhead ?Eh? When you set a limit what are you trying to do? From my experience
most of the time this setting is used by hosting providers to restrict
memory utilization of various PHP scripts to avoid out-of- memory
situations and system abuse.
The most cases I saw were providers or production site wanting to grant
more than 8M (typically 16M) to their customers / users.
I dont call that a "restriction".
You should know this 8M limit is reached very easily.
e.g. lot of popular CMS wont work below this limit as soon as they have
a litle contents.
As such you'd want as accurate measurement
as possible including any overhead and what not, since it is possible
to use the overhead to exceed or in some cases even to bypass the
memory limit entirely.
You miss the point , they don't care about accuracy,
they just don't want the default 8M but more.It looks to me that --enable-memory-limit mixes 2 independant roles :
- setting the amount of available memory, and enabling eventually
local setting,- enabling the evaluation of the process consumption.
That is correct. Memory limit by its nature requires PHP to count the
amount of memory consumed, since such tracking is performed it enabled
PHP to report internally via functions (or via Apache log) about its
memory utilization.
I have no idea how it works internally ...
Are you meaning the default "hard-coded" 8M limit is magic and will not
need to count the amount of memory consumed but any custom setting will ?
--
toggg
Dmitry Stogov wrote:
Right now memory_get[peak_]usage() show the amount of REAL memory that PHP
(Zend Memory Manager) takes from system.Previous memory manager showed size of emalloc()-ed memory without malloc()
overhead.
Also it didn't consider internal caches.Shouldn't we make the old behavior the default here?
Agree.
--
Wbr,
Antony Dovgal
Does anybody else think that memory_get[peak_]usage() should work
more like
before? I would imagine it can be "fixed" fairly easily, though it
probably
needs another variable, and just a little extra overhead of inc./
dec. that
variable more often (but I think that was done in the old MM anyway?).
I think that the current, more precise memory usage indicator is the
correct behavior and should not be changed.
Ilia Alshanetsky