# Memory warning hook - Thread: https://externals.io/message/66473 - Original archive: https://news-web.php.net/php.internals/66473 - Started by nathan@starin.biz on 2013-03-05T18:23:35+00:00 - Messages: 7 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #66473 — nathan@starin.biz — 2013-03-05T18:23:35+00:00 URL: https://externals.io/message/66473#66473 As PHP applications are turning into large frameworks one of the issues arriving is memory management. One of the issues is that many frameworks use sophisticated caching techniques to make accessing the same data quickly, this improves speed it is at the cost of memory. Often the developer knows these areas that cache and often times already have functions in place to clear out the cache, however in the case where PHP is approaching or exceeds memory limits PHP runs the GC then dies if it cannot allocate enough memory. If we implemented "memory warning" triggers or user function that will be called before the GC is executed which allows the user to try and free up some memory on their own. This hopefully would give more flexibility to allowing these advanced caching techniques but at the same time allow the cache to be cleared out in case memory is getting low. Thoughts? Thanks, Software Developer Nathan Bruer --- ## #66474 — Tom Boutell — 2013-03-05T18:25:39+00:00 URL: https://externals.io/message/66473#66474 · In reply to: #66473 Can't you do this already? memory_limit can be fetched via ini_read, and together with memory_get_usage you should be able to check for this sort of thing. Admittedly having to parse memory_limit (which can be in various units) is not perfect. > As PHP applications are turning into large frameworks one of the issues > arriving is memory management. One of the issues is that many frameworks use > sophisticated caching techniques to make accessing the same data quickly, > this improves speed it is at the cost of memory. Often the developer knows > these areas that cache and often times already have functions in place to > clear out the cache, however in the case where PHP is approaching or exceeds > memory limits PHP runs the GC then dies if it cannot allocate enough memory. > If we implemented "memory warning" triggers or user function that will be > called before the GC is executed which allows the user to try and free up > some memory on their own. This hopefully would give more flexibility to > allowing these advanced caching techniques but at the same time allow the > cache to be cleared out in case memory is getting low. > > > > Thoughts? > > > > Thanks, > > Software Developer > > Nathan Bruer > > > -- Tom Boutell P'unk Avenue 215 755 1330 punkave.com window.punkave.com --- ## #66476 — Lazare Inepologlou — 2013-03-05T18:59:50+00:00 URL: https://externals.io/message/66473#66476 · In reply to: #66474 2013/3/5 Tom Boutell > Can't you do this already? memory_limit can be fetched via ini_read, > and together with memory_get_usage you should be able to check for > this sort of thing. Admittedly having to parse memory_limit (which can > be in various units) is not perfect. > This is not the same at all. When are you going to run this code? Memory allocations happen all the time. What Nathan asked for is an event that is triggered when the memory consumption reaches a threshold. However, there is a different solution, which is better IMHO in the case of caches: weak references. A weak reference automatically frees the memory of the object, when the memory is needed. http://php.net/manual/en/book.weakref.php. Having said that, none of these solutions scale up to multiple servers. This is why shared cache systems like memcached are recommended. > > > […] > use > > […] > knows > > […] > exceeds > > […] > memory. > > […] > > > > -- > Tom Boutell > P'unk Avenue > 215 755 1330 > punkave.com > window.punkave.com > > -- > > > > Lazare INEPOLOGLOU Ingénieur Logiciel --- ## #66475 — johannes@schlueters.de — 2013-03-05T19:00:48+00:00 URL: https://externals.io/message/66473#66475 · In reply to: #66473 > As PHP applications are turning into large frameworks one of the issues > arriving is memory management. One of the issues is that many frameworks use > sophisticated caching techniques to make accessing the same data quickly, > this improves speed it is at the cost of memory. Often the developer knows > these areas that cache and often times already have functions in place to > clear out the cache, however in the case where PHP is approaching or exceeds > memory limits PHP runs the GC then dies if it cannot allocate enough memory. > If we implemented "memory warning" triggers or user function that will be > called before the GC is executed which allows the user to try and free up > some memory on their own. This hopefully would give more flexibility to > allowing these advanced caching techniques but at the same time allow the > cache to be cleared out in case memory is getting low. Running the GC is most likely faster than most cleanup routines a user could run, also usually there is not that much stuff cached in PHP scripts. If a PHP script has "tons" of data, which it can easily throw away, in memory this sounds like a smell of an bad architecture. Cache cache-worthy stuff in memcache or such and fetch only the data you need. Also: What should happen if the system runs out of memory while doing the cleanup? Anything sane doesn't sound good either. johannes --- ## #66477 — Sebastian Krebs — 2013-03-05T20:08:08+00:00 URL: https://externals.io/message/66473#66477 · In reply to: #66476 2013/3/5 Lazare Inepologlou > 2013/3/5 Tom Boutell > > > […] > > This is not the same at all. When are you going to run this code? Memory > allocations happen all the time. What Nathan asked for is an event that is > triggered when the memory consumption reaches a threshold. > You can use ticks :) http://php.net/control-structures.declare#control-structures.declare.ticks > > However, there is a different solution, which is better IMHO in the case of > caches: weak references. A weak reference automatically frees the memory of > the object, when the memory is needed. > http://php.net/manual/en/book.weakref.php. > > Having said that, none of these solutions scale up to multiple servers. > This is why shared cache systems like memcached are recommended. > Well, maybe I don't understand, what you are trying to tell, but if you run out of memory, this of course only affects one server on its own. > > > > > > […] > frameworks > > […] > quickly, > > […] > to > > […] > be > > […] > up > > […] > the > > […] > Lazare INEPOLOGLOU > Ingénieur Logiciel > -- github.com/KingCrunch --- ## #66478 — nathan@starin.biz — 2013-03-05T20:14:22+00:00 URL: https://externals.io/message/66473#66478 · In reply to: #66476 >This is not the same at all. When are you going to run this code? Memory allocations happen all the time. What Nathan asked for is an event that is triggered when the memory consumption reaches a >threshold. > >However, there is a different solution, which is better IMHO in the case of >caches: weak references. A weak reference automatically frees the memory of the object, when the memory is needed. >http://php.net/manual/en/book.weakref.php. > >Having said that, none of these solutions scale up to multiple servers. >This is why shared cache systems like memcached are recommended. I agree this probably is a good solution and I personally do use it along with shared memory tools, however there may be cases where the dev may gain more benefit from having a memory-warning installable trigger in place. This would allow things like allowing the dev to release certain cache objects before others or something completely different that I have not thought of yet. > Running the GC is most likely faster than most cleanup routines a user could run, also usually there is not that much stuff cached in PHP scripts. If a PHP script has "tons" of data, which it can easily throw >away, in memory this sounds like a smell of an bad architecture. Cache cache-worthy stuff in memcache or such and fetch only the data you need. > >Also: What should happen if the system runs out of memory while doing the cleanup? Anything sane doesn't sound good either. Yes running the GC is much faster except they are two completely different processes... in my example the dev is keeping references to data for possible future use later on however it's not possible to know when to release these references so php's GC can collect them if the user does not implement something quite juristic like ticks or frequent function calls throughout a code base. >You can use ticks :) > >http://php.net/control-structures.declare#control-structures.declare.ticks Yes Ticks are something useable (like said above) however I have found ticks are clunky, frequently shunned, and you'd be ticking for no reason most of the time. --- ## #66485 — Julien Pauli — 2013-03-06T09:02:08+00:00 URL: https://externals.io/message/66473#66485 · In reply to: #66478 One should try ext/memtrack http://pecl.php.net/package/memtrack Also ext/memprof https://github.com/arnaud-lb/php-memory-profiler/ Julien.Pauli > > […] > allocations happen all the time. What Nathan asked for is an event that is > triggered when the memory consumption reaches a >threshold. > > […] > of > > […] > of the object, when the memory is needed. > > […] > > I agree this probably is a good solution and I personally do use it along > with shared memory tools, however there may be cases where the dev may gain > more benefit from having a memory-warning installable trigger in place. > This would allow things like allowing the dev to release certain cache > objects before others or something completely different that I have not > thought of yet. > > > […] > could run, also usually there is not that much stuff cached in PHP scripts. > If a PHP script has "tons" of data, which it can easily throw >away, in > memory this sounds like a smell of an bad architecture. Cache cache-worthy > stuff in memcache or such and fetch only the data you need. > > […] > cleanup? Anything sane doesn't sound good either. > Yes running the GC is much faster except they are two completely different > processes... in my example the dev is keeping references to data for > possible future use later on however it's not possible to know when to > release these references so php's GC can collect them if the user does not > implement something quite juristic like ticks or frequent function calls > throughout a code base. > > > […] > http://php.net/control-structures.declare#control-structures.declare.ticks > > Yes Ticks are something useable (like said above) however I have found > ticks are clunky, frequently shunned, and you'd be ticking for no reason > most of the time.