# PHP allocating too much memory - Thread: https://externals.io/message/94005 - Original archive: https://news-web.php.net/php.internals/94005 - Started by Peter LeBrun on 2016-06-15T13:01:51+00:00 - Messages: 9 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #94005 — Peter LeBrun — 2016-06-15T13:01:51+00:00 URL: https://externals.io/message/94005#94005 Hi - I'm getting the following error in php7.0.5: Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 140729445144864 bytes) If it's relevant, we're using dblib+freetds. I can't reproduce the issue deterministically; the best I can do is to repeatedly curl an endpoint and something like 30% of the requests will fail due to the above error. The weirdest part about this is that PHP is somehow trying to allocate 140TB of memory. Has anyone seen anything like this? Any insights would be appreciated. Thanks, Peter --- ## #94007 — Jordi Boggiano — 2016-06-15T13:07:37+00:00 URL: https://externals.io/message/94005#94007 · In reply to: #94005 > Hi - > > I'm getting the following error in php7.0.5: > > Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to > allocate 140729445144864 bytes) > > If it's relevant, we're using dblib+freetds. > > I can't reproduce the issue deterministically; the best I can do is to > repeatedly curl an endpoint and something like 30% of the requests will > fail due to the above error. > > The weirdest part about this is that PHP is somehow trying to allocate > 140TB of memory. > > Has anyone seen anything like this? Any insights would be appreciated. I have hit that due to an opcache bug in handling of constant arrays on classes. Something like: class Foo { const BLA = []; } This was fixed in 5.6.15 [1] which came out before PHP7 though, so I'd imagine that isn't the exact problem you are hitting, but it might be related. [1] https://bugs.php.net/bug.php?id=70795 Cheers -- Jordi Boggiano @seldaek - http://seld.be --- ## #94009 — johannes@schlueters.de — 2016-06-15T13:22:44+00:00 URL: https://externals.io/message/94005#94009 · In reply to: #94005 Hi, > I'm getting the following error in php7.0.5: > > Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to > allocate 140729445144864 bytes) The huge number 140729445144864 is most likely consequence of an overflow. Can you try to reproduce this with a debug build? - Those make the error message a bit more verbose by stating which emalloc call caused this. Probably one can receive more info by attaching gdb (or another debugger) to the PHP process i.e. $ gdb -p 1234 # 1234 is the PID of a PHP process (gdb) source /path/to/php-src/.gdbinit (gdb) break zend_mm_safe_error (gdb) run now hit PHP a few times (gdb) bt [dumps system stack trace] (gdb) zbacktrace [dumps PHP-level stack trace, gdb macro defined in gdbinit loaded above] johannes --- ## #94011 — Rowan Collins — 2016-06-15T13:55:50+00:00 URL: https://externals.io/message/94005#94011 · In reply to: #94005 > The weirdest part about this is that PHP is somehow trying to allocate > 140TB of memory. I've seen numbers like that a few times - always around 140TB, but the exact number varies. I assume it's an overflow (or underflow?) somewhere, but the exact mechanism escapes me. (It's close to 2^47, but not very close; I've got examples logged as "low" as 140090229815192, and I think I've seen under 140 trillion.) Apart from sheer curiosity of where this magic number comes from, I wonder if there is some sanity check missing in the memory manager to at least display a different error message... Regards, -- Rowan Collins [IMSoP] --- ## #94013 — Thomas Hruska — 2016-06-15T14:35:18+00:00 URL: https://externals.io/message/94005#94013 · In reply to: #94007 > > > […] I have! You wouldn't, by any chance, have ODBC in the mix? The libraries you mention seem to indicate that you might be using ODBC. There is a known issue with mixing 32-bit and 64-bit binaries with PHP and ODBC. I don't remember all the details, but mixing 32-bit and 64-bit binaries will see things like `NULL` values as not `NULL` behind the scenes and do the wrong thing whereby PHP eventually attempts to allocate a ridiculous amount of RAM (e.g. a -1 value gets translated badly and assumes that it is the amount of RAM to allocate). I've seen allocation requests for upwards of 14 exabytes of RAM before - the amount varies but is always laughable. From what I can tell, `NULL` values in the query results will always trigger the problem and, even if you can guarantee that there are no `NULL` values in ANY of the query results, the issue still arises at a pretty significant rate due to bit-depth differences. Unfortunately, due to the way ODBC itself was created and functions, there is no easy fix within a mixed binary environment. To *correctly* fix the problem, you've got to match the bit-depths of ALL of the binaries involved - that is, either all 32-bit or all 64-bit - which is not always possible. The alternative is to use the ugliest hack ever: Execute a separate PHP process for running each database query that continually retries when it fails and just live with the high failure rates. Guess which solution I used? Fortunately, in my case, the number of queries being run was quite limited and on a set schedule. You might not be so lucky. -- Thomas Hruska CubicleSoft President I've got great, time saving software that you will find useful. http://cubiclesoft.com/ --- ## #94014 — Michael Felt — 2016-06-15T15:34:39+00:00 URL: https://externals.io/message/94005#94014 · In reply to: #94011 > > > […] > > I've seen numbers like that a few times - always around 140TB, but the > exact number varies. I assume it's an overflow (or underflow?) > somewhere, but the exact mechanism escapes me. (It's close to 2^47, > but not very close; I've got examples logged as "low" as > 140090229815192, and I think I've seen under 140 trillion.) In hex: 00007F694C627798 - so apparently 00007F69400000000 is common to all. FYI: I have seen similar issues with mixed environments (32 and 64-bit) - at this point I am surprised that you can even dlopen() both sizes (my OS now refuses to dlopen() 32-bit modules aka shared libraries when 64-bit application and v.v.) > Apart from sheer curiosity of where this magic number comes from, I > wonder if there is some sanity check missing in the memory manager to > at least display a different error message... > > Regards, --- ## #94019 — Trevor Suarez — 2016-06-15T17:50:58+00:00 URL: https://externals.io/message/94005#94019 · In reply to: #94014 > > > > > > […] > In hex: 00007F694C627798 - so apparently 00007F69400000000 is common to > all. > FYI: I have seen similar issues with mixed environments (32 and 64-bit) - > at this point I am surprised that you can even dlopen() both sizes (my OS > now refuses to dlopen() 32-bit modules aka shared libraries when 64-bit > application and v.v.) > > > […] > > > -- > > > > I've seen this bug come up with the SOAP extension. I saw it happen when instantiating a SoapClient or SoapServer when the constructor tries to load a WSDL file under very certain circumstances. If the SOAP WSDL caching is on (if `soap.wsdl_cache = 1`), the WSDL file is cached (or is attempting to be cached) or the WSDL must be downloaded, and the file-system is full, then this crazy overflow can happen. I believe it's due to the WSDL's becoming corrupted due to the file-system halting the write of the file and PHP not cleaning up the improper write. In fact, this is a reported bug: https://bugs.php.net/bug.php?id=62337 Hope that helps! :) - Trevor --- ## #94029 — Peter LeBrun — 2016-06-16T00:21:53+00:00 URL: https://externals.io/message/94005#94029 · In reply to: #94019 Hey everyone, thanks for your help and input. We've narrowed it down to cases where there is string concatenation with a constant, but currently upgrading to 7.0.7 to see if that makes a difference. Enjoy your evening, Peter > > > > […] > somewhere, > > […] > I've seen this bug come up with the SOAP extension. I saw it happen when > instantiating a SoapClient or SoapServer when the constructor tries to load > a WSDL file under very certain circumstances. If the SOAP WSDL caching is > on (if `soap.wsdl_cache = 1`), the WSDL file is cached (or is attempting to > be cached) or the WSDL must be downloaded, and the file-system is full, > then this crazy overflow can happen. I believe it's due to the WSDL's > becoming corrupted due to the file-system halting the write of the file and > PHP not cleaning up the improper write. > > In fact, this is a reported bug: https://bugs.php.net/bug.php?id=62337 > > Hope that helps! :) > > - Trevor --- ## #94032 — Pierre Joye — 2016-06-16T03:11:38+00:00 URL: https://externals.io/message/94005#94032 · In reply to: #94029 Hi > > Hey everyone, thanks for your help and input. We've narrowed it down to > cases where there is string concatenation with a constant, but currently > upgrading to 7.0.7 to see if that makes a difference. Is it possible to open a bug or post a reproduce script here please? > Enjoy your evening, > > Peter > > > > > […] wrote: > > […] allocate > > […] the > > […] think > > […] to > > […] 64-bit) - > > […] (my OS > > […] 64-bit > > […] to at > > […] load > > […] is > > […] attempting to > > […] and > > […]