Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:94013
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain cubiclesoft.com designates 74.208.222.236 as permitted sender)
To: PHP Development <internals@lists.php.net>
References: <CADS+g1pFGsuv5Tfz9a_exmycBKuo4K8erfsWZKySSbtWEnHFhQ@mail.gmail.com>
 <c24fd5dc-53b7-d028-15b6-ba5e026a7508@seld.be>
Message-ID: <6f1890dd-eb41-b685-1fa8-cbd8e8d64333@cubiclesoft.com>
Date: Wed, 15 Jun 2016 07:35:18 -0700
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:11.0) Gecko/20120327
 Thunderbird/11.0.1
MIME-Version: 1.0
In-Reply-To: <c24fd5dc-53b7-d028-15b6-ba5e026a7508@seld.be>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] PHP allocating too much memory
From: thruska@cubiclesoft.com (Thomas Hruska)

On 6/15/2016 6:07 AM, Jordi Boggiano wrote:
> On 15/06/2016 14:01, Peter LeBrun wrote:
>> Hi -
>>
>> I'm getting the following error in php7.0.5:
>>
>> Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to
>> allocate 140729445144864 bytes)
>>
>> If it's relevant, we're using dblib+freetds.
>>
>> I can't reproduce the issue deterministically; the best I can do is to
>> repeatedly curl an endpoint and something like 30% of the requests will
>> fail due to the above error.
>>
>> The weirdest part about this is that PHP is somehow trying to allocate
>> 140TB of memory.
>>
>> Has anyone seen anything like this?  Any insights would be appreciated.

I have!

You wouldn't, by any chance, have ODBC in the mix?  The libraries you 
mention seem to indicate that you might be using ODBC.  There is a known 
issue with mixing 32-bit and 64-bit binaries with PHP and ODBC.  I don't 
remember all the details, but mixing 32-bit and 64-bit binaries will see 
things like NULL values as not NULL behind the scenes and do the wrong 
thing whereby PHP eventually attempts to allocate a ridiculous amount of 
RAM (e.g. a -1 value gets translated badly and assumes that it is the 
amount of RAM to allocate).  I've seen allocation requests for upwards 
of 14 exabytes of RAM before - the amount varies but is always 
laughable.  From what I can tell, NULL values in the query results will 
always trigger the problem and, even if you can guarantee that there are 
no NULL values in ANY of the query results, the issue still arises at a 
pretty significant rate due to bit-depth differences.

Unfortunately, due to the way ODBC itself was created and functions, 
there is no easy fix within a mixed binary environment.  To *correctly* 
fix the problem, you've got to match the bit-depths of ALL of the 
binaries involved - that is, either all 32-bit or all 64-bit - which is 
not always possible.  The alternative is to use the ugliest hack ever: 
Execute a separate PHP process for running each database query that 
continually retries when it fails and just live with the high failure 
rates.  Guess which solution I used?  Fortunately, in my case, the 
number of queries being run was quite limited and on a set schedule. 
You might not be so lucky.

-- 
Thomas Hruska
CubicleSoft President

I've got great, time saving software that you will find useful.

http://cubiclesoft.com/