Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:52894
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain naenius.com designates 83.96.159.14 as permitted sender)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Sat, 04 Jun 2011 23:38:00 +0200
To: PHP Internals <internals@lists.php.net>
Message-ID: <ff98f4d70b6306710582b9debdbb667f@naenius.com>
User-Agent: Roundcube Webmail/0.5.2
Subject: Trying to find out where the memory went
From: mike.vanriel@naenius.com (Mike van Riel)

Dear Internals,

During development of DocBlox I encountered a (for me) unusual 
situation
with regards to memory usage.

I hope you can shed some light on this for me as I do not understand.

The situations is as follows:

I have a php file containing about 53 KLOC (including whitespace and
comments), which is about 2.1MB in size. When I execute the
memory_get_peak_usage after running the token_get_all method on its
content it reports that 232MB of RAM have been used in the process.

I am having trouble understanding how 244003984B (232MB) RAM could be
used.

The following is what I have calculated:
* 640.952B to start with (measured);
* 2.1MB to load the file contents into memory using file_get_contents
* 68 bytes for the resulting array
* 68 bytes for each child array representing a token
* 68 bytes for each element in a token array (which can be either 1 or
3, depending whether it is actually a token or literal)
* 2.1MB in total for the string contents of the token literals /
contents (equivalent to the byte size of the file)

I have used the count method to retrieve the number of tokens (276697)
and come to the following sum (everything is retrieved and calculated 
in
bytes):

640952+2165950+68+(276697*68)+(276697*3*68)+2165950=80234436 = 76M

This is a worst case formula where I assume that every token in the
array consists of 3 elements.

Based on this calculation I would be missing 156MB of memory; anybody
know where that went?

I used the following snippet of code for my tests:

     var_dump(memory_get_peak_usage());
     $tokens = token_get_all(file_get_contents('<PATH>'));
     var_dump(count($tokens));
     var_dump(memory_get_peak_usage());

I hope this mail did not scare anyone ;)

Kind regards,

Mike van Riel