Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:60375
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.217.170 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <002701cd26b5$1cd0bb70$56723250$@thetaphi.de>
References: <4F9E2173.50005@garfieldtech.com>
	<000e01cd269a$06f15230$14d3f690$@php.net>
	<CAEZPtU4kw1HbRJr=zoEY8nFiBYEdNRka0rp2O+fbmtEPM2xaNg@mail.gmail.com>
	<002701cd26b5$1cd0bb70$56723250$@thetaphi.de>
Date: Mon, 30 Apr 2012 06:33:34 -0400
Message-ID: <CAL+t6cGSHU7Q1VE2BkYJcTnYL1a0XJY3eUHOUgGnM7CFi1RzLw@mail.gmail.com>
To: Uwe Schindler <uwe@thetaphi.de>
Cc: Pierre Joye <pierre.php@gmail.com>, Uwe Schindler <thetaphi@php.net>, 
	Larry Garfield <larry@garfieldtech.com>, internals@lists.php.net
Content-Type: text/plain; charset=ISO-8859-1
Subject: Re: [PHP-DEV] readfile() memory usage
From: theanomaly.is@gmail.com (Sherif Ramadan)

> Mmapping of course uses memory, but the memory used here is not from PHP's
> memory manager, it's memory that's already used for the O/S cache. The
> memory mapping used here does not enforce loading the file contents into O/S
> cache; it just gets a virtual address into the O/S cache. If the actual file
> contents are not yet in O/S cache, the O/S will hit a page fault and load
> the pages into memory. Apache Server uses the same mechanism to serve files.
>
> Maybe the "user confusion" about memory usage comes from that fact (they see
> lots of *virtual* memory used by PHP when viewed in top). I know this user
> confusion from my work in the Apache Lucene/Solr project, where one option
> (used on 64 bit operating systems) is to memory-map the whole Lucene
> full-text index. Users then see hundreds of Gigabytes of "virtual memory"
> usage in TOP / Windows Task Manager and are afraid of running their machine
> out of memory. This is always hard to explain to people that are not used to
> the term "virtual memory".
>


That's a very good point and a detailed look at the stack can show
some of the underlying mechanics coming from readfile and how its
pretty much is just implemented like fpassthru delegating a stream, as
you said.

http://i.imgur.com/PWiTv.png


>
>> About php memory usage, one has to use an external tools to actually see
> this
>> memory usage as it is not managed by the zend memory manager.
>
> Of course...
>

Also, running valgrind and taking a closer look at what memory blocks
PHP is allocating here it can be better determined what's leaking and
what isn't, of course...

http://pastebin.com/LSQcUsBL


I can certainly attest to users being deceived by memory readings in
top in the past. It can be very deceiving if you think look at what
free memory top or `free` shows you especially after some huge
allocation. There's a clear difference here though between the Zend
memory manager allocating these blocks in PHP and what's going on with
readfile. The memory the system knows is available to it isn't being
tied up in this case.