Resend. List serv problems.
ap.
---------------------------- Original Message ----------------------------
Subject: Re: PHP memory fragmentation
From: "Andrew Prendergast" ap@tellusion.com
Date: Sun, June 12, 2005 21:09
To: internals@lists.php.net
Please find attached a diff against HEAD & a modified version of the file
implementing my proposed PHP_FCGI_MAX_RAM_MB & PHP_FCGI_MAX_RAM_INCREASE
changes.
Example invocation of PHP with 10 child processes that are automatically
restarted if they increase in size by more than 25%:
PHP_FCGI_CHILDREN=10
PHP_FCGI_MAX_RAM_INCREASE=25
PHP_FCGI_MAX_REQUESTS=20
export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS PHP_FCGI_MAX_RAM_INCREASE
/usr/local/zeus/web/bin/fcgirunner
--user=www --group=www 8002
/usr/src/usr.local/php-5.0.4/sapi/cgi/php -c /usr/local/lib/php.ini
Compile with -DDEBUG_FASTCGI to see whats going on.
I've been testing it for the last 24 hours with various volumes of data
and it definitely does the trick.
Regards,
ap.
On Sat, June 11, 2005 18:36, Andrew Prendergast said:
After a bit more investigation, I would like to change this paragraph:
Once a script finishes executing, although the garbage collector
technically has freed the ram, the pages are still allocated. The next
time the thread executes, there is a whole pile of swapping and if a
memory hungry script gets run again, the memory footprint increases.To:
Once a script finishes executing, although the garbage collector
technically has freed the ram, the pages are still allocated. If the
thread is re-used, even for a small script, much of the previously
swapped out ram is swapped back in due to fragmentation. The memory
footprint of the thread will increase further if a different memory
hungry script is executed, or a small script that has a different
allocation pattern to the first memory eating script is executed.Regards,
ap.
On Sat, June 11, 2005 16:30, Andrew Prendergast said:
Hey Folks,
I'm not on the list so please reply back to my email.
I have been having some problems with memory fragmentation in PHP when
working with large data structures.Once a script finishes executing, although the garbage collector
technically has freed the ram, the pages are still allocated. The next
time the thread executes, there is a whole pile of swapping and if a
memory hungry script gets run again, the memory footprint increases.Now consider what happens when I have 20 processes in an FCGI pool,
each one getting above 300MB in size, its not good.My solution which I have implemented as part of the FCGI SAPI (thanks
to GeorgeS, hartmut, Pierre & Derick for their guidance) is the
introduction
of two new environment variables, PHP_FCGI_MAX_RAM_MB &
PHP_FCGI_MAX_RAM_INCREASE.PHP_FCGI_MAX_RAM_MB allows one to set a limit (in megabytes) of how large
an FCGI process is allowed to grow to.PHP_FCGI_MAX_RAM_INCREASE allows one to set a maximum % increase compared
to when the process first started.If either of these limits are exceeded, the process will be restarted
AFTER script execution has completed (unlike the --enable-memory-limit
stuff).Please get back to me on:
your thoughts on this
an idea of how big an average PHP process should be (mine are 150MB
when they start, 160MB after running some average scripts, and grow to
above 300 MB when working with big data structures and complex object
caching code)how do i get this back into the PHP source tree! (sorry newbie)
Regards,
ap.