Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:4355
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Message-ID: <5.1.0.14.2.20030902234052.0aed8e90@127.0.0.1>
Date: Tue, 02 Sep 2003 23:49:01 +0300
To: "Vesselin Atanasov" <vma1@abv.bg>,"PHP-DEV" <internals@lists.php.net>
In-Reply-To: <JJEIJKBBKGFHEDGHDANNEEHFCAAA.vma1@abv.bg>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Subject: Re: [PHP-DEV] A note about the Zend low level memory manager
  in zend_mm.c
From: andi@zend.com (Andi Gutmans)
References: <JJEIJKBBKGFHEDGHDANNEEHFCAAA.vma1@abv.bg>

Hi,

First of all I'm happy someone is taking a closer look at the memory 
manager. I am aware that it needs some fine tuning.
The first thing I'd like you to try is to change the following line in 
zend_alloc.c
zend_mm_startup(&AG(mm_heap), 256*1024);
to
zend_mm_startup(&AG(mm_heap), 32*1024); // You can also try 16

I didn't change this in the CVS yet because libXML2 2.5.9 (or was it 
2.5.8?) has a bug (memory corruption) which indirectly would cause a crash. 
Best is if you upgrade to 2.5.10 and then changing this value shouldn't 
cause any problems.

Secondly, right now except for the exact match strategy for small blocks, I 
am using a best match strategy. I guess I could still change the insertion 
order (complicating implementation a tiny bit) but do you think it would 
really be useful? As I'm using best fit most likely the age of the block 
won't make much difference.

Last but not least, in the past I did discover that ZEND_MM performed 
better without the fast memory cache in zend_alloc.c That's why there's a 
define there which theoretically turns it off with ZEND_MM. I enabled it 
because of some inconclusive benchmarks. It'd be great if you could post 
your personal benchmarks with this #define turned on and off. Just note 
that different platforms do tend to behave quite differently so benchmarks 
on Linux might be quite different from Solaris. We'd need to benchmark at 
least 2-3 platforms and 2-3 apps to get a real world idea.
Anyway, so change the following:
#ifdef ZEND_MM
#define ZEND_DISABLE_MEMORY_CACHE 0
#else
#define ZEND_DISABLE_MEMORY_CACHE 0
#endif
to:
#ifdef ZEND_MM
#define ZEND_DISABLE_MEMORY_CACHE 1
#else
#define ZEND_DISABLE_MEMORY_CACHE 0
#endif

and let me know how performance and memory usage differ.
Be sure to do any performance tests in non-debug mode.
Thanks,

Andi

At 01:13 PM 9/2/2003 +0300, Vesselin Atanasov wrote:
>Hello.
>The current low-level memory management code in zend_mm.c places the
>free blocks as first on the bucket. This increases the fragmentation
>of newly allocated free blocks. I think it is a good idea to place the
>freed or newly allocated blocks as last on the bucket. Thus when a new
>block is allocated the allocation scheme will give preference to old
>and already fragmented memory segments. The new segments will be used
>only when a request for a big block is made. This will significantly
>decrease the memory fragmentation.
>
>The current scheme uses the newly allocated segments as soon as
>possible because they are placed first on the list. This combined with
>the bad (IMHO) scheme of the high-level memory caching creates enormous
>memory fragmentation. A script that needs about 2MB of memory when run
>several times in a row may cause PHP to allocate about 10 MB of memory.
>
>vesselin
>
>
>--
>PHP Internals - PHP Runtime Development Mailing List
>To unsubscribe, visit: http://www.php.net/unsub.php