Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:46101
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.216.178 as permitted sender)
DomainKey-Status: bad
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=Ars8AmDY33+lZGhzxLwQpK9p1o/HNXyuWwlVqtRMsPGoacczCP+vxY2LVzLeeRfseM
         6YobZ2Vzrfw318Mp5hDgaEA+KgkxEGidkclJaJOf7NgbhCFWYCv3KuFKrw55tPIT1i5q
         DNQOESFSpfR+SG+DX9pkuWyPIXDCUx/k0JctM=
MIME-Version: 1.0
Date: Thu, 19 Nov 2009 18:12:35 +0530
Message-ID: <b65d742b0911190442p4a63832bo940edb13f35d5b34@mail.gmail.com>
To: internals@lists.php.net
Content-Type: text/plain; charset=ISO-8859-1
Subject: Question about buffer size calculation in _php_stream_copy_to_mem_ex
From: yoarvi@gmail.com (Arvind Srinivasan)

I see the following in _php_stream_copy_to_mem_ex in main/streams/streams.c

	int step = CHUNK_SIZE;
	int min_room = CHUNK_SIZE / 4;

        (..stuff deleted..)

	/* avoid many reallocs by allocating a good sized chunk to begin with, if
	 * we can.  Note that the stream may be filtered, in which case the stat
	 * result may be inaccurate, as the filter may inflate or deflate the
	 * number of bytes that we can read.  In order to avoid an upsize followed
	 * by a downsize of the buffer, overestimate by the step size (which is
	 * 2K).  */
	if (php_stream_stat(src, &ssbuf) == 0 && ssbuf.sb.st_size > 0) {
		max_len = ssbuf.sb.st_size + step;
	} else {
		max_len = step;
	}

According to the comment the intention is to overestimate by 2K, but
CHUNK_SIZE = 8192 and that leads quite a bit of over-allocation of
memory.

Is the following a better setting for max_len in the if block?

max_len = (((ssbuf.sb.st_size + min_room)/step) + 1) * step;

The following are sample values that show the current vs new size calculation.
ssbuf.sb.st_size, max_len (current), max_len (new)
(     100,    8292,    8192)
(   1000,    9192,    8192)
(   5000,  13192,    8192)
(   7000,  15192,  16384)
( 13000,  21192,  16384)
( 22000,  30192,  24576)

thanks,
arvi