Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:43228
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain openmv.com from 64.15.152.204 cause and error)
To: "'PHP Developers Mailing List'" <internals@lists.php.net>
Date: Mon, 2 Mar 2009 11:47:36 -0500
Message-ID: <002a01c99b56$97b7fd00$c727f700$@com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_002B_01C99B2C.AEE1F500"
Thread-Index: AcmbVpXRvcgBz3lWRwWZQb1SjgaG2w==
Content-Language: en-ca
Subject: Stream chunk size
From: jbondc@openmv.com ("Jonathan Bond-Caron")

------=_NextPart_000_002B_01C99B2C.AEE1F500
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi everyone, I have a question about streams and the maximum =91chunk =
size=92 of
8192.=20

=20

I=92ve read README.STREAMS and found these slides by Wez:

http://netevil.org/blog/2008/07/slides-php-streams

=20

While trying to write an Amazon S3 stream wrapper and I ran into an =
issue
with large files:=20

=20

$fp =3D fopen('s3://mvtest/large.html', 'r'); // 30 mb

=20

// This is OK

fseek($fp, 10);

echo fread($fp, 100) . "\n"; // 100 bytes

echo fread($fp, 100) . "\n"; // 100 bytes

=20

// This is OK (according to documentation, max 8192 bytes)=20

echo fread($fp, 65536) . "\n"; // 8192 bytes

=20

My issue is I would like to request larger =91chunks=92, something like:

stream_set_chunk_size($fp, 65536);

=20

echo fread($fp, 65536) . "\n"; // 65536 bytes

echo fread($fp, 100000) . "\n"; // 65536 bytes

echo fread($fp, 15) . "\n"; // 15 bytes

=20

Then copying to a file and avoiding memory issues:

=20

$wfp =3D fopen(=91/tmp/large.html=92);

stream_copy_to_stream($fp, $wfp); // read 65536 byte chunks, write =
default
8192 byte chunks

=20

stream_set_chunk_size($wfp, 65536);

stream_copy_to_stream($fp, $wfp); // read & write 65536 byte chunks

copy('s3://mvtest/large.html', '/tmp/large.html'); // read & write =
default
8192 byte chunks

=20

Going through the PHP 5.2 source, it looks like there=92s support for it =
but
at some places the 8192 =91chunk=92 is hardcoded:

=20

#define CHUNK_SIZE     8192

=20

PHPAPI size_t _php_stream_copy_to_stream(php_stream *src, php_stream =
*dest,
size_t maxlen STREAMS_DC TSRMLS_DC)

{

                char buf[CHUNK_SIZE]; =DF  Is there any reason the =
php_stream
*src->chunk_size isn=92t used?

=20

stream_set_chunk_size($fp, 65536); // Would mean src->chunk_size =3D =
65536;

=20

I=92d like to try to write a patch for it, anything that I should know =
about
streams and why the limit is there?


------=_NextPart_000_002B_01C99B2C.AEE1F500--