Hello all,
I'm KishoreKumar Bairi, Google Summer of Code Student for MySQL.
Project: Streaming Enabled MySQL Driver for PHP
Details:
MyBS[1] is a storage engine used along with other engines to manage BLOB
data efficiently.
MyBS maintains a blob repository for each database to store the blobs
belonging to that database. Each blob stored in this repository has a unique
ID which can be used to reference that blob. MyBS has an embedded light
weight webserver which interfaces the blob repository and client.
BLOBs can be stored in this repository by sending HTTP PUT request to the
embedded web server which returns the unique ID that can be stored in
database to refer this blob later. To retrieve this BLOB, HTTP GET request
for referencing URL is sent to the embedded web server.
This task involves addition of two functions to 'mysqli'.
- resource mysqli_get_blob ( mysqli link, string reference )
reference: is the string which used after database part in the referencing
URL.
It can be repository Id (or) "<table>/<column>/<row condition>".
- string mysqli_put_blob ( mysqli link, string file_path, string mode , int
length )
file_path: is the path of the file which is to be stored in a blob field.
mode: mode specifier if the file should be read ASCII mode or binary mode.
link: is the object representing the MySQL database connection.
So, finally this is how the usage will be.
Procedural style:
- resource mysqli_get_blob ( mysqli link , string reference )
- string mysqli_put_blob (mysqli link ,string file_path, string mode , int
length)
Object Oriented style:
(methods)
mysqli {
resource mysqli_get_blob ( string reference )
}
mysqli {
string mysqli_put_blob(string file_path, string mode , int length)
}
Looking forward for suggestions and comments.
[1] http://www.blobstreaming.org/
Regards,
KishoreKumar Bairi.
Hi!
KishoreKumar Bairi schrieb:
- resource mysqli_get_blob ( mysqli link, string reference )
reference: is the string which used after database part in the referencing
URL.
It can be repository Id (or) "<table>/<column>/<row condition>".
PHP streams are a better choice than PHP strings for handling large
string objects. PHP strings are loaded into memory at once. For a
streaming engine you do not want to load large portions of data into
memory unless you really have to.
Why does this function return a resource? What type of resource is it
and what is the resource handle used for?
- string mysqli_put_blob ( mysqli link, string file_path, string mode , int
length )file_path: is the path of the file which is to be stored in a blob field.
mode: mode specifier if the file should be read ASCII mode or binary mode.
PHP strings are binary safe. There are no ASCII mode strings. Why do you
want to distinguish between ASCII and binary strings? Why is the
functionality restricted to existing files? Looks a bit like its
inspired by ftp or Windows?
In general, I think such blob streaming functionality should use PHP
streams. After some discussion with Andrey I'm not even sure if there is
a need to introduce a single new API call to ext/mysqli. Andrey brought
up the idea of using a streams protocol wrapper [1]. This would give you
quite a neat API which is more powerful and flexible than the above
suggestion.
Here's an example in pseudo-code.
Reading data from the blob repository:
$stream = fopen('mysqlblob://user:password@socket/db/table/whateverid',
'r');
while ($chunk = fread($stream, 1024))
// read blob in 1k chunks
echo $chunk;
fclose($stream);
Writing data from a file to the blob repository:
$blob_stream =
fopen('mysqlblob://user:password@host/https=0&id=/db/table/id', 'rw');
$file_stream = fopen('myimage.jpg');
$num_chunks = 0;
while (stream_copy_to_stream($file_stream, $blob_stream, 1024) > 0) {
printf("Writing to stream... %04dkb\n", $num_chunks++);
}
// the unique id generated by the blob engine to identify the blob
$blob_id = fgets($blob_stream);
fclose($file_stream);
fclose($blob_stream);
Generating data in PHP and writing to the blob repository:
$blob_stream = fopen('mysqlblob://user:password@socket/options', 'w');
// create thumbnail
$image = new Imagick('image.jpg');
$image->thumbnailImage(100, 0);
fwrite($blob_stream, $image);
// get unique blog id from the blob engine
$blob_id = fgets($blob_stream);
fclose($blob_stream);
echo $image;
And, and, and... - streams are a very mighty tool. I'm not sure about
the syntax for connections. It could be something like this:
URL:
mysqlblob://user:password@[socket|host]?[param=value[¶m=value...]]
Params:
id - id of the blob to fetch/read
port - MySQL server port
ssl - SSL parameter
ssh - SSH parameter
Whatever you need.
Ulf
[1] http://www.php.net/manual/en/wrappers.php
--
Ulf Wendel, MySQL
Sun Microsystems GmbH, Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering Muenchen: HRB161028
hi Ulf,
PHP streams are a better choice than PHP strings for handling large string
objects. PHP strings are loaded into memory at once. For a streaming engine
you do not want to load large portions of data into memory unless you really
have to.Why does this function return a resource? What type of resource is it and
what is the resource handle used for?
- string mysqli_put_blob ( mysqli link, string file_path, string mode ,
int
length )file_path: is the path of the file which is to be stored in a blob field.
mode: mode specifier if the file should be read ASCII mode or binary mode.
PHP strings are binary safe. There are no ASCII mode strings. Why do you
want to distinguish between ASCII and binary strings? Why is the
functionality restricted to existing files? Looks a bit like its inspired by
ftp or Windows?
That's partaially incorrect. PHP streams are not always binary safe by
default, at least on windows (linux has no 'b' but for API
compatibility reasons afair).
Unless you specify the binary mode explicitly. A common problem is to
run under cgi, cli or apache on windows, the default mode may differ
(apache for example is/was not binary safe)
Cheers,
Pierre
Pierre Joye wrote:
hi Ulf,
PHP streams are a better choice than PHP strings for handling large string
objects. PHP strings are loaded into memory at once. For a streaming engine
you do not want to load large portions of data into memory unless you really
have to.Why does this function return a resource? What type of resource is it and
what is the resource handle used for?
- string mysqli_put_blob ( mysqli link, string file_path, string mode ,
int
length )file_path: is the path of the file which is to be stored in a blob field.
mode: mode specifier if the file should be read ASCII mode or binary mode.
PHP strings are binary safe. There are no ASCII mode strings. Why do you
want to distinguish between ASCII and binary strings? Why is the
functionality restricted to existing files? Looks a bit like its inspired by
ftp or Windows?That's partaially incorrect. PHP streams are not always binary safe by
default, at least on windows (linux has no 'b' but for API
compatibility reasons afair).Unless you specify the binary mode explicitly. A common problem is to
run under cgi, cli or apache on windows, the default mode may differ
(apache for example is/was not binary safe)
Sorry, forgot about the UOS (Unreal OS :) Anyway, a blob is a blob,
binary large object data, so binary should be enforced.
Andrey
Andrey Hristov, Connectors Software Developer, Database Group
Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB161028
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering