# Streaming Enabled MySQL Driver for PHP - Google Summer of Code 2008 - MySQL

- Thread: https://externals.io/message/37910
- Original archive: https://news-web.php.net/php.internals/37910
- Started by KishoreKumar Bairi on 2008-05-26T20:55:47+00:00
- Messages: 4
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #37910 — KishoreKumar Bairi — 2008-05-26T20:55:47+00:00

URL: https://externals.io/message/37910#37910

Hello all,

I'm KishoreKumar Bairi, Google Summer of Code Student for MySQL.

Project: *Streaming Enabled MySQL Driver for PHP*

Details:

MyBS[1] is a storage engine used along with other engines to manage BLOB
data efficiently.
MyBS maintains a blob repository for each database to store the blobs
belonging to that database. Each blob stored in this repository has a unique
ID which can be used to reference that blob. MyBS has an embedded light
weight webserver which interfaces the blob repository and client.

 BLOBs can be stored in this repository by sending HTTP PUT request to the
embedded web server which returns the unique ID that can be stored in
database to refer this blob later. To retrieve this BLOB, HTTP GET request
for referencing URL is sent to the embedded web server.

This task involves addition of two functions to 'mysqli'.

1. resource mysqli_get_blob ( mysqli link, string reference )

reference: is the string which used after database part in the referencing
URL.
It can be repository Id (or) "<table>/<column>/<row condition>".

2. string mysqli_put_blob ( mysqli link, string file_path, string mode , int
length )

file_path: is the path of the file which is to be stored in a blob field.

mode: mode specifier if the file should be read ASCII mode or binary mode.

link: is the object representing the MySQL database connection.
So, finally this is how the usage will be.

Procedural style:
1. resource mysqli_get_blob ( mysqli link , string reference )
2. string mysqli_put_blob (mysqli link ,string file_path, string mode , int
length)

Object Oriented style:

(methods)

mysqli {
   resource mysqli_get_blob ( string reference )
}

mysqli {
   string mysqli_put_blob(string file_path, string mode , int length)
}

Looking forward for suggestions and comments.


[1] http://www.blobstreaming.org/


Regards,
KishoreKumar Bairi.

---

## #37916 — Ulf Wendel — 2008-05-27T13:07:20+00:00

URL: https://externals.io/message/37910#37916 · In reply to: #37910

Hi!

KishoreKumar Bairi schrieb:
> 1. resource mysqli_get_blob ( mysqli link, string reference )
> 
> reference: is the string which used after database part in the referencing
> URL.
> It can be repository Id (or) "<table>/<column>/<row condition>".

PHP streams are a better choice than PHP strings for handling large 
string objects. PHP strings are loaded into memory at once. For a 
streaming engine you do not want to load large portions of data into 
memory unless you really have to.

Why does this function return a resource? What type of resource is it 
and what is the resource handle used for?

> 
> 2. string mysqli_put_blob ( mysqli link, string file_path, string mode , int
> length )
> 
> file_path: is the path of the file which is to be stored in a blob field.
> 
> mode: mode specifier if the file should be read ASCII mode or binary mode.

PHP strings are binary safe. There are no ASCII mode strings. Why do you 
want to distinguish between ASCII and binary strings? Why is the 
functionality restricted to existing files? Looks a bit like its 
inspired by ftp or Windows?

In general, I think such blob streaming functionality should use PHP 
streams. After some discussion with Andrey I'm not even sure if there is 
a need to introduce a single new API call to ext/mysqli. Andrey brought 
up the idea of using a streams protocol wrapper [1]. This would give you 
quite a neat API which is more powerful and flexible than the above 
suggestion.

Here's an example in pseudo-code.

Reading data from the blob repository:

$stream = fopen('mysqlblob://user:password@socket/db/table/whateverid', 
'r');
while ($chunk = fread($stream, 1024))
   // read blob in 1k chunks
   echo $chunk;
fclose($stream);

Writing data from a file to the blob repository:

$blob_stream = 
fopen('mysqlblob://user:password@host/https=0&id=/db/table/id', 'rw');
$file_stream = fopen('myimage.jpg');
$num_chunks = 0;
while (stream_copy_to_stream($file_stream, $blob_stream, 1024) > 0) {
   printf("Writing to stream... %04dkb\n", $num_chunks++);
}
// the unique id generated by the blob engine to identify the blob
$blob_id = fgets($blob_stream);
fclose($file_stream);
fclose($blob_stream);

Generating data in PHP and writing to the blob repository:

$blob_stream = fopen('mysqlblob://user:password@socket/options', 'w');
// create thumbnail
$image = new Imagick('image.jpg');
$image->thumbnailImage(100, 0);
fwrite($blob_stream, $image);
// get unique blog id from the blob engine
$blob_id = fgets($blob_stream);
fclose($blob_stream);
echo $image;

And, and, and... - streams are a very mighty tool. I'm not sure about 
the syntax for connections. It could be something like this:

URL:
   mysqlblob://user:password@[socket|host]?[param=value[&param=value...]]

Params:
  id       - id of the blob to fetch/read
  port     - MySQL server port
  ssl      - SSL parameter
  ssh      - SSH parameter

Whatever you need.

Ulf


[1] http://www.php.net/manual/en/wrappers.php


-- 
Ulf Wendel, MySQL
Sun Microsystems GmbH,   Sonnenallee 1,   D-85551 Kirchheim-Heimstetten
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering     Muenchen: HRB161028

---

## #37917 — Pierre Joye — 2008-05-27T13:16:18+00:00

URL: https://externals.io/message/37910#37917 · In reply to: #37916

hi Ulf,



> PHP streams are a better choice than PHP strings for handling large string
> objects. PHP strings are loaded into memory at once. For a streaming engine
> you do not want to load large portions of data into memory unless you really
> have to.
>
> Why does this function return a resource? What type of resource is it and
> what is the resource handle used for?
>
> > […]
>
> PHP strings are binary safe. There are no ASCII mode strings. Why do you
> want to distinguish between ASCII and binary strings? Why is the
> functionality restricted to existing files? Looks a bit like its inspired by
> ftp or Windows?

That's partaially incorrect. PHP streams are not always binary safe by
default, at least on windows (linux has no 'b' but for API
compatibility reasons afair).

Unless you specify the binary mode explicitly. A common problem is to
run under cgi, cli or apache on windows, the default mode may differ
(apache for example is/was not binary safe)

Cheers,
-- 
Pierre

http://blog.thepimp.net | http://www.libgd.org

---

## #37918 — Andrey Hristov — 2008-05-27T13:21:01+00:00

URL: https://externals.io/message/37910#37918 · In reply to: #37917

Pierre Joye wrote:
> hi Ulf,
> 
> 
> 
> > […]
> 
> That's partaially incorrect. PHP streams are not always binary safe by
> default, at least on windows (linux has no 'b' but for API
> compatibility reasons afair).
> 
> Unless you specify the binary mode explicitly. A common problem is to
> run under cgi, cli or apache on windows, the default mode may differ
> (apache for example is/was not binary safe)

Sorry, forgot about the UOS (Unreal OS :) Anyway, a blob is a blob, 
binary large object data, so binary should be enforced.

Andrey
-- 
Andrey Hristov, Connectors Software Developer, Database Group
Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB161028
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering