# Proposal: php://memory/ (& ://temp/) - Thread: https://externals.io/message/65528 - Original archive: https://news-web.php.net/php.internals/65528 - Started by Ivan Enderlin @ Hoa on 2013-01-31T09:37:53+00:00 - Messages: 14 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #65528 — Ivan Enderlin @ Hoa — 2013-01-31T09:37:53+00:00 URL: https://externals.io/message/65528#65528 Hi internals, The php://memory, and its respectful sibling php://temp, appear very useful when we need to compute streams on-the-fly. They offer a lot of services, like avoiding HDD accesses, increasing speed... and through a straightforward API /à la/ PHP, e.g. fopen('php://memory', 'r+'). However, we always work on the same "bucket"/space of data (per runtime), which prevent us to use two different streams at a time. My proposal is to allow access to many "buckets"/spaces to php://memory and its sibling. I propose the following syntax: php://memory/ php://temp//maxmemory: We append / to the existing syntax. Thus, it would be possible to work on php://memory/foo and php://memory/bar without collisions between them. They would represent two distinct "buckets"/spaces. The implementation does not seem to require a ton of work, and the use cases are numerous regarding the services offered by these wrappers. Thoughts? Best regards. -- Ivan Enderlin Developer of Hoa http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #65529 — Attila Bukor — 2013-01-31T10:00:02+00:00 URL: https://externals.io/message/65528#65529 · In reply to: #65528 Hi Ivan, I've never actually worked with them, but it seems nice. However, we should make redirect php://memory and php://temp to php://memory/default and php://temp/default respectively to avoid break stuff imho. Cheers, r1pp3rj4ck On Thu, Jan 31, 2013 at 10:37 AM, Ivan Enderlin @ Hoa < ivan.enderlin@hoa-project.net> wrote: > Hi internals, > > The php://memory, and its respectful sibling php://temp, appear very > useful when we need to compute streams on-the-fly. They offer a lot of > services, like avoiding HDD accesses, increasing speed... and through a > straightforward API /à la/ PHP, e.g. fopen('php://memory', 'r+'). However, > we always work on the same "bucket"/space of data (per runtime), which > prevent us to use two different streams at a time. My proposal is to allow > access to many "buckets"/spaces to php://memory and its sibling. > > I propose the following syntax: > > php://memory/ > php://temp//maxmemory:<**size> > > We append / to the existing syntax. Thus, it would be possible to work > on php://memory/foo and php://memory/bar without collisions between them. > They would represent two distinct "buckets"/spaces. > > The implementation does not seem to require a ton of work, and the use > cases are numerous regarding the services offered by these wrappers. > > Thoughts? > Best regards. > > -- > Ivan Enderlin > Developer of Hoa > http://hoa-project.net/ > > PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) > http://disc.univ-fcomte.fr/ and http://www.inria.fr/ > > Member of HTML and WebApps Working Group of W3C > http://w3.org/ --- ## #65530 — Ivan Enderlin @ Hoa — 2013-01-31T10:11:59+00:00 URL: https://externals.io/message/65528#65530 · In reply to: #65529 > Hi Ivan, Hi Attila, > I've never actually worked with them, but it seems nice. However, we should > make redirect php://memory and php://temp to php://memory/default and > php://temp/default respectively to avoid break stuff imho. Obviously yes! Sorry, I miss to precise this point. Thank you for the comment. Best regards. -- Ivan Enderlin Developer of Hoa http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #65531 — ALeX — 2013-01-31T10:12:25+00:00 URL: https://externals.io/message/65528#65531 · In reply to: #65528 Hi Ivan, > php://memory/ > php://temp//maxmemory: It's not that hard to write such an function in php. (http://www.php.net/manual/en/function.stream-wrapper-register.php). I even once wrote an handler like "string://data-right-here" which allows to read (only) the "data-right-here" as a file. (I used it for a function which required a file and does not support strings directly, and instead of php://memory all data is freed when no longer used) --- ## #65532 — Chris Wright — 2013-01-31T10:34:41+00:00 URL: https://externals.io/message/65528#65532 · In reply to: #65528 > I propose the following syntax: > > php://memory/ > php://temp//maxmemory: I would very much like to see this as well. Would this also allow you to open multiple pointers to the same bucket? For example would this work? > > […] > > I would very much like to see this as well. > > Would this also allow you to open multiple pointers to the same bucket? > > For example would this work? > > > $fp = fopen('php://memory/foo', 'w+'); > > file_put_contents('php://memory/foo', 'data'); > > rewind($fp); // would this be necessary? > > echo stream_get_contents($fp); // outputs 'data' > Everything else wouldn't make much sense, because why else should you need to define an id for it? Bu whats about file_put_contents('php://memory/foo', 'data'); $fp = fopen('php://memory/foo', 'w+'); 'file_put_contents()' closes the stream after writing. I would expect, that the engine will cleanup the memory (at least sooner or later), but then I open it again. Additional why don't you just pass the open stream around, if you need to access it at multiple places? Regards, Sebastian > > Best regards > Chris Wright > > -- > > > > -- github.com/KingCrunch --- ## #65534 — Ivan Enderlin @ Hoa — 2013-01-31T11:00:23+00:00 URL: https://externals.io/message/65528#65534 · In reply to: #65531 > Hi Ivan, > > > […] > It's not that hard to write such an function in php. > (http://www.php.net/manual/en/function.stream-wrapper-register.php). Did I say it was hard? This is quite obvious to write such a wrapper, but PHP has php://memory. My proposal will save future work to users (and offer better performance because we stay in the PHP land). > I even once wrote an handler like "string://data-right-here" which > allows to read (only) the "data-right-here" as a file. (I used it for > a function which required a file and does not support strings > directly, and instead of php://memory all data is freed when no longer > used) Yes. I also have Hoa\Stringbuffer or some stuff like that. But, again, it will save future work to users. Best regards. -- Ivan Enderlin Developer of Hoa http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #65535 — Ivan Enderlin @ Hoa — 2013-01-31T11:04:23+00:00 URL: https://externals.io/message/65528#65535 · In reply to: #65533 > 2013/1/31 Chris Wright > > > […] > > Everything else wouldn't make much sense, because why else should you need > to define an id for it? > > Bu whats about > > > file_put_contents('php://memory/foo', 'data'); > $fp = fopen('php://memory/foo', 'w+'); > > 'file_put_contents()' closes the stream after writing. I would expect, that > the engine will cleanup the memory (at least sooner or later), but then I > open it again. Yup, I also expect the engine to clean memory when closing php://memory or php://temp, but this is another proposal I think, no? > Additional why don't you just pass the open stream around, if you need to > access it at multiple places? Is this question for me or Chris? If it is for me, I don't want multiple accesses to the same “bucket”/space, I want multiple “buckets”/spaces. Best regards. -- Ivan Enderlin Developer of Hoa http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #65537 — Gustavo Lopes — 2013-01-31T11:18:58+00:00 URL: https://externals.io/message/65528#65537 · In reply to: #65528 On Thu, 31 Jan 2013 10:37:53 +0100, Ivan Enderlin @ Hoa wrote: > The php://memory, and its respectful sibling php://temp, appear very > useful when we need to compute streams on-the-fly. They offer a lot of > services, like avoiding HDD accesses, increasing speed... and through a > straightforward API /à la/ PHP, e.g. fopen('php://memory', 'r+'). > However, we always work on the same "bucket"/space of data (per > runtime), which prevent us to use two different streams at a time. My > proposal is to allow access to many "buckets"/spaces to php://memory and > its sibling. > I'm not sure what you mean here. Each time you open a php://memory stream, you're working on a new "bucket", right? $ cat a.php && php a.php > > > > > […] > Yup, I also expect the engine to clean memory when closing php://memory or > php://temp, but this is another proposal I think, no? > > > Additional why don't you just pass the open stream around, if you need to > > […] > Is this question for me or Chris? A question for everone :) > If it is for me, I don't want multiple accesses to the same > “bucket”/space, I want multiple “buckets”/spaces. So now I am confused: As far as I can see you can open as many memory- or temp-streams as you like. > > > Best regards. > > -- > Ivan Enderlin > Developer of Hoa > http://hoa-project.net/ > > PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) > http://disc.univ-fcomte.fr/ and http://www.inria.fr/ > > Member of HTML and WebApps Working Group of W3C > http://w3.org/ > > > > -- > > > > -- github.com/KingCrunch --- ## #65538 — Chris Wright — 2013-01-31T11:22:00+00:00 URL: https://externals.io/message/65528#65538 · In reply to: #65535 > > > […] > Yup, I also expect the engine to clean memory when closing php://memory or php://temp, but this > is another proposal I think, no? > > […] > Is this question for me or Chris? If it is for me, I don't want multiple > accesses to the same "bucket"/space, I want multiple "buckets"/spaces. My thinking was for stuff like ZipArchive, which does not accept streams, only paths. As it is there is no way to use ZipArchive entirely in memory, you have to write it to disk (it also currently cannot cope with URLs so this alone would not fix it, but that's another story). I admit the use case is narrow, but I do have a specific case where I need to post-process the resulting data before outputting/writing to the final file, this functionality would be a step towards working around this. I'm sure there are other examples of this as well. As ALeX pointed out, there are other ways to work in this manner using custom stream wrappers, and if it would be too difficult/wouldn't make sense to the engine to do this, then so be it :) Best Regards Chris Wright --- ## #65540 — Ivan Enderlin @ Hoa — 2013-01-31T13:55:26+00:00 URL: https://externals.io/message/65528#65540 · In reply to: #65537 > On Thu, 31 Jan 2013 10:37:53 +0100, Ivan Enderlin @ Hoa > wrote: > > > […] > > I'm not sure what you mean here. Each time you open a php://memory > stream, you're working on a new "bucket", right? Yes exact. My bad. But my proposal still persist. Identifying php://memory buckets would be nice to retrieve them. For example: $f1 = fopen('php://memory/foo', 'r+'); fwrite($f1, 'foobar'); unset($f1); $f1prime = fopen('php://memory/foo', 'r+'); rewind($f1); var_dump(stream_get_contents($f1)); // string(6) "foobar" Is it interesting? In this way, we have a similar behavior than php://fd/, file://, http:// etc. Best regards. -- Ivan Enderlin Developer of Hoa http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #65541 — Gustavo Lopes — 2013-01-31T14:35:14+00:00 URL: https://externals.io/message/65528#65541 · In reply to: #65540 On Thu, 31 Jan 2013 14:55:26 +0100, Ivan Enderlin @ Hoa wrote: > > […] > Yes exact. My bad. But my proposal still persist. Identifying > php://memory buckets would be nice to retrieve them. For example: > > $f1 = fopen('php://memory/foo', 'r+'); > fwrite($f1, 'foobar'); > unset($f1); > > $f1prime = fopen('php://memory/foo', 'r+'); > rewind($f1); > var_dump(stream_get_contents($f1)); // string(6) "foobar" > > Is it interesting? In this way, we have a similar behavior than > php://fd/, file://, http:// etc. > No. When you do unset($f1), you're closing the file and the memory is reclaimed. That's how it's supposed to be. Just pass the stream along. If you really need to use operations on the URLs themselves, you can easily write a user wrapper that delegates to static php://memory streams kept statically. -- Gustavo Lopes --- ## #65542 — Ivan Enderlin @ Hoa — 2013-01-31T14:38:54+00:00 URL: https://externals.io/message/65528#65542 · In reply to: #65541 > On Thu, 31 Jan 2013 14:55:26 +0100, Ivan Enderlin @ Hoa > wrote: > > > […] > > No. When you do unset($f1), you're closing the file and the memory is > reclaimed. That's how it's supposed to be. Ok. > Just pass the stream along. If you really need to use operations on > the URLs themselves, you can easily write a user wrapper that > delegates to static php://memory streams kept statically. That's what I did. Thank you for the clarification. -- Ivan Enderlin Developer of Hoa http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/