# feature request : easy shared memory - Thread: https://externals.io/message/66604 - Original archive: https://news-web.php.net/php.internals/66604 - Started by rene7705 on 2013-03-14T07:05:03+00:00 - Messages: 11 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #66604 — rene7705 — 2013-03-14T07:05:03+00:00 URL: https://externals.io/message/66604#66604 Hi. I'd like to build a replacement for SQL (yes, talk about an ambitious project! ;), because the constant transferal of data in and out of SQL from Javascript (where everything might as well be object-oriented and hierarchial) is a pain in the neck. But in order to do so, I'd very much like PHP (the server still controls the data after all) to support shared memory efficiently. Something like sharedmem $bigNestedArray; // $bigNestedArray would be shared accross the entire server and all CPUs on it. as you now have global $bigNestedArray; would be ideal. I bet this would be useful for a host of other applications as well, and fairly easy to implement. I'm an application programmer by trade, or I would hack this in myself. I'd much rather see the PHP development team develop this in properly. I don't think it would require much time, as OS-level shared memory has been easy to implement since the 1990s. Please put this on the agenda, and get back to us in this thread as to when this will be available. --- ## #66605 — rene7705 — 2013-03-14T07:08:23+00:00 URL: https://externals.io/message/66604#66605 · In reply to: #66604 oh, if no PHP threads/processes are active, all sharedmem's would still be allocated by the PHP OS-level process itself, of course, to prevent any overhead as new PHP calls are made that access this sharedmem. > Hi. > > I'd like to build a replacement for SQL (yes, talk about an ambitious > project! ;), because the constant transferal of data in and out of SQL from > Javascript (where everything might as well be object-oriented and > hierarchial) is a pain in the neck. > > But in order to do so, I'd very much like PHP (the server still controls > the data after all) to support shared memory efficiently. > > Something like > > sharedmem $bigNestedArray; // $bigNestedArray would be shared accross the > entire server and all CPUs on it. > > as you now have > > global $bigNestedArray; > > would be ideal. > > I bet this would be useful for a host of other applications as well, and > fairly easy to implement. > > I'm an application programmer by trade, or I would hack this in myself. > > I'd much rather see the PHP development team develop this in properly. I > don't think it would require much time, as OS-level shared memory has been > easy to implement since the 1990s. > > Please put this on the agenda, and get back to us in this thread as to > when this will be available. --- ## #66607 — Chris Stockton — 2013-03-14T08:08:33+00:00 URL: https://externals.io/message/66604#66607 · In reply to: #66604 Might want to look into redis or memcache also, you can get a bit of persistence and distribution that's a bit tougher when your married to an apache instance. -Chris > Hi. > > I'd like to build a replacement for SQL (yes, talk about an ambitious > project! ;), because the constant transferal of data in and out of SQL from > Javascript (where everything might as well be object-oriented and > hierarchial) is a pain in the neck. > > But in order to do so, I'd very much like PHP (the server still controls > the data after all) to support shared memory efficiently. > > Something like > > sharedmem $bigNestedArray; // $bigNestedArray would be shared accross the > entire server and all CPUs on it. > > as you now have > > global $bigNestedArray; > > would be ideal. > > I bet this would be useful for a host of other applications as well, and > fairly easy to implement. > > I'm an application programmer by trade, or I would hack this in myself. > > I'd much rather see the PHP development team develop this in properly. I > don't think it would require much time, as OS-level shared memory has been > easy to implement since the 1990s. > > Please put this on the agenda, and get back to us in this thread as to when > this will be available. --- ## #66606 — Nikita Nefedov — 2013-03-14T11:45:14+00:00 URL: https://externals.io/message/66604#66606 · In reply to: #66604 > Hi. > > I'd like to build a replacement for SQL (yes, talk about an ambitious > project! ;), because the constant transferal of data in and out of SQL > from > Javascript (where everything might as well be object-oriented and > hierarchial) is a pain in the neck. > > But in order to do so, I'd very much like PHP (the server still controls > the data after all) to support shared memory efficiently. > > Something like > > sharedmem $bigNestedArray; // $bigNestedArray would be shared accross the > entire server and all CPUs on it. > > as you now have > > global $bigNestedArray; > > would be ideal. > > I bet this would be useful for a host of other applications as well, and > fairly easy to implement. > > I'm an application programmer by trade, or I would hack this in myself. > > I'd much rather see the PHP development team develop this in properly. I > don't think it would require much time, as OS-level shared memory has > been > easy to implement since the 1990s. > > Please put this on the agenda, and get back to us in this thread as to > when > this will be available. Hi, You can already do it using APC's apc_store() and apc_fetch() functions which let you use shared memory. But of course you should 'commit' every change of the fetched variable. --- ## #66613 — rene7705 — 2013-03-14T14:31:36+00:00 URL: https://externals.io/message/66604#66613 · In reply to: #66606 great! :) this will do nicely. is there much overhead for storing and fetching these variables? (ideally I would like to get a pointer) > > > Hi. > > […] > > Hi, > > You can already do it using APC's apc_store() and apc_fetch() functions > which let you use shared memory. But of course you should 'commit' every > change of the fetched variable. --- ## #66615 — Lazare Inepologlou — 2013-03-14T15:46:56+00:00 URL: https://externals.io/message/66604#66615 · In reply to: #66613 Hello, 2013/3/14 rene7705 > great! :) > > this will do nicely. is there much overhead for storing and fetching these > variables? (ideally I would like to get a pointer) > > Unfortunately there is. Every object stored in APC has to be serialised first and then unserialised on retrieval. You can improve performance by using a binary serialiser (such as igBinary) instead of the default php one. > > On Thu, Mar 14, 2013 at 12:45 PM, Nikita Nefedov > wrote: > > > […] > the > > […] > been > > […] > Lazare INEPOLOGLOU Ingénieur Logiciel --- ## #66616 — Steve Clay — 2013-03-14T15:49:02+00:00 URL: https://externals.io/message/66604#66616 · In reply to: #66613 > (ideally I would like to get a pointer) PHP's environment is torn down after every request, so no matter what the mechanism you generally can't store anything that can't be serialized. See also https://www.google.com/search?q=php+shared+memory Steve Clay -- http://www.mrclay.org/ --- ## #66617 — Bob Weinand — 2013-03-14T16:13:58+00:00 URL: https://externals.io/message/66604#66617 · In reply to: #66615 Am 14.3.2013 um 16:46 schrieb Lazare Inepologlou : > Hello, > > 2013/3/14 rene7705 > > > […] > Unfortunately there is. Every object stored in APC has to be serialised > first and then unserialised on retrieval. You can improve performance by > using a binary serialiser (such as igBinary) instead of the default php one. > > > > > > […] > > > Lazare INEPOLOGLOU > Ingénieur Logiciel Hi! And there is no possibility to store the zval as raw binary data like in memory (deep copy?) So that you only have to copy from ram? And replace the pointers to the place in the string? This must be possible I think. And should be faster. shmop has to be opened on every request and only supports strings. APC, memcache,... can only save under serialized form which is slow. Why not integrate some native shared memory mechanism which is faster than the extensions which always serialize? (or as a new extension… but I really like the proposal to do like "shared $var;" which is (nearly) impossible as an extension) Bob Weinand --- ## #66621 — Rasmus Lerdorf — 2013-03-14T17:14:15+00:00 URL: https://externals.io/message/66604#66621 · In reply to: #66617 > And there is no possibility to store the zval as raw binary data like in memory (deep copy?) > So that you only have to copy from ram? And replace the pointers to the place in the string? > This must be possible I think. And should be faster. > > shmop has to be opened on every request and only supports strings. > APC, memcache,... can only save under serialized form which is slow. APC doesn't serialize most types. Only actual objects need to be serialized because it is the easiest way to fully save and restore objects. eg. calling their __sleep()/__wakeup() magic methods, etc. Arrays are not serialized. -Rasmus --- ## #66622 — Bob Weinand — 2013-03-14T17:28:44+00:00 URL: https://externals.io/message/66604#66622 · In reply to: #66621 Am 14.3.2013 um 18:14 schrieb Rasmus Lerdorf : > > > […] > > APC doesn't serialize most types. Only actual objects need to be > serialized because it is the easiest way to fully save and restore > objects. eg. calling their __sleep()/__wakeup() magic methods, etc. > Arrays are not serialized. > > -Rasmus Thanks, ..., okay, didn't know that. But even now I am in favor of a new keyword as it will be easier to have a reference to the shared memory (written in and reread from memory when modified) than every time refetching it when the shared memory block may have changed in an other program (what could really reduce race-conditions implicitly as as a developer you may forget to refetch the variable from shared memory). Yes, refetching always is already possible with an userland getter/setter, but I don't think it's best practice to do so in PHP... Bob Weinand --- ## #66623 — Larry Garfield — 2013-03-14T17:47:28+00:00 URL: https://externals.io/message/66604#66623 · In reply to: #66622 > Am 14.3.2013 um 18:14 schrieb Rasmus Lerdorf : > > > […] > > Thanks, ..., okay, didn't know that. > > But even now I am in favor of a new keyword as it will be easier to have a reference to the > shared memory (written in and reread from memory when modified) than every time refetching it > when the shared memory block may have changed in an other program (what could really reduce > race-conditions implicitly as as a developer you may forget to refetch the variable from shared > memory). Yes, refetching always is already possible with an userland getter/setter, but I don't think > it's best practice to do so in PHP... > > Bob Weinand Sharing active memory between processes goes against the "shared nothing" design of PHP. The lack of the feature you're describing is itself a feature. :-) If you had real shared memory, then you're now writing a multi-threaded app. Even if you aren't using threads per se it's the same level of potential for spooky action at a distance. If your problem space really requires that (and there certainly are those that do), Java or NodeJs will suit you better because those are built specifically for a persistent-server model, rather than PHP's shared-nothing design. However, in practice most PHP/web applications don't need that, because HTTP is a stateless request/response system. Shared-nothing more closely models what the actual environment is doing, and can still be very performant as long as you don't do anything naive. If you're doing something stateful like Web Sockets, then you can run PHP as a cli application that is its own persistent server rather than as an Apache add-on. For that, look at Ratchet: http://socketo.me/ --Larry Garfield