# Closures serialization - Thread: https://externals.io/message/42452 - Original archive: https://news-web.php.net/php.internals/42452 - Started by Marcin Kurzyna on 2009-01-02T19:56:41+00:00 - Messages: 5 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #42452 — Marcin Kurzyna — 2009-01-02T19:56:41+00:00 URL: https://externals.io/message/42452#42452 Hello internals and a happy new year. I've been meaning to ask this for some time now: why aren't closures serializable? The only on-list discussion about this that i've found is: http://marc.info/?l=php-internals&m=119837318407795&w=2 Personally i'd find serialization of closures very useful (think serializing of callback arrays, serialization of objects that acquired closure somewhere during execution, etc) Also i find that being unable to serialize closure introduces a slight inconsistency. It was possible to serialize "is callable" type (plain array/string) whereas now it's not always the case[1] so programmers have to be aware of it when using some kind of "register callback" behavior. So to my question: why aren't closures serializable? Is it a design choice or implementation issue? I've looked at both zend_closure.c (for closure implementation) and var.c (for serialization) but didn't see anything that could suggest either way. Now for some wild guessing: In the internals post mentioned above it's stated that annonymous function name is along the lines of hashfunction(__FILE__) and per_file_counter; i understand that it's deterministic, and so if i were to serialize just this id (as sting) it should be possible to track back to the actual code after deserialization? i imagine that it would require (in simple terms) getting the original file and "executing/assigning" n-th per_file closure. It should be even simpler when done in the same request and/or with opcode caches where just the id should be sufficient to get the original opcodes and execute. I'll look some more into it myself but i'm not that fluent in C to provide quality patch (not to mention zend api). I'd like to know your opinion whether this could be implemented though. TIA Marcin Kurzyna [1] While it was possible to serialize an array of callbacks before it's impossible with closures assigned, see example below: http://aquarion.hq.crystalpoint.pl/public/php-internals/closure-serialization.php --- ## #42453 — Marcus Boerger — 2009-01-02T20:12:22+00:00 URL: https://externals.io/message/42452#42453 · In reply to: #42452 Hello Marcin, Friday, January 2, 2009, 8:56:41 PM, you wrote: > Hello internals and a happy new year. > I've been meaning to ask this for some time now: why aren't closures > serializable? The only on-list discussion about this that i've found is: > http://marc.info/?l=php-internals&m=119837318407795&w=2 > Personally i'd find serialization of closures very useful (think > serializing of callback arrays, serialization of objects that acquired > closure somewhere during execution, etc) > Also i find that being unable to serialize closure introduces a slight > inconsistency. It was possible to serialize "is callable" type (plain > array/string) whereas now it's not always the case[1] so programmers > have to be aware of it when using some kind of "register callback" behavior. > So to my question: why aren't closures serializable? Is it a design > choice or implementation issue? > I've looked at both zend_closure.c (for closure implementation) and > var.c (for serialization) but didn't see anything that could suggest > either way. > Now for some wild guessing: In the internals post mentioned above it's > stated that annonymous function name is along the lines of > hashfunction(__FILE__) and per_file_counter; i understand that it's > deterministic, and so if i were to serialize just this id (as sting) it > should be possible to track back to the actual code after > deserialization? i imagine that it would require (in simple terms) > getting the original file and "executing/assigning" n-th per_file > closure. It should be even simpler when done in the same request and/or > with opcode caches where just the id should be sufficient to get the > original opcodes and execute. > I'll look some more into it myself but i'm not that fluent in C to > provide quality patch (not to mention zend api). I'd like to know your > opinion whether this could be implemented though. You would need to provide a c level serialization that stores the $this pointer (the easy part) and the zend_function member (not easy). The second part requires storing of all static variables, which again is pretty easy but it also requires to store the function itself, which in case of a user function probably only means the string but then at the time of serialization you only have the compiled opcodes. So you would need APC to serialize the opcodes, which is not yet implemented. Alternatively you could save the implementation string somewhere. And serialize that and compiling it upon unserialization. If you're more interested, c level serialization with multiple serialization inputs is used for: SplObjectStorage::serialize() / `unserialize()`. marcus > TIA > Marcin Kurzyna > [1] While it was possible to serialize an array of callbacks before it's > impossible with closures assigned, see example below: > http://aquarion.hq.crystalpoint.pl/public/php-internals/closure-serialization.php Best regards, Marcus --- ## #42460 — Marcin Kurzyna — 2009-01-03T13:12:55+00:00 URL: https://externals.io/message/42452#42460 · In reply to: #42453 Hi Marcus, > You would need to provide a c level serialization that stores the $this > pointer (the easy part) and the zend_function member (not easy). The > second part requires storing of all static variables, which again is > pretty easy but it also requires to store the function itself, which in > case of a user function probably only means the string but then at the > time of serialization you only have the compiled opcodes. So you would > need APC to serialize the opcodes, which is not yet implemented. > Alternatively you could save the implementation string somewhere. And > serialize that and compiling it upon unserialization. If you're more > interested, c level serialization with multiple serialization inputs is > used for: SplObjectStorage::serialize() / `unserialize()`. thnx for your insight. i've looked at SplObjectStorage::serialize and it seems pretty straight forward - this shouldn't be a problem. as you said it's saving the zend_function member that is tricky. i thought i'd go with saving function string and recompiling it as you suggested (i don't want to mess with APC although i find the idea of opcode serialization much more appealing). anyway it's proving itself, well... tricky ;-) i'm thinking about hooking into zend_do_beging_lambda_function_declaration and saving all until zend_do_end_function_declaration, but i haven't tried that in practice yet (not even sure how to do this at the moment - i'll get to it tonight). thnx marcin --- ## #42461 — Marcin Kurzyna — 2009-01-03T13:13:24+00:00 URL: https://externals.io/message/42452#42461 · In reply to: #42453 Hi Marcus, > You would need to provide a c level serialization that stores the $this > pointer (the easy part) and the zend_function member (not easy). The > second part requires storing of all static variables, which again is > pretty easy but it also requires to store the function itself, which in > case of a user function probably only means the string but then at the > time of serialization you only have the compiled opcodes. So you would > need APC to serialize the opcodes, which is not yet implemented. > Alternatively you could save the implementation string somewhere. And > serialize that and compiling it upon unserialization. If you're more > interested, c level serialization with multiple serialization inputs is > used for: SplObjectStorage::serialize() / `unserialize()`. thnx for your insight. i've looked at SplObjectStorage::serialize and it seems pretty straight forward - this shouldn't be a problem. as you said it's saving the zend_function member that is tricky. i thought i'd go with saving function string and recompiling it as you suggested (i don't want to mess with APC although i find the idea of opcode serialization much more appealing). anyway it's proving itself, well... tricky ;-) i'm thinking about hooking into zend_do_beging_lambda_function_declaration and saving all until zend_do_end_function_declaration, but i haven't tried that in practice yet (not even sure how to do this at the moment - i'll get to it tonight). thnx marcin --- ## #42469 — johannes@schlueters.de — 2009-01-04T13:35:39+00:00 URL: https://externals.io/message/42452#42469 · In reply to: #42460 > as you said it's saving the zend_function member that is tricky. i > thought i'd go with saving function string and recompiling it as you > suggested (i don't want to mess with APC although i find the idea of > opcode serialization much more appealing). anyway it's proving itself, > well... tricky ;-) Two thoughts: - It can be quite tricky to keep the binding of "use"d variables correct - Serialized data soften used as data exchange format (for instance ) the default unserialize shouldn't introduce executable code even though it's not executed by default we should be careful there. And yes, I can understand the need to serialize object structures, including closures, to sessions... johannes