# PHP 5.2: Serialization patch - Thread: https://externals.io/message/26756 - Original archive: https://news-web.php.net/php.internals/26756 - Started by Andrei Zmievski on 2006-12-01T21:43:32+00:00 - Messages: 19 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #26756 — Andrei Zmievski — 2006-12-01T21:43:32+00:00 URL: https://externals.io/message/26756#26756 When serializing binary strings in PHP 6, we have to escape non-ASCII characters and then unescape them on unserialization. This patch adds the unescapement support to PHP 5.2, in order to make it easier to exchange data between PHP 5 and 6. If no one has objections, I will commit soon. -Andrei --- ## #26757 — Ilia Alshanetsky — 2006-12-01T21:51:42+00:00 URL: https://externals.io/message/26756#26757 · In reply to: #26756 Before we commit, let's explore how it will affect existing applications. The last thing I want to do is break the serialization format BC in a minor release. > When serializing binary strings in PHP 6, we have to escape non- > ASCII characters and then unescape them on unserialization. This > patch adds the unescapement support to PHP 5.2, in order to make it > easier to exchange data between PHP 5 and 6. If no one has > objections, I will commit soon. > > -Andrei > > > -- > > Ilia Alshanetsky --- ## #26758 — Marcus Boerger — 2006-12-01T21:53:43+00:00 URL: https://externals.io/message/26756#26758 · In reply to: #26756 Hello Andrei, this will be a problematic bc break as with the patch applied strings suddenly behave pretty different. Until now our policy was to notintroduce forward compatibility breaks. best regards marcus Friday, December 1, 2006, 10:43:32 PM, you wrote: > When serializing binary strings in PHP 6, we have to escape non-ASCII > characters and then unescape them on unserialization. This patch adds > the unescapement support to PHP 5.2, in order to make it easier to > exchange data between PHP 5 and 6. If no one has objections, I will > commit soon. > -Andrei Best regards, Marcus --- ## #26759 — Andrei Zmievski — 2006-12-01T22:02:41+00:00 URL: https://externals.io/message/26756#26759 · In reply to: #26757 We can put it in 5.3 perhaps. -Andrei > Before we commit, let's explore how it will affect existing > applications. The last thing I want to do is break the serialization > format BC in a minor release. > > > > > > […] > > Ilia Alshanetsky --- ## #26760 — Ilia Alshanetsky — 2006-12-01T23:03:07+00:00 URL: https://externals.io/message/26756#26760 · In reply to: #26756 As it stands the current code breaks BC on decoding when the serialized string contains \ characters. For example: Input PHP 5.2 PHP 5.2 w/patch ------------------------------------------------------------------------ ------------------- s:7:"foo\10b"; foo\10b error (NULL returned) s:7:"foo\bar"; foo\bar error (NULL returned) s:5:"\\\\\"; \\\\\ error (NULL returned) Basically any operation involving strings with \ in them, stop working once the patch is applied. There is also the performance drawback to consider, which based on a rudimentary test involving $_SERVER serialization, shows that the new code is roughly 1/2 slower. 5.2: 0.388 5.2 w/patch: 0.652 Ilia --- ## #26823 — Andrei Zmievski — 2006-12-06T17:41:12+00:00 URL: https://externals.io/message/26756#26823 · In reply to: #26760 I don't see a way we can make it work for all the cases. I guess we'll have to leave this task to PHP_Compat. -Andrei > As it stands the current code breaks BC on decoding when the > serialized string contains \ characters. > > For example: > > Input PHP 5.2 PHP 5.2 w/patch > ----------------------------------------------------------------------- > -------------------- > s:7:"foo\10b"; foo\10b error (NULL returned) > s:7:"foo\bar"; foo\bar error (NULL returned) > s:5:"\\\\\"; \\\\\ error (NULL returned) > > Basically any operation involving strings with \ in them, stop working > once the patch is applied. > > There is also the performance drawback to consider, which based on a > rudimentary test involving $_SERVER serialization, shows that the new > code is roughly 1/2 slower. > > 5.2: 0.388 > 5.2 w/patch: 0.652 > > Ilia --- ## #26826 — Thomas Seifert — 2006-12-07T10:25:25+00:00 URL: https://externals.io/message/26756#26826 · In reply to: #26823 From a users point of view: you must be kidding, eh? You want to break all the strings which were stored serialized in the 5.x-series? I can understand that it will break in php6 but not in some minor release. It will cause havoc with a lot of apps which can't read their cached- / meta-data anymore. Also the performance drawback will get you a lot of angry users. Wasn't that just some versions before when the serialization has gotten a lot slower and created lots of problems? thomas Andrei Zmievski schrieb: > I don't see a way we can make it work for all the cases. I guess we'll > have to leave this task to PHP_Compat. > > -Andrei > > > > > […] --- ## #26828 — Robert Cummings — 2006-12-07T11:26:06+00:00 URL: https://externals.io/message/26756#26828 · In reply to: #26826 > > […] > You want to break all the strings which were stored serialized in the > 5.x-series? I can understand that it will break in php6 but not in some > minor release. > It will cause havoc with a lot of apps which can't read their cached- / > meta-data anymore. > > Also the performance drawback will get you a lot of angry users. > Wasn't that just some versions before when the serialization has gotten > a lot slower and created lots of problems? This is a forwards compatibility issue, if we don't care about version less than or equal to 5.2 being able to understand serialized data from future version then there shouldn't be a problem with backwards compatibility. I propose checking the start of the string to be unserialized for the character v, if found then we can parse out a version number up to the first found comma (or some other delimiter). So for example we have echo serialize( array( 1, 2, 3, 4 ) ); -> a:4:{i:0;i:1;i:1;i:2;i:2;i:3;i:3;i:4;} The new system could have: -> v:5.21,a:4:{i:0;i:1;i:1;i:2;i:2;i:3;i:3;i:4;} Then in newer version of unserialize the version of 5.21 can be extracted and the appropriate unserialization can occur. If no version info is found then fall back to original semantics. Cheers, Rob. -- .------------------------------------------------------------. | InterJinn Application Framework - http://www.interjinn.com | :------------------------------------------------------------: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `------------------------------------------------------------' --- ## #26830 — Stefan Esser — 2006-12-07T11:35:28+00:00 URL: https://externals.io/message/26756#26830 · In reply to: #26823 Guys, why can't you simply introduce a new variable type for serialized strings like uppercase S: There is absolutely no need for breaking backward compatibility. If you want to introduce NEW features then introduce them and do not abuse old ones. Stefan Andrei Zmievski schrieb: > I don't see a way we can make it work for all the cases. I guess we'll > have to leave this task to PHP_Compat. > > -Andrei > > > > > […] --- ## #26831 — Zeev Suraski — 2006-12-07T15:17:34+00:00 URL: https://externals.io/message/26756#26831 · In reply to: #26830 At 13:35 07/12/2006, Stefan Esser wrote: >Guys, > >why can't you simply introduce a new variable type for serialized >strings like uppercase S: >There is absolutely no need for breaking backward compatibility. I can't think of a single reason not to use a new type like you suggest. Can anybody think of any drawbacks? Zeev --- ## #26833 — Ilia Alshanetsky — 2006-12-07T15:53:51+00:00 URL: https://externals.io/message/26756#26833 · In reply to: #26831 > At 13:35 07/12/2006, Stefan Esser wrote: > > […] > > I can't think of a single reason not to use a new type like you > suggest. > > Can anybody think of any drawbacks? Well, older version of PHP will be totally incapable of parsing it, creating problems for the many people who use serialized strings as a means of passing data between PHP applications. This would be an especially big problem if you would want to add it to a pre-6.0 release. Ilia Alshanetsky --- ## #26834 — Stefan Esser — 2006-12-07T16:00:23+00:00 URL: https://externals.io/message/26756#26834 · In reply to: #26833 > Well, older version of PHP will be totally incapable of parsing it, > creating problems for the many people who use serialized strings as a > means of passing data between PHP applications. This would be an > especially big problem if you would want to add it to a pre-6.0 release. Uhmm you are missing the point. The point was that PHP 5.2.x can read data serialized with PHP 6. PHP 5.2.x only needs to be able to read it. Generating it is not necessary. At the moment it seems that data serialized in PHP 5.2.x is totally incompatible with PHP 6. Stefan --- ## #26836 — Andrei Zmievski — 2006-12-07T16:51:43+00:00 URL: https://externals.io/message/26756#26836 · In reply to: #26826 Excuse me, I don't quite understand what you're concerned about. The break would occur if you use PHP 6 serialized content in PHP 5, not vice versa. I already said that having the same escape format in PHP 5.2 is not tractable for BC reasons and the support, as such, must be left to PHP_Compat. -Andrei From a users point of view: you must be kidding, eh? > You want to break all the strings which were stored serialized in the > 5.x-series? I can understand that it will break in php6 but not in some > minor release. > It will cause havoc with a lot of apps which can't read their cached- / > meta-data anymore. > > Also the performance drawback will get you a lot of angry users. > Wasn't that just some versions before when the serialization has gotten > a lot slower and created lots of problems? --- ## #26842 — Chad Daelhousen — 2006-12-07T18:01:07+00:00 URL: https://externals.io/message/26756#26842 · In reply to: #26834 Stefan Esser wrote: > > […] > Uhmm you are missing the point. The point was that PHP 5.2.x can read > data serialized with PHP 6. PHP 5.2.x only needs to be able to read it. > Generating it is not necessary. > > At the moment it seems that data serialized in PHP 5.2.x is totally > incompatible with PHP 6. As 6.0 is not, to my knowledge, anywhere near release, what about this: 1. Stick a version number on the front of PHP6-serialized strings. BC can't break inside the 6.x family if there's nothing to be compatible with. 2. Don't change `serialize()` at all in PHP < 6.0.0. No version number, no extra escaping. 3. Add some mechanism to tell PHP6 to generate a PHP5-compatible binary string when serializing. I recommend an optional flag (or flags field) to `serialize()`. 4. Understand the new version number in PHP 5.2.1+. That would not touch existing serialization format. It would keep existing compatibility among the 4.x and 5.x family, and allow future PHPs to unambiguously determine how to parse the string. With the `serialize()` flag, you could explicitly generate old-style data from PHP6 if you really, really wanted to pass it back to PHP 5.2.0 or earlier. -- Chad Daelhousen "Television shepherds with living room sheep/ And I pray" --Temple of the Dog, "Wooden Jesus" --- ## #26874 — Richard Lynch — 2006-12-09T19:55:09+00:00 URL: https://externals.io/message/26756#26874 · In reply to: #26831 > At 13:35 07/12/2006, Stefan Esser wrote: > > […] > > I can't think of a single reason not to use a new type like you > suggest. > > Can anybody think of any drawbacks? Somebody MIGHT have written their own unserializer/serializer which made incorrect assumptions about the internal format, such as 's' being the same as 'S'... So, perhaps, just to be consistent with the data I *think* appears in serialized form, 'u' for 'unicode' would maybe be better than 'S' I confess I have no idea what is actually inside of serialized data, and could be WAY off base, but I stretched far enough to come up with something that could go wrong, because that's the kind of paranoid guy I am. :-) -- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? --- ## #26958 — Andrei Zmievski — 2006-12-14T17:19:44+00:00 URL: https://externals.io/message/26756#26958 · In reply to: #26842 I think 'v' with version info makes sense if we foresee multiple serialization format changes in the future. The steps mentioned below do not seem particularly practical for that reason. I would rather go with a new type 'S' for escaped binary strings in both 6.0 and 5.2.1. -Andrei > As 6.0 is not, to my knowledge, anywhere near release, what about this: > 1. Stick a version number on the front of PHP6-serialized strings. BC > can't break inside the 6.x family if there's nothing to be > compatible with. > 2. Don't change `serialize()` at all in PHP < 6.0.0. No version number, > no > extra escaping. > 3. Add some mechanism to tell PHP6 to generate a PHP5-compatible binary > string when serializing. I recommend an optional flag (or flags field) > to `serialize()`. > 4. Understand the new version number in PHP 5.2.1+. > > That would not touch existing serialization format. It would keep > existing compatibility among the 4.x and 5.x family, and allow future > PHPs to unambiguously determine how to parse the string. With the > `serialize()` flag, you could explicitly generate old-style data from > PHP6 > if you really, really wanted to pass it back to PHP 5.2.0 or earlier. > > -- > Chad Daelhousen > "Television shepherds with living room sheep/ And I pray" > --Temple of the Dog, "Wooden Jesus" --- ## #26968 — Andrei Zmievski — 2006-12-15T00:11:18+00:00 URL: https://externals.io/message/26756#26968 · In reply to: #26958 This is the patch against 5.2.1 that supports both 's' and 'S' in serialized strings with no BC breaks. If there are no objections, I will commit soon. -Andrei --- ## #26972 — Mike Ho — 2006-12-15T03:40:15+00:00 URL: https://externals.io/message/26756#26972 · In reply to: #26968 Andrei, thanks for the patch. When you initially posted a patch for serialization on this list, I believe Ilia did some quick, basic benchmarks and saw that the patched version had a moderate negative performance impact. Do you believe that this performance impact is still there? --Mike > This is the patch against 5.2.1 that supports both 's' and 'S' in > serialized strings with no BC breaks. If there are no objections, I > will commit soon. > > -Andrei > > Andrei, thanks for the patch. > > When you initially posted a patch for serialization on this list, I > believe Ilia did some quick, basic benchmarks and saw that the > patched version had a moderate negative performance impact. > > Do you believe that this performance impact is still there? > > --Mike > > > > > […]