# [RFC] build IDs proposal - Thread: https://externals.io/message/42624 - Original archive: https://news-web.php.net/php.internals/42624 - Started by Stanislav Malyshev on 2009-01-14T22:22:42+00:00 - Messages: 18 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #42624 — Stanislav Malyshev — 2009-01-14T22:22:42+00:00 URL: https://externals.io/message/42624#42624 Hi! As many of you know, one can build PHP engine in a number of ways which are binary-incompatible - debug/no debug, thread-safe/not thread safe, etc. Also, for windows we have now anther dimension - we could build with VC6, VC8, VC9, etc. Loading modules built with another, binary incompatible version, could lead to crashes and malfunctions, thus Zend extension and PHP extension loaders check these options, stored in special structure, to match module ones with engine ones. Now the problem is that when we need to add dimensions, we will have to change structures and maybe even edit some extensions, and as we have more dimensions it becomes less manageable and more "hackish". I think that it would be better if we adopted more clean and scalable solution for that. I propose having one string build ID, which would look something like: "API20071006,NTS,Debug,VC8" and would be rquired to match exactly in the engine and the module. This should be relatively easy to generate when compiling. Pro: 1. Easy, robust and scalable way to match current and any future build flavors 2. You can easily create "private" build flavors when testing binary-incompatible changes 3. Build data is in readable form even inside binary, so binaries could be automatically checked by Unix tools. Contra: 1. Well, it's change :) What do you think? -- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com --- ## #42625 — Pierre Joye — 2009-01-14T22:29:57+00:00 URL: https://externals.io/message/42624#42625 · In reply to: #42624 > Hi! > > As many of you know, one can build PHP engine in a number of ways which are > binary-incompatible - debug/no debug, thread-safe/not thread safe, etc. > Also, for windows we have now anther dimension - we could build with VC6, > VC8, VC9, etc. Loading modules built with another, binary incompatible > version, could lead to crashes and malfunctions, thus Zend extension and PHP > extension loaders check these options, stored in special structure, to match > module ones with engine ones. > Now the problem is that when we need to add dimensions, we will have to > change structures and maybe even edit some extensions, and as we have more > dimensions it becomes less manageable and more "hackish". > > I think that it would be better if we adopted more clean and scalable > solution for that. I propose having one string build ID, which would look > something like: "API20071006,NTS,Debug,VC8" and would be rquired to match > exactly in the engine and the module. This should be relatively easy to > generate when compiling. > Pro: > 1. Easy, robust and scalable way to match current and any future build > flavors > 2. You can easily create "private" build flavors when testing > binary-incompatible changes > 3. Build data is in readable form even inside binary, so binaries could be > automatically checked by Unix tools. > Contra: > 1. Well, it's change :) > > What do you think? As said this morning during our little chat about that, I'm all for it, the sooner (now) the better. Another cons: - if we don't do that, I'll have to add a new element the struct to indentify which compiler/crt was used to build an ext. That was the reasons why this proposal comes in now :) Cheers, -- Pierre http://blog.thepimp.net | http://www.libgd.org --- ## #42626 — Lukas Kahwe Smith — 2009-01-14T23:06:48+00:00 URL: https://externals.io/message/42624#42626 · In reply to: #42624 > Hi! > > As many of you know, one can build PHP engine in a number of ways > which are binary-incompatible - debug/no debug, thread-safe/not > thread safe, etc. Also, for windows we have now anther dimension - > we could build with VC6, VC8, VC9, etc. Loading modules built with > another, binary incompatible version, could lead to crashes and > malfunctions, thus Zend extension and PHP extension loaders check > these options, stored in special structure, to match module ones > with engine ones. > Now the problem is that when we need to add dimensions, we will have > to change structures and maybe even edit some extensions, and as we > have more dimensions it becomes less manageable and more "hackish". > > I think that it would be better if we adopted more clean and > scalable solution for that. I propose having one string build ID, > which would look something like: "API20071006,NTS,Debug,VC8" and > would be rquired to match exactly in the engine and the module. This > should be relatively easy to generate when compiling. > Pro: > 1. Easy, robust and scalable way to match current and any future > build flavors > 2. You can easily create "private" build flavors when testing binary- > incompatible changes > 3. Build data is in readable form even inside binary, so binaries > could be automatically checked by Unix tools. > Contra: > 1. Well, it's change :) > > What do you think? how much work will this be? what kind of risk will it bring? as in i guess its too late for beta1 .. i guess its not something that forces us to stay at alpha, but will that change alone require another beta before we go to RC? regards, Lukas --- ## #42627 — Stanislav Malyshev — 2009-01-14T23:31:07+00:00 URL: https://externals.io/message/42624#42627 · In reply to: #42626 Hi! > how much work will this be? Not much work - I think I can do it in 1 day. > what kind of risk will it bring? not much I guess as it will influence only how extensions are loaded - once extension loads, pretty much the deed is done and that code influences nothing. -- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com --- ## #42628 — Pierre Joye — 2009-01-15T00:08:00+00:00 URL: https://externals.io/message/42624#42628 · In reply to: #42626 > > > how much work will this be? Very little work, see zend_extension.c line 60, we only have to replace this test and update definition for the default header (zend_extension struct used by each ext). > what kind of risk will it bring? None, but it removes many troubles. > as in i guess its too late for beta1 .. i guess its not something that > forces us to stay at alpha, but will that change alone require another beta > before we go to RC? It is not too late for b1. However this change alone does not justify another beta but I think (well I'm sure) that we will have many betas anyway. Cheers, -- Pierre http://blog.thepimp.net | http://www.libgd.org --- ## #42629 — jvlad — 2009-01-15T04:36:22+00:00 URL: https://externals.io/message/42624#42629 · In reply to: #42627 > > […] > > Not much work - I think I can do it in 1 day. Please don't forget to add human-readable error message when the IDs don't match. Better to have different errors for different mismatching parts. It's important because not all administrators are good with understanding realted problems. Maybe some sort of hints on what to do should be there too. -jv --- ## #42630 — Pierre Joye — 2009-01-15T09:11:21+00:00 URL: https://externals.io/message/42624#42630 · In reply to: #42629 > > […] > > Please don't forget to add human-readable error message when the IDs don't > match. > Better to have different errors for different mismatching parts. > It's important because not all administrators are good with understanding > realted problems. > Maybe some sort of hints on what to do should be there too. Human readblke errors are already in place. Cheersm -- Pierre http://blog.thepimp.net | http://www.libgd.org --- ## #42631 — Lukas Kahwe Smith — 2009-01-15T09:26:12+00:00 URL: https://externals.io/message/42624#42631 · In reply to: #42624 > Hi! > > As many of you know, one can build PHP engine in a number of ways > which are binary-incompatible - debug/no debug, thread-safe/not > thread safe, etc. Also, for windows we have now anther dimension - > we could build with VC6, VC8, VC9, etc. Loading modules built with > another, binary incompatible version, could lead to crashes and > malfunctions, thus Zend extension and PHP extension loaders check > these options, stored in special structure, to match module ones > with engine ones. > Now the problem is that when we need to add dimensions, we will have > to change structures and maybe even edit some extensions, and as we > have more dimensions it becomes less manageable and more "hackish". > > I think that it would be better if we adopted more clean and > scalable solution for that. I propose having one string build ID, > which would look something like: "API20071006,NTS,Debug,VC8" and > would be rquired to match exactly in the engine and the module. This > should be relatively easy to generate when compiling. > Pro: > 1. Easy, robust and scalable way to match current and any future > build flavors > 2. You can easily create "private" build flavors when testing binary- > incompatible changes > 3. Build data is in readable form even inside binary, so binaries > could be automatically checked by Unix tools. > Contra: > 1. Well, it's change :) seems to me like we need this and since its quickly doable, lets do it for beta1, right now .. freeze seems to be happening next monday regards, Lukas --- ## #42635 — johannes@schlueters.de — 2009-01-15T10:49:14+00:00 URL: https://externals.io/message/42624#42635 · In reply to: #42624 > I think that it would be better if we adopted more clean and scalable > solution for that. I propose having one string build ID, which would > look something like: "API20071006,NTS,Debug,VC8" and would be rquired to > match exactly in the engine and the module. This should be relatively > easy to generate when compiling. In general: Yes, making this extendable is good. But a few comments: - We should make sure not to over-specify things, some minor things can be irrelevant for 99% of the cases (like the old memory limit: for all extensions, except XDebug, this didn't matter but including it in the version string creates trouble and XDebug would either miss a feature (memory profiling) or not load due to a "symbol not found" ...) - We (usually) guarantee binary compatibility for bug fix releases (z in x.y.z) maybe we could use x.y in the version ID - mapping from API No. to version No. is a pain - While at it: Do we really need _three_ version IDs? (Zend API, PHP API and ZendExtension API...) - Loading a 5.2 ext in 5.3 or vice versa should give a nice error, too (... as long as there's no "symbol no found" at least) johannes --- ## #42640 — jvlad — 2009-01-15T17:31:49+00:00 URL: https://externals.io/message/42624#42640 · In reply to: #42630 > > […] > > Human readblke errors are already in place. > Do you mean this? fprintf(stderr, "%s requires Zend Engine API version %d.\n" "The Zend Engine API version %d which is installed, is outdated.\n\n", new_extension->name, First of all, php error_log may not get this message because it's thrown into stderr. 2nd, perhaps it's clear for developers, that know and care of API versions. What's about server admins who will be installing the extensions? They know filenames and php versions. They may not know extension->name and may now know that Zend Engine API is related to particular php version. Same goes to the php modules loaded in ext/standard/dl.c just my 2cents. -jv --- ## #42641 — Pierre Joye — 2009-01-15T17:43:03+00:00 URL: https://externals.io/message/42624#42641 · In reply to: #42640 > > […] I meant "readable" :) > Do you mean this? > fprintf(stderr, "%s requires Zend Engine API version %d.\n" > "The Zend Engine API version %d which is installed, is outdated.\n\n", > new_extension->name, > First of all, php error_log may not get this message because it's thrown > into stderr. It is not possible at this stage. > What's about server admins who will be installing the extensions? An admin should know that, really. We may improve the documentation if there is no mention of these errors. Cheers, -- Pierre http://blog.thepimp.net | http://www.libgd.org --- ## #42642 — Christian Seiler — 2009-01-15T18:35:12+00:00 URL: https://externals.io/message/42624#42642 · In reply to: #42624 Hi, > I think that it would be better if we adopted more clean and scalable > solution for that. I propose having one string build ID, which would > look something like: "API20071006,NTS,Debug,VC8" and would be rquired to > match exactly in the engine and the module. This should be relatively > easy to generate when compiling. I'm generally in favor of that change, however, I'm not quite sure on the impact of including the Visual Studio version specifier. I don't have that much experience with PHP on Windows but I have successfully built PHP extensions with VC8 (i.e. VS 2005 Express) for PHP 5.2.x. I didn't have any problems whatsoever in using these extensions together with the official PHP builds (VC6) and Zend Core (IIRC also VC6) - as long as both VC runtimes were installed on the corresponding system. Since VC6 is not available for free download I wouldn't be able to build Windows extensions for VC6 builds of PHP anymore if the version string check would inhibit the loading of the extension. Would any of the PHP Windows guys like to comment on this? Viele Grüße, Christian --- ## #42643 — Pierre Joye — 2009-01-15T18:52:07+00:00 URL: https://externals.io/message/42624#42643 · In reply to: #42642 hi, > Would any of the PHP Windows guys like to comment on this? I did, and the reason behind this proposal was about solving the VC* problem for the end users (we have to add another condition to check, which CRT is used) :) Cheers, -- Pierre http://blog.thepimp.net | http://www.libgd.org --- ## #42644 — Christian Seiler — 2009-01-15T19:04:42+00:00 URL: https://externals.io/message/42624#42644 · In reply to: #42643 Hi Pierre, > > […] > > I did, and the reason behind this proposal was about solving the VC* > problem for the end users (we have to add another condition to check, > which CRT is used) :) I'm sorry for asking again, maybe I'm just plain stupid ;-) but I'm not quite sure how to interpret your response (I've no idea what VC* problem you mean anyway, I never had problems with PHP under Windows at all): So, basically, my situation would still work, even with the build ID? To clarify what I'm doing currently: 1. Build PHP with additional extension in ext/$name with VC8 (VS 2005) on machine A (developement machine). 2. Install both VC6 and VC8 runtimes on machine B (production machine) 3. Install official PHP build or Zend Core (compiled with VC6) on machine B 4. Copy the extension DLL (compiled with VC8) to machine B and use it as a PHP extension. (This *does* currently work with PHP 5.2.x - at least for me.) Regards, Christian --- ## #42647 — Pierre Joye — 2009-01-15T19:43:00+00:00 URL: https://externals.io/message/42624#42647 · In reply to: #42644 > Hi Pierre, > > > […] > > I'm sorry for asking again, maybe I'm just plain stupid ;-) but I'm not > quite sure how to interpret your response (I've no idea what VC* problem > you mean anyway, I never had problems with PHP under Windows at all): > > So, basically, my situation would still work, even with the build ID? > To clarify what I'm doing currently: > A VC6 extension is not compatible with a VC9 build (or the other way round). It crashes randomly depending on what the extension does (xdebug for one is really good for this test :). PHP 5.3 will provide VC9 and VC6 builds, we need such a safety check to avoid too much pain for our users and us (I already get many bogus reports about people using VC6 extensions with VC9 builds, or VC9 extensions with VC6 builds). By the way, VS2008 Express is free and 100% compatible with a normal VS2008. You may use it so you can use our VC9 builds directly or provide extensions for it. Cheers, -- Pierre http://blog.thepimp.net | http://www.libgd.org --- ## #42648 — Christian Seiler — 2009-01-15T19:47:48+00:00 URL: https://externals.io/message/42624#42648 · In reply to: #42647 Hi Pierre, > A VC6 extension is not compatible with a VC9 build (or the other way > round). It crashes randomly depending on what the extension does > (xdebug for one is really good for this test :). You wouldn't know about how VC8 fits in here? Anyway, thanks for the pointer to xdebug, I'll try that with my VC6/VC8 mix and see if that crashes. > By the way, VS2008 Express is free and 100% compatible with a normal > VS2008. You may use it so you can use our VC9 builds directly or > provide extensions for it. Yes, I know, but the problem is that it doesn't run on Win2k. ;-) Regards, Christian --- ## #42650 — jvlad — 2009-01-15T22:27:55+00:00 URL: https://externals.io/message/42624#42650 · In reply to: #42641 > I meant "readable" :) > > > […] > > It is not possible at this stage. It is possible at this stage. Just put zend_error() call and you'll see it works and calls regular php error handler. And for example ioncube (it is a known zend extension) uses this "feature". > > > > […] > > An admin should know that, really. I don't see any reason why would all admins learn this core "feature" of php? Why not just make it providing clearer error messages? Such error message like "BogusName requires Blah blah blah Engine Api version XYXYXYX, and Blah blah blah Engine Api version YXYXYXYX is installed, is outdated" is just insane. By seeing these two numbers XYXYXYX and YXYXYXYX, whould anybody non-involved in core php development guess that this relates to php versions? Does that "BogusName" tell anything about real module name that caused the error, despite the fact that error itself in some cases is just not visible. -jv --- ## #42651 — William A. Rowe — 2009-01-15T22:30:39+00:00 URL: https://externals.io/message/42624#42651 · In reply to: #42647 Pierre Joye wrote: > > A VC6 extension is not compatible with a VC9 build (or the other way > round). It crashes randomly depending on what the extension does > (xdebug for one is really good for this test :). This is only true for specific combinations of extensions and 3p libs based on what resources the one grabs that the other releases. E.g. if the 3p lib calls malloc and expects the ext to call free(), those two must use the same msvcr. Similarly, an extension that close()'s a file handle allocated by php core, then those two must use the same. For most well partitioned API's this is simply not a problem. (e.g. for mod_perl which is anything but partitioned, it's a horrible situation.) If the additional info were a diagnostic form, that would be nice. As a roadblock to loading an extension? That's a nonstarter. The day after you ship Studio 9 builds (quit saying VC9 already! Studio 6 shipped VC12 a very long time ago) you recognize people will be also building with Studio 10? That's just the way these things go. So +1 for extra information; I can find that with the forever-distributed depends.exe tool anyways, but -1 to some blanket "enforcement" or for building this into the enforced "fingerprint" of php. And +1 for enforcing things such as zts by way of such a fingerprint.