# 5.4.0 rc6 and release - Thread: https://externals.io/message/57436 - Original archive: https://news-web.php.net/php.internals/57436 - Started by Stas Malyshev on 2012-01-19T22:55:27+00:00 - Messages: 16 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #57436 — Stas Malyshev — 2012-01-19T22:55:27+00:00 URL: https://externals.io/message/57436#57436 Hi! We'll be announcing RC6 very soon, and we're planning the final release for February 2nd. The two weeks in between are meant for testing and finding out any critical issues that could prevent us from releasing 5.4.0. If we have none of these, we will proceed with the release. To ensure this release would not have any "surprises", I am asking again to hold your commits to 5.4 branch unless you have a critical bug fix AND it is approved by the RMs (myself and David). Thanks for your help! -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #57437 — Michael Kliewe — 2012-01-19T23:35:33+00:00 URL: https://externals.io/message/57436#57437 · In reply to: #57436 Hi, great to see progress! I have some small questions, please don't feel offended, I just want to know and understand: - According to this website there are still 94 test failures in 5.4 . Can you confirm all of them are minor problems? http://gcov.php.net/viewer.php?version=PHP_5_4 - There was this problem with 5.3.7 and the `crypt()` bug. Has there been some improvement in the process of handling test failures? For example mark expected failures as expected failures, and fix the tests or the code? Or are the failing tests "stable" since month and all of them are minor problems? - There have been 319 unique failed tests in RC5, reported by user tests. Is someone looking into them and trying to classify and/or fix them? http://qa.php.net/reports/ All in all the number of test failures still feels very high, I would be interested in your opinion. Is this "normal" for big projects like this? Again, please take these questions as normal questions, I don't want to attack anybody, I'm just interested in the process and relatively new to internal PHP development. Thanks for your work on PHP! Michael Am 19.01.2012 23:55, schrieb Stas Malyshev: > Hi! > > We'll be announcing RC6 very soon, and we're planning the final > release for February 2nd. The two weeks in between are meant for > testing and finding out any critical issues that could prevent us from > releasing 5.4.0. > If we have none of these, we will proceed with the release. To ensure > this release would not have any "surprises", I am asking again to hold > your commits to 5.4 branch unless you have a critical bug fix AND it > is approved by the RMs (myself and David). > Thanks for your help! --- ## #57439 — Stas Malyshev — 2012-01-20T00:08:28+00:00 URL: https://externals.io/message/57436#57439 · In reply to: #57437 Hi! > - According to this website there are still 94 test failures in 5.4 . > Can you confirm all of them are minor problems? > http://gcov.php.net/viewer.php?version=PHP_5_4 Most of them appear so, I'll go through them again to be sure and encourage others to do so too and raise red flags if somebody sees something bad there. Unfortunately, some tests are environment-dependent or otherwise have subtle dependencies or structure that make them work on one system and fail on another not because of the bug in PHP but because of the test itself. So, I have 0 fails on my Linux build but 6 fails on my Mac build. Other times some systems may not support some capability, use old version of the library, etc. and the test may not account for that. I do not think it is practical to postpone release until we solve all of such problems, since this being volunteer-driven open-source project this means not having any release schedule at all. I prefer having the schedule even if that means we'd have to release with some known deficiencies. > - There was this problem with 5.3.7 and the `crypt()` bug. Has there been > some improvement in the process of handling test failures? For example > mark expected failures as expected failures, and fix the tests or the > code? Or are the failing tests "stable" since month and all of them are > minor problems? Yes, there was work done on these. Most of those were fixed, but few still remain, especially across various environments (i.e. test may be fine on some but not others). I of course am all for fixing that further and welcome any help on that. > - There have been 319 unique failed tests in RC5, reported by user > tests. Is someone looking into them and trying to classify and/or fix them? > http://qa.php.net/reports/ Non-reproducible failures usually mean the problem is with the test itself, or with the difference of expectations in the test and local environment, not with PHP. It may still be PHP problem, of course, so the person running the test should check it out and submit a bug if appropriate and if it's bad enough, also send a message to this list. > All in all the number of test failures still feels very high, I would be > interested in your opinion. Is this "normal" for big projects like this? I do think it should be reduced, however if the choice is between waiting forever and have release with some bugs, I think it is practical to choose the latter. Of course, if we discover a major problem that makes PHP unusable or seriously impacts many PHP users, it will be dealt with immediately, and had been so in the past, but otherwise we have to work within the realities of a big project with limited resources and realize that while we strive for 0 bugs in every release, it may never be possible. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #57440 — Clint Byrum — 2012-01-20T00:43:22+00:00 URL: https://externals.io/message/57436#57440 · In reply to: #57439 Excerpts from Stas Malyshev's message of Thu Jan 19 16:08:28 -0800 2012: > Hi! > > > […] > > Most of them appear so, I'll go through them again to be sure and > encourage others to do so too and raise red flags if somebody sees > something bad there. > Unfortunately, some tests are environment-dependent or otherwise have > subtle dependencies or structure that make them work on one system and > fail on another not because of the bug in PHP but because of the test > itself. So, I have 0 fails on my Linux build but 6 fails on my Mac > build. Other times some systems may not support some capability, use old > version of the library, etc. and the test may not account for that. These tests should be skipped or marked as XFAIL on platforms they are known to fail on. Better to have no test than one that cannot be relied upon. All supported platforms should pass with 0 fails. These intentional skips should have open bugs that are documented in the test code so that a developer can find out why this test was disabled when trying to make a change covered by the test. > > I do not think it is practical to postpone release until we solve all of > such problems, since this being volunteer-driven open-source project > this means not having any release schedule at all. I prefer having the > schedule even if that means we'd have to release with some known > deficiencies. > Its pretty bad actually. For all of PHP's success, this is something that continues to baffle me, and many others I have talked to who are charged with measuring quality and with patching systems in a timely manner. How better to document unreliable tests than to skip them with something like "SKIPPED - known to fail on Mac". Its precisely this unreliability that forced me to take a conservative approach for Ubuntu 12.04 and recommend to the community that we ship 5.3.9 instead of 5.4.0. I would much rather have the new stuff in, but even if all the tests pass on the machine we run the test suite on, how can we be sure they won't fail in another time zone, or in some other strange configuration? > > […] > > Yes, there was work done on these. Most of those were fixed, but few > still remain, especially across various environments (i.e. test may be > fine on some but not others). I of course am all for fixing that further > and welcome any help on that. > > > […] > > Non-reproducible failures usually mean the problem is with the test > itself, or with the difference of expectations in the test and local > environment, not with PHP. It may still be PHP problem, of course, so > the person running the test should check it out and submit a bug if > appropriate and if it's bad enough, also send a message to this list. > > > […] > > I do think it should be reduced, however if the choice is between > waiting forever and have release with some bugs, I think it is practical > to choose the latter. Of course, if we discover a major problem that > makes PHP unusable or seriously impacts many PHP users, it will be dealt > with immediately, and had been so in the past, but otherwise we have to > work within the realities of a big project with limited resources and > realize that while we strive for 0 bugs in every release, it may never > be possible. All software will have bugs. The test suite, however, should reflect the bits of code that you know work reliably... not the bits of code you know work most of the time. The fact that its all being running regularly is a fantastic improvement. I'd like to see a commitment to getting 100% pass/xfail/skip for every release/tested environment in future releases though. --- ## #57441 — Stas Malyshev — 2012-01-20T00:57:08+00:00 URL: https://externals.io/message/57436#57441 · In reply to: #57440 Hi! > These tests should be skipped or marked as XFAIL on platforms they are > known to fail on. Better to have no test than one that cannot be relied > upon. All supported platforms should pass with 0 fails. These intentional Of course they should, and you (or anybody else) are welcome to make the patches that make them so :) I promise they'll be in 5.4.1. > Its precisely this unreliability that forced me to take a conservative > approach for Ubuntu 12.04 and recommend to the community that we ship > 5.3.9 instead of 5.4.0. I would much rather have the new stuff in, but 5.4.0 is better, not worse, than 5.3.9 in this regard - especially because of the work that was done in 5.4 branch to fix or improve tests that were failing and improve test coverage. I would advise to give your users a choice once 5.4.0 is released and support both packages. On my Linux environment, I have now 0 fails (though I don't run all modules so some of non-default ones may be missing). > even if all the tests pass on the machine we run the test suite on, > how can we be sure they won't fail in another time zone, or in some > other strange configuration? Well, you never can be *sure* - tests can test only things they know about, unless somebody invents a way to make unit test test every possible combination of environments at once. So far I never heard about such tests. > The fact that its all being running regularly is a fantastic improvement. > I'd like to see a commitment to getting 100% pass/xfail/skip for every > release/tested environment in future releases though. I'd like that too. I hope more people will step up and help to make this a reality. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #59163 — ar@ez.no — 2012-03-26T11:44:53+00:00 URL: https://externals.io/message/57436#59163 · In reply to: #57440 > Excerpts from Stas Malyshev's message of Thu Jan 19 16:08:28 -0800 2012: > > […] > > These tests should be skipped or marked as XFAIL on platforms they are > known to fail on. Better to have no test than one that cannot be relied > upon. All supported platforms should pass with 0 fails. These intentional > skips should have open bugs that are documented in the test code so that > a developer can find out why this test was disabled when trying to make > a change covered by the test. > > > […] > > Its pretty bad actually. For all of PHP's success, this is something that > continues to baffle me, and many others I have talked to who are charged > with measuring quality and with patching systems in a timely manner. How > better to document unreliable tests than to skip them with something like > "SKIPPED - known to fail on Mac". > > Its precisely this unreliability that forced me to take a conservative > approach for Ubuntu 12.04 and recommend to the community that we ship > 5.3.9 instead of 5.4.0. I would much rather have the new stuff in, but > even if all the tests pass on the machine we run the test suite on, > how can we be sure they won't fail in another time zone, or in some > other strange configuration? > Given that 12.04 beta2 will be out on Thursday, and the unit tests where fixed before shipping 5.4 (I naively assume), is this in or out? ref: https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-php54 With beta freeze on march 22 I guess the "mistake" of bundeling 5.3 instead of 5.4 is already made, and we will have to live with that for the next 2 years for prod, and the next 7 months for dev. > > > […] > them? > > […] > be > > […] > this? > > […] > > All software will have bugs. The test suite, however, should reflect > the bits of code that you know work reliably... not the bits of code > you know work most of the time. > > The fact that its all being running regularly is a fantastic improvement. > I'd like to see a commitment to getting 100% pass/xfail/skip for every > release/tested environment in future releases though. --- ## #59167 — Pierre Joye — 2012-03-26T17:49:36+00:00 URL: https://externals.io/message/57436#59167 · In reply to: #59163 hi, > With beta freeze on march 22 I guess the "mistake" of bundeling 5.3 instead > of 5.4 is already made, and we will have to live with that for the next 2 > years for prod, and the next 7 months for dev. It was long planed as far as I remember, to use 5.3 and not 5.4 in this ubuntu release. We tried to convince them but sadly failed. Debian however will use or uses already 5.4, if nothing has changed :) -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #59168 — Clint Byrum — 2012-03-26T19:00:13+00:00 URL: https://externals.io/message/57436#59168 · In reply to: #59163 Excerpts from André Rømcke's message of Mon Mar 26 04:44:53 -0700 2012: > > > > […] > > > Given that 12.04 beta2 will be out on Thursday, and the unit tests where > fixed before shipping 5.4 (I naively assume), is this in or out? > ref: https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-php54 > > With beta freeze on march 22 I guess the "mistake" of bundeling 5.3 instead > of 5.4 is already made, and we will have to live with that for the next 2 > years for prod, and the next 7 months for dev. > Our hands are tied, as the security team still does not feel comfortable shipping a PHP without Suhosin. Perhaps more can be done to convince the world that this is a safe thing to do now, but for now, we're taking the extremely conservative stance and shipping 5.3.10 with the Suhosin patch. Thanks everyone for chiming in, and especially thanks to Ondrej for pushing hard to get things tested and rebuilt. --- ## #59170 — johannes@schlueters.de — 2012-03-26T23:09:20+00:00 URL: https://externals.io/message/57436#59170 · In reply to: #59168 > > Our hands are tied, as the security team still does not feel > comfortable shipping a PHP without Suhosin. Perhaps more can be done > to convince the world that this is a safe thing to do now, but for > now, we're taking the extremely conservative stance and shipping > 5.3.10 with the Suhosin patch. > > Thanks everyone for chiming in, and especially thanks to Ondrej for > pushing hard to get things tested and rebuilt. Thinking loud: One could also ship both. Yes this doubles the effort but gives users a choice :-) johannes --- ## #59175 — Clint Byrum — 2012-03-27T06:38:43+00:00 URL: https://externals.io/message/57436#59175 · In reply to: #59170 Excerpts from Johannes Schlüter's message of Mon Mar 26 16:09:20 -0700 2012: > > > […] > > Thinking loud: One could also ship both. Yes this doubles the effort but > gives users a choice :-) This simply won't happen in the main archive of Ubuntu. The whole point of having a version from the archive in an LTS is that it receives security updates for 5 years, regardless of upstream releasing fixes or not. If users want something unsupported, an effort can be made to setup a PPA: https://help.launchpad.net/Packaging/PPA In fact, Ondrej already went through the trouble of creating one for testing purposes: https://launchpad.net/~ondrej/+archive/php5 Ubuntu's paid (by Canonical) security team does not have the resources to support two versions of anything really. Often times two versions of something are provided (like python 2.6 and 2.7) during a transition like we see in PHP right now. However, one is generally in universe, which means it is only supported by the community. I think the lesson here is to get the necessary bits from Suhosin into PHP's core so that users can feel safe when using stock PHP, rather than needing to wait for the good and generous folks at the hardened PHP project to catch up. --- ## #59176 — Pierre Joye — 2012-03-27T06:45:23+00:00 URL: https://externals.io/message/57436#59176 · In reply to: #59175 hi, hi, > I think the lesson here is to get the necessary bits from Suhosin into > PHP's core so that users can feel safe when using stock PHP, rather > than needing to wait for the good and generous folks at the hardened > PHP project to catch up. I disagree. The lesson here is that the Ubuntu's security team should have discussed with us to see what are actually their worries instead of not following what is actually a good move for everyone. Cheers, -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #59177 — Stas Malyshev — 2012-03-27T07:01:35+00:00 URL: https://externals.io/message/57436#59177 · In reply to: #59175 Hi! > I think the lesson here is to get the necessary bits from Suhosin into > PHP's core so that users can feel safe when using stock PHP, rather > than needing to wait for the good and generous folks at the hardened > PHP project to catch up. Unfortunately, the good and generous leader of Suhosin project expressed his complete opposition to the cooperation with PHP team on the topic of getting the features into the core. It still can be done, I guess, but I'm not sure if we will have a volunteer to do it, especially given this situation. As for users not feeling safe using stock PHP, I have a feeling you are overestimating the number of users feeling so. Millions of users are running stock PHP and we got no indication that they are suffering from any particular strong feelings of unsafety. If the security team has any specific concerns, of course, they can be discussed. Without doubt, Suhosin adds a layer of protection, but I do not see why this layer is so absolutely crucial that you are unable to release a version of PHP without it. What would happen is that users would just use third-party packages of 5.4 or build their own - with all issues that follow that. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #59178 — ar@ez.no — 2012-03-27T08:27:40+00:00 URL: https://externals.io/message/57436#59178 · In reply to: #59176 > hi, > hi, > > > > > […] > > I disagree. The lesson here is that the Ubuntu's security team should > have discussed with us to see what are actually their worries instead > of not following what is actually a good move for everyone. > Why? It is common practice to avoid .0 releases, including those from PHP :) Actually, this is not a bad thing at all. It makes sure lots of PHP projects and Frameworks stays on 5.3 as a common base for the next 1-3 years, introp wise that is a good thing. And those that want the extra 5.4 stuff can get it somehow anyway, and as part of the distro in 7 months. > > Cheers, > -- > Pierre > > @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #59179 — Pierre Joye — 2012-03-27T08:29:34+00:00 URL: https://externals.io/message/57436#59179 · In reply to: #59178 > > […] > > > > Why? Non vanilla php. Cheers, -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #59180 — Lester Caine — 2012-03-27T08:53:42+00:00 URL: https://externals.io/message/57436#59180 · In reply to: #59178 André Rømcke wrote: > > […] > > Why? It is common practice to avoid .0 releases, including those from PHP:) > > Actually, this is not a bad thing at all. > It makes sure lots of PHP projects and Frameworks stays on 5.3 as a common > base for the next 1-3 years, introp wise that is a good thing. > > And those that want the extra 5.4 stuff can get it somehow anyway, and as > part of the distro in 7 months. Or another 3 years time ... I've taken over support for a number of php websites across a couple of ISP's. Currently they are running on PHP5.2.10 because that is what has been provided. And the upgrade by one of these has been announced for May ... to 5.3.10 Changing that to 5.4 was ruled out by the ISP because they would need to rerun all of their tests, and they had given up waiting for 5.4 at the end of last year. Real life users need 6 months to bed new things in, so I can well understand Ubuntu's stance. It makes perfect sense NOT to rush something new in. I've got to check all of my customers sites will still work in May so finishing work on moving other sites to 5.4 is on hold .... I've already had the panics by 5.3.10 being installed on another ISP and having to get sites on that machine working again. Nothing has changed on what the SITES are doing, only config settings screwing up BC :( -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #59184 — Stas Malyshev — 2012-03-27T16:44:48+00:00 URL: https://externals.io/message/57436#59184 · In reply to: #59178 Hi! > Actually, this is not a bad thing at all. > It makes sure lots of PHP projects and Frameworks stays on 5.3 as a common > base for the next 1-3 years, introp wise that is a good thing. How this is good? 5.4 is superior to 5.3 in... well, all aspects, why denying people chance to use it for years is a good thing? > And those that want the extra 5.4 stuff can get it somehow anyway, and as > part of the distro in 7 months. People of course will get it "somehow", but then why bother with having Linux distribution? This stuff is open source, people can download it somehow anyway. Obviously, there's some value in getting software as part of organized process, not "somehow". -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227