# libmagic 5.14 upgrade - Thread: https://externals.io/message/66823 - Original archive: https://news-web.php.net/php.internals/66823 - Started by Anatol Belski on 2013-03-26T22:44:06+00:00 - Messages: 11 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #66823 — Anatol Belski — 2013-03-26T22:44:06+00:00 URL: https://externals.io/message/66823#66823 Hi, I've developed a patch for libmagic 5.14 which is available under http://belski.net/phpz/finfo/finfo_5.14_5.patch.gz . For those willing to test please overwrite ext/fileinfo/tests/magic with http://belski.net/phpz/finfo/magic.mgc.gz (not contained in the patch). I've tested it on Linux and Windows, the issues it brings are minimal. The tests failing are test fails due to the lib upgrade. The reason I'm suggesting the patch now is that there was a discussion on IRC about upgrading libmagic for 5.5. We haven't have any upgrades to the filenfo extension on all the branches since about 1 year. That's why tickets like bug #64462 come to the life. David is going to tag the beta2 tomorrow, so there is kind of time pressure. IMHO it'd really make sense to put the upgraded libmagic into 5.5 despite it's beta already. Even 5.4 would come to question. At the moment we have libmagic 5.11 in all the branches. With this we're about a year back in time. Thinking that 5.5 without even have been released yet has two years old libmagic in a year is not nice. 5.4 would also profit from that IMHO. Of course I cant guarantee (as this is a bit sudden) the patch is perfect yet, it has some bugs for sure. However the patch is related only to libmagic, no line of fileinfo ext is touched. I'm sure any possible issues canbe fixed over the next couple of weeks. I'll be working on improvements to it in the next days anyway. What +/- I personally see upgrading this at this time: contra: - there might be bugs, the next release might have not all them fixed - 5.11 is what the latest linux exts have even as dev - older/custom magic files might be incompatible pro: - latest libmagic data - better safety for the future - in a year all that libs are for sure upgraded in the main distros - further upgrades can be better handled Please note that the latest libmagic upgrade was from 5.02 to 5.11 and there was no significant breach in 5.3+. I can just repeat that upgrading 5.4+ would make pretty much sense when looking forward. Cheers Anatol --- ## #66838 — David Soria Parra — 2013-03-27T18:29:14+00:00 URL: https://externals.io/message/66823#66838 · In reply to: #66823 > What +/- I personally see upgrading this at this time: > > contra: > - there might be bugs, the next release might have not all them fixed > - 5.11 is what the latest linux exts have even as dev > - older/custom magic files might be incompatible > > pro: > - latest libmagic data > - better safety for the future > - in a year all that libs are for sure upgraded in the main distros > - further upgrades can be better handled HI Anatol, thanks for putting this patch together. However as we already reached feature freeze and we are will have beta2 tomorrow I think it's a bit too late for so much new code. I cannot really judge the sideeffects of this upgrade and would prefer to be strict about the feature freeze. So please do not update the library unless Stas decides that he wants to do it in 5.4, in that case I have to go with his decision. David --- ## #66841 — Pierre Joye — 2013-03-27T20:35:57+00:00 URL: https://externals.io/message/66823#66841 · In reply to: #66838 > > > […] > > HI Anatol, > > thanks for putting this patch together. However as we already reached > feature freeze and we are will have beta2 tomorrow I think it's a bit > too late for so much new code. I cannot really judge the sideeffects of > this upgrade and would prefer to be strict about the feature freeze. So > please do not update the library unless Stas decides that he wants to do > it in 5.4, in that case I have to go with his decision. We have done that many times in the past for 5.3 and 5.4. It is relatively risk free. Even more for 5.5 during beta phase. It does not add new features but fixes bugs. The good side effect is that we can test it well with 5.5 and back port to 5.3/4 later. Cheers, -- Pierre @pierrejoye --- ## #66843 — David Soria Parra — 2013-03-27T21:09:54+00:00 URL: https://externals.io/message/66823#66843 · In reply to: #66841 > We have done that many times in the past for 5.3 and 5.4. It is > relatively risk free. Even more for 5.5 during beta phase. It does not > add new features but fixes bugs. > > The good side effect is that we can test it well with 5.5 and back > port to 5.3/4 later. I consider it a feature and not a necessary bugfix. In addition anatoly is saying that there are still problems with the patch. Both are reasons for me to believe that upgrading it 1-1.5 month before a final is not worth the risk. I would prefer to be more strict about the "feature freeze" than we were with 5.3 and 5.4. David --- ## #66882 — Anatol Belski — 2013-04-01T17:13:25+00:00 URL: https://externals.io/message/66823#66882 · In reply to: #66843 Stas, I've invested more time and here's almost cleaned up patch http://belski.net/phpz/finfo/finfo_5.14_10.patch.gz The tests pass, valgrind is happy, as well Windows. I've noticed no behaviour change, except - as the data is updated and one might see different (eventually better) results. Please take a look. Regards Anatol On Wed, 2013-03-27 at 22:09 +0100, David Soria Parra wrote: > > > > […] > > I consider it a feature and not a necessary bugfix. In addition anatoly > is saying that there are still problems with the patch. Both are reasons > for me to believe that upgrading it 1-1.5 month before a final is not > worth the risk. I would prefer to be more strict about the "feature > freeze" than we were with 5.3 and 5.4. > > David --- ## #66979 — Anatol Belski — 2013-04-05T12:38:15+00:00 URL: https://externals.io/message/66823#66979 · In reply to: #66882 Stas, how does it look with this one? Please let me know whether it's ok for 5.4. I would go with master otherwise. Thanks Anatol On Mon, 2013-04-01 at 19:13 +0200, Anatol Belski wrote: > Stas, > > I've invested more time and here's almost cleaned up patch > > http://belski.net/phpz/finfo/finfo_5.14_10.patch.gz > > The tests pass, valgrind is happy, as well Windows. I've noticed no > behaviour change, except - as the data is updated and one might see > different (eventually better) results. > > Please take a look. > > Regards > > Anatol > > On Wed, 2013-03-27 at 22:09 +0100, David Soria Parra wrote: > > […] --- ## #66995 — Stas Malyshev — 2013-04-07T00:12:34+00:00 URL: https://externals.io/message/66823#66995 · In reply to: #66882 Hi! > I've invested more time and here's almost cleaned up patch > > http://belski.net/phpz/finfo/finfo_5.14_10.patch.gz > > The tests pass, valgrind is happy, as well Windows. I've noticed no > behaviour change, except - as the data is updated and one might see > different (eventually better) results. Looks ok, but I have one question - we have libmagic.patch file there which as I understand is supposed to be diff between "our" libmagic and upstream libmagic. But this patch does not update it. Is it intentional? Will it be updated later? -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #66998 — Anatoliy Belsky — 2013-04-07T17:44:52+00:00 URL: https://externals.io/message/66823#66998 · In reply to: #66995 Hi, Am Samstag, den 06.04.2013, 17:12 -0700 schrieb Stas Malyshev: > Looks ok, but I have one question - we have libmagic.patch file there > which as I understand is supposed to be diff between "our" libmagic and > upstream libmagic. But this patch does not update it. Is it intentional? > Will it be updated later? Of course I'll update libmagic.patch before commiting, that's just a diff command. Didn't do it as that's redundant to ext/fileinfo/libmagic part in the patch I've sent. Thanks for taking time. Anatol --- ## #67052 — ondrej@sury.org — 2013-04-11T15:27:49+00:00 URL: https://externals.io/message/66823#67052 · In reply to: #66998 Just a curious question – would it be possible to add some code directly to upstream, so you wouldn't have to engulf yet-another-library into PHP source tree? It's always a security nightmare to have various copies of libraries at various places of the system. Ondrej > Hi, > > Am Samstag, den 06.04.2013, 17:12 -0700 schrieb Stas Malyshev: > > > […] > > > Of course I'll update libmagic.patch before commiting, that's just a > diff command. Didn't do it as that's redundant to ext/fileinfo/libmagic > part in the patch I've sent. > > Thanks for taking time. > > Anatol > > > > -- > > > -- Ondřej Surý Just a curious question – would it be possible to add some code > directly to upstream, so you wouldn't have to engulf > yet-another-library into PHP source tree? > > It's always a security nightmare to have various copies of libraries > at various places of the system. Not sure about what modifications are needed in this specific one but usually some patching is needed with libraries that work with files if we want to have them supporting streams, since by default they would just do open() and such. So in this case if we want them to support streams patching seems to be inevitable. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #67055 — Pierre Joye — 2013-04-11T18:12:03+00:00 URL: https://externals.io/message/66823#67055 · In reply to: #67052 > Just a curious question – would it be possible to add some code > directly to upstream, so you wouldn't have to engulf > yet-another-library into PHP source tree? It is a bit of work to support php's stream upstream. I wanted to do it back then but it was simply easier to patch the bundled version. To support stream it would need something like what we do in libgd or something along this line, so we can pass our own handles and ops. > It's always a security nightmare to have various copies of libraries > at various places of the system. I totally agree but not always possible. Same problem with the timezone, users experience matters more too as everyone gets on the same line/stage with a given PHP release and avoids many data related issues. Cheers, -- Pierre @pierrejoye