# [RFC] [Discussion] Resource to object conversion - Thread: https://externals.io/message/121660 - Original archive: https://news-web.php.net/php.internals/121660 - Started by kocsismate90@gmail.com on 2023-11-12T16:47:37+00:00 - Messages: 18 - RFC: Resource to object conversion (Accepted) — https://externals.io/rfc/resource_to_object_conversion.md - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · RFCs at https://externals.io/rfc.md · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #121660 — kocsismate90@gmail.com — 2023-11-12T16:47:37+00:00 URL: https://externals.io/message/121660#121660 Hi Internals, Following my straw poll about the Process resource name, I would like to present an RFC which clarifies the rough timeline and the BC promises of the "resource to object conversion" project. Link: https://wiki.php.net/rfc/resource_to_object_conversion I'm looking forward to your feedback! Regards, Máté --- ## #121666 — Derick Rethans — 2023-11-13T14:51:36+00:00 URL: https://externals.io/message/121660#121666 · In reply to: #121660 >Hi Internals, > >Following my straw poll about the Process resource name, I would like to >present an RFC which clarifies the rough timeline and the BC promises of >the "resource to object conversion" project. > >Link: https://wiki.php.net/rfc/resource_to_object_conversion > >I'm looking forward to your feedback! I don't really have any comments beyond that everything stream related should happen at the same time. I think I would prefer ProcessHandle and all streams related change to happen with 9.0. As to COM, I dont think many people at going to notice this, as I doubt it'd be in much use. Did you do an analyses as to how much either of these changes could break anything? cheers Derick --- ## #121698 — Jakub Zelenka — 2023-11-16T16:13:39+00:00 URL: https://externals.io/message/121660#121698 · In reply to: #121660 > Hi Internals, > > Following my straw poll about the Process resource name, I would like to > present an RFC which clarifies the rough timeline and the BC promises of > the "resource to object conversion" project. > > Link: https://wiki.php.net/rfc/resource_to_object_conversion > > Please could you add a separate vote for primary streams if the resource to object conversion should be done at all (requiring 2/3 votes to be accepted). I will personally vote against this if there is no is_resource change as I think it's just too big BC break even for 9.0 - it will likely require massive update of many code bases. Cheers Jakub --- ## #121720 — Philip Hofstetter — 2023-11-20T04:16:57+00:00 URL: https://externals.io/message/121660#121720 · In reply to: #121698 Hi, > I will personally vote against this if there is no is_resource > change as I think it's just too big BC break even for 9.0 - it will likely > require massive update of many code bases. As someone maintaining a large code-base going back to 5.0.0 and bitten in the past by the resource thing, I wonder whether it was previously discussed to have all these converted objects implement a `Resource` interface and have `is_resource()` check for that. Both the function and maybe also the interface could then also be marked as deprecated, but it would allow for a much more painless transition. Personally, I’m now relying on psalm to detect such issues, so if I had a vote I would selfishly vote yes anyways, but still: for those without static analysis, this would IMHO make things much easier. Philip --- ## #121748 — kocsismate90@gmail.com — 2023-11-21T21:42:47+00:00 URL: https://externals.io/message/121660#121748 · In reply to: #121720 Hi Derick, Jakub, Phiip > Did you do an analyses as to how much either of these changes could break > anything? I updated the RFC with some impact analysis. The numbers support my hypothesis that the conversion of auxiliary stream resources would cause hardly any BC break - at least in case of the top 2000 PHP packages. That's why I believe we can migrate them separately from the primary ones, which are used much more often. I wonder whether it was previously discussed to have all these converted > objects implement a `Resource` interface and have `is_resource()` check for > that. > Yes, the RFC links two threads about this topic: https://externals.io/message/116127 and https://externals.io/message/104361#104369. Even though there's a clear interest in changing how `is_resource()` works, doing so would bring us to a minefield... Please could you add a separate vote for primary streams if the resource to > object conversion should be done at all (requiring 2/3 votes to be > accepted). I will personally vote against this if there is no is_resource change as I > think it's just too big BC break even for 9.0 - it will likely require > massive update of many code bases. Yes, I added the option. > Both the function and maybe also the interface could then also be marked > as deprecated, but it would allow for a much more painless transition. > While I would love to see resources go altogether, deprecating `is_resource()` is way too early. Even if php-src itself manages to sunset all the built-in resources, there will still be lots of third party extensions which will still rely on them. Personally, I’m now relying on psalm to detect such issues, so if I had a > vote I would selfishly vote yes anyways, but still: for those without > static analysis, > this would IMHO make things much easier. > Yeah, using static analysis should be mandatory for mission critical systems, since these tools can easily detect issues like wrong `is_resource()` checks. However, I acknowledge that doing so is not always possible due to some constraints (i.e. budget limits). Fortunately, the possible issues caused by resource to object conversions are usually not too difficult to find out. For example, in case of the Process resource, one has to search for all `proc_open()` invocations, and check whether the return values are correctly checked. According to my experience, the `is_resource()` checks are usually very close to the creation of resources, so they can be fixed easily when needed. Regards, Máté --- ## #121752 — G. P. B. — 2023-11-22T04:32:57+00:00 URL: https://externals.io/message/121660#121752 · In reply to: #121698 > On Sun, Nov 12, 2023 at 4:48 PM Máté Kocsis > wrote: > > > […] > Please could you add a separate vote for primary streams if the resource to > object conversion should be done at all (requiring 2/3 votes to be > accepted). I will personally vote against this if there is no is_resource > change as I think it's just too big BC break even for 9.0 - it will likely > require massive update of many code bases. > What is the point of a major release if we cannot even do such a BC break? We don't even know when PHP 9.0 is going to happen yet. I will also state that I am against changing the semantics of `is_resource()` *unless* we remove support for resources altogether from the engine at the same time so that there is no possible ambiguity. Which frankly is probably something we should be doing, and that people paid by the foundation should convert resources to opaque objects in PECL extensions, and other extensions brought to our attention. And yes, that also means helping Oracle with OCI8 that is getting unbundled. I will try to get round to do ext/dba somewhat soon. Best regards, Gina P. Banyard --- ## #121753 — Aleksander Machniak — 2023-11-22T07:27:54+00:00 URL: https://externals.io/message/121660#121753 · In reply to: #121660 > Link: https://wiki.php.net/rfc/resource_to_object_conversion Is "Primary stream-related resources" section about file pointers? I'd like it to be more clear. Maybe it should even list all functions that will be impacted. I don't see an impact analysis for this section either. -- Aleksander Machniak Kolab Groupware Developer [https://kolab.org] Roundcube Webmail Developer [https://roundcube.net] ---------------------------------------------------- PGP: 19359DC1 # Blog: https://kolabian.wordpress.com --- ## #121754 — Mike Schinkel — 2023-11-22T07:35:59+00:00 URL: https://externals.io/message/121660#121754 · In reply to: #121752 > > > > > > What is the point of a major release if we cannot even do such a BC break? We don't even know when PHP 9.0 is going to happen yet. > > I have been using Go for about four years now and it seems they have gotten the backward compatibility issue nailed, and that pays great dividends in developer confidence in the language, i.e.: https://www.reddit.com/r/golang/comments/17v4xja/anyone_face_issues_when_updating_version_of_go/ They recently explained in depth how they do it: https://go.dev/blog/compat Also see: https://thenewstack.io/how-golang-evolves-without-breaking-programs/ Although Go is compiled and PHP is not, I think there still may be significant insight that can be gained for PHP by studying how Go is handling it and applying any lessons learned. #fwiw - Mike --- ## #121759 — G. P. B. — 2023-11-22T17:08:12+00:00 URL: https://externals.io/message/121660#121759 · In reply to: #121754 > > > What is the point of a major release if we cannot even do such a BC break? > We don't even know when PHP 9.0 is going to happen yet. > > > I have been using Go for about four years now and it seems they have > gotten the backward compatibility issue nailed, and that pays great > dividends in developer confidence in the language, i.e.: > > > https://www.reddit.com/r/golang/comments/17v4xja/anyone_face_issues_when_updating_version_of_go/ > > They recently explained in depth how they do it: > > https://go.dev/blog/compat > > Also see: > > https://thenewstack.io/how-golang-evolves-without-breaking-programs/ > > Although Go is compiled and PHP is not, I think there still may be > significant insight that can be gained for PHP by studying how Go is > handling it and applying any lessons learned. > Go is a "new" programming language, with its 1.0.0 version being from 2012. It was also designed from the ground up. PHP on the other hand wasn't designed, but the language grew organically, and is 28 years old. Comparing it to Go, in my opinion, makes no sense. We should be comparing ourselves to languages of that age or older, the most famous example being Python, which did a major BC break between its version 2 and 3. But Fortran, C, Perl (with Raku), and for sure others have all made changes to the language, recent or not, that break compatibility. Go even has a cave out that they *may* release a Go 2 specification, which does not guarantee any backwards compatibility with Go 1. [1] Even if the current lead engineer says this is "never" going to happen, the cave out still exists. More importantly, it is possible to write cross compatible code, even without changing anything about `is_resource()`, if we convert streams to opaque objects. It might be tedious and one might need to have redundant instanceof checks with `is_resource()` if one does not want to check for a false return, or duplicate checks for closed resources. But it is possible, which was *not* the case for Python 2 and 3 as it changed fundamentally how strings behaved. Finally, I think people would have more confidence in the language if it stopped coming with various foot guns included that need to be explicitly kept in check by using external tools such as static analysis tools, or code style tools. And removing those, or making the language overall more coherent and consistent, requires us to break backwards compatibility. Sincerely, Gina P. Banyard [1] https://go.dev/doc/go1compat: > It is intended that programs written to the Go 1 specification will > continue to compile and run correctly, unchanged, over the lifetime of that > specification. At some indefinite point, a Go 2 specification may arise, > but until that time, Go programs that work today should continue to work > even as future "point" releases of Go 1 arise (Go 1.1, Go 1.2, etc.). --- ## #121760 — Ayesh Karunaratne — 2023-11-22T18:08:56+00:00 URL: https://externals.io/message/121660#121760 · In reply to: #121759 > More importantly, it is possible to write cross compatible code, even > without changing anything about `is_resource()`, if we convert streams to > opaque objects. > It might be tedious and one might need to have redundant instanceof checks > with `is_resource()` if one does not want to check for a false return, or > duplicate checks for closed resources. > Extensions like GD, IMAP, Curl, and FTP were once very "resource heavy", but they already received this migration with next to none disruptions. Shameless plug here (https://php.watch/articles/resource-object) where I track the progress. As these changes were merged in the last couple years, I spent a few hours grepping the Composer top 1000 packages. There were not that many `is_resource` checks, and it was quite easy and straightforward to update them to account for the class objects. I'm supportive of our trajectory so far by slowly but steadily moving resource objects to class objects. I also support that we do not modify `is_resource` to mark resource class objects as a resource. I really look forward to the day that we remove `is_resource` and kill the final resource object in PHP 🥺. --- ## #121764 — Deleu — 2023-11-22T21:52:53+00:00 URL: https://externals.io/message/121660#121764 · In reply to: #121759 > > > > […] > wrote: > > […] > break? > > […] > https://www.reddit.com/r/golang/comments/17v4xja/anyone_face_issues_when_updating_version_of_go/ > > […] > > Go is a "new" programming language, with its 1.0.0 version being from 2012. > It was also designed from the ground up. > > PHP on the other hand wasn't designed, but the language grew organically, > and is 28 years old. > Comparing it to Go, in my opinion, makes no sense. > > We should be comparing ourselves to languages of that age or older, the > most famous example being Python, which did a major BC break between its > version 2 and 3. > But Fortran, C, Perl (with Raku), and for sure others have all made changes > to the language, recent or not, that break compatibility. > > Go even has a cave out that they *may* release a Go 2 specification, which > does not guarantee any backwards compatibility with Go 1. [1] > Even if the current lead engineer says this is "never" going to happen, the > cave out still exists. > > More importantly, it is possible to write cross compatible code, even > without changing anything about `is_resource()`, if we convert streams to > opaque objects. > It might be tedious and one might need to have redundant instanceof checks > with `is_resource()` if one does not want to check for a false return, or > duplicate checks for closed resources. > But it is possible, which was *not* the case for Python 2 and 3 as it > changed fundamentally how strings behaved. > > Finally, I think people would have more confidence in the language if it > stopped coming with various foot guns included that need to be explicitly > kept in check by using external tools such as static analysis tools, or > code style tools. > And removing those, or making the language overall more coherent and > consistent, requires us to break backwards compatibility. > > Sincerely, > > Gina P. Banyard > I sympathise with both sides on this topic. As a Software Engineer, not breaking 10-30 years of BC promise is unsustainable, but as a PHP user BC breaks have a heavy impact on legacy codebase written 10~20 years ago. A lot of discussion has happened around this subject, but unfortunately no consensus is ever reached. I've talked about increasing the stability and the pool of maintainers by providing PHP Packages under PHP namespace ( https://externals.io/message/120335#120354). There were a lot of discussions on language evolution and editions that ultimately didn't go much further. Given how much time has passed and how this subject is always present, I now look at this with the optics that both PHP internals devs and PHP users suffer from the same condition of dealing with legacy. Between 1990 up to 2010, give or take, the way software used to be built vastly differs from how software is built today and these old software can be very hard to decommission given their lack of automation tests and inability to be statically analysed. With today's practices, I believe it's easier to introduce BC breaks that affect software written after 2018. And since I believe that, it's a natural consequence for me to believe that if PHP introduced a new `declare(backward_compatibility=0)`, it could be used for users to signal to the engine that 1) we're writing this file after 2023 and 2) we will cover this file with automation tests and/or static analysis tools. PHP Internals wouldn't need to make a huge big-bang BC break all-at-once. Every year new BC breaks could be introduced affecting only the "new engine version". The idea isn't to build several combination of "PHP Editions" as it was discussed in the past, but rather to have a consensus between PHP Internals and PHP Developers that a new Engine is constantly being developed, this engine will break compatibility with the past 20 years of PHP whenever PHP Internals manage to rebuild something (throughout multiple versions), a migration path assumes users will use of Rector, Static Analysers and PHPUnit and the old engine keeps the BC promise in order to allow old software to keep running, but is expected to degrade in performance and to only support new stuff if it has no added burden to internals. When such assumptions are made about userland, the concept of what's an acceptable BC break should be skewed in favor of helping PHP Internals. -- Marco Deleu --- ## #121772 — Robert Landers — 2023-11-23T08:17:49+00:00 URL: https://externals.io/message/121660#121772 · In reply to: #121764 > > > > > […] > > I sympathise with both sides on this topic. As a Software Engineer, not > breaking 10-30 years of BC promise is unsustainable, but as a PHP user BC > breaks have a heavy impact on legacy codebase written 10~20 years ago. > > A lot of discussion has happened around this subject, but unfortunately no > consensus is ever reached. I've talked about increasing the stability and > the pool of maintainers by providing PHP Packages under PHP namespace ( > https://externals.io/message/120335#120354). There were a lot of > discussions on language evolution and editions that ultimately didn't go > much further. > > Given how much time has passed and how this subject is always present, I > now look at this with the optics that both PHP internals devs and PHP users > suffer from the same condition of dealing with legacy. Between 1990 up to > 2010, give or take, the way software used to be built vastly differs from > how software is built today and these old software can be very hard to > decommission given their lack of automation tests and inability to be > statically analysed. With today's practices, I believe it's easier to > introduce BC breaks that affect software written after 2018. And since I > believe that, it's a natural consequence for me to believe that if PHP > introduced a new `declare(backward_compatibility=0)`, it could be used for > users to signal to the engine that 1) we're writing this file after 2023 > and 2) we will cover this file with automation tests and/or static analysis > tools. PHP Internals wouldn't need to make a huge big-bang BC break > all-at-once. Every year new BC breaks could be introduced affecting only > the "new engine version". The idea isn't to build several combination of > "PHP Editions" as it was discussed in the past, but rather to have a > consensus between PHP Internals and PHP Developers that a new Engine is > constantly being developed, this engine will break compatibility with the > past 20 years of PHP whenever PHP Internals manage to rebuild something > (throughout multiple versions), a migration path assumes users will use of > Rector, Static Analysers and PHPUnit and the old engine keeps the BC > promise in order to allow old software to keep running, but is expected to > degrade in performance and to only support new stuff if it has no added > burden to internals. > > When such assumptions are made about userland, the concept of what's an > acceptable BC break should be skewed in favor of helping PHP Internals. > > -- > Marco Deleu Hey Marco, I think the biggest issue with that strategy is that you end up endorsing projects that PHP cannot control. This may not be a bad thing, but personally, I'd love to see PHP start adopting these kinds of projects as officially maintained by PHP. For example, composer could be forked or brought over to the PHP organization and then if PHP wants to change how autoloading works ... they can just change it and implement a migration path in composer/rector/whatever. All these tools maintained by volunteers over the years would benefit from being able to have important conversations before a feature is even developed; even be able to suggest changes to the feature that would allow for seamless migrations. Imagine installing PHP 9.0 and then just running `php --migrate-from=8.2` and having it run rector + composer (maybe even static analysis before and after to give you issues) migration scripts on an entire codebase. Right now, you need to know what to do and how to use it. You may not even know these tools exist or what to search for to find them, particularly if you are new to PHP. Robert Landers Software Engineer Utrecht NL --- ## #122019 — kocsismate90@gmail.com — 2023-12-20T10:18:56+00:00 URL: https://externals.io/message/121660#122019 · In reply to: #121772 Hey Everyone, Sorry for the radio silence, I was busy with other tasks. However, I managed to improve the RFC in the recent days the following way: - most importantly, I changed the suggested approach of the conversion in case of primary stream resources: the `is_resource()` hack mentioned a few times before would be used in this case. - I added impact analysis for each resource category + listed all functions which return streams (even though they are not impacted). - I added more examples, reasoning etc. Please read the updated proposal because I intend to start the vote shortly after the holiday season. Thanks, Máté --- ## #122020 — Larry Garfield — 2023-12-20T17:39:19+00:00 URL: https://externals.io/message/121660#122020 · In reply to: #122019 > Hey Everyone, > > Sorry for the radio silence, I was busy with other tasks. However, I > managed to improve the RFC in the recent days the following way: > - most importantly, I changed the suggested approach of the conversion in > case of primary stream resources: the `is_resource()` hack mentioned a few > times > before would be used in this case. > - I added impact analysis for each resource category + listed all functions > which return streams (even though they are not impacted). > - I added more examples, reasoning etc. > > Please read the updated proposal because I intend to start the vote shortly > after the holiday season. > > Thanks, > Máté I'm concerned about the 3 way vote on the first item. Requiring a 2/3 majority on a 3-mutually-exclusive option setup doesn't really work, mathematically, and it's very possible for that to result in no conclusion. (Or vote splitting, if 3/4 of voters want to change but split on when, would that result in do-nothing winning?) Since "do nothing" is one of the options, I would recommend an RCV/IRV vote here. The rest of the RFC looks good to me. --Larry Garfield --- ## #122025 — kocsismate90@gmail.com — 2023-12-23T14:34:18+00:00 URL: https://externals.io/message/121660#122025 · In reply to: #122020 Hi Larry, Thanks for your input. I'm fine with adding a separate vote whether votes are ok with the described approaches of converting resources to objects, and then the 3 way vote can be eliminated. Regards, Máté --- ## #122088 — kocsismate90@gmail.com — 2024-01-03T07:36:08+00:00 URL: https://externals.io/message/121660#122088 · In reply to: #122025 Hi Everyone, If there are no further complaints, I intend to start the votes ( https://wiki.php.net/rfc/resource_to_object_conversion) the day after tomorrow. Regards, Máté --- ## #122103 — Larry Garfield — 2024-01-03T15:55:58+00:00 URL: https://externals.io/message/121660#122103 · In reply to: #122088 > Hi Everyone, > > If there are no further complaints, I intend to start the votes ( > https://wiki.php.net/rfc/resource_to_object_conversion) the day after > tomorrow. > > Regards, > Máté "Each vote requires 2/3 majority in order to be accepted. " That's fine for the main vote, but the others are all either/or votes, not yes/no votes, so 2/3 majority doesn't mean anything. Do you mean those are 50/50 votes, or something else? --Larry Garfield --- ## #122105 — kocsismate90@gmail.com — 2024-01-03T23:01:32+00:00 URL: https://externals.io/message/121660#122105 · In reply to: #122103 Hey Larry, That's fine for the main vote, but the others are all either/or votes, not > yes/no votes, so 2/3 majority doesn't mean anything. Do you mean those are > 50/50 votes, or something else? > Thanks for your insight again, You are right, it was a silly bug in the RFC. I've just fixed it so that the vote for the implementation itself is now a primary vote requiring 2/3 majority, while the rest of the votes are now secondary ones requiring a simple majority. Thanks, Máté