Hi Internals,
Following my straw poll about the Process resource name, I would like to
present an RFC which clarifies the rough timeline and the BC promises of
the "resource to object conversion" project.
Link: https://wiki.php.net/rfc/resource_to_object_conversion
I'm looking forward to your feedback!
Regards,
Máté
Hi Internals,
Following my straw poll about the Process resource name, I would like to
present an RFC which clarifies the rough timeline and the BC promises of
the "resource to object conversion" project.Link: https://wiki.php.net/rfc/resource_to_object_conversion
I'm looking forward to your feedback!
I don't really have any comments beyond that everything stream related should happen at the same time.
I think I would prefer ProcessHandle and all streams related change to happen with 9.0.
As to COM, I dont think many people at going to notice this, as I doubt it'd be in much use.
Did you do an analyses as to how much either of these changes could break anything?
cheers
Derick
Hi Internals,
Following my straw poll about the Process resource name, I would like to
present an RFC which clarifies the rough timeline and the BC promises of
the "resource to object conversion" project.Link: https://wiki.php.net/rfc/resource_to_object_conversion
Please could you add a separate vote for primary streams if the resource to
object conversion should be done at all (requiring 2/3 votes to be
accepted). I will personally vote against this if there is no is_resource
change as I think it's just too big BC break even for 9.0 - it will likely
require massive update of many code bases.
Cheers
Jakub
Hi,
I will personally vote against this if there is no is_resource
change as I think it's just too big BC break even for 9.0 - it will likely
require massive update of many code bases.
As someone maintaining a large code-base going back to 5.0.0 and bitten in
the past by the resource thing, I wonder whether it was previously
discussed to have all these converted objects implement a Resource
interface and have is_resource()
check for that.
Both the function and maybe also the interface could then also be marked as
deprecated, but it would allow for a much more painless transition.
Personally, I’m now relying on psalm to detect such issues, so if I had a
vote I would selfishly vote yes anyways, but still: for those without
static analysis, this would IMHO make things much easier.
Philip
Hi Derick, Jakub, Phiip
Did you do an analyses as to how much either of these changes could break
anything?
I updated the RFC with some impact analysis. The numbers support my
hypothesis that the conversion
of auxiliary stream resources would cause hardly any BC break - at least in
case of the top 2000 PHP
packages. That's why I believe we can migrate them separately from the
primary ones, which are used
much more often.
I wonder whether it was previously discussed to have all these converted
objects implement a
Resource
interface and haveis_resource()
check for
that.
Yes, the RFC links two threads about this topic:
https://externals.io/message/116127 and
https://externals.io/message/104361#104369. Even though
there's a clear interest in changing how is_resource()
works, doing so
would bring us to a minefield...
Please could you add a separate vote for primary streams if the resource to
object conversion should be done at all (requiring 2/3 votes to be
accepted).
I will personally vote against this if there is no is_resource change as I
think it's just too big BC break even for 9.0 - it will likely require
massive update of many code bases.
Yes, I added the option.
Both the function and maybe also the interface could then also be marked
as deprecated, but it would allow for a much more painless transition.
While I would love to see resources go altogether, deprecating
is_resource()
is way too early. Even if php-src itself manages to sunset
all the built-in resources,
there will still be lots of third party extensions which will still rely on
them.
Personally, I’m now relying on psalm to detect such issues, so if I had a
vote I would selfishly vote yes anyways, but still: for those without
static analysis,
this would IMHO make things much easier.
Yeah, using static analysis should be mandatory for mission critical
systems, since these tools can easily detect issues like wrong
is_resource()
checks.
However, I acknowledge that doing so is not always possible due to some
constraints (i.e. budget limits). Fortunately, the possible issues
caused by resource
to object conversions are usually not too difficult to find out. For
example, in case of the Process resource, one has to search for all
proc_open()
invocations, and
check whether the return values are correctly checked. According to my
experience, the is_resource()
checks are usually very close to the creation
of resources, so
they can be fixed easily when needed.
Regards,
Máté
On Sun, Nov 12, 2023 at 4:48 PM Máté Kocsis kocsismate90@gmail.com
wrote:Hi Internals,
Following my straw poll about the Process resource name, I would like to
present an RFC which clarifies the rough timeline and the BC promises of
the "resource to object conversion" project.Link: https://wiki.php.net/rfc/resource_to_object_conversion
Please could you add a separate vote for primary streams if the resource to
object conversion should be done at all (requiring 2/3 votes to be
accepted). I will personally vote against this if there is no is_resource
change as I think it's just too big BC break even for 9.0 - it will likely
require massive update of many code bases.
What is the point of a major release if we cannot even do such a BC break?
We don't even know when PHP 9.0 is going to happen yet.
I will also state that I am against changing the semantics of is_resource()
unless we remove support for resources altogether from the engine at the
same time so that there is no possible ambiguity.
Which frankly is probably something we should be doing, and that people
paid by the foundation should convert resources to opaque objects in PECL
extensions, and other extensions brought to our attention.
And yes, that also means helping Oracle with OCI8 that is getting unbundled.
I will try to get round to do ext/dba somewhat soon.
Best regards,
Gina P. Banyard
What is the point of a major release if we cannot even do such a BC break? We don't even know when PHP 9.0 is going to happen yet.
I have been using Go for about four years now and it seems they have gotten the backward compatibility issue nailed, and that pays great dividends in developer confidence in the language, i.e.:
https://www.reddit.com/r/golang/comments/17v4xja/anyone_face_issues_when_updating_version_of_go/
They recently explained in depth how they do it:
Also see:
https://thenewstack.io/how-golang-evolves-without-breaking-programs/
Although Go is compiled and PHP is not, I think there still may be significant insight that can be gained for PHP by studying how Go is handling it and applying any lessons learned.
#fwiw
- Mike
What is the point of a major release if we cannot even do such a BC break?
We don't even know when PHP 9.0 is going to happen yet.I have been using Go for about four years now and it seems they have
gotten the backward compatibility issue nailed, and that pays great
dividends in developer confidence in the language, i.e.:https://www.reddit.com/r/golang/comments/17v4xja/anyone_face_issues_when_updating_version_of_go/
They recently explained in depth how they do it:
Also see:
https://thenewstack.io/how-golang-evolves-without-breaking-programs/
Although Go is compiled and PHP is not, I think there still may be
significant insight that can be gained for PHP by studying how Go is
handling it and applying any lessons learned.
Go is a "new" programming language, with its 1.0.0 version being from 2012.
It was also designed from the ground up.
PHP on the other hand wasn't designed, but the language grew organically,
and is 28 years old.
Comparing it to Go, in my opinion, makes no sense.
We should be comparing ourselves to languages of that age or older, the
most famous example being Python, which did a major BC break between its
version 2 and 3.
But Fortran, C, Perl (with Raku), and for sure others have all made changes
to the language, recent or not, that break compatibility.
Go even has a cave out that they may release a Go 2 specification, which
does not guarantee any backwards compatibility with Go 1. [1]
Even if the current lead engineer says this is "never" going to happen, the
cave out still exists.
More importantly, it is possible to write cross compatible code, even
without changing anything about is_resource()
, if we convert streams to
opaque objects.
It might be tedious and one might need to have redundant instanceof checks
with is_resource()
if one does not want to check for a false return, or
duplicate checks for closed resources.
But it is possible, which was not the case for Python 2 and 3 as it
changed fundamentally how strings behaved.
Finally, I think people would have more confidence in the language if it
stopped coming with various foot guns included that need to be explicitly
kept in check by using external tools such as static analysis tools, or
code style tools.
And removing those, or making the language overall more coherent and
consistent, requires us to break backwards compatibility.
Sincerely,
Gina P. Banyard
[1] https://go.dev/doc/go1compat:
It is intended that programs written to the Go 1 specification will
continue to compile and run correctly, unchanged, over the lifetime of that
specification. At some indefinite point, a Go 2 specification may arise,
but until that time, Go programs that work today should continue to work
even as future "point" releases of Go 1 arise (Go 1.1, Go 1.2, etc.).
More importantly, it is possible to write cross compatible code, even
without changing anything aboutis_resource()
, if we convert streams to
opaque objects.
It might be tedious and one might need to have redundant instanceof checks
withis_resource()
if one does not want to check for a false return, or
duplicate checks for closed resources.
Extensions like GD, IMAP, Curl, and FTP were once very "resource
heavy", but they already received this migration with next to none
disruptions. Shameless plug here
(https://php.watch/articles/resource-object) where I track the
progress. As these changes were merged in the last couple years, I
spent a few hours grepping the Composer top 1000 packages. There were
not that many is_resource
checks, and it was quite easy and
straightforward to update them to account for the class objects.
I'm supportive of our trajectory so far by slowly but steadily moving
resource objects to class objects. I also support that we do not
modify is_resource
to mark resource class objects as a resource. I
really look forward to the day that we remove is_resource
and kill
the final resource object in PHP 🥺.
On Nov 21, 2023 at 11:33 PM, <G. P. B. george.banyard@gmail.com>
wrote:What is the point of a major release if we cannot even do such a BC
break?
We don't even know when PHP 9.0 is going to happen yet.I have been using Go for about four years now and it seems they have
gotten the backward compatibility issue nailed, and that pays great
dividends in developer confidence in the language, i.e.:https://www.reddit.com/r/golang/comments/17v4xja/anyone_face_issues_when_updating_version_of_go/
They recently explained in depth how they do it:
Also see:
https://thenewstack.io/how-golang-evolves-without-breaking-programs/
Although Go is compiled and PHP is not, I think there still may be
significant insight that can be gained for PHP by studying how Go is
handling it and applying any lessons learned.Go is a "new" programming language, with its 1.0.0 version being from 2012.
It was also designed from the ground up.PHP on the other hand wasn't designed, but the language grew organically,
and is 28 years old.
Comparing it to Go, in my opinion, makes no sense.We should be comparing ourselves to languages of that age or older, the
most famous example being Python, which did a major BC break between its
version 2 and 3.
But Fortran, C, Perl (with Raku), and for sure others have all made changes
to the language, recent or not, that break compatibility.Go even has a cave out that they may release a Go 2 specification, which
does not guarantee any backwards compatibility with Go 1. [1]
Even if the current lead engineer says this is "never" going to happen, the
cave out still exists.More importantly, it is possible to write cross compatible code, even
without changing anything aboutis_resource()
, if we convert streams to
opaque objects.
It might be tedious and one might need to have redundant instanceof checks
withis_resource()
if one does not want to check for a false return, or
duplicate checks for closed resources.
But it is possible, which was not the case for Python 2 and 3 as it
changed fundamentally how strings behaved.Finally, I think people would have more confidence in the language if it
stopped coming with various foot guns included that need to be explicitly
kept in check by using external tools such as static analysis tools, or
code style tools.
And removing those, or making the language overall more coherent and
consistent, requires us to break backwards compatibility.Sincerely,
Gina P. Banyard
I sympathise with both sides on this topic. As a Software Engineer, not
breaking 10-30 years of BC promise is unsustainable, but as a PHP user BC
breaks have a heavy impact on legacy codebase written 10~20 years ago.
A lot of discussion has happened around this subject, but unfortunately no
consensus is ever reached. I've talked about increasing the stability and
the pool of maintainers by providing PHP Packages under PHP namespace (
https://externals.io/message/120335#120354). There were a lot of
discussions on language evolution and editions that ultimately didn't go
much further.
Given how much time has passed and how this subject is always present, I
now look at this with the optics that both PHP internals devs and PHP users
suffer from the same condition of dealing with legacy. Between 1990 up to
2010, give or take, the way software used to be built vastly differs from
how software is built today and these old software can be very hard to
decommission given their lack of automation tests and inability to be
statically analysed. With today's practices, I believe it's easier to
introduce BC breaks that affect software written after 2018. And since I
believe that, it's a natural consequence for me to believe that if PHP
introduced a new declare(backward_compatibility=0)
, it could be used for
users to signal to the engine that 1) we're writing this file after 2023
and 2) we will cover this file with automation tests and/or static analysis
tools. PHP Internals wouldn't need to make a huge big-bang BC break
all-at-once. Every year new BC breaks could be introduced affecting only
the "new engine version". The idea isn't to build several combination of
"PHP Editions" as it was discussed in the past, but rather to have a
consensus between PHP Internals and PHP Developers that a new Engine is
constantly being developed, this engine will break compatibility with the
past 20 years of PHP whenever PHP Internals manage to rebuild something
(throughout multiple versions), a migration path assumes users will use of
Rector, Static Analysers and PHPUnit and the old engine keeps the BC
promise in order to allow old software to keep running, but is expected to
degrade in performance and to only support new stuff if it has no added
burden to internals.
When such assumptions are made about userland, the concept of what's an
acceptable BC break should be skewed in favor of helping PHP Internals.
--
Marco Deleu
On Nov 21, 2023 at 11:33 PM, <G. P. B. george.banyard@gmail.com>
wrote:What is the point of a major release if we cannot even do such a BC
break?
We don't even know when PHP 9.0 is going to happen yet.I have been using Go for about four years now and it seems they have
gotten the backward compatibility issue nailed, and that pays great
dividends in developer confidence in the language, i.e.:https://www.reddit.com/r/golang/comments/17v4xja/anyone_face_issues_when_updating_version_of_go/
They recently explained in depth how they do it:
Also see:
https://thenewstack.io/how-golang-evolves-without-breaking-programs/
Although Go is compiled and PHP is not, I think there still may be
significant insight that can be gained for PHP by studying how Go is
handling it and applying any lessons learned.Go is a "new" programming language, with its 1.0.0 version being from 2012.
It was also designed from the ground up.PHP on the other hand wasn't designed, but the language grew organically,
and is 28 years old.
Comparing it to Go, in my opinion, makes no sense.We should be comparing ourselves to languages of that age or older, the
most famous example being Python, which did a major BC break between its
version 2 and 3.
But Fortran, C, Perl (with Raku), and for sure others have all made changes
to the language, recent or not, that break compatibility.Go even has a cave out that they may release a Go 2 specification, which
does not guarantee any backwards compatibility with Go 1. [1]
Even if the current lead engineer says this is "never" going to happen, the
cave out still exists.More importantly, it is possible to write cross compatible code, even
without changing anything aboutis_resource()
, if we convert streams to
opaque objects.
It might be tedious and one might need to have redundant instanceof checks
withis_resource()
if one does not want to check for a false return, or
duplicate checks for closed resources.
But it is possible, which was not the case for Python 2 and 3 as it
changed fundamentally how strings behaved.Finally, I think people would have more confidence in the language if it
stopped coming with various foot guns included that need to be explicitly
kept in check by using external tools such as static analysis tools, or
code style tools.
And removing those, or making the language overall more coherent and
consistent, requires us to break backwards compatibility.Sincerely,
Gina P. Banyard
I sympathise with both sides on this topic. As a Software Engineer, not
breaking 10-30 years of BC promise is unsustainable, but as a PHP user BC
breaks have a heavy impact on legacy codebase written 10~20 years ago.A lot of discussion has happened around this subject, but unfortunately no
consensus is ever reached. I've talked about increasing the stability and
the pool of maintainers by providing PHP Packages under PHP namespace (
https://externals.io/message/120335#120354). There were a lot of
discussions on language evolution and editions that ultimately didn't go
much further.Given how much time has passed and how this subject is always present, I
now look at this with the optics that both PHP internals devs and PHP users
suffer from the same condition of dealing with legacy. Between 1990 up to
2010, give or take, the way software used to be built vastly differs from
how software is built today and these old software can be very hard to
decommission given their lack of automation tests and inability to be
statically analysed. With today's practices, I believe it's easier to
introduce BC breaks that affect software written after 2018. And since I
believe that, it's a natural consequence for me to believe that if PHP
introduced a newdeclare(backward_compatibility=0)
, it could be used for
users to signal to the engine that 1) we're writing this file after 2023
and 2) we will cover this file with automation tests and/or static analysis
tools. PHP Internals wouldn't need to make a huge big-bang BC break
all-at-once. Every year new BC breaks could be introduced affecting only
the "new engine version". The idea isn't to build several combination of
"PHP Editions" as it was discussed in the past, but rather to have a
consensus between PHP Internals and PHP Developers that a new Engine is
constantly being developed, this engine will break compatibility with the
past 20 years of PHP whenever PHP Internals manage to rebuild something
(throughout multiple versions), a migration path assumes users will use of
Rector, Static Analysers and PHPUnit and the old engine keeps the BC
promise in order to allow old software to keep running, but is expected to
degrade in performance and to only support new stuff if it has no added
burden to internals.When such assumptions are made about userland, the concept of what's an
acceptable BC break should be skewed in favor of helping PHP Internals.--
Marco Deleu
Hey Marco,
I think the biggest issue with that strategy is that you end up
endorsing projects that PHP cannot control. This may not be a bad
thing, but personally, I'd love to see PHP start adopting these kinds
of projects as officially maintained by PHP. For example, composer
could be forked or brought over to the PHP organization and then if
PHP wants to change how autoloading works ... they can just change it
and implement a migration path in composer/rector/whatever. All these
tools maintained by volunteers over the years would benefit from being
able to have important conversations before a feature is even
developed; even be able to suggest changes to the feature that would
allow for seamless migrations.
Imagine installing PHP 9.0 and then just running php --migrate-from=8.2
and having it run rector + composer (maybe even
static analysis before and after to give you issues) migration scripts
on an entire codebase.
Right now, you need to know what to do and how to use it. You may not
even know these tools exist or what to search for to find them,
particularly if you are new to PHP.
Robert Landers
Software Engineer
Utrecht NL
Hey Everyone,
Sorry for the radio silence, I was busy with other tasks. However, I
managed to improve the RFC in the recent days the following way:
- most importantly, I changed the suggested approach of the conversion in
case of primary stream resources: theis_resource()
hack mentioned a few
times
before would be used in this case. - I added impact analysis for each resource category + listed all functions
which return streams (even though they are not impacted). - I added more examples, reasoning etc.
Please read the updated proposal because I intend to start the vote shortly
after the holiday season.
Thanks,
Máté
Hey Everyone,
Sorry for the radio silence, I was busy with other tasks. However, I
managed to improve the RFC in the recent days the following way:
- most importantly, I changed the suggested approach of the conversion in
case of primary stream resources: theis_resource()
hack mentioned a few
times
before would be used in this case.- I added impact analysis for each resource category + listed all functions
which return streams (even though they are not impacted).- I added more examples, reasoning etc.
Please read the updated proposal because I intend to start the vote shortly
after the holiday season.Thanks,
Máté
I'm concerned about the 3 way vote on the first item. Requiring a 2/3 majority on a 3-mutually-exclusive option setup doesn't really work, mathematically, and it's very possible for that to result in no conclusion. (Or vote splitting, if 3/4 of voters want to change but split on when, would that result in do-nothing winning?) Since "do nothing" is one of the options, I would recommend an RCV/IRV vote here.
The rest of the RFC looks good to me.
--Larry Garfield
Hi Larry,
Thanks for your input. I'm fine with adding a separate vote whether votes
are ok with the described approaches of converting resources to objects,
and then the 3 way vote can be eliminated.
Regards,
Máté
Hi Everyone,
If there are no further complaints, I intend to start the votes (
https://wiki.php.net/rfc/resource_to_object_conversion) the day after
tomorrow.
Regards,
Máté
Hi Everyone,
If there are no further complaints, I intend to start the votes (
https://wiki.php.net/rfc/resource_to_object_conversion) the day after
tomorrow.Regards,
Máté
"Each vote requires 2/3 majority in order to be accepted. "
That's fine for the main vote, but the others are all either/or votes, not yes/no votes, so 2/3 majority doesn't mean anything. Do you mean those are 50/50 votes, or something else?
--Larry Garfield
Hey Larry,
That's fine for the main vote, but the others are all either/or votes, not
yes/no votes, so 2/3 majority doesn't mean anything. Do you mean those are
50/50 votes, or something else?
Thanks for your insight again, You are right, it was a silly bug in the
RFC. I've just fixed it so that the vote for the implementation itself is
now a primary vote requiring 2/3 majority, while the
rest of the votes are now secondary ones requiring a simple majority.
Thanks,
Máté
Link: https://wiki.php.net/rfc/resource_to_object_conversion
Is "Primary stream-related resources" section about file pointers? I'd
like it to be more clear. Maybe it should even list all functions that
will be impacted. I don't see an impact analysis for this section either.
--
Aleksander Machniak
Kolab Groupware Developer [https://kolab.org]
Roundcube Webmail Developer [https://roundcube.net]
PGP: 19359DC1 # Blog: https://kolabian.wordpress.com