BC breaking changes in PHP 8.1

I just want to point out that PHP does not follow semantic versioning. It
never did and there are no plans to start following it now.

This statement is true.

Breaking changes were made in all previous versions, both major and minor.

This statement may be true, depending on how you define "breaking",
which is always tricky.

The only time when we try to avoid breaking changes is in patch releases.
These are extremely uncommon.

This statement, however, is false. (As are similar statements made by
other people in this thread.) There is an explicit policy to avoid
breaking changes in minor releases here:
https://wiki.php.net/rfc/releaseprocess

Under heading "x.y.z to x.y+1.z", there are bullets for "Backward
compatibility must be kept" and "API compatibility must be kept (userland)".

So, while it may be reasonable to claim that we frequently fail to do
so, we always try to avoid breaking changes in minor releases such as 8.1.

Whether this particular compatibility break is acceptable is a subject
for wider debate. I do sympathise with Matthew's position that it would
be easier for users if the "resource" type was simply removed all in one
go in 9.0. I also sympathise with the position that it's easier for
maintainers to work through this as a longer-term project, with
multiple deliverables.

I also share the concern that there wasn't an RFC introducing this
policy, which in hindsight could have led to a more careful migration
strategy. For instance, should we have implemented a way for
is_resource() and/or get_resource_type() to return expected values for
the new objects?

I have much less sympathy for some of the more extreme responses in this
thread with ridiculous hyperbole like "shooting ourselves in the face".
There is no perfect answer, and we should be able to have a rational
discussion about the pros and cons of different strategies.

Regards,

--
Rowan Tommins
[IMSoP]

Le 22/09/2021 à 16:24, Matthew Weier O'Phinney a écrit :

On Wed, 22 Sept 2021 at 14:30, Matthew Weier O'Phinney <
mweierophinney@gmail.com> wrote:
As somebody who's been contributing to and maintaining OSS libraries
forever (since 2002), the pace of change of PHP is, frankly, ridiculous. I
can keep up with patches. I can keep up with new features. But BC breaks
EVERY YEAR just creates churn. I've spent most of the past 18 months doing
nothing but ensuring libraries work on new PHP versions. I then get users
angry that they aren't getting new features; if I don't update to the
latest PHP version, I get other users angry they can't use the library on
the newer PHP version. And with new PHP versions every year... I
essentially have to update every 2-3 years regardless, and will lose users
if I don't do it every year.

Figure out what the BC breaks are going to be, and do them all at once.
It's far easier for the ecosystem to adapt to a big drop of BC breaks every
3-5 years than it is every year.

Again, I DO like the resource objects feature. But getting a few here and
there over many minor releases is a lot of breakage to track and adapt to.
And that's just the tip of the iceberg.

Hello,

I fully understand your situation, but I want to mitigate, I do maintain
a few OSS tools (and much more internal code where I work for) and I
experienced those PHP upgrades as well (one project when from PHP 5.6 to
7.4 and its platform code served as a basis for a new one using 8.0, for
example, but I have many), and it never was such a pain. Laminas is a
huge framework thought, so it seems more likely that you experience this
pain*10 compared to me. Nevertheless, I have an opposite opinion, I
think that until now, PHP has did an extremely good job in not making
too much BC breaks, and keeping those BC breaks easy to deal with.

Regards,

--

Pierre

As somebody who's been contributing to and maintaining OSS libraries
forever (since 2002), the pace of change of PHP is, frankly, ridiculous. I
can keep up with patches. I can keep up with new features. But BC breaks
EVERY YEAR just creates churn. I've spent most of the past 18 months doing
nothing but ensuring libraries work on new PHP versions. I then get users
angry that they aren't getting new features; if I don't update to the
latest PHP version, I get other users angry they can't use the library on
the newer PHP version. And with new PHP versions every year... I
essentially have to update every 2-3 years regardless, and will lose users
if I don't do it every year.

Figure out what the BC breaks are going to be, and do them all at once.
It's far easier for the ecosystem to adapt to a big drop of BC breaks every
3-5 years than it is every year.

There’s merit to spacing it out - I doubt anyone wants another PHP 7 flag day again.

(Ask the Python people how they feel about moving all the breaking changes to a single release…)

On Wed, 22 Sept 2021 at 15:24, Matthew Weier O'Phinney <
mweierophinney@gmail.com> wrote:

On Wed, 22 Sept 2021 at 14:30, Matthew Weier O'Phinney <
mweierophinney@gmail.com> wrote:

Yesterday, I opened an issue regarding a change in the pgsql extension (
https://bugs.php.net/bug.php?id=81464).

PHP 8.0 introduced the concept of "resource objects". Where previously we
would have resources, and use get_resource_type() when we needed to
differentiate various resources, resource objects give us immutable
objects
instead that allow us to type hint. Personally, I think this is
wonderful!

The rollout for 8.0 was incomplete, however, and only touched on
something
like 4-6 different resource types. Still, a good start.

With PHP 8.1, we're seeing the addition of more of these, and it was
encountering one of those changes that prompted the bug I previously
linked
to.

Here's the issue: while overall, I like the move to resource objects,
introducing them in a MINOR release is hugely problematic.

Previously, you would do constructs such as the following:

if (! is_resource($resource) || get_resource_type($resource) !==

$someSpecificType) {
// skip a test or raise an exception
}

Resource objects, however:

• Return false for is_resource() checks.
• Raise a warning for get_resource_type() checks, and/or report the
  resource object class name — which differs from the previous resource
  names
  in all cases.

This means conditionals like the above BREAK. As a concrete example, I
did
PHP 8.1 updates for laminas-db last week, and assumed our postgres
integration tests were running when we finally had all tests passing
successfully. However, what was really happening was that our test suite
was testing with is_resource() and skipping tests if resources were not
present. We shipped with broken pgsql support as a result, and it wasn't
until test suites in other components started failing that we were able
to
identify the issue.

Further, the "fix" so that the code would work on both 8.1 AND versions
prior to 8.1 meant complicating the conditional, adding a ! $resource instanceof \PgSql\Connection into the mix. The code gets unwieldy very
quickly, and having to do this to support a new minor version was
irritating.

When I opened the aforementioned bug report, it was immediately closed as
"not an issue" with the explanation that it was "documented in
UPGRADING".

This is not an acceptable explanation.

• There was no RFC related to 8.1 indicating these changes were
  happening.
  (In fact, there was no RFC for resource objects in the first place —
  which
  is concerning considering the BC implications!)
• In semantic versioning, existing APIs MUST NOT change in a new minor
  version, only in new major versions.

Reading the UPGRADING guide, there's a HUGE section of backwards
incompatible changes for 8.1 — THIRTY-FOUR of them. Nested in these are
notes of around a half-dozen extensions that once produced resources now
producing resource objects.

The pace of change in PHP is already breathtaking when you consider large
projects (both OSS and in userland); keeping up with new features is in
and
of itself quite a challenge. Introducing BC breaks in minor versions
makes
things harder for everyone, as now you have to figure out not only if
there's new features you want to adopt, but whether or not there are
changes that will actively break your existing code. I strongly feel that
anything in the backwards incompatible section of the UPGRADING guide
should be deferred to 9.0, when people actually expect things to change.

Resource to object conversions have precedent for not being in a major
version:
GMP in PHP 5.6
Hash in PHP 7.2

The fix to how to check these conditions is the same as last year and it
is to check against false, not is_resource nor an instance of the class.
It is true that if you ensure that the resource type is correct it gets
slightly unwieldy but:

if ($resource !== false && ((is_resource($resource) &&
get_resource_type($resource) === $someSpecificType) || $resource instanceof
ClassName) ) {
// skip a test or raise an exception
}

Should be identical to the changes made to support PHP 5.6, 7.2, and 8.0
in regards to the other conversions, and I don't see why it is now an issue.
We also explicitly hold off from pouring more of our time in resources
to object conversions during the Beta cycle of PHP 8.0 as we knew we could
defer this to a later minor version, compared to promotions from E_WARNING
to Exceptions which took a huge amount of last summer.

PHP tries to broadly follow semantic versioning but it never has
adhered to it strictly, and this is not uncommon within programming
languages nomenclature (e.g. Python).

Moreover, many changes to php-src are based, and a result of this
refactoring from resources to objects, so it isn't just a simple "revert
commit", especially as we are in the RC phase.

Albeit this is only my opinion, this is still not a bug, and I frankly
don't think any of it should be reverted as we, the core team, are never
going to be able to convert all of the resource to objects in time for a
major release, except if we decide that each year PHP is getting a major
release and we throw minor versions out of the windows.

With changes like this, though, PHP is already de facto doing major
releases every year.

If internals buys the argument Kamil makes that "PHP does not follow
semantic versioning", then:

• Stop using semantic versioning numbers, as the numbers lead to
  (appropriate) user expectations about stability.
• Or start numbering versions properly, and call each minor a new major,
  so users know what to expect.

I'm sorry but this is nonsense, we don't use semantic version numbers, they
might look like them but they are not.
All of the version names of Python, Golang, Ruby, and TypeScript just to
name a few look like semantic versioning but are not.

Example for Python:
https://docs.python.org/release/3.9.0/whatsnew/3.9.html#removed
Example for Golang: https://golang.org/doc/devel/release#policy
Example for Ruby:
https://github.com/ruby/ruby/blob/v2_5_0/NEWS#label-Compatibility+issues+-28excluding+feature+bug+fixes-29
Example for TypeScript:
https://devblogs.microsoft.com/typescript/announcing-typescript-4-2/#breaking-changes

And a non programming languages example Spring Boot doesn't adhere strictly
to semver either [1]

So I sincerely hope you're also telling these projects to change their
naming scheme, just so that it cannot be mistaken as semver.

As a sanity check I looked at the migration guide of every single version
of PHP from PHP 5.0 onwards, and all had some BC break in a minor version,
so I don't know why you're touting this as an issue now.

As somebody who's been contributing to and maintaining OSS libraries
forever (since 2002), the pace of change of PHP is, frankly, ridiculous. I
can keep up with patches. I can keep up with new features. But BC breaks
EVERY YEAR just creates churn. I've spent most of the past 18 months doing
nothing but ensuring libraries work on new PHP versions. I then get users
angry that they aren't getting new features; if I don't update to the
latest PHP version, I get other users angry they can't use the library on
the newer PHP version. And with new PHP versions every year... I
essentially have to update every 2-3 years regardless, and will lose users
if I don't do it every year.

Figure out what the BC breaks are going to be, and do them all at once.
It's far easier for the ecosystem to adapt to a big drop of BC breaks every
3-5 years than it is every year.

I don't think any of us can look into the future and just "figure out" what
the BC breaks are going to be, because if we could I'm pretty sure we
wouldn't be introducing those issues in the first place.
Moreso, PHP is also an open source project, so people work on what they
want and when they want to, which such "clean-up" tasks might get boring
for folks.
Something I would imagine that you should now, as you've been in the
community for two decades.

I'd also point out that the only reason the BC breaks are now every year is
not because we randomly decided to, but because of the fixed yearly release
schedule which was introduced with PHP 7.
So, should we now go back to a random release cycle where we release a
version when it is deemed ready, and go back to the criticism that the
language is stagnating?

I'll add that some of these BCs are a prerequisite for some features, we
might be able to mitigate it to some extent, see the readonly properties
kafufle with WordPress, but this is not always guaranteed.

Again, I DO like the resource objects feature. But getting a few here and
there over many minor releases is a lot of breakage to track and adapt to.
And that's just the tip of the iceberg.

Again, breakages are not a new thing, so I don't see why you are pointing
this out now.
And FYI the resource to object migration has a tracker [2] and is for the
most part complete, with the exception of streams, which due to the heavy
engine works which needs to be done, can probably only happen for a PHP 9.0
release.

There’s merit to spacing it out - I doubt anyone wants another PHP 7 flag

day again.

(Ask the Python people how they feel about moving all the breaking
changes to a single release…)

I can tell you now a lot of us OSS maintainers would prefer it to yearly
updates.

We are also an OSS project, so my question to you, do you fundamentally
break compatibility between two major versions of your project so that
people cannot upgrade to the new version without a full rewrite?
Because this is what Python 2 to 3 was, and many project did not upgrade
from one version to another and Python has only dropped support for Python
2 after 10 years, and there are still loads of projects which didn't make
the upgrade.

Therefore, why on earth should we as a project shoot ourselves in the face
by making it so daunting to upgrade with a 500 bullet item list of BC
breaks in one version compared to spanning it out over multiple years.

It might be a good idea not to assume, and instead actually do some
scientific polling of users and ecosystem library/framework maintainers.
Based on my conversations with other maintainers in the ecosystem, my
experience is not isolated by any means. On top of that, I get to analyze
the annual Zend user surveys, and the number one reason for people being on
older PHP versions (and > 50% of respondents are, EVERY YEAR) is the cost
of upgrading. Clearly, there's a perception that something is broken with
the current approach, and internals is ignoring it.

How can we ignore something when this seems to be the first time this is a
major complaint?

BTW, another good possibility, recommended by somebody responding to a
twitter thread I started around this issue: work with RectorPHP to provide
a ruleset for each minor release, to ease upgrades. It's far easier than
having to read through an UPGRADING guide and having to figure it out for
yourself. I'd argue these should be in place as soon as a beta is ready.

Sure, where is the money to pay someone to do this?

Best regards,

George P. Banyard

[1]
https://developer.okta.com/blog/2019/12/16/semantic-versioning#phil-webb-spring-boot
[2] https://github.com/php/php-tasks/issues/6

As somebody who's been contributing to and maintaining OSS libraries
forever (since 2002), the pace of change of PHP is, frankly, ridiculous. I
can keep up with patches. I can keep up with new features. But BC breaks
EVERY YEAR just creates churn. I've spent most of the past 18 months doing
nothing but ensuring libraries work on new PHP versions. I then get users
angry that they aren't getting new features; if I don't update to the
latest PHP version, I get other users angry they can't use the library on
the newer PHP version. And with new PHP versions every year... I
essentially have to update every 2-3 years regardless, and will lose users
if I don't do it every year.

I'm sorry you feel this way; this hasn't been my experience nor the
experience of many people which I interact with.

Figure out what the BC breaks are going to be, and do them all at once.
It's far easier for the ecosystem to adapt to a big drop of BC breaks every
3-5 years than it is every year.

We struggle to find consensus, and now you want us to find consensus
years in advance? Sorry, I don't think it's practical.

We can do better, though. I think we could commit to having no BC
breaks in the last minor of a major series e.g. no BC breaks in PHP
7.4. This requires us to agree 2 years in advance when we'll release
the next major. We could perhaps choose this for a fixed schedule,
e.g. we release a new major every X=5 years.

Anyway, I agree with an earlier statement that reverting resource to
object conversions at this stage is not feasible.

Le dim. 26 sept. 2021 à 14:42, Jordi Boggiano j.boggiano@seld.be a écrit :

I'm surprised that is_resource() returns false for resource objects,
does anyone knows why it wouldn't return true in such case ?

This is a very weird behavior, I'd expect it to return true, moreover
this is the most annoying detail of this BC break, in my opinion.

I have lots of code using is_resource() but I usually don't bother
checking for type (maybe I'm doing it wrong). But I'd also expect
get_resource_type() to be deprecated somehow and return
backward-compatible values for converted resources to objects.

Right, having is_resource return true for resourcey objects would have
prevented all resource-related errors I ran into with php8.0 migration I
believe. That'd be a welcome improvement/fix for 8.1.

I'm not sure about this one: is_resource($foo) ? get_resource_type() is
pretty common. Also this would break some expectations with gettype($foo)

Nicolas

I'm surprised that is_resource() returns false for resource objects,
does anyone knows why it wouldn't return true in such case ?

This is a very weird behavior, I'd expect it to return true, moreover
this is the most annoying detail of this BC break, in my opinion.

I have lots of code using is_resource() but I usually don't bother
checking for type (maybe I'm doing it wrong). But I'd also expect
get_resource_type() to be deprecated somehow and return
backward-compatible values for converted resources to objects.

Right, having is_resource return true for resourcey objects would have
prevented all resource-related errors I ran into with php8.0 migration I
believe. That'd be a welcome improvement/fix for 8.1.

that was proposed but purity won if I remember correctly.

That's kind of defeat the initial goal (attempt) of yearly releases. Now I
feel like we introduced more, small or big, bc in minor releases in the
name of purity. That counter balance the yearly release with no bc from a
migration pace pov.

best,
Pierre

Might one possible way forward be to work with the PHPUnit folks to make deprecations not a test-breaking issue, but report them side-channel somehow? (Like "risky" tests now pass, but have an extra note on them.) That seems like it would resolve the "deprecations get turned into breaking changes immediately" problem, at least.

There is already a "convertDeprecationsToExceptions" configuration
setting, but it defaults to "true":
https://phpunit.readthedocs.io/en/stable/configuration.html

I disagree with that default, but presumably some people think it is a
sensible one.

Regards,

--
Rowan Tommins
[IMSoP]