Coercive Scalar Type Hints RFC - BC breaks

9 years ago by francois@php.net — view source

unread

Hi,

I ran the PHP core 'make test' on a basic configuration, still running a significant number of 8785 tests.

Note: the restrictions on numeric string conversions (leading chars/zeroes, trailing chars) are not simulated yet, but they shouldn't change the results very much.

The ruleset corresponding to the coercive RFC, as it is published today, globally broke 7.3% of tests. From these 7.3%, 5.3% correspond to the disabled conversions from null to scalar types, which will probably have to be reintroduced, as it was suspected. So, if we consider that these conversions are re-enabled, the resulting coercive ruleset breaks 1.9% of the tests, which is rather positive, IMO, because there is still room for improvements.

More details soon.

Regards

François

9 years ago by Yasuo Ohgaki — view source

unread

Hi Francois,

On Tue, Feb 24, 2015 at 1:17 PM, François Laupretre francois@php.net
wrote:

I ran the PHP core 'make test' on a basic configuration, still running a
significant number of 8785 tests.

Note: the restrictions on numeric string conversions (leading
chars/zeroes, trailing chars) are not simulated yet, but they shouldn't
change the results very much.

The ruleset corresponding to the coercive RFC, as it is published today,
globally broke 7.3% of tests. From these 7.3%, 5.3% correspond to the
disabled conversions from null to scalar types, which will probably have to
be reintroduced, as it was suspected. So, if we consider that these
conversions are re-enabled, the resulting coercive ruleset breaks 1.9% of
the tests, which is rather positive, IMO, because there is still room for
improvements.

More details soon.

I'm guessing most failures are due to E_DEPRECATED.
I suggest to test without E_DEPRECATED, then it would
be fair figure for the patch. Could you run test without
E_DEPRECATED? We can use -c option for almost all
tests, I think.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

9 years ago by francois@php.net — view source

unread

I don’t understand which E_DEPRECATED you’re talking about. The E_DEPRECATED mechanism proposed in the RFC is not implemented yet. But, yes, all these failures will raise E_DEPRECATED, as none of them appears in PHP 5. In such a case, hiding E_DEPRECATED will remove every failure.

Regards

François

De : yohgaki@gmail.com [mailto:yohgaki@gmail.com] De la part de Yasuo Ohgaki
Envoyé : mardi 24 février 2015 05:32
À : francois
Cc : Pierre Joye; Zeev Suraski; Anthony Ferrara; PHP internals
Objet : Re: [PHP-DEV] Coercive Scalar Type Hints RFC - BC breaks

Hi Francois,

I ran the PHP core 'make test' on a basic configuration, still running a significant number of 8785 tests.

Note: the restrictions on numeric string conversions (leading chars/zeroes, trailing chars) are not simulated yet, but they shouldn't change the results very much.

More details soon.

I'm guessing most failures are due to E_DEPRECATED.

I suggest to test without E_DEPRECATED, then it would

be fair figure for the patch. Could you run test without

E_DEPRECATED? We can use -c option for almost all

tests, I think.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

9 years ago by Yasuo Ohgaki — view source

unread

Hi Francois,

On Tue, Feb 24, 2015 at 2:01 PM, François Laupretre francois@php.net
wrote:

I don’t understand which E_DEPRECATED you’re talking about. The
E_DEPRECATED mechanism proposed in the RFC is not implemented yet. But,
yes, all these failures will raise E_DEPRECATED, as none of them appears in
PHP 5. In such a case, hiding E_DEPRECATED will remove every failure.

OK. I missed current patch status. Then it seems to early for testing.
I think we need result w/o E_DEPRECATED errors.
I suggest to wait a little more.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

9 years ago by francois@php.net — view source

unread

I think I was not clear. Every 'break' I report here won't generate anything more than E_DEPRECATED in the final implementation. By definition. Ne need to wait for a better patch.

François

-----Message d'origine-----
De : yohgaki@gmail.com [mailto:yohgaki@gmail.com] De la part de Yasuo
Ohgaki
Envoyé : mardi 24 février 2015 06:09
À : francois
Cc : Pierre Joye; Zeev Suraski; Anthony Ferrara; PHP internals
Objet : Re: [PHP-DEV] Coercive Scalar Type Hints RFC - BC breaks

Hi Francois,

On Tue, Feb 24, 2015 at 2:01 PM, François Laupretre francois@php.net
wrote:

I don’t understand which E_DEPRECATED you’re talking about. The
E_DEPRECATED mechanism proposed in the RFC is not implemented yet.
But,
yes, all these failures will raise E_DEPRECATED, as none of them appears in
PHP 5. In such a case, hiding E_DEPRECATED will remove every failure.

OK. I missed current patch status. Then it seems to early for testing.
I think we need result w/o E_DEPRECATED errors.
I suggest to wait a little more.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

9 years ago by Pierre Joye — view source

unread

I think I was not clear. Every 'break' I report here won't generate anything more than E_DEPRECATED in the final implementation. By definition. Ne need to wait for a better patch.

How do you test apps with actual production data? What a given code is
being fed with in production, everywhere, using random
application&users specific data or inputs.

Obviously it is easy failed casts, but what do your tests cover? I do
not worry much about what we know but what we do not know. And
changing the casting rules opens a Pandora box.

9 years ago by francois@php.net — view source

unread

Hi,

De : Pierre Joye [mailto:pierre.php@gmail.com]

How do you test apps with actual production data? What a given code is
being fed with in production, everywhere, using random
application&users specific data or inputs.

I personally have no codebase to test. These results come from a simple PHP core 'make test, probably not representative of a real application, I agree.

I expected ML members to try the tool on their codebase and return numbers but, as usual, after everyone asked for a BC break evaluation tool, now it is here and nobody uses it.

I cannot do more than providing the tool. If nobody wants to use it, we'll have to make a decision based on the few not so representative numbers we have. But I won't accept arguments on potential huge BC break if none is able to give 10 minutes to evaluate the real risk.

Regards

François

9 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: François Laupretre [mailto:francois@php.net]
Sent: Wednesday, February 25, 2015 3:20 AM
To: 'Pierre Joye'
Cc: 'Yasuo Ohgaki'; 'Zeev Suraski'; 'Anthony Ferrara'; 'PHP internals'
Subject: RE: [PHP-DEV] Coercive Scalar Type Hints RFC - BC breaks

Hi,

De : Pierre Joye [mailto:pierre.php@gmail.com]

How do you test apps with actual production data? What a given code is
being fed with in production, everywhere, using random
application&users specific data or inputs.

I personally have no codebase to test. These results come from a simple
PHP
core 'make test, probably not representative of a real application, I
agree.

I expected ML members to try the tool on their codebase and return
numbers but, as usual, after everyone asked for a BC break evaluation
tool,
now it is here and nobody uses it.

I know that I will be testing it with a number of real world apps.

Let's not set the bar here higher than what we have for all other RFCs.
It's not as if we test the entire planet codebase every time we make a
potentially dangerous change, and PHP 7 has plenty of those.
We test a (fairly representative) sample of real world code through some
real world apps, framework tests and our own test suite; Additional list
members and users will test it with their apps and provide feedback, etc.

Zeev