Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:116046
MIME-Version: 1.0
References: <CANzZr6kvQP26317=mnHbP49L+y6_PQp=d+uG=dKQTMe4JF+k5g@mail.gmail.com>
 <CAESVnVr6x+1r375wEEGM6FdahkoNEaBv7ncu-d3HgXJN2wHunw@mail.gmail.com>
In-Reply-To: <CAESVnVr6x+1r375wEEGM6FdahkoNEaBv7ncu-d3HgXJN2wHunw@mail.gmail.com>
Date: Wed, 15 Sep 2021 16:33:07 -0400
Message-ID: <CANzZr6=Qt+HwkMF3epb8km9NaknRKYAyAQ+Hx9fL+fyW54c5xQ@mail.gmail.com>
To: Sara Golemon <pollita@php.net>
Cc: PHP Internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="000000000000c4697805cc0e9bae"
Subject: Re: [PHP-DEV] run-tests SKIPIF caching can be problematic for
 third-party extensions
From: jmikola@gmail.com (Jeremy Mikola)

--000000000000c4697805cc0e9bae
Content-Type: text/plain; charset="UTF-8"

On Wed, Sep 15, 2021 at 1:59 PM Sara Golemon <pollita@php.net> wrote:

>
> If the phpt file outputs "skip nocache" then that will skip without
> caching on all versions, won't it?  In 8.1 because it has the explicit
> 'nocache' provided, and in older versions because they don't cache anyway.
>
> Am I missing something?
>

If I'm reading the most recent patch correctly [^1], the SKIPIF result is
always cached if the output does not start with "nocache". If the output
starts with "nocache" (ignoring whatever may follow), the entire result is
replaced with an empty string. Given that, it seems like it's not possible
to opt out of caching and skip a test. Perhaps that's by design if the
cache is only intended to memoize SKIPIF blocks that do result in a test
skip.

But the question remains: how do we portably opt out of caching without
skipping a test? In PHP 7.4 (and presumably earlier versions),
run-tests.php appears to ignore unrecognized SKIPIF output [^2]; however,
8.0 is more strict and will flag unrecognized output as borked [^3]. I
think Nikita alluded to that logic in his most recent patch [^1], since the
same validation exists in 8.1; however, I don't think backwards
compatibility (e.g. running the same test on both 8.0 and 8.1) was being
considered there.

[^1]:
https://github.com/php/php-src/commit/0074a1d4e3a85d0d63118e7a30f4b7ed6da64695
[^2]: https://github.com/php/php-src/blob/php-7.4.23/run-tests.php#L2127
[^3]: https://github.com/php/php-src/blob/php-8.0.10/run-tests.php#L2200

-- 
jeremy mikola

--000000000000c4697805cc0e9bae--