Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:116146
MIME-Version: 1.0
References: <CANzZr6kvQP26317=mnHbP49L+y6_PQp=d+uG=dKQTMe4JF+k5g@mail.gmail.com>
 <CAF+90c8fPuMw+s1y_RGr6DjiDvch2LKFwJiviJctaBMo29jKuQ@mail.gmail.com>
In-Reply-To: <CAF+90c8fPuMw+s1y_RGr6DjiDvch2LKFwJiviJctaBMo29jKuQ@mail.gmail.com>
Date: Thu, 23 Sep 2021 16:23:15 +0200
Message-ID: <CAF+90c80VSPyjLhCM03=gScw8V5yfBtsMnFTH8VEByTmQQT-cg@mail.gmail.com>
To: Jeremy Mikola <jmikola@gmail.com>
Cc: PHP Internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="0000000000001948e305ccaa6034"
Subject: Re: [PHP-DEV] run-tests SKIPIF caching can be problematic for
 third-party extensions
From: nikita.ppv@gmail.com (Nikita Popov)

--0000000000001948e305ccaa6034
Content-Type: text/plain; charset="UTF-8"

On Thu, Sep 23, 2021 at 4:10 PM Nikita Popov <nikita.ppv@gmail.com> wrote:

> On Wed, Sep 15, 2021 at 7:23 PM Jeremy Mikola <jmikola@gmail.com> wrote:
>
>> I just discovered that run-tests.php was changed to cache SKIPIF
>> evaluation
>> since 8.1.0beta3[1]. I believe ext-mongodb ran into the same issue as
>> mysqli[2], as we use SKIPIF to check that database contents are clean
>> going
>> into a test. The present solution in core is to check SKIPIF output for
>> "nocache" [3,4] and allow individual tests to opt out of caching; however,
>> I'm worried that won't be portable for third-party extensions, where we
>> test with run-tests.php for each version of PHP that we support.
>>
>> Is there still time to consider an alternative solution? If "nocache"
>> needs
>> to remain in place for mysqli, perhaps 8.1.0's run-tests.php can be
>> enhanced to consult an environment variable (for maximum BC and playing
>> nice with `make test`) that disables caching entirely.
>>
>
> I'm not sure I understand the setup you're using. The "nocache" tag that
> mysqli uses covers the situation where part of the test setup is done in
> the SKIPIF section -- because checking whether the test should be skipped
> requires doing the same setup as the actual test. If the test does get
> skipped, it's still fine to cache that.
>
> Now, it sounds like your situation is different from that, and you
> actually have SKIPIF sections where first one test should be skipped, but
> then a later test with identical SKIPIF shouldn't be skipped. Is that
> correct? What changes between the time these two tests run?
>

Independently of that question (which is about whether "nocache" is
sufficient if we ignore cross-version compatibility issue) I do agree that
an option to disable the skip cache entirely would make sense. Would
https://github.com/php/php-src/pull/7510 work for you?

Regards,
Nikita

--0000000000001948e305ccaa6034--