Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:116145
MIME-Version: 1.0
References: <CANzZr6kvQP26317=mnHbP49L+y6_PQp=d+uG=dKQTMe4JF+k5g@mail.gmail.com>
In-Reply-To: <CANzZr6kvQP26317=mnHbP49L+y6_PQp=d+uG=dKQTMe4JF+k5g@mail.gmail.com>
Date: Thu, 23 Sep 2021 16:10:39 +0200
Message-ID: <CAF+90c8fPuMw+s1y_RGr6DjiDvch2LKFwJiviJctaBMo29jKuQ@mail.gmail.com>
To: Jeremy Mikola <jmikola@gmail.com>
Cc: PHP Internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="00000000000015d49905ccaa33d0"
Subject: Re: [PHP-DEV] run-tests SKIPIF caching can be problematic for
 third-party extensions
From: nikita.ppv@gmail.com (Nikita Popov)

--00000000000015d49905ccaa33d0
Content-Type: text/plain; charset="UTF-8"

On Wed, Sep 15, 2021 at 7:23 PM Jeremy Mikola <jmikola@gmail.com> wrote:

> I just discovered that run-tests.php was changed to cache SKIPIF evaluation
> since 8.1.0beta3[1]. I believe ext-mongodb ran into the same issue as
> mysqli[2], as we use SKIPIF to check that database contents are clean going
> into a test. The present solution in core is to check SKIPIF output for
> "nocache" [3,4] and allow individual tests to opt out of caching; however,
> I'm worried that won't be portable for third-party extensions, where we
> test with run-tests.php for each version of PHP that we support.
>
> Is there still time to consider an alternative solution? If "nocache" needs
> to remain in place for mysqli, perhaps 8.1.0's run-tests.php can be
> enhanced to consult an environment variable (for maximum BC and playing
> nice with `make test`) that disables caching entirely.
>

I'm not sure I understand the setup you're using. The "nocache" tag that
mysqli uses covers the situation where part of the test setup is done in
the SKIPIF section -- because checking whether the test should be skipped
requires doing the same setup as the actual test. If the test does get
skipped, it's still fine to cache that.

Now, it sounds like your situation is different from that, and you actually
have SKIPIF sections where first one test should be skipped, but then a
later test with identical SKIPIF shouldn't be skipped. Is that correct?
What changes between the time these two tests run?

Regards,
Nikita

--00000000000015d49905ccaa33d0--