Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:97350
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain belski.net from 85.214.73.107 cause and error)
To: "'Daniel Morris'" <daniel@honestempire.com>,
	<internals@lists.php.net>
Cc: "Matt Ficken (Insight Global)"@mail.klapt.com
References: <1481144234.5762.811846497.711AD419@webmail.messagingengine.com>
In-Reply-To: <1481144234.5762.811846497.711AD419@webmail.messagingengine.com>
Date: Fri, 9 Dec 2016 18:16:44 +0100
Message-ID: <042001d25240$0611f370$1235da50$@belski.net>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
Thread-Index: AQFD19j8CYi/eqjnYTtqmjiTMdbaRaIcuVCA
Content-Language: en-us
Subject: RE: [PHP-DEV] Improvements to the PHP test suite
From: anatol.php@belski.net ("Anatol Belski")

Hi,

> -----Original Message-----
> From: Daniel Morris [mailto:daniel@honestempire.com]
> Sent: Wednesday, December 7, 2016 9:57 PM
> To: internals@lists.php.net
> Subject: [PHP-DEV] Improvements to the PHP test suite
> 
> Since around 2011 there's been an item on the ideas list on php.net:
> (https://wiki.php.net/ideas/runtests) to improve the PHP test suite, I was
> thinking as one of my first major contributions I would like to take a
stab at this,
> there's no upwards of 11,000 tests as compared to the
> 8000 or so when the idea was originally proposed, and it looks like some
work
> was initially started but it's likely now outdated and I think it would be
better to
> start from a blank sheet.
> 
> Would this be of interest for 7.2? My main goals would be to maintain
> backwards compatibility with all the existing tests, allow it to be run on
the
> existing CI infrastructure, be cross-platform, and for it to be part of
the internals
> repository, with initial work being done outside of the repository with a
view to
> merging for the 7.2 release.
> 
> A question I have is that is it better that it continues to be a written
in PHP, or is
> the internals team open to perhaps writing the test runner in another
language,
> keeping the same flags that run-tests.php currently uses, but choosing a
> language which could improve the performance of the suite.
> 
> Thoughts appreciated always.
> Daniel Morris.
> 
There's a tool called PFTT, written in Java http://git.php.net/?p=pftt2.git
. It's currently used for testing of the Windows builds. I'm not sure it'd
run out of the box on Linux, but that's for sure an implementation question.
The main advantage of such a tool is, that the test suite is run with
various SAPIs. The tool has also prepared ways to run the test cases from
arbitrary PHP apps like Symfony, Drupal, Joomla, etc. While being powerful,
it is a great complement to the simplicity of the current test approach in
the PHP core.

But the simplicity itself is an advantage, too. Fe, the current Appveyor
integration I'm doing, is based on the pure PHP test suite - this ensures
the essentials for both build and functionality, as indeed the big part of
the functionality is not SAPI dependent. Also, with the CI mechanisms,
provided for free, there are various limitations like the build and test run
time, so it's is not always possible to pass all the desired test scenarios.
It of course depends on what you're intended to do - my thought is just,
that simplicity matters as much as complexity does, and usage in various
environment should be taken into account.

Regards

Anatol