Since around 2011 there's been an item on the ideas list on php.net:
(https://wiki.php.net/ideas/runtests) to improve the PHP test suite, I
was thinking as one of my first major contributions I would like to take
a stab at this, there's no upwards of 11,000 tests as compared to the
8000 or so when the idea was originally proposed, and it looks like some
work was initially started but it's likely now outdated and I think it
would be better to start from a blank sheet.
Would this be of interest for 7.2? My main goals would be to maintain
backwards compatibility with all the existing tests, allow it to be run
on the existing CI infrastructure, be cross-platform, and for it to be
part of the internals repository, with initial work being done outside
of the repository with a view to merging for the 7.2 release.
A question I have is that is it better that it continues to be a written
in PHP, or is the internals team open to perhaps writing the test runner
in another language, keeping the same flags that run-tests.php currently
uses, but choosing a language which could improve the performance of the
suite.
Thoughts appreciated always.
Daniel Morris.
--
Daniel Morris
daniel@honestempire.com
Hi,
-----Original Message-----
From: Daniel Morris [mailto:daniel@honestempire.com]
Sent: Wednesday, December 7, 2016 9:57 PM
To: internals@lists.php.net
Subject: [PHP-DEV] Improvements to the PHP test suiteSince around 2011 there's been an item on the ideas list on php.net:
(https://wiki.php.net/ideas/runtests) to improve the PHP test suite, I was
thinking as one of my first major contributions I would like to take a
stab at this,
there's no upwards of 11,000 tests as compared to the
8000 or so when the idea was originally proposed, and it looks like some
work
was initially started but it's likely now outdated and I think it would be
better to
start from a blank sheet.Would this be of interest for 7.2? My main goals would be to maintain
backwards compatibility with all the existing tests, allow it to be run on
the
existing CI infrastructure, be cross-platform, and for it to be part of
the internals
repository, with initial work being done outside of the repository with a
view to
merging for the 7.2 release.A question I have is that is it better that it continues to be a written
in PHP, or is
the internals team open to perhaps writing the test runner in another
language,
keeping the same flags that run-tests.php currently uses, but choosing a
language which could improve the performance of the suite.Thoughts appreciated always.
Daniel Morris.
There's a tool called PFTT, written in Java http://git.php.net/?p=pftt2.git
. It's currently used for testing of the Windows builds. I'm not sure it'd
run out of the box on Linux, but that's for sure an implementation question.
The main advantage of such a tool is, that the test suite is run with
various SAPIs. The tool has also prepared ways to run the test cases from
arbitrary PHP apps like Symfony, Drupal, Joomla, etc. While being powerful,
it is a great complement to the simplicity of the current test approach in
the PHP core.
But the simplicity itself is an advantage, too. Fe, the current Appveyor
integration I'm doing, is based on the pure PHP test suite - this ensures
the essentials for both build and functionality, as indeed the big part of
the functionality is not SAPI dependent. Also, with the CI mechanisms,
provided for free, there are various limitations like the build and test run
time, so it's is not always possible to pass all the desired test scenarios.
It of course depends on what you're intended to do - my thought is just,
that simplicity matters as much as complexity does, and usage in various
environment should be taken into account.
Regards
Anatol