Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:99103
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain schlueters.de from 84.19.169.162 cause and error)
Message-ID: <1495103485.9756.58.camel@schlueters.de>
To: Marco Pivetta <ocramius@gmail.com>
Cc: PHP Internals List <internals@lists.php.net>, Sammy Kaye Powers
	 <me@sammyk.me>
Date: Thu, 18 May 2017 12:31:25 +0200
In-Reply-To: <CADyq6sKVSgry1AwsqL2EBr96ot35Hoy37pB4ri4wEZeaAWEwiQ@mail.gmail.com>
References: 
	<CALYT1Yrs-9Xr0tyGGWdVBsm0r6h+w+GZcLT0UbVCSMrBYZ01bQ@mail.gmail.com>
	 <b62db45d-6b77-f5ec-ac0f-096687d3efe7@fleshgrinder.com>
	 <CADyq6s+=DCk1LiBc3wjUBRdf-mnkgQEqRmGoUeLc+XEjqjk6Qg@mail.gmail.com>
	 <1495067758.9756.47.camel@schlueters.de>
	 <CADyq6sKVSgry1AwsqL2EBr96ot35Hoy37pB4ri4wEZeaAWEwiQ@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 8bit
Subject: Re: [PHP-DEV] Implement formal process for run-tests.php refactor
From: johannes@schlueters.de (Johannes =?ISO-8859-1?Q?Schl=FCter?=)

On Do, 2017-05-18 at 12:08 +0200, Marco Pivetta wrote:
> On Thu, May 18, 2017 at 2:35 AM, Johannes Schlüter <johannes@schluete
> rs.de> wrote:
> > On Mi, 2017-05-17 at 23:30 +0200, Marco Pivetta wrote:
> > > Is Sebastian copied in here? Why can't we just use the super-
> > battle-
> > > tested
> > > PHPUnit? It supports phpt and a ton of plugins, plus everyone
> > uses it
> > > and
> > > is familiar with it.
> > 
> > PHPUnit is huge. run-tests is a small script in a single file which
> > I
> > can quickly edit. For PHPUnit I have multiple files and need
> > tooling to
> > phar them up.
> You don't need to phar them up - just run a typical composer
> installation, like everyone else, and like every PHP package running
> CI on travis-ci/circle-ci/continuous-php ever built in the last 5
> years.
> At this point, composer and phpunit are such big players that
> breaking them or breaking PHP makes no difference at all for the
> user-base.

I love composer. But we're not talking application development here,
but runtime development, where I'm sometimes testing PHP on some
embedded or otherwise strange platform. Keeping things simple has a
value there.

> > PHPUnit (according to it's website) also has more dependencies (DOM
> > and
> > JSON) which are not included in --disable-all. used
> Ah yes, that is indeed a problem. I think these deps are used in some
> assertions only, so it may be possible to drop them if the assertions
> are segregated somewhere else.
> 
> > I'm not sure if PHPUnit meanwhile supports redirect tests as in
> > PDO.
> Unsure - got a link to those?

ext/pdo_*/tests/common.phpt

> > Is there a performance comparison? - make test runs for a loooong
> > time.
> > A difference might have an impact. (both ways round ;) )
> PHPUnit has plugins to parallelize and aggregate results too - the
> point of using a battle-tested and commonly used solution is
> precisely the benefits you get from community support.

phpUnit might be battle tested with parallelization, the PHP test suite
isn't. There are tons of tests using the same resource (i.e. different
mysqli and pdo_mysql tests might use the same tables some other tests
use same port for some socket stuff etc.) Would be great if we get
parallelization, though.

Oh, and while thinking about performance: run-tests.php integrates with
tools like valgrind ... I guess phpUnit doesn't do that either, yet.

johannes