Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:61845
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: unknown (pb1.pair.com: domain php.net does not designate 85.214.73.107 as permitted sender)
Message-ID: <97767b567717f761f673c9561efda831.squirrel@webmail.klapt.com>
Date: Fri, 27 Jul 2012 22:07:29 +0200
To: internals@lists.php.net
Reply-To: ab@php.net
User-Agent: SquirrelMail/1.4.21
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Importance: Normal
Subject: PHP test suite
From: ab@php.net ("Anatoliy Belsky")

Hi internals,

after working for some time with the PHP test suite I've realized - it
needs an improvement of a special kind. What I'm talking about is the
server based testing. Very urgent cases have shown them up working with
APC. Most of the current bugs reported only reproduceable with tests where
multiple requests are involved. Where tests work fine on the first
request, all the subsequent requests produce memory leaks or other errors.
That's where PHP 5.4 with its built-in cli server comes into the game. As
how it works now - a CLI executable is run only once and due to its nature
only one pure PHP lifecircle is tested - MINIT-RINIT-RSHUTDOWN-MSHUTDOWN.

For now what I did - based my APC tests on sapi/cli/tests/php_cli_server*.
This has some limitations however. As such, this test don't care about any
valgrind params usually could be passed with make test TESTS=-m. Well,
this can be solved with a custom env var. Another thing - this tests
doesn't produce any coverage information, i suspect because the only way
it can be run now is using exec (talking about linux) shell command only.
You can convince yourself here
http://gcov.php.net/PHP_5_4/lcov_html/var/php_gcov/PHP_5_4/sapi/cli/index.php
despite built-in cli server has 17 tests, no code coverage is shown.
Several tries to fix this (including fork) didn't give me a solution to
produce coverage for APC either. However, a test runnt manually adds to
the coverage data.

Please look at this examples illustrating how the built-in CLI server is
used now with APC
http://svn.php.net/viewvc/pecl/apc/trunk/tests/server_test.inc?view=markup
http://svn.php.net/viewvc/pecl/apc/trunk/tests/apc54_002.phpt?view=markup

What I see now many particular tests do - relying on some external hosts
hoping them being online forever and/or a mashine running test havig
internet available. Nevertheless many tests make more sense having an
opposing party available (say a database extension for instance), another
plenty would require just a mock. Therefore the idea I have to bring here
- we need an improvement on the standard test suites since PHP 5.4
allowing to use the built-in CLI server native inside a phpt (and may be a
TCP server?). This should care about valgrind, about coverage, windows
compatibility as well as about beyond things available in the test suite.
Most likely it should be run from the make file, so on an upper lever to
run-tests.php. Having this would give more qualitative tests and at the
end of the day - more quality for PHP. It could be looking like --SERVER--
block in phpt or what ever.

That's it, an idea I wantend just throw into the air here :)

Regards

Anatoliy