# PHP test suite

- Thread: https://externals.io/message/61845
- Original archive: https://news-web.php.net/php.internals/61845
- Started by Anatoliy Belsky on 2012-07-27T20:07:29+00:00
- Messages: 6
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #61845 — Anatoliy Belsky — 2012-07-27T20:07:29+00:00

URL: https://externals.io/message/61845#61845

Hi internals,

after working for some time with the PHP test suite I've realized - it
needs an improvement of a special kind. What I'm talking about is the
server based testing. Very urgent cases have shown them up working with
APC. Most of the current bugs reported only reproduceable with tests where
multiple requests are involved. Where tests work fine on the first
request, all the subsequent requests produce memory leaks or other errors.
That's where PHP 5.4 with its built-in cli server comes into the game. As
how it works now - a CLI executable is run only once and due to its nature
only one pure PHP lifecircle is tested - MINIT-RINIT-RSHUTDOWN-MSHUTDOWN.

For now what I did - based my APC tests on sapi/cli/tests/php_cli_server*.
This has some limitations however. As such, this test don't care about any
valgrind params usually could be passed with make test TESTS=-m. Well,
this can be solved with a custom env var. Another thing - this tests
doesn't produce any coverage information, i suspect because the only way
it can be run now is using exec (talking about linux) shell command only.
You can convince yourself here
http://gcov.php.net/PHP_5_4/lcov_html/var/php_gcov/PHP_5_4/sapi/cli/index.php
despite built-in cli server has 17 tests, no code coverage is shown.
Several tries to fix this (including fork) didn't give me a solution to
produce coverage for APC either. However, a test runnt manually adds to
the coverage data.

Please look at this examples illustrating how the built-in CLI server is
used now with APC
http://svn.php.net/viewvc/pecl/apc/trunk/tests/server_test.inc?view=markup
http://svn.php.net/viewvc/pecl/apc/trunk/tests/apc54_002.phpt?view=markup

What I see now many particular tests do - relying on some external hosts
hoping them being online forever and/or a mashine running test havig
internet available. Nevertheless many tests make more sense having an
opposing party available (say a database extension for instance), another
plenty would require just a mock. Therefore the idea I have to bring here
- we need an improvement on the standard test suites since PHP 5.4
allowing to use the built-in CLI server native inside a phpt (and may be a
TCP server?). This should care about valgrind, about coverage, windows
compatibility as well as about beyond things available in the test suite.
Most likely it should be run from the make file, so on an upper lever to
run-tests.php. Having this would give more qualitative tests and at the
end of the day - more quality for PHP. It could be looking like --SERVER--
block in phpt or what ever.

That's it, an idea I wantend just throw into the air here :)

Regards

Anatoliy

---

## #61857 — johannes@schlueters.de — 2012-07-28T14:09:16+00:00

URL: https://externals.io/message/61845#61857 · In reply to: #61845

> 
> after working for some time with the PHP test suite I've realized - it
> needs an improvement of a special kind. What I'm talking about is the
> server based testing. Very urgent cases have shown them up working
> with APC. Most of the current bugs reported only reproduceable with
> tests where multiple requests are involved. 

For that I once created a special SAPI, which can also do
multi-threading: 
  https://github.com/johannes/pconn-sapi
  http://schlueters.de/blog/archives/166-Testing-persistent-connection-and-thread-safety-features-in-PHP.html

johannes

---

## #61859 — Rasmus Lerdorf — 2012-07-28T17:08:17+00:00

URL: https://externals.io/message/61845#61859 · In reply to: #61857

> 
> > […]
> 
> For that I once created a special SAPI, which can also do
> multi-threading: 
>   https://github.com/johannes/pconn-sapi
>   http://schlueters.de/blog/archives/166-Testing-persistent-connection-and-thread-safety-features-in-PHP.html

While that is cool, one of the reasons we added the built-in web server
was to use it in exactly this sort of scenario. So Anatoliy, go for it.
Make whatever changes you see fit to make it easy to do a --SERVER--
type mechanism in a phpt file. Or perhaps there are some volunteers here
who have ideas and you can team up and either flash it out in a wiki doc
or on irc.

-Rasmus

---

## #61860 — Ferenc Kovacs — 2012-07-28T17:32:24+00:00

URL: https://externals.io/message/61845#61860 · In reply to: #61859

> 
> > […]
> http://schlueters.de/blog/archives/166-Testing-persistent-connection-and-thread-safety-features-in-PHP.html
>
> While that is cool, one of the reasons we added the built-in web server
> was to use it in exactly this sort of scenario. So Anatoliy, go for it.
> Make whatever changes you see fit to make it easy to do a --SERVER--
> type mechanism in a phpt file. Or perhaps there are some volunteers here
> who have ideas and you can team up and either flash it out in a wiki doc
> or on irc.
>
> -Rasmus
>
> --
> 
> 
>
>
ccing the qa list, albeit I guess most people who follow that also follow
internals also.

-- 
Ferenc Kovács
@Tyr43l - http://tyrael.hu

---

## #61861 — johannes@schlueters.de — 2012-07-28T18:24:50+00:00

URL: https://externals.io/message/61845#61861 · In reply to: #61859

> > […]
> 
> While that is cool, one of the reasons we added the built-in web server
> was to use it in exactly this sort of scenario. So Anatoliy, go for it.
> Make whatever changes you see fit to make it easy to do a --SERVER--
> type mechanism in a phpt file. Or perhaps there are some volunteers here
> who have ideas and you can team up and either flash it out in a wiki doc
> or on irc.

I agree the server is a good way for that, I mostly shared my thing so
people maybe can take ideas from there, not to stop people doing
undoubtedly useful things.

johannes

---

## #61862 — Rasmus Lerdorf — 2012-07-28T18:33:39+00:00

URL: https://externals.io/message/61845#61862 · In reply to: #61861

> 
> > […]
> 
> I agree the server is a good way for that, I mostly shared my thing so
> people maybe can take ideas from there, not to stop people doing
> undoubtedly useful things.

Yup, sorry I did realize that. But I wanted to make sure Anatoliy didn't
take your reply as a "do it like this instead" response.

-Rasmus