New run-tests.php feature

13 years ago by Hannes Magnusson — view source

unread

Throw qa and internals@ into the loop to.

I'd also like to move the XFAIL section (printed out in the end) above
the FAIL section.
We have bucketloads of xfailed tests, I actually have to scroll up to
see the failed tests which is really annoying and silly.

Anyone have strong opinion on the printout order?

-Hannes

---------- Forwarded message ----------
From: Hannes Magnusson bjori@php.net
Date: Thu, Sep 1, 2011 at 13:32
Subject: Re: [PHP-CVS] svn: /php/php-src/
branches/PHP_5_3/run-tests.php branches/PHP_5_4/run-tests.php
trunk/run-tests.php
To: Pierre Joye pierre.php@gmail.com
Cc: php-cvs@lists.php.net

I did want to use -v at first, but that has a completely different
meaning already.
Its really only useful when looking at what is going on in a single
testcase to debug run-tests :]

-Hannes

ah right, I miss this part of the patch. I would suggest -v instead
and add a level of verbosity (less means no output but failed tests
for example).

It is, -g

so, you can either do $ SHOW_ONLY_GROUPS=FAIL,WARN make test
or via run-tets, $ TEST_PHP_EXECUTABLE=auto php run-tests.php -g XFAIL,SKIP

-Hannes

good idea!

please add it as a command line option as well, maybe using a verbose option

bjori Thu, 01 Sep 2011 11:00:51 +0000

Revision: http://svn.php.net/viewvc?view=revision&revision=315967

Log:
Seeing thousands of PASS tests flying by is meaningless.
Add an option to only print certain result groups.

Changed paths:
U php/php-src/branches/PHP_5_3/run-tests.php
U php/php-src/branches/PHP_5_4/run-tests.php
U php/php-src/trunk/run-tests.php

Modified: php/php-src/branches/PHP_5_3/run-tests.php

--- php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966)
+++ php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967)
@@ -207,6 +207,12 @@
$DETAILED = 0;
}

+if (getenv('SHOW_ONLY_GROUPS')) {

$SHOW_ONLY_GROUPS = explode(",", getenv('SHOW_ONLY_GROUPS'));
+} else {

$SHOW_ONLY_GROUPS = array();
+}

// Check whether user test dirs are requested.
if (getenv('TEST_PHP_USER')) {
$user_tests = explode (',', getenv('TEST_PHP_USER'));
@@ -560,6 +566,9 @@
case 'd':
$ini_overwrites[] = $argv[++$i];
break;

case 'g':

$SHOW_ONLY_GROUPS = explode(",", $argv[++$i]);;

break;
//case 'h'
case '--keep-all':
foreach($cfgfiles as $file) {
@@ -675,6 +684,9 @@
-d foo=bar Pass -d option to the php binary (Define INI entry foo
with value 'bar').

-g Comma seperated list of groups to show during test run

(e.x. FAIL,SKIP).

-m Test for memory leaks with Valgrind.

-p <php> Specify PHP executable to run.
@@ -2426,16 +2438,24 @@
function show_test($test_idx, $shortname)
{
global $test_cnt;

global $line_length;

echo "TEST $test_idx/$test_cnt [$shortname]\r";

$str = "TEST $test_idx/$test_cnt [$shortname]\r";

$line_length = strlen($str);

echo $str;
flush();
}

function show_result($result, $tested, $tested_file, $extra = '', $temp_filenames = null)
{

global $html_output, $html_file, $temp_target, $temp_urlbase;

global $html_output, $html_file, $temp_target, $temp_urlbase, $line_length, $SHOW_ONLY_GROUPS;

echo "$result $tested [$tested_file] $extra\n";

if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {

echo "$result $tested [$tested_file] $extra\n";

} else {

// Write over the last line to avoid random trailing chars on next echo

echo str_repeat(" ", $line_length), "\r";

}

if ($html_output) {

Modified: php/php-src/branches/PHP_5_4/run-tests.php

--- php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966)
+++ php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967)
@@ -207,6 +207,12 @@
$DETAILED = 0;
}

+if (getenv('SHOW_ONLY_GROUPS')) {

$SHOW_ONLY_GROUPS = explode(",", getenv('SHOW_ONLY_GROUPS'));
+} else {

$SHOW_ONLY_GROUPS = array();
+}

// Check whether user test dirs are requested.
if (getenv('TEST_PHP_USER')) {
$user_tests = explode (',', getenv('TEST_PHP_USER'));
@@ -560,6 +566,9 @@
case 'd':
$ini_overwrites[] = $argv[++$i];
=C2? break;

case 'g':

$SHOW_ONLY_GROUPS = explode(",", $argv[++$i]);;

break;
//case 'h'
case '--keep-all':
foreach($cfgfiles as $file) {
@@ -675,6 +684,9 @@
-d foo=bar Pass -d option to the php binary (Define INI entry foo
with value 'bar').

-g Comma seperated list of groups to show during test run

(e.x. FAIL,SKIP).

-m Test for memory leaks with Valgrind.

-p <php> Specify PHP executable to run.
@@ -2426,16 +2438,24 @@
function show_test($test_idx, $shortname)
{
global $test_cnt;

global $line_length;

echo "TEST $test_idx/$test_cnt [$shortname]\r";

$str = "TEST $test_idx/$test_cnt [$shortname]\r";

$line_length = strlen($str);

echo $str;
flush();
}

function show_result($result, $tested, $tested_file, $extra = '', $temp_filenames = null)
{

global $html_output, $html_file, $temp_target, $temp_urlbase;

global $html_output, $html_file, $temp_target, $temp_urlbase, $line_length, $SHOW_ONLY_GROUPS;

echo "$result $tested [$tested_file] $extra\n";

if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {

echo "$result $tested [$tested_file] $extra\n";

} else {

// Write over the last line to avoid random trailing chars on next echo

echo str_repeat(" ", $line_length), "\r";

}

if ($html_output) {

Modified: php/php-src/trunk/run-tests.php

--- php/php-src/trunk/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966)
+++ php/php-src/trunk/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967)
@@ -207,6 +207,12 @@
$DETAILED = 0;
}

+if (getenv('SHOW_ONLY_GROUPS')) {

$SHOW_ONLY_GROUPS = explode(",", getenv('SHOW_ONLY_GROUPS'));
+} else {

$SHOW_ONLY_GROUPS = array();
+}

// Check whether user test dirs are requested.
if (getenv('TEST_PHP_USER')) {
$user_tests = explode (',', getenv('TEST_PHP_USER'));
@@ -560,6 +566,9 @@
case 'd':
$ini_overwrites[] = $argv[++$i];
break;

case 'g':

$SHOW_ONLY_GROUPS = explode(",", $argv[++$i]);;

break;
//case 'h'
case '--keep-all':
foreach($cfgfiles as $file) {
@@ -675,6 +684,9 @@
-d foo=bar Pass -d option to the php binary (Define INI entry foo
with value 'bar').

-g Comma seperated list of groups to show during test run

(e.x. FAIL,SKIP).

-m Test for memory leaks with Valgrind.

-p <php> Specify PHP executable to run.
@@ -2426,16 +2438,24 @@
function show_test($test_idx, $shortname)
{
global $test_cnt;

global $line_length;

echo "TEST $test_idx/$test_cnt [$shortname]\r";

$str = "TEST $test_idx/$test_cnt [$shortname]\r";

$line_length = strlen($str);

echo $str;
flush();
}

function show_result($result, $tested, $tested_file, $extra = '', $temp_filenames = null)
{

global $html_output, $html_file, $temp_target, $temp_urlbase;

global $html_output, $html_file, $temp_target, $temp_urlbase, $line_length, $SHOW_ONLY_GROUPS;

echo "$result $tested [$tested_file] $extra\n";

if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {

echo "$result $tested [$tested_file] $extra\n";

} else {

// Write over the last line to avoid random trailing chars on next echo

echo str_repeat(" ", $line_length), "\r";

}

if ($html_output) {

--
PHP CVS Mailing List (http://www.php.net/)

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

--
PHP CVS Mailing List (http://www.php.net/)

13 years ago by johannes@schlueters.de — view source

unread

Hi,

Throw qa and internals@ into the loop to.

I'd also like to move the XFAIL section (printed out in the end) above
the FAIL section.
We have bucketloads of xfailed tests, I actually have to scroll up to
see the failed tests which is really annoying and silly.

Good thing to do!

johannes

Anyone have strong opinion on the printout order?

-Hannes

---------- Forwarded message ----------
From: Hannes Magnusson bjori@php.net
Date: Thu, Sep 1, 2011 at 13:32
Subject: Re: [PHP-CVS] svn: /php/php-src/
branches/PHP_5_3/run-tests.php branches/PHP_5_4/run-tests.php
trunk/run-tests.php
To: Pierre Joye pierre.php@gmail.com
Cc: php-cvs@lists.php.net

I did want to use -v at first, but that has a completely different
meaning already.
Its really only useful when looking at what is going on in a single
testcase to debug run-tests :]

-Hannes
ah right, I miss this part of the patch. I would suggest -v instead
and add a level of verbosity (less means no output but failed tests
for example).
It is, -g

so, you can either do $ SHOW_ONLY_GROUPS=FAIL,WARN make test
or via run-tets, $ TEST_PHP_EXECUTABLE=auto php run-tests.php -g XFAIL,SKIP

-Hannes
good idea!

please add it as a command line option as well, maybe using a verbose option
bjori Thu, 01 Sep 2011 11:00:51 +0000

Revision: http://svn.php.net/viewvc?view=revision&revision=315967

Log:
Seeing thousands of PASS tests flying by is meaningless.
Add an option to only print certain result groups.

Changed paths:
U php/php-src/branches/PHP_5_3/run-tests.php
U php/php-src/branches/PHP_5_4/run-tests.php
U php/php-src/trunk/run-tests.php

Modified: php/php-src/branches/PHP_5_3/run-tests.php

--- php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966)
+++ php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967)
@@ -207,6 +207,12 @@
$DETAILED = 0;
}

+if (getenv('SHOW_ONLY_GROUPS')) {
  $SHOW_ONLY_GROUPS = explode(",", getenv('SHOW_ONLY_GROUPS'));
+} else {
  $SHOW_ONLY_GROUPS = array();
+}

// Check whether user test dirs are requested.
if (getenv('TEST_PHP_USER')) {
$user_tests = explode (',', getenv('TEST_PHP_USER'));
@@ -560,6 +566,9 @@
case 'd':
$ini_overwrites[] = $argv[++$i];
break;
                          case 'g':
                                  $SHOW_ONLY_GROUPS = explode(",", $argv[++$i]);;
                                  break;
                         //case 'h'
                         case '--keep-all':
                                 foreach($cfgfiles as $file) {
@@ -675,6 +684,9 @@
-d foo=bar Pass -d option to the php binary (Define INI entry foo
with value 'bar').
-g Comma seperated list of groups to show during test run
           (e.x. FAIL,SKIP).
-m Test for memory leaks with Valgrind.

-p <php> Specify PHP executable to run.
@@ -2426,16 +2438,24 @@
function show_test($test_idx, $shortname)
{
global $test_cnt;
  global $line_length;
  echo "TEST $test_idx/$test_cnt [$shortname]\r";
  $str = "TEST $test_idx/$test_cnt [$shortname]\r";
  $line_length = strlen($str);
  echo $str;
 `flush()`;
}

function show_result($result, $tested, $tested_file, $extra = '', $temp_filenames = null)
{
  global $html_output, $html_file, $temp_target, $temp_urlbase;
  global $html_output, $html_file, $temp_target, $temp_urlbase, $line_length, $SHOW_ONLY_GROUPS;
  echo "$result $tested [$tested_file] $extra\n";
  if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {
          echo "$result $tested [$tested_file] $extra\n";
  } else {
          // Write over the last line to avoid random trailing chars on next echo
          echo str_repeat(" ", $line_length), "\r";
  }

 if ($html_output) {
Modified: php/php-src/branches/PHP_5_4/run-tests.php

--- php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966)
+++ php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967)
@@ -207,6 +207,12 @@
$DETAILED = 0;
}

+if (getenv('SHOW_ONLY_GROUPS')) {
  $SHOW_ONLY_GROUPS = explode(",", getenv('SHOW_ONLY_GROUPS'));
+} else {
  $SHOW_ONLY_GROUPS = array();
+}

// Check whether user test dirs are requested.
if (getenv('TEST_PHP_USER')) {
$user_tests = explode (',', getenv('TEST_PHP_USER'));
@@ -560,6 +566,9 @@
case 'd':
$ini_overwrites[] = $argv[++$i];
break;
                          case 'g':
                                  $SHOW_ONLY_GROUPS = explode(",", $argv[++$i]);;
                                  break;
                         //case 'h'
                         case '--keep-all':
                                 foreach($cfgfiles as $file) {
@@ -675,6 +684,9 @@
-d foo=bar Pass -d option to the php binary (Define INI entry foo
with value 'bar').
-g Comma seperated list of groups to show during test run
           (e.x. FAIL,SKIP).
-m Test for memory leaks with Valgrind.

-p <php> Specify PHP executable to run.
@@ -2426,16 +2438,24 @@
function show_test($test_idx, $shortname)
{
global $test_cnt;
  global $line_length;
  echo "TEST $test_idx/$test_cnt [$shortname]\r";
  $str = "TEST $test_idx/$test_cnt [$shortname]\r";
  $line_length = strlen($str);
  echo $str;
 `flush()`;
}

function show_result($result, $tested, $tested_file, $extra = '', $temp_filenames = null)
{
  global $html_output, $html_file, $temp_target, $temp_urlbase;
  global $html_output, $html_file, $temp_target, $temp_urlbase, $line_length, $SHOW_ONLY_GROUPS;
  echo "$result $tested [$tested_file] $extra\n";
  if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {
          echo "$result $tested [$tested_file] $extra\n";
  } else {
          // Write over the last line to avoid random trailing chars on next echo
          echo str_repeat(" ", $line_length), "\r";
  }

 if ($html_output) {
Modified: php/php-src/trunk/run-tests.php

--- php/php-src/trunk/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966)
+++ php/php-src/trunk/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967)
@@ -207,6 +207,12 @@
$DETAILED = 0;
}

+if (getenv('SHOW_ONLY_GROUPS')) {
  $SHOW_ONLY_GROUPS = explode(",", getenv('SHOW_ONLY_GROUPS'));
+} else {
  $SHOW_ONLY_GROUPS = array();
+}

// Check whether user test dirs are requested.
if (getenv('TEST_PHP_USER')) {
$user_tests = explode (',', getenv('TEST_PHP_USER'));
@@ -560,6 +566,9 @@
case 'd':
$ini_overwrites[] = $argv[++$i];
break;
                          case 'g':
                                  $SHOW_ONLY_GROUPS = explode(",", $argv[++$i]);;
                                  break;
                         //case 'h'
                         case '--keep-all':
                                 foreach($cfgfiles as $file) {
@@ -675,6 +684,9 @@
-d foo=bar Pass -d option to the php binary (Define INI entry foo
with value 'bar').
-g Comma seperated list of groups to show during test run
           (e.x. FAIL,SKIP).
-m Test for memory leaks with Valgrind.

-p <php> Specify PHP executable to run.
@@ -2426,16 +2438,24 @@
function show_test($test_idx, $shortname)
{
global $test_cnt;
  global $line_length;
  echo "TEST $test_idx/$test_cnt [$shortname]\r";
  $str = "TEST $test_idx/$test_cnt [$shortname]\r";
  $line_length = strlen($str);
  echo $str;
 `flush()`;
}

function show_result($result, $tested, $tested_file, $extra = '', $temp_filenames = null)
{
  global $html_output, $html_file, $temp_target, $temp_urlbase;
  global $html_output, $html_file, $temp_target, $temp_urlbase, $line_length, $SHOW_ONLY_GROUPS;
  echo "$result $tested [$tested_file] $extra\n";
  if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {
          echo "$result $tested [$tested_file] $extra\n";
  } else {
          // Write over the last line to avoid random trailing chars on next echo
          echo str_repeat(" ", $line_length), "\r";
  }

 if ($html_output) {
--
PHP CVS Mailing List (http://www.php.net/)
--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org
--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

--
PHP CVS Mailing List (http://www.php.net/)

13 years ago by Ferenc Kovacs — view source

unread

Throw qa and internals@ into the loop to.

I'd also like to move the XFAIL section (printed out in the end) above
the FAIL section.
We have bucketloads of xfailed tests, I actually have to scroll up to
see the failed tests which is really annoying and silly.

Anyone have strong opinion on the printout order?

yeah, strongly agree

--
Ferenc Kovács
@Tyr43l - http://tyrael.hu

13 years ago by Alexey Shein — view source

unread

2011/9/1 Ferenc Kovacs tyra3l@gmail.com:

Throw qa and internals@ into the loop to.

I'd also like to move the XFAIL section (printed out in the end) above
the FAIL section.
We have bucketloads of xfailed tests, I actually have to scroll up to
see the failed tests which is really annoying and silly.

Anyone have strong opinion on the printout order?

yeah, strongly agree

Great addition.
While you're at it, could you also add an option to print a list of
skipped tests (maybe above XFAIL section), that's also currently can't
be done with other options in run-tests.php.
It could also be useful because some skipped tests maybe skipped for
the wrong reason and be actually failed. It's also a great spot to
notice obsolete tests that need to be removed from the suite and make
running all process faster.

--
Regards,
Shein Alexey

13 years ago by Hannes Magnusson — view source

unread

2011/9/1 Ferenc Kovacs tyra3l@gmail.com:

Throw qa and internals@ into the loop to.

I'd also like to move the XFAIL section (printed out in the end) above
the FAIL section.
We have bucketloads of xfailed tests, I actually have to scroll up to
see the failed tests which is really annoying and silly.

Anyone have strong opinion on the printout order?

yeah, strongly agree

Great addition.
While you're at it, could you also add an option to print a list of
skipped tests (maybe above XFAIL section), that's also currently can't
be done with other options in run-tests.php.
It could also be useful because some skipped tests maybe skipped for
the wrong reason and be actually failed. It's also a great spot to
notice obsolete tests that need to be removed from the suite and make
running all process faster.

Depending on how you run the test suite, you could get buuuuuuuunch of
skipped tests..
Unsure if its that good idea to include them in the report.

To see the skipped tests while running the test suite, and nothing
else, you can use the new switch/environment variable as mentioned
earlier in this thread..

-Hannes

13 years ago by johannes@schlueters.de — view source

unread

Great addition.
While you're at it, could you also add an option to print a list of
skipped tests (maybe above XFAIL section), that's also currently can't
be done with other options in run-tests.php.
It could also be useful because some skipped tests maybe skipped for
the wrong reason and be actually failed. It's also a great spot to
notice obsolete tests that need to be removed from the suite and make
running all process faster.

Let's open a can of worms: When designing this I'd like to keep in mind
a feature I had on the list for the rewrite of run-tests by Zoe (anybody
knows the status of that?) which is skipping entire directories with a
single skipif.
Benefit there is that for instance we don't have to create two PHP
process for each and every test and don't have to try connecting to
databases two times for each test (during skipif and the actual test).

johannes

--
Regards,
Shein Alexey

13 years ago by Hannes Magnusson — view source

unread

2011/9/1 Johannes Schlüter johannes@schlueters.de:

Great addition.
While you're at it, could you also add an option to print a list of
skipped tests (maybe above XFAIL section), that's also currently can't
be done with other options in run-tests.php.
It could also be useful because some skipped tests maybe skipped for
the wrong reason and be actually failed. It's also a great spot to
notice obsolete tests that need to be removed from the suite and make
running all process faster.

Let's open a can of worms: When designing this I'd like to keep in mind
a feature I had on the list for the rewrite of run-tests by Zoe (anybody
knows the status of that?) which is skipping entire directories with a
single skipif.
Benefit there is that for instance we don't have to create two PHP
process for each and every test and don't have to try connecting to
databases two times for each test (during skipif and the actual test).

Create a magical "run-me-first" file which run-tests will execute
first, and if it fails it will skip all the tests in that directory..
should be relatively easy to implement.

Being able to run the tests for two directories in parallel however..
that would be awesome.

-Hannes

13 years ago by Ferenc Kovacs — view source

unread

2011/9/1 Hannes Magnusson bjori@php.net:

2011/9/1 Johannes Schlüter johannes@schlueters.de:

Great addition.
While you're at it, could you also add an option to print a list of
skipped tests (maybe above XFAIL section), that's also currently can't
be done with other options in run-tests.php.
It could also be useful because some skipped tests maybe skipped for
the wrong reason and be actually failed. It's also a great spot to
notice obsolete tests that need to be removed from the suite and make
running all process faster.

Let's open a can of worms: When designing this I'd like to keep in mind
a feature I had on the list for the rewrite of run-tests by Zoe (anybody
knows the status of that?) which is skipping entire directories with a
single skipif.
Benefit there is that for instance we don't have to create two PHP
process for each and every test and don't have to try connecting to
databases two times for each test (during skipif and the actual test).

Create a magical "run-me-first" file which run-tests will execute
first, and if it fails it will skip all the tests in that directory..
should be relatively easy to implement.

Being able to run the tests for two directories in parallel however..
that would be awesome.

-Hannes

Hi guys, I think it would be useful to add this to the issue tracker,
and if nobody else, I will look into this.

--
Ferenc Kovács
@Tyr43l - http://tyrael.hu

13 years ago by johannes@schlueters.de — view source

unread

Create a magical "run-me-first" file which run-tests will execute
first, and if it fails it will skip all the tests in that directory..
should be relatively easy to implement.

See patch at
http://schlueters.de/~johannes/php/run-tests-directory-skip.diff ;-)

This will look for a file called SKIP, which has to be a PHP file, in
each directory while looking for tests. There's at least one bug left:
INI options aren't passed to the SKIP run so shared modules etc. will
not be found. If somebody could extend it accordingly: great. :-)

Being able to run the tests for two directories in parallel however..
that would be awesome.

Ack. While that's a bit more complex as we can't run tests for a native
database and the corresponding PDO driver at the same time, most likely.
So we need some locking mechanism.

johannes

13 years ago by Kalle Sommer Nielsen — view source

unread

2011/9/1 Hannes Magnusson bjori@php.net:

Create a magical "run-me-first" file which run-tests will execute
first, and if it fails it will skip all the tests in that directory..
should be relatively easy to implement.

Being able to run the tests for two directories in parallel however..
that would be awesome.

I reckon Felipe had a patch for this back in the early 5.3 development
days, Felipe?

--
regards,

Kalle Sommer Nielsen
kalle@php.net

13 years ago by Felipe Pena — view source

unread

2011/9/1 Kalle Sommer Nielsen kalle@php.net:

Hi

2011/9/1 Hannes Magnusson bjori@php.net:

Create a magical "run-me-first" file which run-tests will execute
first, and if it fails it will skip all the tests in that directory..
should be relatively easy to implement.

Being able to run the tests for two directories in parallel however..
that would be awesome.

I reckon Felipe had a patch for this back in the early 5.3 development
days, Felipe?

--
regards,

Kalle Sommer Nielsen
kalle@php.net

Here's the patch: http://felipe.ath.cx/diff/run-tests.diff

--
Regards,
Felipe Pena

13 years ago by Hannes Magnusson — view source

unread

2011/9/1 Kalle Sommer Nielsen kalle@php.net:

Hi

2011/9/1 Hannes Magnusson bjori@php.net:

Create a magical "run-me-first" file which run-tests will execute
first, and if it fails it will skip all the tests in that directory..
should be relatively easy to implement.

Being able to run the tests for two directories in parallel however..
that would be awesome.

I reckon Felipe had a patch for this back in the early 5.3 development
days, Felipe?

Here's the patch: http://felipe.ath.cx/diff/run-tests.diff

Hmh. This patch seems to be similar to johannes' patch, being able to
skip entire directories based on a "configure file".
I don't see a way to execute a check though.
f.e. running the mysql tests won't do you any good unless you have the
environment properly setup, so the skipif section needs to be able to
check if it can connect to a db, and if not then you can skip the
entire folder..

-Hannes