Hi,
looking for tasks to invest a little spare time in the weeks to come I had
a look at
http://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from the
previous version are reused, and fixed if needed.
Looking at the 5.5 source tree all the phpt files seem in place... Aren't
the tests just currently not executed on gcov.php.net?
Thanks for any insight!
Greetings
Nico
On Wed, Jun 4, 2014 at 10:06 PM, Nicolai Scheer nicolai.scheer@gmail.com
wrote:
Hi,
looking for tasks to invest a little spare time in the weeks to come I had
a look athttp://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from the
previous version are reused, and fixed if needed.Looking at the 5.5 source tree all the phpt files seem in place... Aren't
the tests just currently not executed on gcov.php.net?Thanks for any insight!
Greetings
Nico
Yeah, something is definitely wrong:
http://gcov.php.net/PHP_5_4/lcov_html/
vs
http://gcov.php.net/PHP_5_5/lcov_html/
and it we dates are correct, the last gcov build for 5.6 was more than a
week ago:
http://gcov.php.net/
http://gcov.php.net/PHP_5_6/lcov_html/
ps: I've cc'ed Nuno as he is the man who knows our gcov setup.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
On Wed, Jun 4, 2014 at 10:06 PM, Nicolai Scheer nicolai.scheer@gmail.com
wrote:Hi,
looking for tasks to invest a little spare time in the weeks to come I had
a look athttp://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from the
previous version are reused, and fixed if needed.Looking at the 5.5 source tree all the phpt files seem in place... Aren't
the tests just currently not executed on gcov.php.net?Thanks for any insight!
Greetings
Nico
Yeah, something is definitely wrong:
http://gcov.php.net/PHP_5_4/lcov_html/
vs
http://gcov.php.net/PHP_5_5/lcov_html/and it we dates are correct, the last gcov build for 5.6 was more than a
week ago:
http://gcov.php.net/
http://gcov.php.net/PHP_5_6/lcov_html/ps: I've cc'ed Nuno as he is the man who knows our gcov setup.
I will ask by the way. Aren't any parts duplicate of what we've got with
travis/QA reports now? I mean statistics about passed/failed tests etc.
It is probably good to have all such data in one place, but not when
tools are not working properly. It also means two systems to maintain
and in my opinion leads to confusion as the results are not equal.
Regards,
Maciej.
On Wed, Jun 4, 2014 at 10:06 PM, Nicolai Scheer nicolai.scheer@gmail.com
wrote:
Hi,
looking for tasks to invest a little spare time in the weeks to come I
had
a look athttp://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from the
previous version are reused, and fixed if needed.Looking at the 5.5 source tree all the phpt files seem in place... Aren't
the tests just currently not executed on gcov.php.net?Thanks for any insight!
Greetings
Nico
Yeah, something is definitely wrong:
http://gcov.php.net/PHP_5_4/lcov_html/
vs
http://gcov.php.net/PHP_5_5/lcov_html/and it we dates are correct, the last gcov build for 5.6 was more than a
week ago:
http://gcov.php.net/
http://gcov.php.net/PHP_5_6/lcov_html/ps: I've cc'ed Nuno as he is the man who knows our gcov setup.
I will ask by the way. Aren't any parts duplicate of what we've got with
travis/QA reports now? I mean statistics about passed/failed tests etc. It
is probably good to have all such data in one place, but not when tools are
not working properly. It also means two systems to maintain and in my
opinion leads to confusion as the results are not equal.Regards,
Maciej.--
my idea with jenkins was to have everything in one place, so build and test
php-src on multiple platforms(including windows) and also test userland
projects with the resulting binaries and also have coverity reports (not
for every commit, but periodically) but that was low prio, as gcov was
already there:
http://grokbase.com/p/php/php-internals/11btzq29x8/php-dev-ci-for-5-4
but stuff happen, and I never managed to finish what I was planning.
having travis is a bit redundant, as it does less than what jenkins does
(or can), and only supports linux with one distribution and only 64bit, but
it doesn't really require any maintenance from our part and "just works"
and also supports testing pull requests, which would require a bit more
work on our part if we want to do that on our jenkins environment (not the
PR building, but doing that in an isolated and secure way).
btw, I'm currently working on two pull request for our travis setup: one is
for changing our current travis project from php to C, which saves some
build time, as the travis box won't need to build php 5.4 which we doesn't
use at all, and it will also mean that we will install our dependencies for
ourselves explicitly instead of using those installed by default for the
travis php 5.4.
the other pull request is to do cross-compile 32bit builds on travis
through ia32-libs.
that would help us at least cover one platform/distro.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
On Wed, Jun 4, 2014 at 10:06 PM, Nicolai Scheer nicolai.scheer@gmail.com
wrote:
Hi,
looking for tasks to invest a little spare time in the weeks to come I
had
a look athttp://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from the
previous version are reused, and fixed if needed.Looking at the 5.5 source tree all the phpt files seem in place... Aren't
the tests just currently not executed on gcov.php.net?Thanks for any insight!
Greetings
Nico
Yeah, something is definitely wrong:
http://gcov.php.net/PHP_5_4/lcov_html/
vs
http://gcov.php.net/PHP_5_5/lcov_html/and it we dates are correct, the last gcov build for 5.6 was more than a
week ago:
http://gcov.php.net/
http://gcov.php.net/PHP_5_6/lcov_html/ps: I've cc'ed Nuno as he is the man who knows our gcov setup.
I will ask by the way. Aren't any parts duplicate of what we've got with
travis/QA reports now? I mean statistics about passed/failed tests etc. It
is probably good to have all such data in one place, but not when tools are
not working properly. It also means two systems to maintain and in my
opinion leads to confusion as the results are not equal.Regards,
Maciej.--
my idea with jenkins was to have everything in one place, so build and test
php-src on multiple platforms(including windows) and also test userland
projects with the resulting binaries and also have coverity reports (not
for every commit, but periodically) but that was low prio, as gcov was
already there:
http://grokbase.com/p/php/php-internals/11btzq29x8/php-dev-ci-for-5-4
but stuff happen, and I never managed to finish what I was planning.
having travis is a bit redundant, as it does less than what jenkins does
(or can), and only supports linux with one distribution and only 64bit, but
it doesn't really require any maintenance from our part and "just works"
and also supports testing pull requests, which would require a bit more
work on our part if we want to do that on our jenkins environment (not the
PR building, but doing that in an isolated and secure way).
Nice idea indeed. It would be good to have such tools in one place.
However, as far as I understand, it is abandoned now.
btw, I'm currently working on two pull request for our travis setup: one is
for changing our current travis project from php to C, which saves some
build time, as the travis box won't need to build php 5.4 which we doesn't
use at all, and it will also mean that we will install our dependencies for
ourselves explicitly instead of using those installed by default for the
travis php 5.4.
the other pull request is to do cross-compile 32bit builds on travis
through ia32-libs.
that would help us at least cover one platform/distro.
If we're currently focused on using travis ci maybe it would be good to
at least disable some duplicated functionalities from gcov.php.net? I'm
not certainly sure which have better and more up-to-date replacements
now but I thought about:
- compile results
- test failures
- expected test failures
- valgrind
- skipped tests - not sure it has replacement, but it seems that gcov
also supports only one distro in 64x architecture, so it's not a big help
What do you think?
Greetings,
Maciej.
On Wed, Jun 4, 2014 at 10:06 PM, Nicolai Scheer <
wrote:
Hi,
looking for tasks to invest a little spare time in the weeks to come I
had
a look athttp://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from
the
previous version are reused, and fixed if needed.Looking at the 5.5 source tree all the phpt files seem in place...
Aren't
the tests just currently not executed on gcov.php.net?Thanks for any insight!
Greetings
Nico
Yeah, something is definitely wrong:
http://gcov.php.net/PHP_5_4/lcov_html/
vs
http://gcov.php.net/PHP_5_5/lcov_html/and it we dates are correct, the last gcov build for 5.6 was more than a
week ago:
http://gcov.php.net/
http://gcov.php.net/PHP_5_6/lcov_html/ps: I've cc'ed Nuno as he is the man who knows our gcov setup.
I will ask by the way. Aren't any parts duplicate of what we've got
with
travis/QA reports now? I mean statistics about passed/failed tests etc.
It
is probably good to have all such data in one place, but not when tools
are
not working properly. It also means two systems to maintain and in my
opinion leads to confusion as the results are not equal.Regards,
Maciej.--
my idea with jenkins was to have everything in one place, so build and
test
php-src on multiple platforms(including windows) and also test userland
projects with the resulting binaries and also have coverity reports (not
for every commit, but periodically) but that was low prio, as gcov was
already there:
http://grokbase.com/p/php/php-internals/11btzq29x8/php-dev-ci-for-5-4
but stuff happen, and I never managed to finish what I was planning.
having travis is a bit redundant, as it does less than what jenkins does
(or can), and only supports linux with one distribution and only 64bit,
but
it doesn't really require any maintenance from our part and "just works"
and also supports testing pull requests, which would require a bit more
work on our part if we want to do that on our jenkins environment (not the
PR building, but doing that in an isolated and secure way).Nice idea indeed. It would be good to have such tools in one place.
However, as far as I understand, it is abandoned now.btw, I'm currently working on two pull request for our travis setup: one
is
for changing our current travis project from php to C, which saves some
build time, as the travis box won't need to build php 5.4 which we doesn't
use at all, and it will also mean that we will install our dependencies
for
ourselves explicitly instead of using those installed by default for the
travis php 5.4.
the other pull request is to do cross-compile 32bit builds on travis
through ia32-libs.
that would help us at least cover one platform/distro.If we're currently focused on using travis ci maybe it would be good to at
least disable some duplicated functionalities from gcov.php.net? I'm not
certainly sure which have better and more up-to-date replacements now but I
thought about:
- compile results
- test failures
- expected test failures
- valgrind
- skipped tests - not sure it has replacement, but it seems that gcov also
supports only one distro in 64x architecture, so it's not a big helpWhat do you think?
Greetings,
Maciej.
I think it makes sense to see the number of executed/skipped/failed/etc.
tests with the coverage report as they are directly related (if you add
more tests that usually increase the coverage, if you remove some, the
coverage will also drop).
Plus we didn't really had any complaints about having that (most people
doesn't even know about gcov).
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
I think it makes sense to see the number of executed/skipped/failed/etc.
tests with the coverage report as they are directly related (if you add
more tests that usually increase the coverage, if you remove some, the
coverage will also drop).
Plus we didn't really had any complaints about having that (most people
doesn't even know about gcov).
Hmm, you're right. So ignore this discussion, please.
Thanks,
Maciej.
Hi,
looking for tasks to invest a little spare time in the weeks to come I
had
a look athttp://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from the
previous version are reused, and fixed if needed.Looking at the 5.5 source tree all the phpt files seem in place... Aren't
the tests just currently not executed on gcov.php.net?Thanks for any insight!
Greetings
Nico
Yeah, something is definitely wrong:
http://gcov.php.net/PHP_5_4/lcov_html/
vs
http://gcov.php.net/PHP_5_5/lcov_html/and it we dates are correct, the last gcov build for 5.6 was more than a
week ago:
http://gcov.php.net/
http://gcov.php.net/PHP_5_6/lcov_html/ps: I've cc'ed Nuno as he is the man who knows our gcov setup.
Thanks for CC'ing me.
Short version: the problem is fixed. The next round of builds will have
correct coverage data (including PHP_HEAD that will be available within a
few hours).
Long version: There was a bug in the makefile that was triggered when the
Zend opcode cache was commited. A few days ago I tried to fix it, but ended
up introducing another bug. Anyway, I believe my last commit fixed the
problem in Makefile.gcov, and therefore the next round of builds should
finally have correct data.
Nuno
P.S.: BTW, regarding the existence of gcov.php.net, when it was created
there were no off-the-shelf solutions, and therefore we created our own.
It's not awsome, but works fine. Of course I'm fine with killing it when a
replacement becomes available.
Hi!
Hi,
looking for tasks to invest a little spare time in the weeks to come I
had
a look athttp://gcov.php.net/viewer.php?version=PHP_5_5
The test coverage is suspiciously low...
Test coverage of 5.4 is quite high, so is coverage for 5.3.
I would have guessed, that for a new major php version all tests from the
previous version are reused, and fixed if needed.Looking at the 5.5 source tree all the phpt files seem in place... Aren't
the tests just currently not executed on gcov.php.net?Thanks for any insight!
Greetings
Nico
Yeah, something is definitely wrong:
http://gcov.php.net/PHP_5_4/lcov_html/
vs
http://gcov.php.net/PHP_5_5/lcov_html/and it we dates are correct, the last gcov build for 5.6 was more than a
week ago:
http://gcov.php.net/
http://gcov.php.net/PHP_5_6/lcov_html/ps: I've cc'ed Nuno as he is the man who knows our gcov setup.
Thanks for CC'ing me.
Short version: the problem is fixed. The next round of builds will have
correct coverage data (including PHP_HEAD that will be available within a
few hours).Long version: There was a bug in the makefile that was triggered when the
Zend opcode cache was commited. A few days ago I tried to fix it, but
ended up introducing another bug. Anyway, I believe my last commit fixed
the problem in Makefile.gcov, and therefore the next round of builds should
finally have correct data.Nuno
P.S.: BTW, regarding the existence of gcov.php.net, when it was created
there were no off-the-shelf solutions, and therefore we created our own.
It's not awsome, but works fine. Of course I'm fine with killing it when a
replacement becomes available.
Thanks for looking into this issue and resolving it :)
Looking forward to checking the results.
Greetings
Nico
P.S.: BTW, regarding the existence of gcov.php.net, when it was created
there were no off-the-shelf solutions, and therefore we created our own.
It's not awsome, but works fine. Of course I'm fine with killing it when
a replacement becomes available.
To be clear. I don't think that gcov.php.net is useless/bad. I'm
impressed how much work was done to create own system like this one. I
was just wondering if we don't duplicate some functionalities.
Also, good to know, that bug is fixed, thanks.
Regards,
Maciej.