Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:57441
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain sugarcrm.com designates 207.97.245.113 as permitted sender)
Message-ID: <4F18BBE4.7010106@sugarcrm.com>
Date: Thu, 19 Jan 2012 16:57:08 -0800
Organization: SugarCRM
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20111222 Thunderbird/9.0.1
MIME-Version: 1.0
To: Clint Byrum <clint@ubuntu.com>
CC: internals <internals@lists.php.net>
References: <4F189F5F.20109@sugarcrm.com> <4F18A8C5.9020301@phpgangsta.de> <4F18B07C.2010402@sugarcrm.com> <1327019609-sup-8204@fewbar.com>
In-Reply-To: <1327019609-sup-8204@fewbar.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] 5.4.0 rc6 and release
From: smalyshev@sugarcrm.com (Stas Malyshev)

Hi!

> These tests should be skipped or marked as XFAIL on platforms they are
> known to fail on. Better to have no test than one that cannot be relied
> upon. All supported platforms should pass with 0 fails. These intentional

Of course they should, and you (or anybody else) are welcome to make the 
patches that make them so :) I promise they'll be in 5.4.1.

> Its precisely this unreliability that forced me to take a conservative
> approach for Ubuntu 12.04 and recommend to the community that we ship
> 5.3.9 instead of 5.4.0. I would much rather have the new stuff in, but

5.4.0 is better, not worse, than 5.3.9 in this regard - especially 
because of the work that was done in 5.4 branch to fix or improve tests 
that were failing and improve test coverage. I would advise to give your 
users a choice once 5.4.0 is released and support both packages. On my 
Linux environment, I have now 0 fails (though I don't run all modules so 
some of non-default ones may be missing).

> even if all the tests pass on the machine we run the test suite on,
> how can we be sure they won't fail in another time zone, or in some
> other strange configuration?

Well, you never can be *sure* - tests can test only things they know 
about, unless somebody invents a way to make unit test test every 
possible combination of environments at once. So far I never heard about 
such tests.

> The fact that its all being running regularly is a fantastic improvement.
> I'd like to see a commitment to getting 100% pass/xfail/skip for every
> release/tested environment in future releases though.

I'd like that too. I hope more people will step up and help to make this 
a reality.
-- 
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227