Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:59212
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain sugarcrm.com designates 67.192.241.143 as permitted sender)
Message-ID: <4F75048E.7050102@sugarcrm.com>
Date: Thu, 29 Mar 2012 17:55:42 -0700
Organization: SugarCRM
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
MIME-Version: 1.0
To: Alexey Shein <confik@gmail.com>
CC: PHP Internals <internals@lists.php.net>
References: <CAAAa2sh3AbFcWa42aGgBh4DnzOnShKjY=P=2y2k6VCK6ZzqVhg@mail.gmail.com> <4F74DFF6.7010507@sugarcrm.com> <CAAAa2sijkWvoD8+wk-r066Du09vWkWAAO0NX+1joCwim=otFYA@mail.gmail.com>
In-Reply-To: <CAAAa2sijkWvoD8+wk-r066Du09vWkWAAO0NX+1joCwim=otFYA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] Change all XFAIL tests to FAIL
From: smalyshev@sugarcrm.com (Stas Malyshev)

Hi!

> The difference started from 5.3.9 release when we start to pay *much
> more* attention to tests.
> You now can cleanly see failing tests, it's not that huge list, so
> it's big difference.

Yes, and removing XFAILs would kill that advantage.

> The main idea I'm trying to say is that it's comfortable to live with
> XFAILs. That's why they live by years. They don't get make any
> pressure, we don't have a release rule "No failing tests", so they go

You talk about "making pressure", but when date fails were sitting in
the tests as FAILs, they didn't make any "pressure" and nobody was
fixing them. And if we had rule of "no failing tests", we'd have no
releases for years now, because nobody is fixing those tests and bugs
behind them. You want to fix them? Go ahead, no problem. But if there's
nobody to fix them - what's the use to put them in FAILs and prevent us
from seeing issues that *are* going to be fixed?

> We have 3 failing tests and 35 xfails, I don't see any tons of fails
> here. Sorry, if I sound like a broken record, but if we need to fix
> those, we need to make more noise about that.

OK, you made noise. Let's see how many of those 35 xfails get fixed,
let's say, in a month. How many you would predict it would be?

> XFAIL - expected to fail test. If it's fails - then it's ok. That's
> how I understand it. Failing test should not be ok, it's an error. If
> you get used to not paying attention on failing tests, you're in
> dangerous situation. It's like a fairy tale about boy that cried

Nobody already *is* paying attention, so it's not an "if", it's a fact.
It's a sad fact, but still a fact. And it's not result of the XFAILs,
because this situation predates XFAILs and was there before we moved
such tests to XFAILs.

> About incomplete, well, it seems it doesn't suite here much, it's
> about that test is not fully written or finished.

If your test is not finished, do it in a fork. By the time the feature
gets merged into main branches, it should be complete enough to run the
tests.
-- 
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227