Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:57273
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain mit.edu designates 18.7.68.35 as permitted sender)
Content-Type: text/plain; charset=UTF-8
Cc: "Edward Z. Yang" <ezyang@php.net>, internals <internals@lists.php.net>
To: Pierre Joye <pierre.php@gmail.com>
In-reply-to: <CAEZPtU5XQ_tSq=YpNwN-h3tjcB-VyeOfhFxhMmchchi9VeTBVg@mail.gmail.com>
References: <1325788652-sup-3243@ezyang> <CAEZPtU5XQ_tSq=YpNwN-h3tjcB-VyeOfhFxhMmchchi9VeTBVg@mail.gmail.com>
Date: Sun, 08 Jan 2012 11:41:50 -0500
Message-ID: <1326039782-sup-7389@ezyang>
User-Agent: Sup/git
Content-Transfer-Encoding: 8bit
Subject: Re: [PHP-DEV] Bug 48417 and PHP 5.4
From: ezyang@MIT.EDU ("Edward Z. Yang")

Hello Pierre,

The whole situation is a slightly complicated.  One question to ask is: "Is
there code in PHP 5.3 that worked, which now no longer works in PHP 5.4?" The
answer to this question is *yes*, as seen by this example:

    var_dump(iconv("UTF-8", "ISO-8859-1//IGNORE", "\xE4\xB8\xAD"));

Now, the reason why this worked in PHP 5.3 was because of a "bug", namely
#52211, so the situation in PHP 5.3 was "two wrongs make a right" (except,
of course, in the case where more than 8000 bytes are involved.)  Fixing one
of the bugs causes "//IGNORE" to stop working entirely.  So if you ask
another question, "Are there less bugs in PHP 5.4 than in PHP 5.3", the answer
is also yes.

Now, consider the perspective of a library developer using iconv //IGNORE.  In
PHP 5.3, effective workarounds exist: I can fix #52211 by checking for captured
errors and returning false instead, I can fix #48147 by splitting my input into
8kb chunks (mumble utf-8 boundaries) and feeding it to iconv individually.

In PHP 5.4, I am left with no such redress. Because iconv will now always
return false upon an input, regardless of whether or not //IGNORE is set,
I can no longer do conversions with invalid inputs.  It's worth repeating:
#48147 is no longer "iconv with //IGNORE cuts string", it's "iconv with
//IGNORE doesn't work". At all. I *guarantee* you large amounts of PHP code
will be affected.

It seems it would be better to wait for a complete fix, than to fix it partially
and leave some (widely used) functionality completely broken.

Is this a bug in glibc?  I sincerely hope I can convince Ulrich Drepper
so, since I agree that his API is completely off the wall:

    http://sources.redhat.com/bugzilla/show_bug.cgi?id=13541

But as it stands, Ulrich does not seem to believe that there is any bug (see
that he's closed both of my upstream bugs; and, since //IGNORE is wholly
undocumented, there is not really any spec I can hold him to), so it would seem
most advisable for PHP to play nice, since most users are not in the position
to patch their glibc installation.

Cheers,
Edward

Excerpts from Pierre Joye's message of Sun Jan 08 07:36:03 -0500 2012:
> hi,
> 
> I closed one bug (unrelated to what you have) and added a comment to
> the /ignore issue. I do not see a bug in PHP but if you have any info
> that shows that PHP causes this problem, then please add them to the
> bug so we can fix it, if not I will bogus it as there is no bug in php
> but in iconv (glibc or other implementations).
> 
> On Thu, Jan 5, 2012 at 7:42 PM, Edward Z. Yang <ezyang@php.net> wrote:
> > Hello all,
> >
> > I know it seems like there always is infinite pile of work to do before
> > the PHP 5.4 release comes out, but I think this bug is worth looking at.
> >
> > https://bugs.php.net/bug.php?id=52211
> >
> > I suggest reverting r300894, since this causes //IGNORE to stop working.
> > Preserving functionality should probably be more important.
> >
> > The underlying cause of this bug is this one:
> >
> > https://bugs.php.net/bug.php?id=48147
> >
> > But I don't think this feature is going to get implemented in time.
> >
> > Edward
> >
> > --
> > PHP Internals - PHP Runtime Development Mailing List
> > To unsubscribe, visit: http://www.php.net/unsub.php
> >
>