Bug 48417 and PHP 5.4

Hey Pierre,

I was wondering if you'd be willing to pitch a rollback of r300894
post-code-freeze. If not, I'll start writing error detection code
to accomodate for PHP 5.4 :^)

Thanks,
Edward

Excerpts from Edward Z. Yang's message of Sun Jan 08 11:41:50 -0500 2012:

Hello Pierre,

The whole situation is a slightly complicated. One question to ask is: "Is
there code in PHP 5.3 that worked, which now no longer works in PHP 5.4?" The
answer to this question is yes, as seen by this example:

var_dump(iconv("UTF-8", "ISO-8859-1//IGNORE", "\xE4\xB8\xAD"));

Now, the reason why this worked in PHP 5.3 was because of a "bug", namely
#52211, so the situation in PHP 5.3 was "two wrongs make a right" (except,
of course, in the case where more than 8000 bytes are involved.) Fixing one
of the bugs causes "//IGNORE" to stop working entirely. So if you ask
another question, "Are there less bugs in PHP 5.4 than in PHP 5.3", the answer
is also yes.

Now, consider the perspective of a library developer using iconv //IGNORE. In
PHP 5.3, effective workarounds exist: I can fix #52211 by checking for captured
errors and returning false instead, I can fix #48147 by splitting my input into
8kb chunks (mumble utf-8 boundaries) and feeding it to iconv individually.

In PHP 5.4, I am left with no such redress. Because iconv will now always
return false upon an input, regardless of whether or not //IGNORE is set,
I can no longer do conversions with invalid inputs. It's worth repeating:
#48147 is no longer "iconv with //IGNORE cuts string", it's "iconv with
//IGNORE doesn't work". At all. I guarantee you large amounts of PHP code
will be affected.

It seems it would be better to wait for a complete fix, than to fix it partially
and leave some (widely used) functionality completely broken.

Is this a bug in glibc? I sincerely hope I can convince Ulrich Drepper
so, since I agree that his API is completely off the wall:

http://sources.redhat.com/bugzilla/show_bug.cgi?id=13541

But as it stands, Ulrich does not seem to believe that there is any bug (see
that he's closed both of my upstream bugs; and, since //IGNORE is wholly
undocumented, there is not really any spec I can hold him to), so it would seem
most advisable for PHP to play nice, since most users are not in the position
to patch their glibc installation.

Cheers,
Edward

Excerpts from Pierre Joye's message of Sun Jan 08 07:36:03 -0500 2012:

hi,

I closed one bug (unrelated to what you have) and added a comment to
the /ignore issue. I do not see a bug in PHP but if you have any info
that shows that PHP causes this problem, then please add them to the
bug so we can fix it, if not I will bogus it as there is no bug in php
but in iconv (glibc or other implementations).

Hello all,

I know it seems like there always is infinite pile of work to do before
the PHP 5.4 release comes out, but I think this bug is worth looking at.

https://bugs.php.net/bug.php?id=52211

I suggest reverting r300894, since this causes //IGNORE to stop working.
Preserving functionality should probably be more important.

The underlying cause of this bug is this one:

https://bugs.php.net/bug.php?id=48147

But I don't think this feature is going to get implemented in time.

Edward