Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:36185
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain stephenbach.net from 141.161.88.35 cause and error)
To: internals@lists.php.net
Date: Mon, 17 Mar 2008 23:51:12 -0400
User-Agent: KMail/1.9.6
MIME-Version: 1.0
Content-Type: Multipart/Mixed;
  boundary="Boundary-00=_wwz3HR3mBxB69Ag"
Message-ID: <200803172351.12440.stephen@stephenbach.net>
Subject: [PATCH] Bug 43477 - Unicode error mode ignored
From: stephen@stephenbach.net (Stephen Bach)

--Boundary-00=_wwz3HR3mBxB69Ag
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Attached is a simple proposed patch that fixes Bug 43477. Basically, the code 
that set the error mode of the ICU converter was giving it an instruction 
(the context parameter) to only skip or substitute if the code point was not 
represented in the new encoding. However, it still was returning an error for 
illegal sequences.

The test suite returns the same results with or without the patch. Test also 
attached.

-Stephen Bach

--Boundary-00=_wwz3HR3mBxB69Ag
Content-Type: text/x-diff;
  charset="utf-8";
  name="bug43477.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename="bug43477.patch"

Index: Zend/zend_unicode.c
===================================================================
RCS file: /repository/ZendEngine2/zend_unicode.c,v
retrieving revision 1.37
diff -u -r1.37 zend_unicode.c
--- Zend/zend_unicode.c	31 Dec 2007 07:12:07 -0000	1.37
+++ Zend/zend_unicode.c	15 Mar 2008 23:37:36 -0000
@@ -47,16 +47,16 @@
 
 		case ZEND_CONV_ERROR_SKIP:
 			if (direction == ZEND_FROM_UNICODE)
-				ucnv_setFromUCallBack(conv, UCNV_FROM_U_CALLBACK_SKIP, UCNV_SKIP_STOP_ON_ILLEGAL, NULL, NULL, &status);
+				ucnv_setFromUCallBack(conv, UCNV_FROM_U_CALLBACK_SKIP, NULL, NULL, NULL, &status);
 			else
-				ucnv_setToUCallBack(conv, UCNV_TO_U_CALLBACK_SKIP, UCNV_SKIP_STOP_ON_ILLEGAL, NULL, NULL, &status);
+				ucnv_setToUCallBack(conv, UCNV_TO_U_CALLBACK_SKIP, NULL, NULL, NULL, &status);
 			break;
 
 		case ZEND_CONV_ERROR_SUBST:
 			if (direction == ZEND_FROM_UNICODE)
-				ucnv_setFromUCallBack(conv, UCNV_FROM_U_CALLBACK_SUBSTITUTE, UCNV_SUB_STOP_ON_ILLEGAL, NULL, NULL, &status);
+				ucnv_setFromUCallBack(conv, UCNV_FROM_U_CALLBACK_SUBSTITUTE, NULL, NULL, NULL, &status);
 			else
-				ucnv_setToUCallBack(conv, UCNV_TO_U_CALLBACK_SUBSTITUTE, UCNV_SUB_STOP_ON_ILLEGAL, NULL, NULL, &status);
+				ucnv_setToUCallBack(conv, UCNV_TO_U_CALLBACK_SUBSTITUTE, NULL, NULL, NULL, &status);
 			break;
 
 		case ZEND_CONV_ERROR_ESCAPE_UNICODE:

--Boundary-00=_wwz3HR3mBxB69Ag
Content-Type: text/plain;
  charset="utf-8";
  name="bug43477.phpt"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename="bug43477.phpt"

--TEST--
Bug #43477 (Unicode error mode)
--FILE--
<?php
var_dump(unicode_decode(b"\xF8", 'UTF-8', U_CONV_ERROR_SKIP));
?>
--EXPECT--
unicode(0) ""

--Boundary-00=_wwz3HR3mBxB69Ag--