Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:56020
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain zend.com designates 212.199.177.89 as permitted sender)
Message-ID: <4EB25BF3.7040703@zend.com>
Date: Thu, 3 Nov 2011 13:16:35 +0400
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110928 Fedora/3.1.15-1.fc14 Lightning/1.0b3pre Thunderbird/3.1.15
MIME-Version: 1.0
To: Yasuo Ohgaki <yohgaki@ohgaki.net>
CC: <internals@lists.php.net>
References: <CAGa2bXb4EjSRAevwnx=RCJayHe0DiLbcLWxP3Zp79xC=V==GqQ@mail.gmail.com> <4EB23E3D.3010908@zend.com> <CAGa2bXamS5re6w4wHtWq4=SyWWdroSQh1vrECJqe_QVT9yw_TA@mail.gmail.com>
In-Reply-To: <CAGa2bXamS5re6w4wHtWq4=SyWWdroSQh1vrECJqe_QVT9yw_TA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: quoted-printable
Subject: Re: [PHP-DEV] Zend Multibyte support
From: dmitry@zend.com (Dmitry Stogov)

I suppose PHP can't autodetect SJIS encoding and needs a hint

for 5.4

$ php -d zend.multibyte=3D1 -d zend.script_encoding=3DSJIS sjis.php

5.3 must be compiled with --enable-zend-multibyte, then

$ php -d mbstring.script_encoding=3DSJIS for 5.3

(I've just tested 5.4 but not 5.3. Just don't have 5.3 compiled with=20
--enable-zend-multibyte).

Thanks. Dmitry.


On 11/03/2011 12:12 PM, Yasuo Ohgaki wrote:
> Hi Dimity&  Rui,
>
> I was trying to see if PHP 5.4 and trunk was also affected by this bug =
report.
>
> --with-zend-multibyte and --enable-debug reports LEAK with run-test.php
> https://bugs.php.net/bug.php?id=3D60194
>
> So I configured as "./configure --enable-debug"
>
> Sorry for being a lazy reader. I could turn on zend.multibyte with
> "--enable-mbstring".
> Now I see PHP 5.3 doesn't have dependency to mbstring, but PHP 5.4+ has=
.
> I haven't read the last sentence.Thanks.
>
> It looks like php-src-5.4 doesn't has leaks as PHP 5.3 does.
> (I haven't completed the test, since it seems zend.multibyte is NOT wor=
king)
>
> $ TEST_PHP_EXECUTABLE=3D./sapi/cli/php ./run-tests.php -m -c ./php.ini
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> PHP         : ./sapi/cli/php
> PHP_SAPI    : cli
> PHP_VERSION : 5.4.0RC1-dev
> ZEND_VERSION: 2.4.0
> PHP_OS      : Linux - Linux dev.inter.es-i.jp 2.6.35.14-2m.mo7.x86_64
> #1 SMP Mon Sep 12 11:09:50 JST 2011 x86_64
> INI actual  : /home/yohgaki/ext/svn/oss/php.net/php-src-5.4/php.ini
> More .INIs  :
> CWD         : /home/yohgaki/ext/svn/oss/php.net/php-src-5.4
> Extra dirs  :
> VALGRIND    : valgrind-3.6.1
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> TIME START 2011-11-03 16:36:57
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> PASS EXPECT [tests/run-test/test001.phpt]
> PASS EXPECTF [tests/run-test/test002.phpt]
> PASS EXPECTREGEX [tests/run-test/test003.phpt]
> .....
>
> However, I got the same parse error, as Rui mentioned, with SJIS source=
 script.
>
> sjis.php
> ----
> <?php
> echo '=E8=A1=A8';
> ----
> (You cannot copy&paste, since this would be UTF-8. If you need
>   file, let me know. I'll directly mail you.)
>
> =E8=A1=A8 is valid SJIS char code and has \ as second byte.
> Since PHP complains with parse error, it seems zend.multibyte is not wo=
rking on
> PHP 5.4.
>
> $ ./sapi/cli/php -i | grep multibyte
> zend.multibyte =3D>  On =3D>  On
> $ ./sapi/cli/php /usr/local/apache2.0/htdocs/sjis.php
> PHP Parse error:  syntax error, unexpected ''=EF=BF=BD\';'
> (T_ENCAPSED_AND_WHITESPACE) in /usr/local/apache2.0/htdocs/sjis.php on
> line 2
>
> It seems like working with zend.multibyte=3DOff.
>
> Regards,
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>
>
> On Thu, Nov 3, 2011 at 4:09 PM, Dmitry Stogov<dmitry@zend.com>  wrote:
>> Hi Yasuo,
>>
>> how did you see that "Zend Multibyte Support" weren't enabled?
>>
>> $ sapi/cli/php -d zend.multibyte=3D1 -i | grep -i multibyte
>>
>> Zend Multibyte Support =3D>  provided by mbstring
>> zend.multibyte =3D>  On =3D>  On ******
>> Multibyte Support =3D>  enabled
>> Multibyte string engine =3D>  libmbfl
>> Multibyte (japanese) regex support =3D>  enabled
>> Multibyte regex (oniguruma) backtrack check =3D>  On
>> Multibyte regex (oniguruma) version =3D>  4.7.1
>>
>> Thanks. Dmitry.
>>
>>
>> On 11/03/2011 04:32 AM, Yasuo Ohgaki wrote:
>>>
>>> Hi all,
>>>
>>> I noticed that "Zend Multibyte Support" won't be on with
>>>
>>> ./sapi/cli/php -d zend.multibyte=3D1
>>> nor
>>> zend.multibyte=3Don (in php.ini)
>>>
>>> This happens both php-src and php-src-5.4.
>>>
>>> According to php.ini-production from php-src:
>>> ; If enabled, scripts may be written in encodings that are incompatib=
le
>>> with
>>> ; the scanner.  CP936, Big5, CP949 and Shift_JIS are the examples of =
such
>>> ; encodings.  To use this feature, mbstring extension must be enabled=
.
>>> ; Default: Off
>>> ;zend.multibyte =3D Off
>>>
>>> I thought it became runtime option.
>>> Is this a bug or am I missing something?
>>>
>>> --
>>> Yasuo Ohgaki
>>> yohgaki@ohgaki.net
>>>
>>
>>