PHP 5.2.1RC3 Released

18 years ago by Ilia Alshanetsky — view source

unread

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The 2nd release candidate for PHP 5.2.1 is now available for
download. The tarballs can be found here:

http://downloads.php.net/ilia/php-5.2.1RC3.tar.bz2
(d3889eda8c3471ce7cf2adb35a4de736)
http://downloads.php.net/ilia/php-5.2.1RC3.tar.gz
(c5b3e5540d1951d4c4b976b8a39c09ab)

and the Win32 binaries will be available in short order.

Since the last release, there are over 20 different bug fixes
resolving some annoying engine issues such as the tempval leak inside
foreach(). We do not anticipate any regressions to be introduced by
this RC, but I would still like to ask everyone to take a few minutes
and test it against their code base. If you come across any issues
please report them at http://bugs.php.net/.

Depending on the stability of this release it may either be followed
by a final release or another RC, therefor your feedback is critical
to determining whether or not the code is stable enough to warrant
the 5.2.1 final.

Ilia Alshanetsky
5.2 Release Master

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFFsBJOLKekh381/CERAq/MAKCKg7buIpB9hXWrtnfHN7hQPB6akgCeJy8q
Q/HBl5G15afjTjebEaILHlE=
=tsFU
-----END PGP SIGNATURE

18 years ago by Edin Kadribasic — view source

unread

Windows build is ready for download too:

http://downloads.php.net/edink/php-5.2.1RC3-Win32.zip
(6dd32e4a2ec2dd67efe42edcba3fdee5)

http://downloads.php.net/edink/php-5.2.1RC3-win32-installer.msi
(50c20632cc232da6ae13c302339df0b5)

http://downloads.php.net/edink/pecl-5.2.1RC3-Win32.zip
(fa31e1c0ad54514b19357b52aadbb9e5 )

http://downloads.php.net/edink/php-debug-pack-5.2.1RC3-Win32.zip
(cd86bc20eb524fd7e72ec2d43e0de13b)

And the FastCGI non-thread safe build
(WARNING: the build below can only be used with CLI/CGI/FastCGI)

http://downloads.php.net/edink/php-5.2.1RC3-nts-Win32.zip
(61b6d9146b871b72f26cc2a44d294941)

http://downloads.php.net/edink/pecl-5.2.1RC3-nts-Win32.zip
(e4331988dc8e692dc5bb5f079f54d776)

http://downloads.php.net/edink/php-debug-pack-5.2.1RC3-nts-Win32.zip
(5c83f3ec7d7fdaa202aac565d38864fe)

Edin

Ilia Alshanetsky wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The 2nd release candidate for PHP 5.2.1 is now available for download.
The tarballs can be found here:

http://downloads.php.net/ilia/php-5.2.1RC3.tar.bz2
(d3889eda8c3471ce7cf2adb35a4de736)
http://downloads.php.net/ilia/php-5.2.1RC3.tar.gz
(c5b3e5540d1951d4c4b976b8a39c09ab)

and the Win32 binaries will be available in short order.

Since the last release, there are over 20 different bug fixes
resolving some annoying engine issues such as the tempval leak inside
foreach(). We do not anticipate any regressions to be introduced by
this RC, but I would still like to ask everyone to take a few minutes
and test it against their code base. If you come across any issues
please report them at http://bugs.php.net/.

Depending on the stability of this release it may either be followed
by a final release or another RC, therefor your feedback is critical
to determining whether or not the code is stable enough to warrant the
5.2.1 final.

Ilia Alshanetsky
5.2 Release Master

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFFsBJOLKekh381/CERAq/MAKCKg7buIpB9hXWrtnfHN7hQPB6akgCeJy8q
Q/HBl5G15afjTjebEaILHlE=
=tsFU
-----END PGP SIGNATURE

18 years ago by steve — view source

unread

Hi, a couple of things (and maybe I need some help, and everything is
assuming windows):

Has something changed with realpath() (at least on windows, my only
test)??? In 5.2.0 if the file does not exist, it returns false, but in
5.2.1RC2 and 5.2.1RC3 it returns the path to a non-existing file.
(Used fastcgi/nts version to test)
I love having the nts versions. That way I can use FastCGI on my
dev machine and have it setup like the servers. And a bit faster.
However, RC3 run times are sooo much longer than RC2 that the FastCGI
server often ends up canceling and gives the browser a Internal
Service Error 500...
When using RC3 as a handler (thread safe version), phpinfo reports
PHP 5.2.0 but under apache2handler is says 5.2.1RC3....??? And it no
longer has the real_path bug above. I no longer have any 5.2.0 files
around, and even tried restarting the machine to no effect. I always
install manually, never using the msi installer.

Hope this report helps in any way.

Thanks!

-s

Windows build is ready for download too:

http://downloads.php.net/edink/php-5.2.1RC3-Win32.zip
(6dd32e4a2ec2dd67efe42edcba3fdee5)

http://downloads.php.net/edink/php-5.2.1RC3-win32-installer.msi
(50c20632cc232da6ae13c302339df0b5)

http://downloads.php.net/edink/pecl-5.2.1RC3-Win32.zip
(fa31e1c0ad54514b19357b52aadbb9e5 )

http://downloads.php.net/edink/php-debug-pack-5.2.1RC3-Win32.zip
(cd86bc20eb524fd7e72ec2d43e0de13b)

And the FastCGI non-thread safe build
(WARNING: the build below can only be used with CLI/CGI/FastCGI)

http://downloads.php.net/edink/php-5.2.1RC3-nts-Win32.zip
(61b6d9146b871b72f26cc2a44d294941)

http://downloads.php.net/edink/pecl-5.2.1RC3-nts-Win32.zip
(e4331988dc8e692dc5bb5f079f54d776)

http://downloads.php.net/edink/php-debug-pack-5.2.1RC3-nts-Win32.zip
(5c83f3ec7d7fdaa202aac565d38864fe)

Edin

Ilia Alshanetsky wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The 2nd release candidate for PHP 5.2.1 is now available for download.
The tarballs can be found here:

http://downloads.php.net/ilia/php-5.2.1RC3.tar.bz2
(d3889eda8c3471ce7cf2adb35a4de736)
http://downloads.php.net/ilia/php-5.2.1RC3.tar.gz
(c5b3e5540d1951d4c4b976b8a39c09ab)

and the Win32 binaries will be available in short order.

Since the last release, there are over 20 different bug fixes
resolving some annoying engine issues such as the tempval leak inside
foreach(). We do not anticipate any regressions to be introduced by
this RC, but I would still like to ask everyone to take a few minutes
and test it against their code base. If you come across any issues
please report them at http://bugs.php.net/.

Depending on the stability of this release it may either be followed
by a final release or another RC, therefor your feedback is critical
to determining whether or not the code is stable enough to warrant the
5.2.1 final.

Ilia Alshanetsky
5.2 Release Master

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFFsBJOLKekh381/CERAq/MAKCKg7buIpB9hXWrtnfHN7hQPB6akgCeJy8q
Q/HBl5G15afjTjebEaILHlE=
=tsFU
-----END PGP SIGNATURE

18 years ago by johannes@php.net — view source

unread

Hi steve,

When using RC3 as a handler (thread safe version), phpinfo reports
PHP 5.2.0 but under apache2handler is says 5.2.1RC3....??? And it no
longer has the real_path bug above. I no longer have any 5.2.0 files
around, and even tried restarting the machine to no effect. I always
install manually, never using the msi installer.

This looks like you're having some old php5ts.dll around which is being
loaded by the sapi instead of the new one. Please check again for that
file.

johannes

18 years ago by Antony Dovgal — view source

unread

Hi, a couple of things (and maybe I need some help, and everything is
assuming windows):

Has something changed with realpath() (at least on windows, my only
test)??? In 5.2.0 if the file does not exist, it returns false, but in
5.2.1RC2 and 5.2.1RC3 it returns the path to a non-existing file.
(Used fastcgi/nts version to test)

Please fill a report @ bugs.php.net.
Thanks in advance.

I love having the nts versions. That way I can use FastCGI on my
dev machine and have it setup like the servers. And a bit faster.
However, RC3 run times are sooo much longer than RC2 that the FastCGI
server often ends up canceling and gives the browser a Internal
Service Error 500...

When using RC3 as a handler (thread safe version), phpinfo reports
PHP 5.2.0 but under apache2handler is says 5.2.1RC3....???

Most likely you forgot to delete old php5ts.dll.

And it no
longer has the real_path bug above. I no longer have any 5.2.0 files
around, and even tried restarting the machine to no effect. I always
install manually, never using the msi installer.

--
Wbr,
Antony Dovgal

18 years ago by steve — view source

unread

Has something changed with realpath() (at least on windows, my only

test)??? In 5.2.0 if the file does not exist, it returns false, but in
5.2.1RC2 and 5.2.1RC3 it returns the path to a non-existing file.
(Used fastcgi/nts version to test)

Please fill a report @ bugs.php.net.
Thanks in advance.

Since I have the thread safe version of rc3 running now, and it is not
there, i suspect that it might be a build issue. Do the two use or
link to different core libraries?

When using RC3 as a handler (thread safe version), phpinfo reports
PHP 5.2.0 but under apache2handler is says 5.2.1RC3....???

Most likely you forgot to delete old php5ts.dll.

Yes, found that, thanks... I knew I had to have done something wrong...

-steve

18 years ago by Tomas Kuliavas — view source

unread

The 2nd release candidate for PHP 5.2.1 is now available for download.
The tarballs can be found here:

http://downloads.php.net/ilia/php-5.2.1RC3.tar.bz2
(d3889eda8c3471ce7cf2adb35a4de736)
http://downloads.php.net/ilia/php-5.2.1RC3.tar.gz
(c5b3e5540d1951d4c4b976b8a39c09ab)

and the Win32 binaries will be available in short order.

Since the last release, there are over 20 different bug fixes resolving
some annoying engine issues such as the tempval leak inside foreach().
We do not anticipate any regressions to be introduced by this RC, but I
would still like to ask everyone to take a few minutes and test it
against their code base. If you come across any issues please report
them at http://bugs.php.net/.

Depending on the stability of this release it may either be followed by
a final release or another RC, therefor your feedback is critical to
determining whether or not the code is stable enough to warrant the
5.2.1 final.

Bugfix for #39993 introduced LC_CTYPE variable corruption.

Comments in bug report stated that patch is not correct, yet patch was
commited to cvs.

Is it that difficult to replace strcasecmp with locale independent case
insensitive comparison function?

--
Tomas

18 years ago by Antony Dovgal — view source

unread

The 2nd release candidate for PHP 5.2.1 is now available for download.
The tarballs can be found here:

http://downloads.php.net/ilia/php-5.2.1RC3.tar.bz2
(d3889eda8c3471ce7cf2adb35a4de736)
http://downloads.php.net/ilia/php-5.2.1RC3.tar.gz
(c5b3e5540d1951d4c4b976b8a39c09ab)

and the Win32 binaries will be available in short order.

Since the last release, there are over 20 different bug fixes resolving
some annoying engine issues such as the tempval leak inside foreach().
We do not anticipate any regressions to be introduced by this RC, but I
would still like to ask everyone to take a few minutes and test it
against their code base. If you come across any issues please report
them at http://bugs.php.net/.

Depending on the stability of this release it may either be followed by
a final release or another RC, therefor your feedback is critical to
determining whether or not the code is stable enough to warrant the
5.2.1 final.

Bugfix for #39993 introduced LC_CTYPE variable corruption.

Please provide more details on this.

Comments in bug report stated that patch is not correct, yet patch was
commited to cvs.

Is it that difficult to replace strcasecmp with locale independent case
insensitive comparison function?

We're open for patches. Feel free to send one.

--
Wbr,
Antony Dovgal

18 years ago by Tomas Kuliavas — view source

unread

The 2nd release candidate for PHP 5.2.1 is now available for download.
The tarballs can be found here:

http://downloads.php.net/ilia/php-5.2.1RC3.tar.bz2
(d3889eda8c3471ce7cf2adb35a4de736)
http://downloads.php.net/ilia/php-5.2.1RC3.tar.gz
(c5b3e5540d1951d4c4b976b8a39c09ab)

and the Win32 binaries will be available in short order.

Since the last release, there are over 20 different bug fixes resolving
some annoying engine issues such as the tempval leak inside foreach().
We do not anticipate any regressions to be introduced by this RC, but I
would still like to ask everyone to take a few minutes and test it
against their code base. If you come across any issues please report
them at http://bugs.php.net/.

Depending on the stability of this release it may either be followed by
a final release or another RC, therefor your feedback is critical to
determining whether or not the code is stable enough to warrant the
5.2.1 final.

Bugfix for #39993 introduced LC_CTYPE variable corruption.

Please provide more details on this.

Comments in bug report stated that patch is not correct, yet patch was
commited to cvs.

From http://bugs.php.net/bug.php?id=39993

Test script:

<?php
// Set all locale vars
var_dump(setlocale(LC_ALL,'tr_TR.UTF-8'));
// get default timezone
var_dump(date_default_timezone_get());
// get current `LC_CTYPE` value
var_dump(setlocale(LC_CTYPE,0));

Expected result:

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(11) "tr_TR.UTF-8"

Result after Iliaa's patch:

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(1) "C"

Patch was applied in ext/date/lib/parse_tz.c revision 1.20.2.6.2.10.
http://cvs.php.net/viewvc.cgi/php-src/ext/date/lib/parse_tz.c?view=log

OS - Linux Debian Etch. glibc 2.3.6.

Tested PHP versions:

php 5.2.0-8 debian - timezone database corruption error
php5.2-200701011130 - timezone database corruption error
php5.2-200701241930 - LC_CTYPE environment variable is modified after
date_default_timezone_set/get calls.

If you set LC_CTYPE to C, you might break gettext translations that don't
set codeset with bind_textdomain_codeset().

Is it that difficult to replace strcasecmp with locale independent case
insensitive comparison function?

We're open for patches. Feel free to send one.

I am PHP developer and don't code in C. In PHP script locale insensitive
strtolower/strtoupper can be done with one preg_replace call.

All timezone identifiers are written in plain US-ASCII. Lowercase ASCII
characters are in 0x61-0x7A range, uppercase characters are in 0x41-0x5A.
Maybe you can add 0x20 to all 0x41-0x5A byte values and use strcmp()
instead of strcasecmp().

--
Tomas

18 years ago by Antony Dovgal — view source

unread

Test script:

<?php
// Set all locale vars
var_dump(setlocale(LC_ALL,'tr_TR.UTF-8'));
// get default timezone
var_dump(date_default_timezone_get());
// get current LC_CTYPE value
var_dump(setlocale(LC_CTYPE,0));

Expected result:

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(11) "tr_TR.UTF-8"

Result after Iliaa's patch:

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(1) "C"

Ok, this is now fixed.

Is it that difficult to replace strcasecmp with locale independent case
insensitive comparison function?

We're open for patches. Feel free to send one.

I am PHP developer and don't code in C. In PHP script locale insensitive
strtolower/strtoupper can be done with one preg_replace call.

Making PCRE a requirement for ext/date is not an option.

All timezone identifiers are written in plain US-ASCII. Lowercase ASCII
characters are in 0x61-0x7A range, uppercase characters are in 0x41-0x5A.
Maybe you can add 0x20 to all 0x41-0x5A byte values and use strcmp()
instead of strcasecmp().

Looks way overcomplicated comparing to the Ilia's solution.

--
Wbr,
Antony Dovgal

18 years ago by Tomas Kuliavas — view source

unread

Is it that difficult to replace strcasecmp with locale independent
case insensitive comparison function?

We're open for patches. Feel free to send one.

I am PHP developer and don't code in C. In PHP script locale insensitive
strtolower/strtoupper can be done with one preg_replace call.

Making PCRE a requirement for ext/date is not an option.

All timezone identifiers are written in plain US-ASCII. Lowercase ASCII
characters are in 0x61-0x7A range, uppercase characters are in
0x41-0x5A.
Maybe you can add 0x20 to all 0x41-0x5A byte values and use strcmp()
instead of strcasecmp().

Looks way overcomplicated comparing to the Ilia's solution.

PHP needs locale insensitive strcasecmp() or your developers will continue
hitting string comparison issues in Turkish and Azerbaijani. Ilia patched
it in timezone parsing code. Some day other strcasecmp will pop in other
place.

Switching locale everytime you want to compare strings is a hack and not a
bug fix.

--
Tomas

18 years ago by Antony Dovgal — view source

unread

Is it that difficult to replace strcasecmp with locale independent
case insensitive comparison function?

We're open for patches. Feel free to send one.

I am PHP developer and don't code in C. In PHP script locale insensitive
strtolower/strtoupper can be done with one preg_replace call.

Making PCRE a requirement for ext/date is not an option.

All timezone identifiers are written in plain US-ASCII. Lowercase ASCII
characters are in 0x61-0x7A range, uppercase characters are in
0x41-0x5A.
Maybe you can add 0x20 to all 0x41-0x5A byte values and use strcmp()
instead of strcasecmp().

Looks way overcomplicated comparing to the Ilia's solution.

PHP needs locale insensitive strcasecmp() or your developers will continue
hitting string comparison issues in Turkish and Azerbaijani. Ilia patched
it in timezone parsing code. Some day other strcasecmp will pop in other
place.

Switching locale everytime you want to compare strings is a hack and not a
bug fix.

I did not and I do not like it either, but it fixes the problem and does not introduce any new issues (anymore).
If you think it's wrong and you have a better patch - send it to the list and we'll discuss it together.

--
Wbr,
Antony Dovgal

18 years ago by Derick Rethans — view source

unread

Is it that difficult to replace strcasecmp with locale
independent case insensitive comparison function?

We're open for patches. Feel free to send one.

I am PHP developer and don't code in C. In PHP script locale
insensitive strtolower/strtoupper can be done with one preg_replace
call.

Making PCRE a requirement for ext/date is not an option.

All timezone identifiers are written in plain US-ASCII. Lowercase
ASCII characters are in 0x61-0x7A range, uppercase characters are
in 0x41-0x5A. Maybe you can add 0x20 to all 0x41-0x5A byte values
and use strcmp() instead of strcasecmp().

Looks way overcomplicated comparing to the Ilia's solution.

PHP needs locale insensitive strcasecmp() or your developers will
continue hitting string comparison issues in Turkish and Azerbaijani.
Ilia patched it in timezone parsing code. Some day other strcasecmp
will pop in other place.

Switching locale everytime you want to compare strings is a hack and
not a bug fix.

I actually agree there and it was one of the reasons why I didn't like
this patch. However, it is the quickest way to do this for now.

regards,
Derick

--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

18 years ago by Ilia Alshanetsky — view source

unread

PHP needs locale insensitive strcasecmp() or your developers will
continue
hitting string comparison issues in Turkish and Azerbaijani. Ilia
patched
it in timezone parsing code. Some day other strcasecmp will pop in
other
place.

Switching locale everytime you want to compare strings is a hack
and not a
bug fix.

Its a solution that works. There are very few instances where such
behavior is needed and in php6 it won't be relevant anyway since
posix locales will not be used.

Ilia Alshanetsky

18 years ago by Nuno Lopes — view source

unread

PHP needs locale insensitive strcasecmp() or your developers will
continue
hitting string comparison issues in Turkish and Azerbaijani. Ilia
patched
it in timezone parsing code. Some day other strcasecmp will pop in other
place.

Switching locale everytime you want to compare strings is a hack and not
a
bug fix.

Its a solution that works. There are very few instances where such
behavior is needed and in php6 it won't be relevant anyway since posix
locales will not be used.

This is not true at all.. I even think this is a big misconception of PHP 6.
Although this has nothing to do with the issue mentioned above, we will
still need setlocale() in PHP 6 because of externals libs. Even the bundled
PCRE uses locales, and that is very handful (e.g. locale-dependent '\w').
So, POSIX locales will be deprecated for string handling, date formatting
and such things, but they will still be used by external libs.

Nuno

18 years ago by Ilia Alshanetsky — view source

unread

Nuno,

The idea is that while internal libs will use it, PHP will do all the
user-locale handling internally.

PHP needs locale insensitive strcasecmp() or your developers will
continue
hitting string comparison issues in Turkish and Azerbaijani. Ilia
patched
it in timezone parsing code. Some day other strcasecmp will pop
in other
place.

Switching locale everytime you want to compare strings is a hack
and not a
bug fix.

Its a solution that works. There are very few instances where such
behavior is needed and in php6 it won't be relevant anyway since
posix locales will not be used.

This is not true at all.. I even think this is a big misconception
of PHP 6.
Although this has nothing to do with the issue mentioned above, we
will still need setlocale() in PHP 6 because of externals libs.
Even the bundled PCRE uses locales, and that is very handful (e.g.
locale-dependent '\w'). So, POSIX locales will be deprecated for
string handling, date formatting and such things, but they will
still be used by external libs.

Nuno

Ilia Alshanetsky

18 years ago by Ilia Alshanetsky — view source

unread

PCRE should operate in UTF-8 mode.

So are you saying that php will automatically issue setlocale() calls?
e.g.:
<?php
i18n_loc_set_default('pt_PT'); // or whatever that is called these
days
//now it magically calls setlocale('pt_PT', 'pt_PT.iso8859-1',
'portuguese'); -- linux/bsd/windows

echo preg_match('/^\w+$/', "àáãçêõ"); //prints 1?
?>

Nuno,

The idea is that while internal libs will use it, PHP will do all
the user-locale handling internally.

PHP needs locale insensitive strcasecmp() or your developers
will continue
hitting string comparison issues in Turkish and Azerbaijani.
Ilia patched
it in timezone parsing code. Some day other strcasecmp will
pop in other
place.

Switching locale everytime you want to compare strings is a
hack and not a
bug fix.

Its a solution that works. There are very few instances where
such behavior is needed and in php6 it won't be relevant anyway
since posix locales will not be used.

This is not true at all.. I even think this is a big
misconception of PHP 6.
Although this has nothing to do with the issue mentioned above,
we will still need setlocale() in PHP 6 because of externals
libs. Even the bundled PCRE uses locales, and that is very
handful (e.g. locale-dependent '\w'). So, POSIX locales will be
deprecated for string handling, date formatting and such things,
but they will still be used by external libs.

Nuno

Ilia Alshanetsky

18 years ago by Pierre — view source

unread

PCRE should operate in UTF-8 mode.

How does UTF-8 help to make it locale compliant?

--Pierre

18 years ago by Andrei Zmievski — view source

unread

Because with UTF-8, PCRE already knows the uppercase and lowercase
equivalents, without having to rely on the POSIX locales.

-Andrei

PCRE should operate in UTF-8 mode.

How does UTF-8 help to make it locale compliant?

--Pierre

18 years ago by Nuno Lopes — view source

unread

But how do I match only portuguese letters? you'll (always) need posix
locales..
I don't think forcing preg_* function to utf-8 is a good idea, but anyway I
haven't looked enough to PHP 6 (yet) to produce a strong opinion.

Nuno

Because with UTF-8, PCRE already knows the uppercase and lowercase
equivalents, without having to rely on the POSIX locales.

-Andrei

PCRE should operate in UTF-8 mode.

How does UTF-8 help to make it locale compliant?

--Pierre

18 years ago by Andrei Zmievski — view source

unread

I've been thinking about how to not force UTF-8 in PCRE for PHP 6, and
it's not that simple. This is mainly due to preg_replace(), because it
allows array() parameters that can contain mixed IS_UNICODE and
IS_STRING values. I hope you realize though, that in UTF-8 mode PCRE
does not care about POSIX locales, even in PHP 5.

By the way, I think ICU regexp extension, when implemented, will let
you match Portuguese characters in UTF-8 strings.

-Andrei

But how do I match only portuguese letters? you'll (always) need posix
locales..
I don't think forcing preg_* function to utf-8 is a good idea, but
anyway I haven't looked enough to PHP 6 (yet) to produce a strong
opinion.

Nuno

Because with UTF-8, PCRE already knows the uppercase and lowercase
equivalents, without having to rely on the POSIX locales.

-Andrei

PCRE should operate in UTF-8 mode.

How does UTF-8 help to make it locale compliant?

--Pierre

18 years ago by Nuno Lopes — view source

unread

I've been thinking about how to not force UTF-8 in PCRE for PHP 6, and
it's not that simple. This is mainly due to preg_replace(), because it
allows array() parameters that can contain mixed IS_UNICODE and IS_STRING
values. I hope you realize though, that in UTF-8 mode PCRE does not care
about POSIX locales, even in PHP 5.

I haven't though on that, but can't you simply reject mixing of unicode and
binary strings?

By the way, I think ICU regexp extension, when implemented, will let you
match Portuguese characters in UTF-8 strings.

I wasn't aware of that API.. anyway it is probably slower than pcre+locales
(because it uses unicode propertie table lookups)

Yes, UTF-8 covers many aspects but does it know about words, white
spaces (not sure if ws are always the same) and other locale specific
issues? generally, not only pcre. Maybe it is more something for ICU
directly, as you said later in this thread.

That's not really a problem with pcre, as it supports unicode character
properties. It isn't documented in phpdoc (don't look at me :P), but it
looks like:
\pL
where L is one of (from http://pcre.org/pcre.txt):
L Letter
Ll Lower case letter
N Number
Nd Decimal number
Nl Letter number
No Other number
P Punctuation
Zs Space separator
(...)

Nuno

18 years ago by Pierre — view source

unread

Because with UTF-8, PCRE already knows the uppercase and lowercase
equivalents, without having to rely on the POSIX locales.

Yes, UTF-8 covers many aspects but does it know about words, white
spaces (not sure if ws are always the same) and other locale specific
issues? generally, not only pcre. Maybe it is more something for ICU
directly, as you said later in this thread.

--Pierre

18 years ago by Andrei Zmievski — view source

unread

Please do not confuse an encoding (UTF-8) with full locale information
(which is what CLDR provides).

-Andrei

Because with UTF-8, PCRE already knows the uppercase and lowercase
equivalents, without having to rely on the POSIX locales.

Yes, UTF-8 covers many aspects but does it know about words, white
spaces (not sure if ws are always the same) and other locale specific
issues? generally, not only pcre. Maybe it is more something for ICU
directly, as you said later in this thread.

--Pierre

PHP 5.2.1RC3 Released

Test script:

<?php // Set all locale vars var_dump(setlocale(LC_ALL,'tr_TR.UTF-8')); // get default timezone var_dump(date_default_timezone_get()); // get current LC_CTYPE value var_dump(setlocale(LC_CTYPE,0));

Expected result:

string(11) "tr_TR.UTF-8" string(15) "Europe/Helsinki" string(11) "tr_TR.UTF-8"

Result after Iliaa's patch:

string(11) "tr_TR.UTF-8" string(15) "Europe/Helsinki" string(1) "C"

Test script:

<?php // Set all locale vars var_dump(setlocale(LC_ALL,'tr_TR.UTF-8')); // get default timezone var_dump(date_default_timezone_get()); // get current LC_CTYPE value var_dump(setlocale(LC_CTYPE,0));

Expected result:

string(11) "tr_TR.UTF-8" string(15) "Europe/Helsinki" string(11) "tr_TR.UTF-8"

Result after Iliaa's patch:

string(11) "tr_TR.UTF-8" string(15) "Europe/Helsinki" string(1) "C"

<?php
// Set all locale vars
var_dump(setlocale(LC_ALL,'tr_TR.UTF-8'));
// get default timezone
var_dump(date_default_timezone_get());
// get current `LC_CTYPE` value
var_dump(setlocale(LC_CTYPE,0));

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(11) "tr_TR.UTF-8"

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(1) "C"

<?php
// Set all locale vars
var_dump(setlocale(LC_ALL,'tr_TR.UTF-8'));
// get default timezone
var_dump(date_default_timezone_get());
// get current `LC_CTYPE` value
var_dump(setlocale(LC_CTYPE,0));

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(11) "tr_TR.UTF-8"

string(11) "tr_TR.UTF-8"
string(15) "Europe/Helsinki"
string(1) "C"