# What is the use of "unicode.semantics" in PHP 6? - Thread: https://externals.io/message/30173 - Original archive: https://news-web.php.net/php.internals/30173 - Started by Jani Taskinen on 2007-06-14T13:53:47+00:00 - Messages: 195 - Page 1 of 8: messages 1 to 25 (25 per page, add `?page=N` to the URL) - Next page: https://externals.io/message/30173.md?page=2 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #30173 — Jani Taskinen — 2007-06-14T13:53:47+00:00 URL: https://externals.io/message/30173#30173 During Derick's talk about PHP 6 at PHP Vikinger, I started to wonder what exactly was the reasoning behind adding something like "unicode.semantics" option. Derick didn't remember, neither did I. Apparently it's another one of these "register_globals" or "magic_quotes_*" directives we'll remove in PHP 7? :D I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? Just stay with PHP 5 then.. --Jani --- ## #30174 — Scott MacVicar — 2007-06-14T14:00:19+00:00 URL: https://externals.io/message/30173#30174 · In reply to: #30173 I have to agree, I think giving the ability to toggle Unicode support is going to add more confusion and grief for application developers, especially with it being PHP_INI_SYSTEM. The reason for the option as far I can remember was to do with performance when you were working with binary strings, this may no longer be valid. Scott Jani Taskinen wrote: > During Derick's talk about PHP 6 at PHP Vikinger, I started to wonder > what exactly was the reasoning behind adding something like > "unicode.semantics" option. Derick didn't remember, neither did I. > > Apparently it's another one of these "register_globals" or > "magic_quotes_*" directives we'll remove in PHP 7? :D > > I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? > Just stay with PHP 5 then.. > > --Jani --- ## #30175 — Pierre — 2007-06-14T14:00:57+00:00 URL: https://externals.io/message/30173#30175 · In reply to: #30173 Hi Jani, > During Derick's talk about PHP 6 at PHP Vikinger, I started to wonder > what exactly was the reasoning behind adding something like > "unicode.semantics" option. Derick didn't remember, neither did I. > > Apparently it's another one of these "register_globals" or > "magic_quotes_*" directives we'll remove in PHP 7? :D > > I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? > Just stay with PHP 5 then.. Apparently there is a certain amount of users (undefined/undefinable masses) who like to use PHP6 new features (?) without unicode. The arguments were about the incompatibilities (the zend fatal errors are likely to cause more troubles :) and performance. I was one who likes to have a single mode: unicode. --Pierre --- ## #30176 — Derick Rethans — 2007-06-14T14:11:30+00:00 URL: https://externals.io/message/30173#30176 · In reply to: #30175 > Hi Jani, > > > > […] > > Apparently there is a certain amount of users (undefined/undefinable > masses) who like to use PHP6 new features (?) without unicode. I guess you're using the "?" to point out that there are no new features (besides Unicode)? :) > The arguments were about the incompatibilities (the zend fatal errors > are likely to cause more troubles :) and performance. > > I was one who likes to have a single mode: unicode. Yup, here as well. regards, Derick --- ## #30177 — Pierre — 2007-06-14T14:15:13+00:00 URL: https://externals.io/message/30173#30177 · In reply to: #30176 > > > > […] > > I guess you're using the "?" to point out that there are no new features > (besides Unicode)? :) You guessed right :) There will be new features besides unicode (and related, like unicode text mode in stream or unicode FS) but nothing that will not work with php5 as well, as far as I can see (and know for "my" extensions). --Pierre --- ## #30205 — johannes@php.net — 2007-06-14T14:28:07+00:00 URL: https://externals.io/message/30173#30205 · In reply to: #30173 Hi Jani, > During Derick's talk about PHP 6 at PHP Vikinger, I started to wonder > what exactly was the reasoning behind adding something like > "unicode.semantics" option. Derick didn't remember, neither did I. The reason was to "keep BC" > Apparently it's another one of these "register_globals" or > "magic_quotes_*" directives we'll remove in PHP 7? :D > > I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? > Just stay with PHP 5 then.. - The ini Setting changes the behaviour of the code in a quite drastic way. This is even worse then magic_quotes which could be fixed by using some prepending file removing/adding slashes depending on the setting. - The UG(unicode) checks in the code make maintenance way harder. - This feature doesn't bring BC - there will still be enough BC breaks. - I guess we're adding a few thousand UG(unicode) checks during each request which certainly cost a bit performance Conclusion: Let's remove that damn setting. johannes --- ## #30179 — Rasmus Lerdorf — 2007-06-14T15:02:31+00:00 URL: https://externals.io/message/30173#30179 · In reply to: #30173 Jani Taskinen wrote: > During Derick's talk about PHP 6 at PHP Vikinger, I started to wonder > what exactly was the reasoning behind adding something like > "unicode.semantics" option. Derick didn't remember, neither did I. > > Apparently it's another one of these "register_globals" or > "magic_quotes_*" directives we'll remove in PHP 7? :D > > I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? > Just stay with PHP 5 then.. That's exactly why we need the toggle. We don't want to encourage people to stay with an older version. We have enough trouble getting people from 4 to 5 today, why build in an automatic excuse for people to stay with 5 when all development moves to 6? If all their PHP 5-based code works flawlessly in PHP 6, the adoption of PHP 6 will be quicker. -Rasmus --- ## #30180 — Tomas Kuliavas — 2007-06-14T15:08:21+00:00 URL: https://externals.io/message/30173#30180 · In reply to: #30175 > > […] > > Apparently there is a certain amount of users (undefined/undefinable > masses) who like to use PHP6 new features (?) without unicode. The > arguments were about the incompatibilities (the zend fatal errors are > likely to cause more troubles :) and performance. > > I was one who likes to have a single mode: unicode. Changes made in PHP6 unicode_semantics=on are not backwards compatible with PHP4 and PHP5. Same scripts (for example, reading string in bytes) work in PHP4 and PHP5. They won't work in unicode_semantics=on. PHP6 places very strict checks on string variables. PHP script writers are not used to it. If developers have to maintain compatibility with PHP6 unicode_semantics=on, they will have to do that in separate code branch. PHP6 code will break with `E_PARSE` errors in older PHP installs. I don't like it, because I can't turn that thing off. Interpreter is trying to outsmart me without knowing my coding environment. If you need multibyte string support, you have mbstring extension. If you want unicode function and variable names, you should remember that they won't work in international coding environment. International developers must use something they all understand. It means ASCII and English function names. Will you understand purpose of the function, when function name is written in Russian, Chinese or Arabic? -- Tomas --- ## #30183 — johannes@php.net — 2007-06-14T15:23:14+00:00 URL: https://externals.io/message/30173#30183 · In reply to: #30173 Hi Jani, > During Derick's talk about PHP 6 at PHP Vikinger, I started to wonder > what exactly was the reasoning behind adding something like > "unicode.semantics" option. Derick didn't remember, neither did I. The reason was to "keep BC" > Apparently it's another one of these "register_globals" or > "magic_quotes_*" directives we'll remove in PHP 7? :D > > I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? > Just stay with PHP 5 then.. - The ini Setting changes the behaviour of the code in a quite drastic way. This is even worse then magic_quotes which could be fixed by using some prepending file removing/adding slashes depending on the setting. - The UG(unicode) checks in the code make maintenance way harder. - This feature doesn't bring BC - there will still be enough BC breaks. - I guess we're adding a few thousand UG(unicode) checks during each request which certainly cost a bit performance Conclusion: Let's remove that damn setting. johannes --- ## #30182 — Pierre — 2007-06-14T15:33:57+00:00 URL: https://externals.io/message/30173#30182 · In reply to: #30179 > Jani Taskinen wrote: > > […] > > That's exactly why we need the toggle. We don't want to encourage > people to stay with an older version. We have enough trouble getting > people from 4 to 5 today, why build in an automatic excuse for people to > stay with 5 when all development moves to 6? If all their PHP 5-based > code works flawlessly in PHP 6, the adoption of PHP 6 will be quicker. As a side note, we had the same thoughts about php5, it did not work. --Pierre --- ## #30184 — Rasmus Lerdorf — 2007-06-14T15:42:37+00:00 URL: https://externals.io/message/30173#30184 · In reply to: #30182 Pierre wrote: > > > […] > > As a side note, we had the same thoughts about php5, it did not work. Not really. Nothing in PHP 5 was designed to break compatibility with PHP 4. However in PHP 6 there are just some things that cannot be made backward compatible in Unicode mode without being completely inconsistent with how Unicode should work. -Rasmus --- ## #30187 — Pierre — 2007-06-14T16:03:17+00:00 URL: https://externals.io/message/30173#30187 · In reply to: #30184 Hi Rasmus, > > […] > > Not really. Nothing in PHP 5 was designed to break compatibility with > PHP 4. In theory, I agree with you here, we were very careful about BC. However in practice, there was a few troubles which made a smooth migration (like running php4 code directly in php5) a little pain until 5.1. But then the new fatal errors brought other troubles... However my point was about making design choices to minimize the migration momentum. It did not work for PHP5. It worked (I can be wrong, it is too old) well for php3 to php4 because there was really a huge improvement in the core language (begin of "OO" support, even if it was not perfect) and its features in general (extensions). I fear that Unicode is not that appealing for almost all users, even those who actually need it, they already rely on mb_string and other home made solutions. But I tend to be too much realistic/pessimist. --Pierre --- ## #30188 — Stanislav Malyshev — 2007-06-14T17:41:42+00:00 URL: https://externals.io/message/30173#30188 · In reply to: #30173 > I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? To get late static binding and namespaces, of course ;) -- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/ --- ## #30193 — Cristian Rodriguez — 2007-06-15T01:44:59+00:00 URL: https://externals.io/message/30173#30193 · In reply to: #30173 > I mean, if PHP 6 is about unicode, why upgrade to PHP 6 and disable it? I think if unicode.semantics remains PHP_INI_SYSTEM it is useless as most users ( people that runs in shared hosting servers) will simple not be able to turn it on, as well hosting companies will keep it off because turning it on will break applications. So, either make it at least PHP_INI_PER_DIR or remove it all togeteher ( aka.. always behave like unicode.semantics= On) --- ## #30194 — Rasmus Lerdorf — 2007-06-15T01:55:16+00:00 URL: https://externals.io/message/30173#30194 · In reply to: #30193 Cristian Rodriguez wrote: > > > > […] > > I think if unicode.semantics remains PHP_INI_SYSTEM it is useless as > most users ( people that runs in shared hosting servers) will simple > not be able to turn it on, as well hosting companies will keep it off > because turning it on will break applications. > > So, either make it at least PHP_INI_PER_DIR or remove it all togeteher > ( aka.. always behave like unicode.semantics= On) Those same shared hosting companies would never upgrade to PHP 6 if we forced unicode semantics on them breaking legacy apps and that would force us to maintain PHP 5 forever. We recognize that some people are simply not going to make the effort to make their apps unicode aware anytime soon and that fact could easily splinter the project across the unicode line. If that unicode line becomes synonymous with PHP 5 vs. PHP 6 we are in trouble. I would much rather have people on the same codebase so we can move everyone ahead on other features and keep the unicode vs. non-unicode battle to a configuration setting within that one codebase. I really don't want to get into the situation where we are backporting features to PHP 5 a couple of years from now. And we obviously did consider making it PER_DIR, but that is really complicated. The current cries that the unicode.semantics check is complicating code are dull compared to what they would be if we allowed a single process to switch back and forth potentially on the same scripts. -Rasmus --- ## #30251 — Peter Brodersen — 2007-06-19T14:13:31+00:00 URL: https://externals.io/message/30173#30251 · In reply to: #30194 On Fri, 15 Jun 2007 02:55:16 +0100, in php.internals rasmus@lerdorf.com (Rasmus Lerdorf) wrote: >Those same shared hosting companies would never upgrade to PHP 6 if we >forced unicode semantics on them breaking legacy apps and that would >force us to maintain PHP 5 forever. On the other hand I feel a bit sad that if I want to write perfectly good portable PHP 6 code that is only intended to work under PHP 6 I still have to check for different configuration settings. I think we were really close to get out of the if(get_magic_quotes_gpc()) "requirement" but now it has been replaced with a new one. Even if a developer would write (portable) PHP 6 only code. Of course, configurations could contain a lot of other obscure settings that might have influence on the script but none as widespread as the difference in magic_quotes settings. -- - Peter Brodersen --- ## #30252 — Rasmus Lerdorf — 2007-06-19T14:26:57+00:00 URL: https://externals.io/message/30173#30252 · In reply to: #30251 Peter Brodersen wrote: > On Fri, 15 Jun 2007 02:55:16 +0100, in php.internals > rasmus@lerdorf.com (Rasmus Lerdorf) wrote: > > > […] > > On the other hand I feel a bit sad that if I want to write perfectly > good portable PHP 6 code that is only intended to work under PHP 6 I > still have to check for different configuration settings. > > I think we were really close to get out of the > if(get_magic_quotes_gpc()) "requirement" but now it has been replaced > with a new one. Even if a developer would write (portable) PHP 6 only > code. > > > Of course, configurations could contain a lot of other obscure > settings that might have influence on the script but none as > widespread as the difference in magic_quotes settings. But this is no different from writing code that will work on both PHP 5 and PHP 6. The only difference is that instead of checking for PHP 5 you will be checking for Unicode. Like I said, we don't want the Unicode decision to be synonymous with PHP 5 vs. PHP 6 because then the non-Unicode folks will never get the benefits of the non-Unicode improvements in PHP 6 and we would be forced to support PHP 5 for a lot longer. We really stretch our already thing resources in order to support multiple branches, so anything we can do to get as many people as possible onto the same codebase helps us a lot. -Rasmus --- ## #30253 — Peter Brodersen — 2007-06-19T14:39:45+00:00 URL: https://externals.io/message/30173#30253 · In reply to: #30252 On Tue, 19 Jun 2007 07:26:57 -0700, in php.internals rasmus@lerdorf.com (Rasmus Lerdorf) wrote: > > […] > >But this is no different from writing code that will work on both PHP 5 >and PHP 6. The only difference is that instead of checking for PHP 5 >you will be checking for Unicode. Yeah, but that's pretty much the problem :) If I want to create code that has to work under different versions of PHP then it's a no-brainer that I have to check for different settings (and maybe even create userland functions to emulate native functions in later versions of PHP). The more BC the code has to maintain the more settings does one have to check for. But if I want to code to one specific version of PHP it bothers me that I even here has to take in consideration that PHP still comes in different flavors, even under one version. If the unicode setting would be on for most of the installments I might just create assumptions on this setting - pretty much the same way that I have dared to assume that magic_quotes_runtime and magic_quotes_sybase is off (as I have never encountered them on in generel setups such as webhosting companies). But my fear is that exactly the webhosting companies would end up with different settings and might even have to create two products for their customer: "Do you want the PHP 6 with unicode setting on or off?". I'm just worried that PHP 6 is the new NULL: PHP6 != PHP6 :-) -- - Peter Brodersen --- ## #30255 — Pierre — 2007-06-19T15:08:43+00:00 URL: https://externals.io/message/30173#30255 · In reply to: #30252 > But this is no different from writing code that will work on both PHP 5 > and PHP 6. The only difference is that instead of checking for PHP 5 > you will be checking for Unicode. Like I said, we don't want the > Unicode decision to be synonymous with PHP 5 vs. PHP 6 because then the > non-Unicode folks will never get the benefits of the non-Unicode > improvements in PHP 6 and we would be forced to support PHP 5 for a lot > longer. We really stretch our already thing resources in order to > support multiple branches, so anything we can do to get as many people > as possible onto the same codebase helps us a lot. Just as a last (hopefully) comment, even if nothing seemed to have an influence, no matter how many we are to prefer a unicode only mode (so far only you are in favour of it, maybe Andree too but I don't remember his opinion on this topic :). The gain we hope to have by keeping a non unicode mode is about having more users moving to PHP6. I would like to know why it will work better than with php5, any thoughts? And let forget that maintaining (and develop/implement) these two modes will obviously take more time. Cheers, --Pierre --- ## #30257 — Lukas Kahwe Smith — 2007-06-19T15:19:22+00:00 URL: https://externals.io/message/30173#30257 · In reply to: #30255 Pierre wrote: > > > > […] > > Just as a last (hopefully) comment, even if nothing seemed to have an > influence, no matter how many we are to prefer a unicode only mode (so > far only you are in favour of it, maybe Andree too but I don't > remember his opinion on this topic :). > > The gain we hope to have by keeping a non unicode mode is about having > more users moving to PHP6. I would like to know why it will work > better than with php5, any thoughts? > > And let forget that maintaining (and develop/implement) these two > modes will obviously take more time. I agree, we tried out best in PHP5 to provide support for PHP4 and it seems this has not been overly successful. Why will this be any easier for PHP6? Maybe we should try a different approach. Lets not hold ourselves back. Lets break BC where it makes sense. Projects like PEAR etc. could always claim that its also PHP5 compatible without truly moving over. If we make a clean cut, this will not work anymore and instead we have an opportunity to clean up, improve performance a bit by removing unicode off hacks and let users migrate or stay. Sooner or later they will be attracted by new shiny stuff and then they will make a true effort to migrate over instead of these half migrations that happened with PHP5 "adoption". regards, Lukas --- ## #30258 — Rasmus Lerdorf — 2007-06-19T15:27:55+00:00 URL: https://externals.io/message/30173#30258 · In reply to: #30255 Pierre wrote: > > > > […] > > Just as a last (hopefully) comment, even if nothing seemed to have an > influence, no matter how many we are to prefer a unicode only mode (so > far only you are in favour of it, maybe Andree too but I don't > remember his opinion on this topic :). Uh, this was agreed upon by everyone involved in the design of the Unicode support. So saying I am the only one is extremely misleading. I may be the only one explaining why the decision was reached, but I am certainly not the only one in favour of it. > The gain we hope to have by keeping a non unicode mode is about having > more users moving to PHP6. I would like to know why it will work > better than with php5, any thoughts? By not providing it, we ensure that a large number of people will not move to PHP 6. At least by providing it we give ourselves a chance. I think if we drop it we are basically giving up and we will be maintaining 2 code bases for the next 10 years. Do we really want that? > And let forget that maintaining (and develop/implement) these two > modes will obviously take more time. More time than maintaining separate Unicode and non-Unicode code bases in difference branches? -Rasmus --- ## #30260 — Pierre — 2007-06-19T15:49:33+00:00 URL: https://externals.io/message/30173#30260 · In reply to: #30258 > Pierre wrote: > > […] > > Uh, this was agreed upon by everyone involved in the design of the > Unicode support. So saying I am the only one is extremely misleading. > I may be the only one explaining why the decision was reached, but I am > certainly not the only one in favour of it. Sorry, I did not know that there was many "externals" people involved in the unicode design Who was involved? Almost all "core" developers I asked are against (anyone having a different view, please step in). > > […] > > By not providing it, we ensure that a large number of people will not > move to PHP 6. At least by providing it we give ourselves a chance. I > think if we drop it we are basically giving up and we will be > maintaining 2 code bases for the next 10 years. Do we really want that? We maintain three branches since a couple of years, having only two is a real progress. PHP4's one is dying, it was about time. I can live with two branches and code base for php-src, I have to maintain three or more branches in many pecl extensions anyway :) > > […] > > More time than maintaining separate Unicode and non-Unicode code bases > in difference branches? Yes, code base is cleaner when the two are in two separate branches, it is easier to merge too. That's a feeling only, I never timed each merge or change :) --Pierre --- ## #30262 — Tomas Kuliavas — 2007-06-19T16:27:49+00:00 URL: https://externals.io/message/30173#30262 · In reply to: #30257 > Pierre wrote: > > […] > > I agree, we tried out best in PHP5 to provide support for PHP4 and it > seems this has not been overly successful. Why will this be any easier > for PHP6? Maybe we should try a different approach. Lets not hold > ourselves back. Lets break BC where it makes sense. Projects like PEAR > etc. could always claim that its also PHP5 compatible without truly > moving over. If we make a clean cut, this will not work anymore and > instead we have an opportunity to clean up, improve performance a bit by > removing unicode off hacks and let users migrate or stay. Sooner or > later they will be attracted by new shiny stuff and then they will make > a true effort to migrate over instead of these half migrations that > happened with PHP5 "adoption". Nope. If I can't turn off unicode_semantics, I will ask end users to use PHP5 or write manual about running two PHP versions on one host. I can't update code to work on PHP6 unicode_semantics=on, because it affects lots of functions and updates will break backwards compatibility with PHP4 and some PHP5 versions. I suspect that I won't be able to set stream encoding, because I will have to read the stream in order to know encoding of 8bit data. In some cases data is provided by third party and character set information can be incorrect. In some cases same stream can output data in different character sets. In some cases automatic charset conversions performed by PHP libraries might break verification of received data. I won't be attracted with new shiny stuff. Last shiny stuff that I can use is available in PHP 5.1.0. I don't care about Unicode support, because it breaks things. I suspect that PHP6 Unicode extension won't give me controls that I have in PHP5 and PHP4 strings. PHP6 Unicode support is not designed for international environment. It is designed for nationalized environments and allows PHP script developers to code in their native language. Code written in French, Russian, Arabic, Japanese or Chinese is not international. Only some people can read it. Only some people can see difference between ァ() and ィ(). If I have to debug code written in Japanese or Arabic, language is the main barrier in understanding the code. -- Tomas --- ## #30263 — Stefan Walk — 2007-06-19T16:49:13+00:00 URL: https://externals.io/message/30173#30263 · In reply to: #30262 > I don't care about Unicode support, because it breaks things. I suspect > that PHP6 Unicode extension won't give me controls that I have in PHP5 and > PHP4 strings. PHP6 Unicode support is not designed for international > environment. It is designed for nationalized environments and allows PHP > script developers to code in their native language. Code written in > French, Russian, Arabic, Japanese or Chinese is not international. Only > some people can read it. Only some people can see difference between ァ() > and ィ(). If I have to debug code written in Japanese or Arabic, language > is the main barrier in understanding the code. You're spreading FUD about PHP6s unicode support. Writing code in your own native language has nothing to do with the unicode support in PHP6. You can already do that in PHP4 if you use utf-8, since any sequence of codepoints > 127 translates to a byte-sequence that is a valid identifier for php. Have a look at http://www.icu-project.org/ to see what the unicode features in php6 are really about. Regards, Stefan (who also thinks that the switch is of no use, unicode_semantics should be on all the time. And at least it shouldn't be off in php.ini-dist and php.ini-recommended) --- ## #30264 — Stanislav Malyshev — 2007-06-19T17:27:26+00:00 URL: https://externals.io/message/30173#30264 · In reply to: #30262 > PHP4 strings. PHP6 Unicode support is not designed for international > environment. It is designed for nationalized environments and allows PHP > script developers to code in their native language. Code written in The goal of PHP 6 unicode support is definitely to allow internationalized development. Japanese function names are of much less importance than global support for unicode data and i18n/l10n functions - such as collation, localized formats etc. -- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com --- Next page: https://externals.io/message/30173.md?page=2