# Unicode progress [Was: unicode.semantics adinfinitum]

- Thread: https://externals.io/message/37869
- Original archive: https://news-web.php.net/php.internals/37869
- Started by Steph Fox on 2008-05-25T19:12:41+00:00
- Messages: 1
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #37869 — Steph Fox — 2008-05-25T19:12:41+00:00

URL: https://externals.io/message/37869#37869

Hi again Johannes,

Last one on this subject I promise...

> If the script encoding doesn't match up with whatever's set in INI, I 
> don't get as far as that stuff anyway:
>
> Warning: Illegal or truncated character in input: offset 0, state=0 in 
> C:\sandbox\php-src\Debug_TS\help.php on line 5

You were right to point out that runtime encoding isn't reliable. (Now I 
finally got the thing working in scripts here - there's something invisible 
going on with UTF-8 somewhere down the line that had me confused for a 
while.)

A call to zend_unicode_to_string_ex() that explicitly passes the fallback 
converter should be good enough:

      } else if (c == 'S' && Z_TYPE_PP(arg) != IS_NULL /* `NULL` is ok */) {
       UErrorCode status = U_ZERO_ERROR;

       zend_unicode_to_string_ex(UG(fallback_encoding_conv), p, pl, 
Z_USTRVAL_PP(arg), Z_USTRLEN_PP(arg), &status);

       if (U_FAILURE(status)) {
        efree(p);
        return "strictly a binary string";
       }
       break;
      }

It doesn't make sense to set unicode.fallback_encoding at script level 
because - as I wrote earlier - the parser may respond differently depending 
on its value. If your INI fallback setting clashes with your script encoding 
you *can't* override it locally. That being the case, I don't see how a 
conversion that uses the fallback encoding can be wrong here.

- Steph