# Proposed change in json_encode+JSON_UNESCAPED_UNICODE behavior

- Thread: https://externals.io/message/90095
- Original archive: https://news-web.php.net/php.internals/90095
- Started by Eddie Kohler on 2016-01-05T14:55:24+00:00
- Messages: 3
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #90095 — Eddie Kohler — 2016-01-05T14:55:24+00:00

URL: https://externals.io/message/90095#90095

Hi,

I'm proposing a small change in the behavior of `json_encode(str,
JSON_UNESCAPED_UNICODE)` around the issue of line terminators.

The U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR
characters are allowed unescaped in JSON strings, but *not* allowed unescaped
in Javascript. This is widely considered a minor wart in the JSON specification.
<https://medium.com/joys-of-javascript/json-js-42a28471221d>

As a result, the `JSON_UNESCAPED_UNICODE` flag is dangerous to use when
generating HTML. For example, this will generate a Javascript error ("Unexpected
token ILLEGAL") in the user's browser:

```
$x = mb_convert_encoding('&#x2028;', 'UTF-8', 'HTML-ENTITIES');
echo '<script>x = ', json_encode($x, JSON_UNESCAPED_UNICODE), ';</script>';
```

The proposal is for `json_encode(..., JSON_UNESCAPED_UNICODE)` to
escape the U+2028 and U+2029 characters as \u2028 and \u2029. A new flag,
JSON_UNESCAPED_LINE_TERMINATORS, preserves the former behavior.

It's important to note that this change *only* affects the non-default
 `JSON_UNESCAPED_UNICODE` flag.

Jakub Zelenka approves of this change, which we've discussed on Github
<https://github.com/php/php-src/pull/1701>, but since it is a small change in
behavior, he asked me to email internals in case anyone objects.

Thanks all,
Eddie Kohler

---

## #90141 — Andrea Faulds — 2016-01-05T19:45:45+00:00

URL: https://externals.io/message/90095#90141 · In reply to: #90095

Hi Eddie,

Eddie Kohler wrote:
> The U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR
> characters are allowed unescaped in JSON strings, but *not* allowed unescaped
> in Javascript. This is widely considered a minor wart in the JSON specification.
> <https://medium.com/joys-of-javascript/json-js-42a28471221d>
>
> As a result, the `JSON_UNESCAPED_UNICODE` flag is dangerous to use when
> generating HTML. For example, this will generate a Javascript error ("Unexpected
> token ILLEGAL") in the user's browser:
>
> ```
> $x = mb_convert_encoding('&#x2028;', 'UTF-8', 'HTML-ENTITIES');
> echo '<script>x = ', json_encode($x, JSON_UNESCAPED_UNICODE), ';</script>';
> ```
>
> The proposal is for `json_encode(..., JSON_UNESCAPED_UNICODE)` to
> escape the U+2028 and U+2029 characters as \u2028 and \u2029. A new flag,
> JSON_UNESCAPED_LINE_TERMINATORS, preserves the former behavior.
>
> It's important to note that this change *only* affects the non-default
> `JSON_UNESCAPED_UNICODE` flag.

This sounds reasonable. I'd like to ask, though, does this mean that 
without that flag, U+2028 and U+2029 are always escaped?

Thanks.
-- 
Andrea Faulds
https://ajf.me/

---

## #90144 — Eddie Kohler — 2016-01-05T20:05:22+00:00

URL: https://externals.io/message/90095#90144 · In reply to: #90141

Yes, without the `JSON_UNESCAPED_UNICODE` flag, all characters with
Unicode values >= 0x80 are escaped. That's the default behavior.



> Hi Eddie,
>
> Eddie Kohler wrote:
> > […]
>
>
> This sounds reasonable. I'd like to ask, though, does this mean that without
> that flag, U+2028 and U+2029 are always escaped?
>
> Thanks.
> --
> Andrea Faulds
> https://ajf.me/