Removing basic types from our JSON parser

16 years ago by mike — view source — reply

unread

Fri, Dec 12, 2008 at 7:50 PM, Scott MacVicar scott@macvicar.net wrote:

The problem here is that none of the other JSON parsers in any other
language support this and more importantly the browsers [2] which are now
adding native JSON support. Users are frequently expecting the result from a
json_encode in PHP to just work with JSON.parse() on the client side. [3]

I have a patch for this that implements the RFC exactly and makes secure
json_encode only works with objects and arrays.

definately +1

we use json both encoding and decoding to interact with javascript and
ensuring that PHP's built-in json is standards compliant is an
absolute must. -please- make sure it works both encoding php -> js,
decoding js -> php for all relevant types in both languages. thank you
:)

16 years ago by Mark Karpeles — view source — reply

unread

Le samedi 13 décembre 2008 à 03:50 +0000, Scott MacVicar a écrit :

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this allows
one to use json_encode / json_decode on any of our scalar types. Omar
correctly identified #38680 as not a bug but it appears that Ilia
added support for this anyway violating the RFC [1]. Maybe there was a
reason for this but I'm not sure why?

The reason was to "make json_decode() able to decode anything encoded by
json_encode()". I believe many people may be depending on this (I saw a
few codes).

If json_decode() is made to only accept arrays/struct, the same should
be done to json_encode().

For reference I saw people use json_encode() to pass a string to
javascript into their page while avoiding bugs/XSS with stuff like
</script>.

var foo = <?=json_encode($my_string)?>;
... (yes, they maybe heared somewhere that JSON is not javascript, I
told 'em too).

I also saw people using json_encode/json_decode as an alternate for
serialize/unserialize.

Also, reading [2], I see about stringify "If value is an object or
array, the structure will be visited recursively to determine the
serialization of each membr or element.", this seems to assume that
"value" can be something else than "an object or array".

At least on Firefox 3.2, it is not the case.

The problem here is that none of the other JSON parsers in any other
language support this and more importantly the browsers [2] which are
now adding native JSON support. Users are frequently expecting the
result from a json_encode in PHP to just work with JSON.parse() on the
client side. [3]

I did some tests with firefox minefield (3.2pre) :

javascript:alert(JSON.stringify({foo: "bar"}));
=> {"foo":"bar"}

javascript:alert(JSON.stringify(true));
=> Invalid argument

javascript:alert(JSON.stringify("hello world"));
=> Invalid argument

javascript:alert(JSON.stringify(["hello world"]));
=> ["hello world"]

javascript:alert(JSON.parse("["hello world"]"));
=> hello world
(bug? or alert() behaviour when receiving an array?)

javascript:alert(JSON.parse(""hello world""));
=> Error parsing JSON

javascript:alert(JSON.parse("null"));
=> Error parsing JSON

I have a patch for this that implements the RFC exactly and makes
secure json_encode only works with objects and arrays.

This would be a change for 5.3+ and we should just document in 5.2
that what we did was a bad idea.

I'm +0 on this, because it will break PHP code on one side, but will
break on browser stuff on the other side.

I guess making 5.3 throwing E_WARNING (or E_DEPRECATED, if someone reads
it) and removing this support in HEAD could be the default behaviour
based on how we handled most features we wanted to remove when people
started relying on those.

Anyway, maintaining two versions of the JSON parser is not something we
want (I believe) so this should be removed as soon as possible. I'm just
not sure about breaking this "features" this fast.

Mark

[1] - http://www.ietf.org/rfc/rfc4627.txt?number=4627
[2] - http://wiki.ecmascript.org/doku.php?id=es3.1:json_support
[3] - https://developer.mozilla.org/en/Using_JSON_in_Firefox

16 years ago by Rasmus Lerdorf — view source — reply

unread

Mark Karpeles wrote:

Le samedi 13 décembre 2008 à 03:50 +0000, Scott MacVicar a écrit :

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this allows
one to use json_encode / json_decode on any of our scalar types. Omar
correctly identified #38680 as not a bug but it appears that Ilia
added support for this anyway violating the RFC [1]. Maybe there was a
reason for this but I'm not sure why?

The reason was to "make json_decode() able to decode anything encoded by
json_encode()". I believe many people may be depending on this (I saw a
few codes).

If json_decode() is made to only accept arrays/struct, the same should
be done to json_encode().

For reference I saw people use json_encode() to pass a string to
javascript into their page while avoiding bugs/XSS with stuff like
</script>.

var foo = <?=json_encode($my_string)?>;
... (yes, they maybe heared somewhere that JSON is not javascript, I
told 'em too).

I also saw people using json_encode/json_decode as an alternate for
serialize/unserialize.

Also, reading [2], I see about stringify "If value is an object or
array, the structure will be visited recursively to determine the
serialization of each membr or element.", this seems to assume that
"value" can be something else than "an object or array".

At least on Firefox 3.2, it is not the case.

I think it does more than assume that. The spec specifically says that
the value can be any Javascript value in the prototype:

JSON.stringify(value, replacer, space)

value any JavaScript value, usually an object or array.

replacer an optional parameter that determines how object
values are stringified for objects without a toJSON
method. It can be a function or an array.

space an optional parameter that specifies the indentation
of nested structures. If it is omitted, the text will
be packed without extra whitespace. If it is a number,
it will specify the number of spaces to indent at each
level. If it is a string (such as '\t' or ' '),
it contains the characters used to indent at each level.

I think we should follow the spec, not individual browser implementations.

-Rasmus

16 years ago by Uwe Schindler — view source — reply

unread

For reference I saw people use json_encode() to pass a string to
javascript into their page while avoiding bugs/XSS with stuff like
</script>.

var foo = <?=json_encode($my_string)?>;
... (yes, they maybe heared somewhere that JSON is not javascript, I
told 'em too).

This is not correct. JSON is valid JavaScript, but the other way round is
not correct. Not every Javascript is JSON. So it is perfectly legal, to do
this.

I think, this is really cool, using json_encode() as something like
"htmlspecialchars" for correctly encoding strings inside JS for security. I
use it, too.

To the problem with basic types: I like the way of maybe giving a option to
the function. If you want to be standards conformant, you have to remove the
encoding parameter (which is also there), too. JSON has to be UTF-8, if you
want to be conform to the specs. But for the example above (secure encoding
of JS strings with json_encode), it is perfectly legal to use another
encoding (in the above case, the encoding of the HTML page).

Uwe

16 years ago by Stan Vassilev | FM — view source — reply

unread

var foo = <?=json_encode($my_string)?>;
... (yes, they maybe heared somewhere that JSON is not javascript, I
told 'em too).

JSON is a strict subset of JavaScript, and the above scenario is something I
also do often.

Since PHP generates web pages, any time I need to generate a JS literal
json_encode() is the natural equivalent of var_export(.., true) for PHP
literals.

Since it may not match the RFC apparently, I suggest a flag as some other
people said:

function json_encode($var, $allowScalar = false) {}

This way we gain best of both worlds without disrupting either specs nor
existing workflows.

Regards,
Stan Vassilev

16 years ago by Rasmus Lerdorf — view source — reply

unread

Stan Vassilev | FM wrote:

var foo = <?=json_encode($my_string)?>;
... (yes, they maybe heared somewhere that JSON is not javascript, I
told 'em too).

JSON is a strict subset of JavaScript, and the above scenario is
something I also do often.

Since PHP generates web pages, any time I need to generate a JS literal
json_encode() is the natural equivalent of var_export(.., true) for PHP
literals.

Since it may not match the RFC apparently, I suggest a flag as some
other people said:

function json_encode($var, $allowScalar = false) {}

I'm not sure if that is the right way of looking at it though. I think
the only question is how to represent the scalar types, not whether they
should be allowed. As far as I am concerned it is obvious that they be
allowed and the JSON spec is quite explicit that JSON can be used to
represent scalar types:

JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays).

The only question is how to serialize these into a JSON-text string.
Again, from the RFC:

A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.

A JSON text is a serialized object or array.

This last statement is the issue. When serializing a basic type into a
JSON text string, the RFC says we need to wrap it inside an array or
object. So, if we are going to add an option to json_encode(), that's
what the option should do. Specify whether, and perhaps how, to wrap
basic types, not whether to allow or disallow them.

I'll talk to Douglas on Monday if he is in the office and get his opinion.

-Rasmus

16 years ago by moo.tinys — view source — reply

unread

I'm not sure if that is the right way of looking at it though. I think
the only question is how to represent the scalar types, not whether they
should be allowed. As far as I am concerned it is obvious that they be
allowed and the JSON spec is quite explicit that JSON can be used to
represent scalar types:

JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays).

The only question is how to serialize these into a JSON-text string.
Again, from the RFC:

A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.

A JSON text is a serialized object or array.

This last statement is the issue. When serializing a basic type into a
JSON text string, the RFC says we need to wrap it inside an array or
object. So, if we are going to add an option to json_encode(), that's
what the option should do. Specify whether, and perhaps how, to wrap
basic types, not whether to allow or disallow them.

I'll talk to Douglas on Monday if he is in the office and get his opinion.

-Rasmus
cool.
let's see how :D. Crockford explain it :D
but before he can react on this thread, i guess Rasmus is right
understanding the RFC.
json force your encoder to output the following in encoded form

array: "multiple of unnamed" values
object: "multiple of named" values
so it's right to "wrap it" in object or array (u can say "remove basic types")

json_encode(array(1)); => [1]
json_encode(array(1, 2, 3, "1")); => [1,2,3,"1"]

json_encode(array('value1' => 1)); => {"value1":1}
json_encode(array('a'=>1, 'b'=>2, 'c'=>3, 'd'=>"1")); => {"a":1,
"b":2,"c":3,"d":"1"}

i'd prefer 2 new functions: js_encode (or js_export?) js_decode which
support basic types instead of having an extra optional parameter

16 years ago by mike — view source — reply

unread

Sorry for top posting. I see no reason why php should not interop with
javascript 100% as it should. I do not see a reason to create new
functions, parameters, etc. The existing should be expected to both
encode and decode json with javascript properly. If there is a
mismatch right now, then it should be fixed.

I'm not sure if that is the right way of looking at it though. I
think
the only question is how to represent the scalar types, not whether
they
should be allowed. As far as I am concerned it is obvious that
they be
allowed and the JSON spec is quite explicit that JSON can be used to
represent scalar types:

JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays).

The only question is how to serialize these into a JSON-text string.
Again, from the RFC:

A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.

A JSON text is a serialized object or array.

This last statement is the issue. When serializing a basic type
into a
JSON text string, the RFC says we need to wrap it inside an array or
object. So, if we are going to add an option to json_encode(),
that's
what the option should do. Specify whether, and perhaps how, to wrap
basic types, not whether to allow or disallow them.

I'll talk to Douglas on Monday if he is in the office and get his
opinion.

-Rasmus
cool.
let's see how :D. Crockford explain it :D
but before he can react on this thread, i guess Rasmus is right
understanding the RFC.
json force your encoder to output the following in encoded form

array: "multiple of unnamed" values

object: "multiple of named" values
so it's right to "wrap it" in object or array (u can say "remove
basic types")

json_encode(array(1)); => [1]
json_encode(array(1, 2, 3, "1")); => [1,2,3,"1"]

json_encode(array('value1' => 1)); => {"value1":1}
json_encode(array('a'=>1, 'b'=>2, 'c'=>3, 'd'=>"1")); => {"a":1,
"b":2,"c":3,"d":"1"}

i'd prefer 2 new functions: js_encode (or js_export?) js_decode which
support basic types instead of having an extra optional parameter

16 years ago by Rasmus Lerdorf — view source — reply

unread

mike wrote:

Sorry for top posting. I see no reason why php should not interop with
javascript 100% as it should. I do not see a reason to create new
functions, parameters, etc. The existing should be expected to both
encode and decode json with javascript properly. If there is a mismatch
right now, then it should be fixed.

There is no mismatch with Javascript now at all. We spit out perfectly
valid Javascript. There is only a slight mismatch with the JSON RFC
which defines a stricter subset of Javascript than we are using.

-Rasmus

16 years ago by mike — view source — reply

unread

I can't give specifics (I'm mobile and I don't remember where the code
would be anyway) but we had an issue with php and javascript exchange
via json where the php output using json_encode and we had to wrap it
with additional braces for it to be usable. Sorry for being vague,
this thread reminded me of that which is why I wanted to voice off.

Thanks for the clarification (and thank you so much for PHP :))

mike wrote:

Sorry for top posting. I see no reason why php should not interop
with
javascript 100% as it should. I do not see a reason to create new
functions, parameters, etc. The existing should be expected to both
encode and decode json with javascript properly. If there is a
mismatch
right now, then it should be fixed.

There is no mismatch with Javascript now at all. We spit out
perfectly
valid Javascript. There is only a slight mismatch with the JSON RFC
which defines a stricter subset of Javascript than we are using.

-Rasmus

16 years ago by Rasmus Lerdorf — view source — reply

unread

Yes, but that would be going to a json_parse thing somewhere, not pure
Javascript.

-Rasmus

mike wrote:

I can't give specifics (I'm mobile and I don't remember where the code
would be anyway) but we had an issue with php and javascript exchange
via json where the php output using json_encode and we had to wrap it
with additional braces for it to be usable. Sorry for being vague, this
thread reminded me of that which is why I wanted to voice off.

Thanks for the clarification (and thank you so much for PHP :))

mike wrote:

Sorry for top posting. I see no reason why php should not interop with
javascript 100% as it should. I do not see a reason to create new
functions, parameters, etc. The existing should be expected to both
encode and decode json with javascript properly. If there is a mismatch
right now, then it should be fixed.

There is no mismatch with Javascript now at all. We spit out perfectly
valid Javascript. There is only a slight mismatch with the JSON RFC
which defines a stricter subset of Javascript than we are using.

-Rasmus

16 years ago by mike — view source — reply

unread

Do you mean a json_parse in javascript (like a better eval() I
assume)? Thats what we were doing. Json is not supposed to be pure
javascript either, but a serialized representation of variables like
php's serialize(). So I would not expect a json string to be usable
without using eval or something else in javascript first. We had to
modify the php json_encode output to be wrapped with a couple extra
braces for it to be able to be consumed properly on javascript (or
maybe it was the other way around) - I just want to make sure we won't
have to make special things like that in the future.

Yes, but that would be going to a json_parse thing somewhere, not pure
Javascript.

-Rasmus

mike wrote:

I can't give specifics (I'm mobile and I don't remember where the
code
would be anyway) but we had an issue with php and javascript exchange
via json where the php output using json_encode and we had to wrap it
with additional braces for it to be usable. Sorry for being vague,
this
thread reminded me of that which is why I wanted to voice off.

Thanks for the clarification (and thank you so much for PHP :))

On Dec 13, 2008, at 10:39 PM, Rasmus Lerdorf rasmus@lerdorf.com
wrote:

mike wrote:

Sorry for top posting. I see no reason why php should not interop
with
javascript 100% as it should. I do not see a reason to create new
functions, parameters, etc. The existing should be expected to both
encode and decode json with javascript properly. If there is a
mismatch
right now, then it should be fixed.

There is no mismatch with Javascript now at all. We spit out
perfectly
valid Javascript. There is only a slight mismatch with the JSON RFC
which defines a stricter subset of Javascript than we are using.

-Rasmus

16 years ago by Rasmus Lerdorf — view source — reply

unread

mike wrote:

Do you mean a json_parse in javascript (like a better eval() I assume)?
Thats what we were doing. Json is not supposed to be pure javascript
either, but a serialized representation of variables like php's
serialize(). So I would not expect a json string to be usable without
using eval or something else in javascript first. We had to modify the
php json_encode output to be wrapped with a couple extra braces for it
to be able to be consumed properly on javascript (or maybe it was the
other way around) - I just want to make sure we won't have to make
special things like that in the future.

Eh? Read what you wrote there. If json wasn't pure javascript, how in
the world would eval() work on it?

Douglas Crockford likes to say that he didn't invent json, he discovered
it. It was already there because it is just javascript. He just
started using it.

will always work. The only question is what sort of variable foo will
end up being. The RFC says we have to wrap basic types in an array or
object, while currently we let the basic types through without the wrapper.

-Rasmus

16 years ago by mike — view source — reply

unread

Eh? Read what you wrote there. If json wasn't pure javascript, how in
the world would eval() work on it?

Sorry. I guess I meant it didn't execute by itself, but needed to be
interpreted using something like eval and/or thrown into a variable
before it was useful. Otherwise it's just text.

I'm not sure why we had to do that workaround I previously mentioned,
and I cannot find a repository with that old code, but nowadays
json_encode seems to work seamlessly for all the data we exchange
(which are strings, ints, arrays, no objects, possibly not even
booleans) and just wanted to voice off if there was any possible
incompatibility that may be introduced here.

I'll go back in my corner now :)

16 years ago by Rasmus Lerdorf — view source — reply

unread

mike wrote:

Eh? Read what you wrote there. If json wasn't pure javascript, how in
the world would eval() work on it?

Sorry. I guess I meant it didn't execute by itself, but needed to be
interpreted using something like eval and/or thrown into a variable
before it was useful. Otherwise it's just text.

I'm not sure why we had to do that workaround I previously mentioned,
and I cannot find a repository with that old code, but nowadays
json_encode seems to work seamlessly for all the data we exchange
(which are strings, ints, arrays, no objects, possibly not even
booleans) and just wanted to voice off if there was any possible
incompatibility that may be introduced here.

I'll go back in my corner now :)

I thought I explained that a few times now. JSON is defined in the RFC
as a subset of Javascript, so a "JSON Parser" doesn't need to have all
the capabilities of the javascript parser. Things break if you pass the
output of json_encode() to a JSON parser that follows the RFC to the letter.

When we do json_encode(123) we spit out just: 123
or json_encode("abc") we produce just: "abc"

this is obviously valid Javascript so both eval and direct injection
into a script block (which is the same thing) will work fine. But the
RFC says we should be wrapping either array or object notation around
it. As in [123] and ["abc"] to make it valid JSON. Direct injection
into <script>var foo = <?php echo json_encode("abc")?>;</script> will
still work, of course, the only change is that foo now becomes an array
with a single string element as opposed to just the string itself, so
making this change will break existing javascript that relies on PHP
producing unwrapped basic types.

Now, in most cases when you are passing stuff around via json, you don't
do that for a single value, so in most cases you end up passing an array
anyway, and in that case nothing changes.

-Rasmus

16 years ago by Uwe Schindler — view source — reply

unread

<script> var foo = <?php echo json_encode($foo)?>; </script>
will always work. The only question is what sort of variable foo will
end up being. The RFC says we have to wrap basic types in an array or
object, while currently we let the basic types through without the
wrapper.

He problem her eis: A lot of people use in a way and assume, that a basic
type is serialized as a basic type in JSON/JS. If we change this, it could
break a lot of apps.

And the above is really nice to encode PHP strings to a secure JS variant.
It is hard to do with addslashes etc. So I think, we should not change the
way it works currently.

I think in the docs for json_encode() should be a warning: "JSON officially
only supports Objects and Arrays as outer/top-level/... type for JSON
serializations. So the argument of json_encode must be a PHP object or array
to be fully JSON conformant. If you provide a simple type (string, number,
Boolean), json_encode will generate a valid java script, but some conformant
JSON parser may not be able to read it." How about this?

Uwe

16 years ago by David Grudl — view source — reply

unread

-------- Původní zpráva --------
Předmět: [PHP-DEV] Removing basic types from our JSON parser
Od: thetaphi@php.net (Uwe Schindler)
Komu:
Datum: 14.12.2008 10:40

<script> var foo = <?php echo json_encode($foo)?>; </script>
will always work. The only question is what sort of variable foo will
end up being. The RFC says we have to wrap basic types in an array or
object, while currently we let the basic types through without the
wrapper.

He problem her eis: A lot of people use in a way and assume, that a basic
type is serialized as a basic type in JSON/JS. If we change this, it could
break a lot of apps.

+1

In PHP there is no function for escaping JS strings and json_encode is
useful compensation. I use it this way very often.

David Grudl

16 years ago by karpeles@ookoo.org — view source — reply

unread

Le dimanche 14 décembre 2008 à 00:39 -0800, Rasmus Lerdorf a écrit :

<script> var foo = <?php echo json_encode($foo)?>; </script>
will always work. The only question is what sort of variable foo will
end up being. The RFC says we have to wrap basic types in an array or
object, while currently we let the basic types through without the wrapper.

I believe that for most people it will feel more natural if foo's type
is not changed to an object or an array.

However I notice with current Firefox (I don't have MSIE, and I'm too
lazy to read the whole ECMA Script specs) that an array with only one
element is represented by JavaScript the same way.

([ "foo" ]).toString().charAt(1)
"foo".toString().charAt(1)

document.write([ "foo" ])
document.write("foo")

Both works the same in each case. Even alert() will display the value of
the first element.

IMO it might be possible to wrap values into arrays without breaking too
much JS code (bools and other scalars will still not work as expected,
and of course an array will not have strings functions) as I think most
uses of json_encode() are for strings escaping.

Mark

16 years ago by Ilia Alshanetsky — view source — reply

unread

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this allows
one to use json_encode / json_decode on any of our scalar types.
Omar correctly identified #38680 as not a bug but it appears that
Ilia added support for this anyway violating the RFC [1]. Maybe
there was a reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to allow
encoding of basic types via json_encode, it reduces the code when
communicating to/form JavaScript to avoid type detection. While it
does violate the RFC IMO the convenience of the feature is definitely
worth it.

Ilia Alshanetsky

16 years ago by Lukas Kahwe Smith — view source — reply

unread

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this
allows one to use json_encode / json_decode on any of our scalar
types. Omar correctly identified #38680 as not a bug but it appears
that Ilia added support for this anyway violating the RFC [1].
Maybe there was a reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to allow
encoding of basic types via json_encode, it reduces the code when
communicating to/form JavaScript to avoid type detection. While it
does violate the RFC IMO the convenience of the feature is
definitely worth it.

well seems to me like its safer to stick with RFC as the default. if
at all this should be optional. JSON is a standard and this requires
everybody to follow it. so using something else should be a conscious
decision that is also visible during code review.

regards,
Lukas Kahwe Smith
mls@pooteeweet.org

16 years ago by Ilia Alshanetsky — view source — reply

unread

Lukas,

Reading the spec it clearly says in the pre-amble that JSON can
represent both primitive and complex data types

From RFC:
"JSON can represent four primitive types (strings, numbers, booleans,
and null) and two structured types (objects and arrays)."

The RFC also says that when it comes to parsers anything that matches
the JSON grammar is valid, it even goes further that accepting
"extensions" on top of the grammar is fine as well

RFC:
"A JSON parser MUST accept all texts that conform to the JSON grammar.
A JSON parser MAY accept non-JSON forms or extensions."

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this
allows one to use json_encode / json_decode on any of our scalar
types. Omar correctly identified #38680 as not a bug but it
appears that Ilia added support for this anyway violating the RFC
[1]. Maybe there was a reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to
allow encoding of basic types via json_encode, it reduces the code
when communicating to/form JavaScript to avoid type detection.
While it does violate the RFC IMO the convenience of the feature is
definitely worth it.

well seems to me like its safer to stick with RFC as the default. if
at all this should be optional. JSON is a standard and this requires
everybody to follow it. so using something else should be a
conscious decision that is also visible during code review.

regards,
Lukas Kahwe Smith
mls@pooteeweet.org

Ilia Alshanetsky

16 years ago by Scott MacVicar — view source — reply

unread

The JSON grammar explicitly says that it's not supported.

JSON Grammar
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
JSON-text = object / array

As you can see the base type should be object / array.
To further test this I checked the python, ruby and perl
implementations and none of them support primitive types that aren't
in an array or object.
I then went and looked at the various JavaScript libraries and the
browser implementations and they all rejected the primitive types and
threw an error when trying to decode it.
Scott

Lukas,

Reading the spec it clearly says in the pre-amble that JSON can
represent both primitive and complex data types

From RFC:
"JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays)."

The RFC also says that when it comes to parsers anything that
matches the JSON grammar is valid, it even goes further that
accepting "extensions" on top of the grammar is fine as well

RFC:
"A JSON parser MUST accept all texts that conform to the JSON
grammar. A JSON parser MAY accept non-JSON forms or extensions."

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this
allows one to use json_encode / json_decode on any of our scalar
types. Omar correctly identified #38680 as not a bug but it
appears that Ilia added support for this anyway violating the RFC
[1]. Maybe there was a reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to
allow encoding of basic types via json_encode, it reduces the code
when communicating to/form JavaScript to avoid type detection.
While it does violate the RFC IMO the convenience of the feature
is definitely worth it.

well seems to me like its safer to stick with RFC as the default.
if at all this should be optional. JSON is a standard and this
requires everybody to follow it. so using something else should be
a conscious decision that is also visible during code review.

regards,
Lukas Kahwe Smith
mls@pooteeweet.org

Ilia Alshanetsky

16 years ago by Ilia Alshanetsky — view source — reply

unread

Please re-read my comments again. First of all RFC says that basic
types are support even identifies them by name, secondly it says you
MAY extend the syntax. What other languages do is fine and up to them
to implement, unless you want to make a case for making PHP a copy of
[language of your choice] then your argument has little merit IMO.

The JSON grammar explicitly says that it's not supported.

JSON Grammar
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
JSON-text = object / array

As you can see the base type should be object / array.
To further test this I checked the python, ruby and perl
implementations and none of them support primitive types that aren't
in an array or object.
I then went and looked at the various JavaScript libraries and the
browser implementations and they all rejected the primitive types
and threw an error when trying to decode it.
Scott

Lukas,

Reading the spec it clearly says in the pre-amble that JSON can
represent both primitive and complex data types

From RFC:
"JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays)."

The RFC also says that when it comes to parsers anything that
matches the JSON grammar is valid, it even goes further that
accepting "extensions" on top of the grammar is fine as well

RFC:
"A JSON parser MUST accept all texts that conform to the JSON
grammar. A JSON parser MAY accept non-JSON forms or extensions."

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this
allows one to use json_encode / json_decode on any of our scalar
types. Omar correctly identified #38680 as not a bug but it
appears that Ilia added support for this anyway violating the
RFC [1]. Maybe there was a reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to
allow encoding of basic types via json_encode, it reduces the
code when communicating to/form JavaScript to avoid type
detection. While it does violate the RFC IMO the convenience of
the feature is definitely worth it.

well seems to me like its safer to stick with RFC as the default.
if at all this should be optional. JSON is a standard and this
requires everybody to follow it. so using something else should be
a conscious decision that is also visible during code review.

regards,
Lukas Kahwe Smith
mls@pooteeweet.org

Ilia Alshanetsky

Ilia Alshanetsky

16 years ago by Rob Richards — view source — reply

unread

It also says:

Generators

A JSON generator produces JSON text. The resulting text MUST
strictly conform to the JSON grammar.

It says "JSON text" not JSON-text. There is no errata about this, so
intended or not, PHP follows the spec.

Rob

Scott MacVicar wrote:

The JSON grammar explicitly says that it's not supported.

JSON Grammar
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
JSON-text = object / array

As you can see the base type should be object / array.
To further test this I checked the python, ruby and perl
implementations and none of them support primitive types that aren't
in an array or object.
I then went and looked at the various JavaScript libraries and the
browser implementations and they all rejected the primitive types and
threw an error when trying to decode it.
Scott

Lukas,

Reading the spec it clearly says in the pre-amble that JSON can
represent both primitive and complex data types

From RFC:
"JSON can represent four primitive types (strings, numbers, booleans,
and null) and two structured types (objects and arrays)."

The RFC also says that when it comes to parsers anything that matches
the JSON grammar is valid, it even goes further that accepting
"extensions" on top of the grammar is fine as well

RFC:
"A JSON parser MUST accept all texts that conform to the JSON
grammar. A JSON parser MAY accept non-JSON forms or extensions."

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this
allows one to use json_encode / json_decode on any of our scalar
types. Omar correctly identified #38680 as not a bug but it
appears that Ilia added support for this anyway violating the RFC
[1]. Maybe there was a reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to
allow encoding of basic types via json_encode, it reduces the code
when communicating to/form JavaScript to avoid type detection.
While it does violate the RFC IMO the convenience of the feature is
definitely worth it.

well seems to me like its safer to stick with RFC as the default. if
at all this should be optional. JSON is a standard and this requires
everybody to follow it. so using something else should be a
conscious decision that is also visible during code review.

regards,
Lukas Kahwe Smith
mls@pooteeweet.org

Ilia Alshanetsky

16 years ago by Rasmus Lerdorf — view source — reply

unread

Lukas Kahwe Smith wrote:

Hi All,

Basic types were added to our JSON decoder to PHP 5.2.1, this allows
one to use json_encode / json_decode on any of our scalar types. Omar
correctly identified #38680 as not a bug but it appears that Ilia
added support for this anyway violating the RFC [1]. Maybe there was
a reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to allow
encoding of basic types via json_encode, it reduces the code when
communicating to/form JavaScript to avoid type detection. While it
does violate the RFC IMO the convenience of the feature is definitely
worth it.

well seems to me like its safer to stick with RFC as the default. if at
all this should be optional. JSON is a standard and this requires
everybody to follow it. so using something else should be a conscious
decision that is also visible during code review.

I still don't see where in the RFC it says that only arrays and
objects can be represented.

I can bring forth the power of Douglas Crockford (who sits next to me at
Yahoo) to put an end to this one on Monday.

-Rasmus

16 years ago by Christian Seiler — view source — reply

unread

Hi,

Basic types were added to our JSON decoder to PHP 5.2.1, this allows
one to use json_encode / json_decode on any of our scalar types. Omar
correctly identified #38680 as not a bug but it appears that Ilia
added support for this anyway violating the RFC [1]. Maybe there was a
reason for this but I'm not sure why?

PHP is typeless language, IMO its conductive to this design to allow
encoding of basic types via json_encode, it reduces the code when
communicating to/form JavaScript to avoid type detection. While it does
violate the RFC IMO the convenience of the feature is definitely worth it.

I agree that it's convenient for exporting variables to Javascript. On
the other hand, one should always be lenient when accepting input but
strict when generating output. Having json_encode encode non-arrays and
non-objects by default will probably allow some people to shoot
themselves in the foot when interfacing with other JSON applications.

Thus I'd propose the following:

json_encode already posesses an $options parameter that allows one to
specify certain options for encoding as a bitmask. My proposal would be
to add another option PHP_JSON_ENCODE_BASIC to allow the encoding basic
types.

When set, json_encode("hi") and json_encode(42) will work without any
problem.

When not set, I propose the following behaviour when passing a string or
integer etc.:

PHP 5.3: An E_DEPRECATED-Warning which mentions the new option but
still return the result.
PHP 6: Return false or whatever it currently returns when encoding
wasn't possible and throw an E_WARNING.

Regards,
Christian