JSON float number as string

Hi all,

Hi Jakub,

I would like to add a new option to JSON for dealing with large floats.
The
use case is mainly for decoder but can be used for encoder as well.

JSON_FLOAT_AS_STRING
decode: all float values will be decoded as string

• It's often an issue for very large float values with many fractional
  digits that are coming from platforms that support larger float
  representation than double. In that case the conversion is lost and there
  is no way how to get it back (see http://bugs.php.net/68456 [pls ignore
  my
  initial dump comments when I didn't get the issue :)] and an example of
  the
  lost precision here http://3v4l.org/80iCh ). Converting the value to
  string
  keep the precision and resolves the problem.

encode: all float values will be encoded as string

• re-using the constant for encoder makes sense if PHP creates JSON for
  platform that support lower float type (e.g. C float) and the precision
  loss is not acceptable

I think that this is more a bugfix as the precision is lost without any
way
how to get it back (except pre-processing json string with regular
expression). I would like to add it to 5.6.x if there are no objections?

Converting int/float to PHP native type is obvious bug.
We must consider compatibility for released versions, so
+1 for adding JSON_FLOAT_AS_STRING option.

"int" should be fixed also.
http://3v4l.org/95dHM

So option may be JSON_SCALAR_AS_STRING or
additional JSON_INT_AS_STRING.

PHP7 should have string scalars by default. IMHO.

I didn't notice this bug before. This bug make me worry that new PHP7
type hint will create a lot of type casting bugs because even C programmer
makes this kind of design mistake...

If it's possible, coercive STH for weak type hint mode will help to reduce
type casting bugs. IMHO.

Same effects but totally unrelated topics. All functions dealing with
large external numbers had the same issues, since ever. It has nothing
to do with STH.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

Hi Jakub,

"int" should be fixed also.
http://3v4l.org/95dHM

We have already fix for this: JSON_BIGINT_AS_STRING (
http://3v4l.org/vYXUk )

Excellent.

So option may be JSON_SCALAR_AS_STRING or
additional JSON_INT_AS_STRING.

I was actually thinking about JSON_INT_AS_STRING anyway as it might be
useful for type consistency in decoding (when decoding number that are
close to bigint limit) and allowing safe encoding from 64bit to 32bit.
However this is a bit more controversial so I plan to open a new thread
about it.

There are too many options (4 options to be exact) for JSON to work safely
under HTML context currently. If user would not like to loose int/float
information, 6 options are needed. Number of options are better to be
reduced, so big +1 for string raw data by default. It may be better to have

JSON_HEX_HTML_CHARS = JSON_HEX_TAG | JSON_HEX_AMP | JSON_HEX_APOS |
JSON_HEX_QUOT

also.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

Hi!

The encoding was just about re-using it. I wouldn't probably propose
such constant if it was just for encoding (the main purpose is decoding
though). I just thought that it could be a good idea to have some usage
for encoder if it's added. It seemed to me better than just ignore it
completely for encoder. What do you think?

I'd have encoder to ignore it.

I didn't mean it for json_encode ( apology for that as it might have
seemed that is related just to json_encode). I meant it just as "a sort
of" bug for json_decode as we loose information without giving user any
way how to prevent it or at least some note in documentation about that.

It's not actually a bug - as I said, nobody who knows anything about how
computers do numbers expects exact representation of floats. That said,
ability to somehow accept numbers which don't fit current precision
may be useful in some corner cases, especially when you are
interoperating with systems with different number sizes. As the old
principle goes, be liberal in what you accept and conservative in what
you produce. json_decode() option would fit this principle well.

--
Stas Malyshev
smalyshev@gmail.com

Hi Jakub,

Could you add "json_encode" type hint also?

Large int/float must be treated as string or GMP object (we don't have
GMP float yet, though)

Currently, there is no way to encode large "numeric" JSON data correctly

when it is stored as
string.

string json_encode ( mixed $value [, int $options = 0 [, int $depth = 512
[, array $type_spec = NULL]]] )

where $type_spec specifies correct data representation. i.e. numeric,
string, bool

json_encode() type hint should work like Zeev's type hint.
i.e. No data type conversion, but validate its content.

Again this is a bit off-topic. I think that it would be good if you could
create a bug report for this. It would be great to add an example how the
$type_spec should work (e.g. if you mean something like
http://json-schema.org/examples.html )

Yes.
We need something like this to make jason_encode() work as it should.

FR is added.
https://bugs.php.net/bug.php?id=69422

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

Hi,

On Tue, Apr 14, 2015 at 7:33 AM, Alexey Zakhlestin indeyets@gmail.com
wrote:

Feels a bit hackish
I think it is possible to introduce an overall better solution

We can expose result of json-tokenizing as a tree of objects:

JSON\Object
JSON\Array
JSON\String
JSON\Number
JSON\False
JSON\True
JSON\Null

then, it would be possible to introduce any number of experimental
userland implementations like the one proposed here

I decided to elaborate

<?php
namespace JSON;

abstract class Value
{
    public function toJson();
    public function toNative();  // this would do, whatever

json_decode does
}

class Object extends Value implements \ArrayAccess, \IteratorAggregate
{
}

class Array extends Value implements \ArrayAccess, \IteratorAggregate
{
}

abstract class Literal extends Value
{
    public function toString();
    public function __toString()
    {
        return $this->toString();
    }
}

class String extends Literal
{
}

class Number extends Literal
{
    public function toInt();
    public function toFloat();
}

class False extends Literal
{
}

class True extends Literal
{
}

class Null extends Literal
{
}

So, in case of Number, there would be a way to get the value the way it
was stored in document using toString(), use toInt() or toFloat() to get a
value with possible precision loss or use toNative(), which would be
“smart” and return int or float using the same logic, which json_decode()
uses now.

It would work the other way round too. Object::toJson() would return
json-representation of the tree.

This could of course can't be introduced as a default (at least not in 7)
as it's a huge BC break. I don't like introducing too many new flags either
(it might be the case that you would like to use this just for objects but
not for Literals). Also it can't be mixed (meaning you can't have some
values as int and some as Json\Number). However I think this could work
nicely with json-scheme type hints and using that only if explicitly
specified in the scheme. The only problem is that it's not a small feature
so it will probably take some time to make that happen.

JSON_FLOAT_AS_STRING is meant like a quick fix for something that is
causing issue. However I it's a bit "hackish" as it applies on all floats
which is not very nice. Maybe it will really make sense to wait for
json-scheme. More I look into more I think that it would be a better
solution.

Cheers

Jakub