# RFC Idea - is_json - looking for feedback - Thread: https://externals.io/message/118310 - Original archive: https://news-web.php.net/php.internals/118310 - Started by juan carlos morales on 2022-07-29T14:26:37+00:00 - Messages: 22 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #118310 — juan carlos morales — 2022-07-29T14:26:37+00:00 URL: https://externals.io/message/118310#118310 I am following the RFC guideline for the first time. ( https://wiki.php.net/rfc/howto) As suggested there, I am here to get a feeling from you, regarding the following RFC for PHP. # Change (draft): New function in php called like: is_json(string $string): bool ## Description ### Parameters string $string -> string to find out if is a valid JSON or not ### Return Returns a bool. The function is capable to determine if the passed string is a valid JSON (true) or not (false). # Why this function ? At the moment the only way to determine if a JSON-string is valid we have to execute the `json_decode()` function. The drawback about this, is that `json_decode()` generates an in memory an object/array (depending on parameters) while parsing the string; this leads to a memory usage that is not needed (because we use memory for creating the object/array) and also can cause an error for reaching the memory-limit of the php process. Sometimes we just need to know is the string is a valid json or not, and nothing else. # Do we need something like this? If a check to an string is valid JSON then for sure I will have to use it in my code either as an object or an array. Well that is not true. There are plenty of cases where you just need to check if a string is a valid json and that is it. Just looking into stackoverflow will give you an idea about how many people is looking for something like this in an efficient way. # Who would develop the RFC ? I would develop the RFC with the help of the community if needed. I already have a first version of the is_json() function tested and ready for review. Thanks in advance, I am looking forward to hear your opinion on this. Kind Regards, Juan Carlos Morales. --- ## #118312 — michal.brzuchalski@gmail.com — 2022-07-29T19:31:46+00:00 URL: https://externals.io/message/118310#118312 · In reply to: #118310 Hi Juan, pt., 29 lip 2022, 16:26 użytkownik juan carlos morales < dev.juan.morales@gmail.com> napisał: > I am following the RFC guideline for the first time. ( > https://wiki.php.net/rfc/howto) > > As suggested there, I am here to get a feeling from you, regarding the > following RFC for PHP. > > # Change (draft): > > New function in php called like: > > is_json(string $string): bool > > ## Description > ### Parameters > string $string -> string to find out if is a valid JSON or not > > ### Return > Returns a bool. The function is capable to determine if the passed string > is a valid JSON (true) or not (false). > > # Why this function ? > > At the moment the only way to determine if a JSON-string is valid we have > to execute the `json_decode()` function. > > The drawback about this, is that `json_decode()` generates an in memory an > object/array (depending on parameters) while parsing the string; this leads > to a memory usage that is not needed (because we use memory for creating > the object/array) and also can cause an error for reaching the memory-limit > of the php process. > Personally I'd vote NO. Cheers, Michał Marcin Brzuchalski --- ## #118313 — David Rodrigues — 2022-07-29T20:17:39+00:00 URL: https://externals.io/message/118310#118313 · In reply to: #118312 I was about to say NO, but after being completely your argument, the idea makes sense. Initially I thought about using `json_decode()` with error capture, but the idea that it would overload memory makes perfect sense, compared to a simple structure analysis, if that is indeed the user's intention. The performance would also be absurdly better. What worries me above is the misuse of the function, like checking if is_json() === true and using `json_decode()` right after. However, I believe this can be easily optimized by the engine itself. My vote is YES. Atenciosamente, David Rodrigues Em sex., 29 de jul. de 2022 às 16:32, Michał Marcin Brzuchalski < michal.brzuchalski@gmail.com> escreveu: > Hi Juan, > > pt., 29 lip 2022, 16:26 użytkownik juan carlos morales < > dev.juan.morales@gmail.com> napisał: > > > […] > leads > > […] > memory-limit > > […] > > Personally I'd vote NO. > > Cheers, > Michał Marcin Brzuchalski --- ## #118314 — juan carlos morales — 2022-07-29T20:55:24+00:00 URL: https://externals.io/message/118310#118314 · In reply to: #118313 El vie, 29 jul 2022 a las 22:18, David Rodrigues () escribió: > I was about to say NO, but after being completely your argument, the idea > makes sense. > > Initially I thought about using `json_decode()` with error capture, but the > idea that it would overload memory makes perfect sense, compared to a > simple structure analysis, if that is indeed the user's intention. The > performance would also be absurdly better. > > What worries me above is the misuse of the function, like checking if > is_json() === true and using `json_decode()` right after. However, I believe > this can be easily optimized by the engine itself. > > My vote is YES. > > > Atenciosamente, > David Rodrigues > > > Em sex., 29 de jul. de 2022 às 16:32, Michał Marcin Brzuchalski < > michal.brzuchalski@gmail.com> escreveu: > > > […] > string > > […] > have > > […] > an > > […] > creating > > […] > Thanks for the feedback *** To: Michał Marcin Brzuchalski Thanks for the opinion, but it would be useful for me to know if there is a reason behind the "no" that could help me get better. *** To: David Rodrigues Gracias. Regarding "performance", I did some benchmarks. I tested is_json() vs `json_decode()` with a json-string with 1 million key-values; is_json was 100% faster than `json_decode()`, plus memory usage with is_json() was flat! , yes , flat. `memory_get_usage()` before and after is_json() returned the same value; with `json_decode()` I had to change the memory limit setting otherwise memory reaches max allowed before finishing parsing it. Regarding use cases, I can say that this idea came up , as in my current company , our product heavily uses `json_decode()` to know if an string is a valid json, and compromises the memory usage once in a while, plus, performance decreases; that is why this RFC-Idea. The good thing is that I already have something for this. Also in stack overflow, honestly, take a look, and check how much people is needing this. Unit tests could heavily benefit from such a functionality. A quick look in github for php code with a function called "is_json()" exposes tons of projects where developers are writting their own "is_json()" function relaying in `json_decode()` to make it work. Etc. :) Stay in contact David. and once again, thanks! --- ## #118315 — Jordan LeDoux — 2022-07-30T01:50:08+00:00 URL: https://externals.io/message/118310#118315 · In reply to: #118310 On Fri, Jul 29, 2022 at 7:27 AM juan carlos morales < dev.juan.morales@gmail.com> wrote: > # Why this function ? > > At the moment the only way to determine if a JSON-string is valid we have > to execute the `json_decode()` function. > > The drawback about this, is that `json_decode()` generates an in memory an > object/array (depending on parameters) while parsing the string; this leads > to a memory usage that is not needed (because we use memory for creating > the object/array) and also can cause an error for reaching the memory-limit > of the php process. > > Sometimes we just need to know is the string is a valid json or not, and > nothing else. > You say that you have a first-pass at the implementation done. I'd be curious to see it. My initial thought was that in order to validate the string, you likely need to allocate extra memory as part of the validation that depends on the string size. You'd definitely save the overhead of a ZVAL, but for such an object that overhead is likely negligible. So I guess my question would be: in the actual implementation that lands, how much memory would this actually save compared to `json_decode()`? This seems like it would make the RFC tricky, as the purported benefit of the RFC depends very tightly on the implementation that lands. Jordan --- ## #118316 — Aleksander Machniak — 2022-07-30T06:05:23+00:00 URL: https://externals.io/message/118310#118316 · In reply to: #118310 > is_json(string $string): bool `json_decode()` has $depth argument, I think is_json() probably also should. And I'm not sure about JSON_INVALID_UTF8_IGNORE flag. My point is that if you use these with `json_decode()` you might also need to use these with is_json(). On the other hand maybe the function should be as simple as possible, but I'd like it to be considered. -- Aleksander Machniak Kolab Groupware Developer [https://kolab.org] Roundcube Webmail Developer [https://roundcube.net] ---------------------------------------------------- PGP: 19359DC1 # Blog: https://kolabian.wordpress.com --- ## #118317 — Dusk — 2022-07-30T06:29:53+00:00 URL: https://externals.io/message/118310#118317 · In reply to: #118316 > `json_decode()` has $depth argument, I think is_json() probably also should. And I'm not sure about JSON_INVALID_UTF8_IGNORE flag. > > My point is that if you use these with `json_decode()` you might also need to use these with is_json() How about if validation were a flag to `json_decode()` -- perhaps JSON_VALIDATE_ONLY -- which made it return a placeholder value like true instead of the decoded structure? (It can't return null on success; that's used for errors if `JSON_THROW_ON_ERROR` isn't set.) --- ## #118318 — juan carlos morales — 2022-07-30T06:37:09+00:00 URL: https://externals.io/message/118310#118318 · In reply to: #118315 Just want to clarify that when I mentioned the use of memory, I wrote down the function "memory_get_usage()", which basically gives us the memory handle by php that is related to the memory_limit INI setting. Now I will provide a benchmark I have done really quick: # Code used (I have the implementation of is_json() done already) ) escribió: > > > On Fri, Jul 29, 2022 at 7:27 AM juan carlos morales < > dev.juan.morales@gmail.com> wrote: > > > […] > > You say that you have a first-pass at the implementation done. I'd be > curious to see it. My initial thought was that in order to validate the > string, you likely need to allocate extra memory as part of the validation > that depends on the string size. You'd definitely save the overhead of a > ZVAL, but for such an object that overhead is likely negligible. > > So I guess my question would be: in the actual implementation that lands, > how much memory would this actually save compared to `json_decode()`? This > seems like it would make the RFC tricky, as the purported benefit of the > RFC depends very tightly on the implementation that lands. > > Jordan --- ## #118319 — juan carlos morales — 2022-07-30T06:44:17+00:00 URL: https://externals.io/message/118310#118319 · In reply to: #118317 El sáb, 30 jul 2022 a las 8:30, Dusk () escribió: > > > […] > should. And I'm not sure about JSON_INVALID_UTF8_IGNORE flag. > > […] > to use these with is_json() > > How about if validation were a flag to `json_decode()` -- perhaps > JSON_VALIDATE_ONLY -- which made it return a placeholder value like true > instead of the decoded structure? (It can't return null on success; that's > used for errors if `JSON_THROW_ON_ERROR` isn't set.) > -- > > To unsubscribe, visit: https://www.php.net/unsub.php > > I thought about that too, and yes can also be a valid approach that would not interfeer with my development. What I personally dont like about that approach is that a developer is able to specify flags that are related to exclusively with result generation, and when they get mixed with JSON_VALIDATE_ONLY would not make sense. for example: json_decode($jsonStringHere, false, 512, `JSON_BIGINT_AS_STRING` | JSON_INVALID_UTF8_SUBSTITUTE | `JSON_OBJECT_AS_ARRAY` | `JSON_THROW_ON_ERROR` | JSON_VALIDATE_ONLY); So the end result, is not clear by reading the line itself, I mean , the mechanism (of having another constant in `json_decode()`) is not enough to make the function self-expressive. --- ## #118320 — Oleksii Bulba — 2022-07-30T09:57:48+00:00 URL: https://externals.io/message/118310#118320 · In reply to: #118318 > $memoryAfter = memory_get_usage(true) / 1024 / 1024; I see that you used `memory_get_usage` that shows memory usage at the time of the function call. As far as I understand, your function does not return any value, so I suspect that it is obvious that the memory usage after the function call is the same as before. But what is the actual memory usage during the function call? Can you run the same benchmark but with `memory_get_peak_usage` function to see how your function uses memory? # $memoryAfter = memory_get_peak_usage(true) / 1024 / 1024; Also, I'm concerned if it would be better to name the function `is_json_valid`? On Sat, Jul 30, 2022 at 10:37 AM juan carlos morales < dev.juan.morales@gmail.com> wrote: > Just want to clarify that when I mentioned the use of memory, I wrote down > the function "memory_get_usage()", which basically gives us the memory > handle by php that is related to the memory_limit INI setting. > > Now I will provide a benchmark I have done really quick: > > # Code used (I have the implementation of is_json() done already) > > // make sure you set your memory limit to -1 before running this code > // Here we create a very very very big json string, really big > $limit = 1000000; > $jsonString = '{ "test": { "foo": "bar" },'; > > for ($i=0; $i < $limit; $i++) { > $jsonString .= " \"test$i\": { \"foo\": { \"test\" : { \"foo\" : { > \"test\" : { \"foo\" : \"bar\" }}}}},"; > } > > $jsonString .= ' "testXXXX": { "foo": "replaceme" } }'; > //{ "test" : { "foo" : "bar" }}} > > $memoryBefore = memory_get_usage(true) / 1024 / 1024; > echo "Megas used before call: " . $memoryBefore . PHP_EOL; > > $start = microtime(true); > > json_decode($jsonString, null, $limit, 0); > //<------------------ un/comment to show/hide results for `json_decode()` > //is_json($jsonString); > //<------------------ un/comment to show/hide results for is_json() > > $memoryAfter = memory_get_usage(true) / 1024 / 1024; > echo "Megas used after call: " . $memoryAfter . PHP_EOL; > > echo "Difference: " . ($memoryAfter - $memoryBefore) . PHP_EOL; > > echo "Time: " . (microtime(true) - $start) . " seconds" . PHP_EOL; > return; > > # Results > ## `json_decode()` > Megas used before call: 79.23828125 > Megas used after call: 3269.23828125 > Difference: 3190 > Time: 12.091101884842 seconds > > ## is_json() > Megas used before call: 79.23828125 > Megas used after call: 79.23828125 > Difference: 0 > Time: 5.4537169933319 seconds > > > And yes, I am open to share the implementation, but after I write the RFC. > > Thanks for taking your time to give me a feedback. > > El sáb, 30 jul 2022 a las 3:50, Jordan LeDoux () > escribió: > > > […] > have > > […] > validation > > […] --- ## #118321 — juan carlos morales — 2022-07-30T10:07:00+00:00 URL: https://externals.io/message/118310#118321 · In reply to: #118320 El sáb., 30 de julio de 2022 11:58, Oleksii Bulba escribió: > > […] > > I see that you used `memory_get_usage` that shows memory usage at the time > of the function call. > As far as I understand, your function does not return any value, > so I suspect that it is obvious that the memory usage after the function > call is the same as before. > But what is the actual memory usage during the function call? > Can you run the same benchmark but with `memory_get_peak_usage` function > to see how your function uses memory? > > # $memoryAfter = memory_get_peak_usage(true) / 1024 / 1024; > > Also, I'm concerned if it would be better to name the function > `is_json_valid`? > > On Sat, Jul 30, 2022 at 10:37 AM juan carlos morales < > dev.juan.morales@gmail.com> wrote: > > > […] > Hello Jordan, thanks for the feedback. I think the benchmark talks by itself (also for the memory Save question). Also by the fact that in order to run it for `json_decode()` rhe memory limit needs to be super high or -1 (no limit, not a good idea in production right?) The advantage here is to be able to parse huge strings without reaching the memory limit set in the INI settings. I take It as a "IF THIS IS AS GOOD AS IT SEEMS THEN YES" :D Once again... Thanks --- ## #118322 — Nikita Popov — 2022-07-30T14:01:09+00:00 URL: https://externals.io/message/118310#118322 · In reply to: #118310 On Fri, Jul 29, 2022 at 4:27 PM juan carlos morales < dev.juan.morales@gmail.com> wrote: > I am following the RFC guideline for the first time. ( > https://wiki.php.net/rfc/howto) > > As suggested there, I am here to get a feeling from you, regarding the > following RFC for PHP. > > # Change (draft): > > New function in php called like: > > is_json(string $string): bool > > ## Description > ### Parameters > string $string -> string to find out if is a valid JSON or not > > ### Return > Returns a bool. The function is capable to determine if the passed string > is a valid JSON (true) or not (false). > > # Why this function ? > > At the moment the only way to determine if a JSON-string is valid we have > to execute the `json_decode()` function. > > The drawback about this, is that `json_decode()` generates an in memory an > object/array (depending on parameters) while parsing the string; this leads > to a memory usage that is not needed (because we use memory for creating > the object/array) and also can cause an error for reaching the memory-limit > of the php process. > > Sometimes we just need to know is the string is a valid json or not, and > nothing else. > > # Do we need something like this? If a check to an string is valid JSON > then for sure I will have to use it in my code either as an object or an > array. > > Well that is not true. There are plenty of cases where you just need to > check if a string is a valid json and that is it. Just looking into > stackoverflow will give you an idea about how many people is looking for > something like this in an efficient way. > Could you please give some specific examples where the proposed functionality would be useful? Regards, Nikita --- ## #118323 — David Gebler — 2022-07-30T14:48:23+00:00 URL: https://externals.io/message/118310#118323 · In reply to: #118322 > On Fri, Jul 29, 2022 at 4:27 PM juan carlos morales < > dev.juan.morales@gmail.com> wrote: > > > […] > leads > > […] > memory-limit > > […] > In the last 15 years, the only time I've ever needed to know if a string is valid JSON is if I'm about to decode or otherwise parse it as JSON. If I'm decoding what I expect to be a large JSON blob, such that memory usage might be a concern, personally I use https://github.com/salsify/jsonstreamingparser but the point is userland solutions are possible. What I'm asking is what's the practical use for this proposed function? Where are you likely to need to know if a string is valid JSON but not have to (try to, with error handling) parse it almost immediately afterwards anyway? Unless there is some fairly commonplace use case for this I'm not thinking of, you're going to be using that extra memory, or using a streaming parser, at some point in your script regardless. If there is genuine demand for it, I'd be in favour (I'm not a voting member so kind of moot but...), otherwise I'm generally against introducing new core functions which are either edge-case or can be perfectly well dealt with via userland code. Cheers -Dave > > […] > > Could you please give some specific examples where the proposed > functionality would be useful? > > Regards, > Nikita --- ## #118324 — juan carlos morales — 2022-07-30T14:53:16+00:00 URL: https://externals.io/message/118310#118324 · In reply to: #118323 El sáb., 30 de julio de 2022 16:48, David Gebler escribió: > > > > > > […] > > In the last 15 years, the only time I've ever needed to know if a string > is valid JSON is if I'm about to decode or otherwise parse it as JSON. If > I'm decoding what I expect to be a large JSON blob, such that memory usage > might be a concern, personally I use > https://github.com/salsify/jsonstreamingparser but the point is userland > solutions are possible. > > What I'm asking is what's the practical use for this proposed function? > Where are you likely to need to know if a string is valid JSON but not have > to (try to, with error handling) parse it almost immediately afterwards > anyway? Unless there is some fairly commonplace use case for this I'm not > thinking of, you're going to be using that extra memory, or using a > streaming parser, at some point in your script regardless. If there is > genuine demand for it, I'd be in favour (I'm not a voting member so kind of > moot but...), otherwise I'm generally against introducing new core > functions which are either edge-case or can be perfectly well dealt with > via userland code. > > Cheers > > -Dave > > > > […] > Thanks everybody for the feedback. I think I got to a point that I need to write another e-mail with all my arguments and use cases for this. That is why I kindly ask you all for patience, and I Will write again soon. It will be a long e-mail for sure, and when It comes , I ask you all to take the time to read it. Thanks in advance. --- ## #118325 — Deleu — 2022-07-30T22:55:54+00:00 URL: https://externals.io/message/118310#118325 · In reply to: #118323 > > > What I'm asking is what's the practical use for this proposed function? > Where are you likely to need to know if a string is valid JSON but not have > to (try to, with error handling) parse it almost immediately afterwards > anyway? > I'm still on the fence over the general idea, but I thought I could at least address this question in particular. I can definitely see it's usefulness on public HTTP API ingesting data (specially large data) where if the payload is a valid JSON, it gets stored and processed by a background queue, making it so that the HTTP layer can either reject the request with a 4xx status or accept it and only truly decode it on a separate process that may even be hosted by a separate server with larger memory size. --- ## #118326 — juan carlos morales — 2022-07-31T13:14:27+00:00 URL: https://externals.io/message/118310#118326 · In reply to: #118325 El dom, 31 jul 2022 a las 0:56, Deleu () escribió: > > > > > > […] > > I'm still on the fence over the general idea, but I thought I could at > least address this question in particular. > > I can definitely see it's usefulness on public HTTP API ingesting data > (specially large data) where if the payload is a valid JSON, it gets stored > and processed by a background queue, making it so that the HTTP layer can > either reject the request with a 4xx status or accept it and only truly > decode it on a separate process that may even be hosted by a separate > server with larger memory size. > Before starting, I want to thank all for taking time from your time, to give me a feedback, I sincerely respect that, so ... thanks! Sorry for the long message, but I have the feeling that ... this is it, is now ... or it will not be, at least not now, so ... here my best effort; even though, I dont put aside the possibility that I might have an offuscated view/opinion regarding my proposal, so is possible that I might not see the things right ... you know .. I am human after all. So, enough of prologue and let's start .... # Why would I have code that checks if a json-string is a valid json, if I will not use the content inside it? (or something like that) - Some of you , asked me this question. - That depends on the infinite amout of use cases that the human brain can imagine, that is why I say that is not the right approach to discuss this topic. # Why this change then? - Is web related, ergo, is PHP related. - It does not add complexity into PHP, with this functionality we are using the existing JSON parser that exists in PHP at the moment, the only thing we do is to create an interface between "userland" and the parser itself, without the need to use memory as `json_decode()` is using to check if a string is a valid json-string or not. ## Proposed functionality (proposed, subject to changes for sure, as now I have only a working and dirty prototype for this) function is_json(string $json, int $flags): bool {} Returns: `TRUE` if json-string is valid json, otherwise returns FALSE. Exceptions: Optionally set, for the same or subset of exceptions of the actual `json_decode()`, like the one Syntax Error for example. So far this is it, subject to changes for sure. # Real open-source project over github using `json_decode()` just to check if a json-string is valid ... and nothing else than that. - I provide here use cases from major projects, where they ned to have code that has to be able to check if an string is a valid JSON and nothing else than that. - Please check the link to see full code - I provide here some small snippets from the link I provide - We are not discussing here how they implemented the code, what I want to show here is that there is a need to check if an string is a valid JSON-string without th need to create an object/array out of it. - Also, we are not discussing if they are mejor projects or not. They are listed in github as major projects, with stars and a big community of users and developers maintaining them. - On some snippets I wrote some notes too, because on some of them is not obvious how a funciton like the one I propose could be useful. ## Symfony Framework https://github.com/symfony/symfony/blob/870eeb975feb1abb4b8a1722e1fd57beeab2b230/src/Symfony/Component/Validator/Constraints/JsonValidator.php ``` class JsonValidator extends ConstraintValidator ... ... ... ``` ## Laravel Framework https://github.com/laravel/framework/blob/302a579f00ebcb2573f481054cbeadad9c970605/src/Illuminate/Validation/Concerns/ValidatesAttributes.php ``` public function validateJson($attribute, $value) { if (is_array($value)) { return false; } if (! is_scalar($value) && ! is_null($value) && ! method_exists($value, '__toString')) { return false; } json_decode($value); return `json_last_error()` === JSON_ERROR_NONE; } ``` https://github.com/laravel/framework/blob/61eac9cae4717699ecb3941b16c3d775820d4ca2/src/Illuminate/Support/Str.php ``` public static function isJson($value) { ``` ## Magento https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/app/code/Magento/User/Block/Role/Grid/User.php ``` private function getJSONString($input) { $output = json_decode($input); return $output ? $this->_jsonEncoder->encode($output) : '{}'; } ``` https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/lib/internal/Magento/Framework/DB/DataConverter/SerializedToJson.php ``` protected function isValidJsonValue($value) { if (in_array($value, ['null', 'false', '0', '""', '[]']) || (json_decode($value) !== null && `json_last_error()` === JSON_ERROR_NONE) ) { return true; } //JSON last error reset json_encode([]); return false; } ``` https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/lib/internal/Magento/Framework/Serialize/JsonValidator.php ``` public function isValid($string) { if ($string !== false && $string !== null && $string !== '') { json_decode($string); if (json_last_error() === JSON_ERROR_NONE) { return true; } } return false; } ``` ## getgrav https://github.com/getgrav/grav/blob/3e7f67f589267e61f823d19824f3ee1b9a8a38ff/system/src/Grav/Common/Data/Validation.php ``` public static function validateJson($value, $params) { return (bool) (@json_decode($value)); } ``` ## Symfony / http-kernel https://github.com/symfony/http-kernel/blob/94986633e4c3e7facb7defbd094a2e1170486ab5/DataCollector/RequestDataCollector.php ``` public function getPrettyJson() { $decoded = json_decode($this->getContent()); //<------ here they decode, just to check if is valid json-string or not, that is th reason of this line. return \JSON_ERROR_NONE === `json_last_error()` ? json_encode($decoded, \JSON_PRETTY_PRINT) : null; } ``` ## Respect / Validation https://github.com/Respect/Validation/blob/3dcd859d986f1b586b5539ea19962723ab7352ed/library/Rules/Json.php ``` final class Json extends AbstractRule { /** * {@inheritDoc} */ public function validate($input): bool { if (!is_string($input) || $input === '') { return false; } json_decode($input); return `json_last_error()` === JSON_ERROR_NONE; } } ``` https://github.com/Respect/Validation/blob/3dcd859d986f1b586b5539ea19962723ab7352ed/library/Rules/Json.php ``` final class Json extends AbstractRule { /** * {@inheritDoc} */ public function validate($input): bool { if (!is_string($input) || $input === '') { return false; } json_decode($input); return `json_last_error()` === JSON_ERROR_NONE; } } ``` ## humhub https://github.com/humhub/humhub/blob/26d7e2667a9317057abe335a056ac8e8f4d675fb/protected/humhub/modules/web/security/controllers/ReportController.php ``` public function actionIndex() { Yii::$app->response->statusCode = 204; if(!SecuritySettings::isReportingEnabled()) { return; } $json_data = file_get_contents('php://input'); if ($json_data = json_decode($json_data)) { //<----- here they `json_decode()` just to check if is valid json-string only, am I right? $json_data = json_encode($json_data, `JSON_PRETTY_PRINT` | JSON_UNESCAPED_SLASHES); $json_data = preg_replace('/\'nonce-[^\']*\'/', "'nonce-xxxxxxxxxxxxxxxxxxxxxxxx'", $json_data); Yii::error($json_data, 'web.security'); } } ``` ## Prestashop https://github.com/PrestaShop/PrestaShop/blob/24f9e510ecb0cb002ac3f4834f3210e8d9359899/classes/Validate.php ``` public static function isJson($string) { json_decode($string); return `json_last_error()` == JSON_ERROR_NONE; } ``` ## Wordpress CLI https://github.com/wp-cli/wp-cli/blob/f3e4b0785aa3d3132ee73be30aedca8838a8fa06/php/utils.php ``` function is_json( $argument, $ignore_scalars = true ) { if ( ! is_string( $argument ) || '' === $argument ) { return false; } if ( $ignore_scalars && ! in_array( $argument[0], [ '{', '[' ], true ) ) { return false; } json_decode( $argument, $assoc = true ); return `json_last_error()` === JSON_ERROR_NONE; } ``` ## JOOMLA CMS https://github.com/joomla/joomla-cms/blob/09d14c65f25f9bc76f2698e69c4d7b35f43bc848/libraries/src/Form/Field/AccessiblemediaField.php ``` if (\is_string($value)) { json_decode($value); //<---------------------------------------------------------- HERE // Check if value is a valid JSON string. if ($value !== '' && `json_last_error()` !== JSON_ERROR_NONE) { /** * If the value is not empty and is not a valid JSON string, * it is most likely a custom field created in Joomla 3 and * the value is a string that contains the file name. */ if (is_file(JPATH_ROOT . '/' . $value)) { $value = '{"imagefile":"' . $value . '","alt_text":""}'; } else { $value = ''; } } ``` # Stackoverflow questions related to this ## In PHP, this question is one of the most high ranked questions related to json && php in stackoverflow, "Fastest way to check if a string is JSON in PHP?" ### The question https://stackoverflow.com/questions/6041741/fastest-way-to-check-if-a-string-is-json-in-php Viewed 484k times ### The ranking https://stackoverflow.com/questions/tagged/php%20json?sort=MostVotes&edited=true ## Person asking how to do exactly this, also providing a real use case; eventhough in python, the programming language is not important. https://stackoverflow.com/questions/5508509/how-do-i-check-if-a-string-is-valid-json-in-python ## Someone has also doing exactly this , in JAVA https://stackoverflow.com/questions/3679479/check-if-file-is-json-java --- ## #118327 — Larry Garfield — 2022-07-31T15:40:26+00:00 URL: https://externals.io/message/118310#118327 · In reply to: #118326 > Before starting, I want to thank all for taking time from your time, to > give me a feedback, I sincerely respect that, so ... thanks! > > Sorry for the long message, but I have the feeling that ... this is it, is > now ... or it will not be, at least not now, so ... here my best effort; > even though, I dont put aside the possibility that I might have an > offuscated view/opinion regarding my proposal, so is possible that I might > not see the things right ... you know .. I am human after all. > > So, enough of prologue and let's start .... > > # Why would I have code that checks if a json-string is a valid json, if I > will not use the content inside it? (or something like that) > > - Some of you , asked me this question. > - That depends on the infinite amout of use cases that the human brain can > imagine, that is why I say that is not the right approach to discuss this > topic. > > # Why this change then? > > - Is web related, ergo, is PHP related. > - It does not add complexity into PHP, with this functionality we are using > the existing JSON parser that exists in PHP at the moment, the only thing > we do is to create an interface between "userland" and the parser itself, > without the need to use memory as `json_decode()` is using to check if a > string is a valid json-string or not. > > > ## Proposed functionality (proposed, subject to changes for sure, as now I > have only a working and dirty prototype for this) > > function is_json(string $json, int $flags): bool {} > > Returns: > `TRUE` if json-string is valid json, otherwise returns FALSE. > > Exceptions: > Optionally set, for the same or subset of exceptions of the actual > `json_decode()`, like the one Syntax Error for example. > > > So far this is it, subject to changes for sure. > > > # Real open-source project over github using `json_decode()` just to check if > a json-string is valid ... and nothing else than that. > > - I provide here use cases from major projects, where they ned to have code > that has to be able to check if an string is a valid JSON and nothing else > than that. > - Please check the link to see full code > - I provide here some small snippets from the link I provide > - We are not discussing here how they implemented the code, what I want to > show here is that there is a need to check if an string is a valid > JSON-string without th need to create an object/array out of it. > - Also, we are not discussing if they are mejor projects or not. They are > listed in github as major projects, with stars and a big community of users > and developers maintaining them. > - On some snippets I wrote some notes too, because on some of them is not > obvious how a funciton like the one I propose could be useful. > > > ## Symfony Framework > > https://github.com/symfony/symfony/blob/870eeb975feb1abb4b8a1722e1fd57beeab2b230/src/Symfony/Component/Validator/Constraints/JsonValidator.php > > ``` > class JsonValidator extends ConstraintValidator > ... > ... > ... > ``` > > ## Laravel Framework > > https://github.com/laravel/framework/blob/302a579f00ebcb2573f481054cbeadad9c970605/src/Illuminate/Validation/Concerns/ValidatesAttributes.php > > ``` > public function validateJson($attribute, $value) > { > if (is_array($value)) { > return false; > } > > if (! is_scalar($value) && ! is_null($value) && ! > method_exists($value, '__toString')) { > return false; > } > > json_decode($value); > > return `json_last_error()` === JSON_ERROR_NONE; > } > ``` > > https://github.com/laravel/framework/blob/61eac9cae4717699ecb3941b16c3d775820d4ca2/src/Illuminate/Support/Str.php > > ``` > public static function isJson($value) > { > ``` > > > ## Magento > > https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/app/code/Magento/User/Block/Role/Grid/User.php > > ``` > private function getJSONString($input) > { > $output = json_decode($input); > return $output ? $this->_jsonEncoder->encode($output) : '{}'; > } > ``` > > https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/lib/internal/Magento/Framework/DB/DataConverter/SerializedToJson.php > > ``` > protected function isValidJsonValue($value) > { > if (in_array($value, ['null', 'false', '0', '""', '[]']) > || (json_decode($value) !== null && `json_last_error()` === > JSON_ERROR_NONE) > ) { > return true; > } > //JSON last error reset > json_encode([]); > return false; > } > ``` > > https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/lib/internal/Magento/Framework/Serialize/JsonValidator.php > > ``` > public function isValid($string) > { > if ($string !== false && $string !== null && $string !== '') { > json_decode($string); > if (json_last_error() === JSON_ERROR_NONE) { > return true; > } > } > return false; > } > ``` > > ## getgrav > > https://github.com/getgrav/grav/blob/3e7f67f589267e61f823d19824f3ee1b9a8a38ff/system/src/Grav/Common/Data/Validation.php > > ``` > public static function validateJson($value, $params) > { > return (bool) (@json_decode($value)); > } > ``` > > > ## Symfony / http-kernel > > https://github.com/symfony/http-kernel/blob/94986633e4c3e7facb7defbd094a2e1170486ab5/DataCollector/RequestDataCollector.php > > ``` > public function getPrettyJson() > { > $decoded = json_decode($this->getContent()); > //<------ here they decode, just to check if is valid json-string or not, > that is th reason of this line. > > return \JSON_ERROR_NONE === `json_last_error()` ? > json_encode($decoded, \JSON_PRETTY_PRINT) : null; > } > ``` > > ## Respect / Validation > > https://github.com/Respect/Validation/blob/3dcd859d986f1b586b5539ea19962723ab7352ed/library/Rules/Json.php > > ``` > final class Json extends AbstractRule > { > /** > * {@inheritDoc} > */ > public function validate($input): bool > { > if (!is_string($input) || $input === '') { > return false; > } > > json_decode($input); > > return `json_last_error()` === JSON_ERROR_NONE; > } > } > ``` > > https://github.com/Respect/Validation/blob/3dcd859d986f1b586b5539ea19962723ab7352ed/library/Rules/Json.php > > ``` > final class Json extends AbstractRule > { > /** > * {@inheritDoc} > */ > public function validate($input): bool > { > if (!is_string($input) || $input === '') { > return false; > } > > json_decode($input); > > return `json_last_error()` === JSON_ERROR_NONE; > } > } > ``` > > ## humhub > > https://github.com/humhub/humhub/blob/26d7e2667a9317057abe335a056ac8e8f4d675fb/protected/humhub/modules/web/security/controllers/ReportController.php > > > ``` > public function actionIndex() > { > Yii::$app->response->statusCode = 204; > > if(!SecuritySettings::isReportingEnabled()) { > return; > } > > $json_data = file_get_contents('php://input'); > if ($json_data = json_decode($json_data)) { > //<----- here they `json_decode()` just to check if > is valid json-string only, am I right? > $json_data = json_encode($json_data, `JSON_PRETTY_PRINT` | > JSON_UNESCAPED_SLASHES); > $json_data = preg_replace('/\'nonce-[^\']*\'/', > "'nonce-xxxxxxxxxxxxxxxxxxxxxxxx'", $json_data); > Yii::error($json_data, 'web.security'); > } > } > ``` > > ## Prestashop > > https://github.com/PrestaShop/PrestaShop/blob/24f9e510ecb0cb002ac3f4834f3210e8d9359899/classes/Validate.php > > ``` > public static function isJson($string) > { > json_decode($string); > > return `json_last_error()` == JSON_ERROR_NONE; > } > ``` > > > ## Wordpress CLI > > https://github.com/wp-cli/wp-cli/blob/f3e4b0785aa3d3132ee73be30aedca8838a8fa06/php/utils.php > > ``` > function is_json( $argument, $ignore_scalars = true ) { > if ( ! is_string( $argument ) || '' === $argument ) { > return false; > } > > if ( $ignore_scalars && ! in_array( $argument[0], [ '{', '[' ], true ) > ) { > return false; > } > > json_decode( $argument, $assoc = true ); > > return `json_last_error()` === JSON_ERROR_NONE; > } > ``` > > > ## JOOMLA CMS > > https://github.com/joomla/joomla-cms/blob/09d14c65f25f9bc76f2698e69c4d7b35f43bc848/libraries/src/Form/Field/AccessiblemediaField.php > > ``` > if (\is_string($value)) { > json_decode($value); > //<---------------------------------------------------------- HERE > > // Check if value is a valid JSON string. > if ($value !== '' && `json_last_error()` !== JSON_ERROR_NONE) { > /** > * If the value is not empty and is not a valid JSON string, > * it is most likely a custom field created in Joomla 3 and > * the value is a string that contains the file name. > */ > if (is_file(JPATH_ROOT . '/' . $value)) { > $value = '{"imagefile":"' . $value . '","alt_text":""}'; > } else { > $value = ''; > } > } > > ``` > > > > # Stackoverflow questions related to this > > ## In PHP, this question is one of the most high ranked questions related > to json && php in stackoverflow, "Fastest way to check if a string is JSON > in PHP?" > > ### The question > > https://stackoverflow.com/questions/6041741/fastest-way-to-check-if-a-string-is-json-in-php > Viewed 484k times > > ### The ranking > > https://stackoverflow.com/questions/tagged/php%20json?sort=MostVotes&edited=true > > ## Person asking how to do exactly this, also providing a real use case; > eventhough in python, the programming language is not important. > > https://stackoverflow.com/questions/5508509/how-do-i-check-if-a-string-is-valid-json-in-python > > ## Someone has also doing exactly this , in JAVA > > https://stackoverflow.com/questions/3679479/check-if-file-is-json-java So the core argument, it seems, is "there's lots of user-space implementations already, hence demand, and it would be better/faster/stronger/we-have-the-technology to do it in C." Thus another, arguably more important benchmark would be a C implementation compared to a userspace implementation of the same algorithm. Presumably your C code is doing some kind of stream-based validation with braces/quotes matching rather than a naive "try and parse and see if it breaks." We would need to see benchmarks of the same stream-based validation in C vs PHP, as that's the real distinction. That a stream validator would be more memory efficient than a full parser is not at all surprising, but that's also not a fair comparison. As for the benchmarks themselves, do not use `memory_get_usage()`; as noted, it shows the memory usage at that time, not ever. What you want is `memory_get_peak_usage()`, which gets the highest the memory usage has gotten in that script run. Or, even better, use PHPBench with separate sample methods to compare various different implementations. It will handle all the "run many times and average the results and throw out outliers" and such for you. It's quite a flexible tool once you get the hang of it. I'll also note that it would be to your benefit to share the working C code as a patch/PR already. If accepted it would be released open source anyway, so letting people see the proposed code now can only help your case; unless the code is awful, in which case showing it later would only waste your time and everyone else's discussing it in the abstract before the implementation could be reviewed. --Larry Garfield --- ## #118331 — juan carlos morales — 2022-07-31T18:22:49+00:00 URL: https://externals.io/message/118310#118331 (I received an email from MAILER-DAEMON@lists.php.net , I had to cut the message history as the mail server complained because the email got too long) Larry, here is the same benchmark including memory_get_peak_usage as requested (more after the benchmark) ----- Benchmark including memory_get_peak_usage ----- # `json_decode()` Megas used before call: 79.23828125 PEAK Megas used before call: 128.00390625 //<---- Megas used after call: 3269.23828125 PEAK Megas used after call: 3309.23828125 //<---- Difference: 3190 PEAK Difference: 3181.234375 //<----- Time: 12.109144926071 seconds # is_json() Megas used before call: 79.23828125 PEAK Megas used before call: 128.00390625 //<---- Megas used after call: 79.23828125 PEAK Megas used after call: 128.00390625 //<---- Difference: 0 PEAK Difference: 0 //<----- Time: 5.4504480361938 seconds I want to clarify, as I previously mentioned, I did not develop any parser at all, the parser already exists in PHP and is use by `json_decode()`, in short words , what I propose is to create a new function, as an interface to THE EXISTING PARSER, only, and only, to check if a string is a valid json-string or not. I already provided a lot, but a lot of real usages of this, and at the moment the only way to do it (without using any 3rd package) is by using `json_decode()` , and the drawback is that `json_decode()` generates an object/array while parsing the string provided affecting the memory associated with the php process; and sometimes you just need to check if an string is valid json-string or not and nothing else; I have provided code from major open-source projects, and also THE question in stack overflow, which is viewed near 500k times, which indicates somehow the number of users interested in something like this. And yes, we can use a 3rd party package, but in companies, using a 3rd party package is not that simple, believe me. ------------ Regarding publishing the a PR with my implementation, is out of context at the moment, because we are discussing if a functionality like this should be included or not, and nothing else. Also I am pretty sure that if this gets a YES from the community (so far seems so), and I show my implementation, someone will tell me that it could be done in a different way , and is totally fine, is great actually. But now, at this moment, everything is about the functionality. ------------- I also want you to know that I really, really appreciate the time you took to participate on this and being direct and honest ..... THANKS A LOT (I really mean it). ============== People I have made my best effort providing you all with as much information and answers as possible. At the end is up to the community, if YES ... great , if NO ... great anyway, it has being a beautiful trip so far, but I cannot lie to you, I would really like to see whats next. I send you all my best wishes. Juan --- ## #118332 — juan carlos morales — 2022-07-31T19:46:11+00:00 URL: https://externals.io/message/118310#118332 Hi Juan, > > Thanks for your efforts, and examples. > > You are solving a common use case, in frameworks, libs, and general > userland. > > This much needed function will see good perf/memory improvements once > adoption of is_json() hits popular libs and frameworks. > > I will be voting in favor of this addition, subject to reviewing the > implementation and full RFC. > > Thanks for the effort. Looking forward to seeing it merged :) > > Many thanks, > Paul Dragoonis > > > > > […] Hello Paul, and sorry, I totally missed your message, thanks for the kind feedback, you made my day. So, if I get it correctly, I have green light to move on with the RFC process right? so ... I will move on. Once everything is done I will ping again. Regards, Juan PD: I had to re-send the message as it was too long, ... I am using a mailing list for the first time ever :) --- ## #118334 — Juan Basso — 2022-08-01T02:16:42+00:00 URL: https://externals.io/message/118310#118334 · In reply to: #118332 I think the arguments are good enough for the RFC. However, I would like to propose using a different API. Instead of creating a new global function that could conflict with existing applications, use filter_var validations since it is basically a validation. You can do something like filter_var($possibleJSON, FILTER_VALIDATE_JSON). Over time, if there is a need for extra parameters you can still use the filter_var flags or options (3rd parameter) to indicate if you want just JSON Object, JSON Array, JSON scalar, etc. Juan Basso On Sun, Jul 31, 2022 at 3:46 PM juan carlos morales < dev.juan.morales@gmail.com> wrote: > Hi Juan, > > […] > Hello Paul, and sorry, I totally missed your message, thanks for the kind > feedback, you made my day. > > So, if I get it correctly, I have green light to move on with the RFC > process right? so ... I will move on. Once everything is done I will ping > again. > > Regards, > > Juan > > PD: I had to re-send the message as it was too long, ... I am using a > mailing list for the first time ever :) --- ## #118335 — Jordan LeDoux — 2022-08-01T05:19:17+00:00 URL: https://externals.io/message/118310#118335 · In reply to: #118331 On Sun, Jul 31, 2022 at 11:23 AM juan carlos morales < dev.juan.morales@gmail.com> wrote: > ----- Benchmark including memory_get_peak_usage ----- > > # `json_decode()` > > Megas used before call: 79.23828125 > PEAK Megas used before call: 128.00390625 //<---- > Megas used after call: 3269.23828125 > PEAK Megas used after call: 3309.23828125 //<---- > Difference: 3190 > PEAK Difference: 3181.234375 //<----- > Time: 12.109144926071 seconds > > # is_json() > > Megas used before call: 79.23828125 > PEAK Megas used before call: 128.00390625 //<---- > Megas used after call: 79.23828125 > PEAK Megas used after call: 128.00390625 //<---- > Difference: 0 > PEAK Difference: 0 //<----- > Time: 5.4504480361938 seconds > This is actually suggesting a compelling case to me if this held up under close inspection of the code. You've certainly shown that it solves a problem that userspace is sort of half-solving, due to the limitations of the language around memory. > I want to clarify, as I previously mentioned, I did not develop any parser > at all, the parser already exists in PHP and is use by `json_decode()`, in > short words , what I propose is to create a new function, as an interface > to THE EXISTING PARSER, only, and only, to check if a string is a valid > json-string or not. > This is curious... the existing parser *should* need to allocate the memory. This makes me wonder if the above performance would hold up under scrutiny, *but* you have fully convinced me at the point (not that I have a vote) that it's worth the RFC to explore it. > Regarding publishing the a PR with my implementation, is out of context at > the moment, because we are discussing if a functionality like this should > be included or not, and nothing else. Also I am pretty sure that if this > gets a YES from the community (so far seems so), and I show my > implementation, someone will tell me that it could be done in a different > way , and is totally fine, is great actually. But now, at this moment, > everything is about the functionality. > Ah, okay. That I understand, even if I find it a tad frustrating. You want to understand if people are on-board with the merits of the idea without it being attached to your particular proof-of-concept code. If I'm reading the feedback correctly though, I think that any votes the RFC received would be heavily dependent on the eventual implementation and its performance though. So I think the RFC will be pretty closely tied to an implementation regardless of your efforts. Jordan --- ## #118339 — David Gebler — 2022-08-01T17:35:54+00:00 URL: https://externals.io/message/118310#118339 · In reply to: #118327 On Sun, Jul 31, 2022 at 4:41 PM Larry Garfield wrote: > So the core argument, it seems, is "there's lots of user-space > implementations already, hence demand, and it would be > better/faster/stronger/we-have-the-technology to do it in C." > There's innumerable features implemented in userland which would be faster/stronger/better done in C. I'm not convinced this alone is a sufficient basis to introduce a new core function. There are also userland JSON streaming parsers which are memory efficient, I've used the one I linked to parse JSON files over 1GB no problem. And I'm not saying I'm against this proposal, I'm just covering devil's advocate here - but while I can accept the number of userland implementations for "validate string as JSON" out there clearly show some demand / use cases for doing this, are there equally numerous examples of issues raised on these product repositories demonstrating userland implementations have commonly been insufficient, encountered OOM errors or otherwise caused problems? This might be an RFC to fix a problem very, very few people have. > > Thus another, arguably more important benchmark would be a C > implementation compared to a userspace implementation of the same > algorithm. Presumably your C code is doing some kind of stream-based > validation with braces/quotes matching rather than a naive "try and parse > and see if it breaks." We would need to see benchmarks of the same > stream-based validation in C vs PHP, as that's the real distinction. That > a stream validator would be more memory efficient than a full parser is not > at all surprising, but that's also not a fair comparison. > > As for the benchmarks themselves, do not use `memory_get_usage()`; as noted, > it shows the memory usage at that time, not ever. What you want is > `memory_get_peak_usage()`, which gets the highest the memory usage has gotten > in that script run. Or, even better, use PHPBench with separate sample > methods to compare various different implementations. It will handle all > the "run many times and average the results and throw out outliers" and > such for you. It's quite a flexible tool once you get the hang of it. > > I'll also note that it would be to your benefit to share the working C > code as a patch/PR already. If accepted it would be released open source > anyway, so letting people see the proposed code now can only help your > case; unless the code is awful, in which case showing it later would only > waste your time and everyone else's discussing it in the abstract before > the implementation could be reviewed. > > --Larry Garfield > > -- > > To unsubscribe, visit: https://www.php.net/unsub.php