# POST, content-type: application/json and json_decode - Thread: https://externals.io/message/63212 - Original archive: https://news-web.php.net/php.internals/63212 - Started by Ivan Enderlin @ Hoa on 2012-09-21T11:05:42+00:00 - Messages: 16 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #63212 — Ivan Enderlin @ Hoa — 2012-09-21T11:05:42+00:00 URL: https://externals.io/message/63212#63212 Hello, If PHP receives a HTTP request with the method POST and with the header Content-Type: application/x-www-form-encoded, then, it automatically parses the request body to populate an array in $_POST. If the Content-Type is different (e.g. text/plain or application/json), the request body is reachable by reading php://input. Well, it is ok. But is there any plans to consider application/json by parsing the request body and populate the result in $_POST (with the help of `json_decode()` maybe)? If so, I would like to propose a patch but I don't find in the source code where request body is caugth and parsed (for POST). Any ideas? Maybe a RFC would also be welcome to complete my suggestion? Thanks. -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #63213 — Starry Sky — 2012-09-21T11:31:36+00:00 URL: https://externals.io/message/63212#63213 · In reply to: #63212 Hi, This may lead to a BC break. If the post body contains a variable with the same name in the JSON, the $_POST will be confusing. On Fri, Sep 21, 2012 at 7:05 PM, Ivan Enderlin @ Hoa < ivan.enderlin@hoa-project.net> wrote: > Hello, > > If PHP receives a HTTP request with the method POST and with the header > Content-Type: application/x-www-form-**encoded, then, it automatically > parses the request body to populate an array in $_POST. If the Content-Type > is different (e.g. text/plain or application/json), the request body is > reachable by reading php://input. Well, it is ok. > > But is there any plans to consider application/json by parsing the request > body and populate the result in $_POST (with the help of `json_decode()` > maybe)? > > If so, I would like to propose a patch but I don't find in the source code > where request body is caugth and parsed (for POST). Any ideas? > Maybe a RFC would also be welcome to complete my suggestion? > > Thanks. > > -- > Ivan Enderlin > Developer of Hoa > http://hoa.42/ or http://hoa-project.net/ > > PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) > http://disc.univ-fcomte.fr/ and http://www.inria.fr/ > > Member of HTML and WebApps Working Group of W3C > http://w3.org/ --- ## #63214 — Ivan Enderlin @ Hoa — 2012-09-21T11:32:59+00:00 URL: https://externals.io/message/63212#63214 · In reply to: #63213 > Hi, Hi, > This may lead to a BC break. > If the post body contains a variable with the same name in the JSON, the > $_POST will be confusing. Can you give me an example please? -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #63215 — Starry Sky — 2012-09-21T11:38:20+00:00 URL: https://externals.io/message/63212#63215 · In reply to: #63214 Ignore my former post please.. I forgot the "Content-Type: application/json" in your proposal. On Fri, Sep 21, 2012 at 7:32 PM, Ivan Enderlin @ Hoa < ivan.enderlin@hoa-project.net> wrote: > > > > […] > Hi, > > This may lead to a BC break. > > […] > Can you give me an example please? > > > -- > Ivan Enderlin > Developer of Hoa > http://hoa.42/ or http://hoa-project.net/ > > PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) > http://disc.univ-fcomte.fr/ and http://www.inria.fr/ > > Member of HTML and WebApps Working Group of W3C > http://w3.org/ --- ## #63216 — Ferenc Kovacs — 2012-09-21T11:44:02+00:00 URL: https://externals.io/message/63212#63216 · In reply to: #63212 On Fri, Sep 21, 2012 at 1:05 PM, Ivan Enderlin @ Hoa < ivan.enderlin@hoa-project.net> wrote: > Hello, > > If PHP receives a HTTP request with the method POST and with the header > Content-Type: application/x-www-form-**encoded, then, it automatically > parses the request body to populate an array in $_POST. If the Content-Type > is different (e.g. text/plain or application/json), the request body is > reachable by reading php://input. Well, it is ok. > > But is there any plans to consider application/json by parsing the request > body and populate the result in $_POST (with the help of `json_decode()` > maybe)? > > If so, I would like to propose a patch but I don't find in the source code > where request body is caugth and parsed (for POST). Any ideas? > Maybe a RFC would also be welcome to complete my suggestion? > > Thanks. > > please watch out to not reintroduce CVE-2011-4885, afair we discussed about that json_decode also vulnerable to the hash collision, but I don't remember seeing any fix committed to json_decode. depending on how would you extract the json encoded variables, this would make possible to bypass the protection of max_input_vars limits. -- Ferenc Kovács @Tyr43l - http://tyrael.hu --- ## #63217 — Ivan Enderlin @ Hoa — 2012-09-21T11:57:25+00:00 URL: https://externals.io/message/63212#63217 · In reply to: #63216 > On Fri, Sep 21, 2012 at 1:05 PM, Ivan Enderlin @ Hoa < > ivan.enderlin@hoa-project.net> wrote: > > > […] > please watch out to not reintroduce CVE-2011-4885, afair we discussed about > that json_decode also vulnerable to the hash collision, but I don't > remember seeing any fix committed to json_decode. > depending on how would you extract the json encoded variables, this would > make possible to bypass the protection of max_input_vars limits. Laruence has opened a bug with some patches: https://bugs.php.net/bug.php?id=60655. What is the state of this bug? I don't understand very well the hash collision problem. Any links? -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #63218 — Ferenc Kovacs — 2012-09-21T12:08:06+00:00 URL: https://externals.io/message/63212#63218 · In reply to: #63217 On Fri, Sep 21, 2012 at 1:57 PM, Ivan Enderlin @ Hoa < ivan.enderlin@hoa-project.net> wrote: > > > > > […] > Laruence has opened a bug with some patches: https://bugs.php.net/bug.php? > **id=60655 . What is the state of > this bug? > > I don't understand very well the hash collision problem. Any links? > > you should find everything googling for the CVE id(CVE-2011-4885). basically it was an inefficient handling of the colliding haskeys, which doesn't happen frequently by accident, but a malicious attacker with a small crafted request was able to send a bunch of input variables which will all collide, and triggering that slow codepath, which results in a DOS. see https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2011-4885 and for the theory of the attack here http://www.cs.rice.edu/~scrosby/hash/CrosbyWallach_UsenixSec2003.pdf -- Ferenc Kovács @Tyr43l - http://tyrael.hu --- ## #63219 — Ivan Enderlin @ Hoa — 2012-09-21T12:10:16+00:00 URL: https://externals.io/message/63212#63219 · In reply to: #63218 > On Fri, Sep 21, 2012 at 1:57 PM, Ivan Enderlin @ Hoa < > ivan.enderlin@hoa-project.net> wrote: > > > […] > you should find everything googling for the CVE id(CVE-2011-4885). > basically it was an inefficient handling of the colliding haskeys, which > doesn't happen frequently by accident, but a malicious attacker with a > small crafted request was able to send a bunch of input variables which > will all collide, and triggering that slow codepath, which results in a DOS. > see https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2011-4885 and for the > theory of the attack here > http://www.cs.rice.edu/~scrosby/hash/CrosbyWallach_UsenixSec2003.pdf Ok thanks, got it. Well, Laruence? :-) -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #63220 — jpauli — 2012-09-21T14:16:42+00:00 URL: https://externals.io/message/63212#63220 · In reply to: #63212 On Fri, Sep 21, 2012 at 1:05 PM, Ivan Enderlin @ Hoa wrote: > Hello, > > If PHP receives a HTTP request with the method POST and with the header > Content-Type: application/x-www-form-encoded, then, it automatically parses > the request body to populate an array in $_POST. If the Content-Type is > different (e.g. text/plain or application/json), the request body is > reachable by reading php://input. Well, it is ok. > > But is there any plans to consider application/json by parsing the request > body and populate the result in $_POST (with the help of `json_decode()` > maybe)? > > If so, I would like to propose a patch but I don't find in the source code > where request body is caugth and parsed (for POST). Any ideas? > Maybe a RFC would also be welcome to complete my suggestion? > > Thanks. Hi ! Reading and parsing post data function is defined as a SAPI struct function pointer. You should look at sapi_module_struct definition (http://lxr.php.net/xref/PHP_5_4/main/SAPI.h#251). When a request comes in, sapi_activate() is called. It then calls sapi_read_post_data() http://lxr.php.net/xref/PHP_5_4/main/SAPI.c#459 that itself invokes handlers. Default handlers are defined here : http://lxr.php.net/xref/PHP_5_4/main/php_content_types.c#29 and for POST, by default, sapi_read_standard_form_data() is called (defined here http://lxr.php.net/xref/PHP_5_4/main/SAPI.c#253) This function is in fact just a bridge, it tells PHP to call sapi.read_post() which is a function pointer defined by each SAPI. Finally, later on, sapi.default_post_reader() is called (again, a function pointer defined by each SAPI) I agree it's not a very trivial part though :p Julien.P --- ## #63221 — Ivan Enderlin @ Hoa — 2012-09-21T14:21:13+00:00 URL: https://externals.io/message/63212#63221 · In reply to: #63220 > On Fri, Sep 21, 2012 at 1:05 PM, Ivan Enderlin @ Hoa > wrote: > > […] > Hi ! > > Reading and parsing post data function is defined as a SAPI struct > function pointer. > You should look at sapi_module_struct definition > (http://lxr.php.net/xref/PHP_5_4/main/SAPI.h#251). > > When a request comes in, sapi_activate() is called. It then calls > sapi_read_post_data() http://lxr.php.net/xref/PHP_5_4/main/SAPI.c#459 > that itself invokes handlers. > Default handlers are defined here : > http://lxr.php.net/xref/PHP_5_4/main/php_content_types.c#29 and for > POST, by default, sapi_read_standard_form_data() is called (defined > here http://lxr.php.net/xref/PHP_5_4/main/SAPI.c#253) > This function is in fact just a bridge, it tells PHP to call > sapi.read_post() which is a function pointer defined by each SAPI. > Finally, later on, sapi.default_post_reader() is called (again, a > function pointer defined by each SAPI) > > I agree it's not a very trivial part though :p No it's clear. Thanks for clarifying this! -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #63226 — Ivan Enderlin @ Hoa — 2012-09-24T14:21:33+00:00 URL: https://externals.io/message/63212#63226 · In reply to: #63220 > On Fri, Sep 21, 2012 at 1:05 PM, Ivan Enderlin @ Hoa > wrote: > > […] > Hi ! > > Reading and parsing post data function is defined as a SAPI struct > function pointer. > You should look at sapi_module_struct definition > (http://lxr.php.net/xref/PHP_5_4/main/SAPI.h#251). > > When a request comes in, sapi_activate() is called. It then calls > sapi_read_post_data() http://lxr.php.net/xref/PHP_5_4/main/SAPI.c#459 > that itself invokes handlers. > Default handlers are defined here : > http://lxr.php.net/xref/PHP_5_4/main/php_content_types.c#29 and for > POST, by default, sapi_read_standard_form_data() is called (defined > here http://lxr.php.net/xref/PHP_5_4/main/SAPI.c#253) > This function is in fact just a bridge, it tells PHP to call > sapi.read_post() which is a function pointer defined by each SAPI. > Finally, later on, sapi.default_post_reader() is called (again, a > function pointer defined by each SAPI) I have some questions. Julien told me in private that he didn't know, hope other could reply. By reading the code, I understand that each SAPI is responsible of parsing the body of a POST request if it is application/x-www-form-urlencoded. Am I right? In SAPI.c, I read that it delegates this task to sapi_module.read_post by giving SG(request_info).post_data (please, see main/SAPI.c#266). It implies that if we would like to add the support of application/json, then we have to do it for each SAPI? Not very nice :-/. When I will locate the code to edit, I'm planning to use php_json_decode_ex & co. (please, see ext/json/json.c#633), but the hash collision risk is always present. In every case, `json_decode()` must be protected again hash collision (please, see https://bugs.php.net/60655). Thank you for your help, Best regards. -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #63228 — jpauli — 2012-09-24T14:48:37+00:00 URL: https://externals.io/message/63212#63228 · In reply to: #63226 On Mon, Sep 24, 2012 at 4:21 PM, Ivan Enderlin @ Hoa wrote: > > > […] > > I have some questions. Julien told me in private that he didn't know, hope > other could reply. > > By reading the code, I understand that each SAPI is responsible of parsing > the body of a POST request if it is application/x-www-form-urlencoded. Am I > right? In SAPI.c, I read that it delegates this task to > sapi_module.read_post by giving SG(request_info).post_data (please, see > main/SAPI.c#266). > > It implies that if we would like to add the support of application/json, > then we have to do it for each SAPI? Not very nice :-/. > > When I will locate the code to edit, I'm planning to use php_json_decode_ex > & co. (please, see ext/json/json.c#633), but the hash collision risk is > always present. In every case, `json_decode()` must be protected again hash > collision (please, see https://bugs.php.net/60655). The hash collision is a behavior our HashTable implementation carries anyway. Removing it can only be done safely by changing the hash algorithm, which is not such an easy task in case of PHP. I suggest you start your POC writting the json handler without worrying about hash collisions :) Julien.P --- ## #63229 — Ivan Enderlin @ Hoa — 2012-09-24T14:50:10+00:00 URL: https://externals.io/message/63212#63229 · In reply to: #63228 > On Mon, Sep 24, 2012 at 4:21 PM, Ivan Enderlin @ Hoa > wrote: > > […] > The hash collision is a behavior our HashTable implementation carries anyway. > Removing it can only be done safely by changing the hash algorithm, > which is not such an easy task in case of PHP. > > I suggest you start your POC writting the json handler without > worrying about hash collisions :) It was my goal, yup. -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/ --- ## #63231 — Andrew Faulds — 2012-09-24T18:09:09+00:00 URL: https://externals.io/message/63212#63231 · In reply to: #63212 > Hello, > > If PHP receives a HTTP request with the method POST and with the > header Content-Type: application/x-www-form-encoded, then, it > automatically parses the request body to populate an array in $_POST. > If the Content-Type is different (e.g. text/plain or > application/json), the request body is reachable by reading > php://input. Well, it is ok. > > But is there any plans to consider application/json by parsing the > request body and populate the result in $_POST (with the help of > `json_decode()` maybe)? I think perhaps that's the wrong approach. It's valid JSON to have a list literal at the top level as well, I think. Perhaps we should introduce a new $_JSON variable or some such thing? $_BODY? > > If so, I would like to propose a patch but I don't find in the source > code where request body is caugth and parsed (for POST). Any ideas? > Maybe a RFC would also be welcome to complete my suggestion? > > Thanks. > -- Andrew Faulds http://ajf.me/ --- ## #63233 — Andrew Faulds — 2012-09-24T18:11:47+00:00 URL: https://externals.io/message/63212#63233 · In reply to: #63231 > I think perhaps that's the wrong approach. It's valid JSON to have a > list literal at the top level as well, Excuse my brief brain lapse. PHP arrays are fine for sequential data as well, and you could also have any other JSON literal value in $_POST. (Although, decoding objects to arrays would be inconvenient, I'd much prefer to decode to objects, and doing that might be confusing considering $_POST is usually an associative array...) -- Andrew Faulds http://ajf.me/ --- ## #63234 — Ivan Enderlin @ Hoa — 2012-09-25T08:02:36+00:00 URL: https://externals.io/message/63212#63234 · In reply to: #63231 > > > […] > I think perhaps that's the wrong approach. It's valid JSON to have a > list literal at the top level as well, I think. Perhaps we should > introduce a new $_JSON variable or some such thing? $_BODY? $_BODY === php://input. -- Ivan Enderlin Developer of Hoa http://hoa.42/ or http://hoa-project.net/ PhD. student at DISC/Femto-ST (Vesontio) and INRIA (Cassis) http://disc.univ-fcomte.fr/ and http://www.inria.fr/ Member of HTML and WebApps Working Group of W3C http://w3.org/