Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:95361
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain ohgaki.net designates 180.42.98.130 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <267d257c-a093-d25b-02ea-737b535fa1e2@gmail.com>
References: <CAGa2bXarw2jTj0yhXpywWNQOJLR2pdhfaqSdwj_btT2giEL15Q@mail.gmail.com>
 <CAGa2bXY-vmEgEFrE0OxQPSx=FKHeHdCayrDXxtTkHUVaJbV+-A@mail.gmail.com>
 <CA+kxMuRiOBQpmTeKqNyV8rX0GKCLrYixi--y5TcYUkdqpT746w@mail.gmail.com>
 <CAGa2bXa78fkZw8gtepmBDu+VDwHW_WiDDv65=zzgLTFqYxF+DA@mail.gmail.com>
 <CADyq6s+Y-OeKEyMus5s7xp9MqY7Fq7udqd5BRwtyGXbYCY1Uag@mail.gmail.com>
 <CADyq6sJuHCrLOoL22MWHv+cfrQQx9z=avswCpDVmnuyBCAchsA@mail.gmail.com>
 <CADyq6sKs=e_1Pc_CCLuThsU-qsvRseQcpx=sWKB_uhWsg=aRmQ@mail.gmail.com>
 <7795ca21-bd70-fe65-9519-af95fdfee33f@gmail.com> <CAGa2bXZMrO__dVd=qVoKjuPmcvVJ-q4xa8XKc0+aj1kH7Xd2Fw@mail.gmail.com>
 <fe3ccc4c-e81e-34fc-3257-19bd943e4abe@gmail.com> <CAGa2bXYy_tNiRfmy=jTsy-GALN4t8SDfxZ5vZocVCMw_7mMO6Q@mail.gmail.com>
 <40279244-a1ba-2680-8a14-89708bcd1852@gmail.com> <CAGa2bXYfaAXHay0txn46ui6T+nJ78ey=zCm9tKuWU2vCpEKTpg@mail.gmail.com>
 <ba10221f-717c-3c8c-9768-23d6a4cf0386@gmail.com> <CAGa2bXZ8CiuZ1ofHSu5psQcdcFy5ad6+JU9oagc_q40QsFkW7g@mail.gmail.com>
 <267d257c-a093-d25b-02ea-737b535fa1e2@gmail.com>
Date: Mon, 22 Aug 2016 13:21:10 +0900
Message-ID: <CAGa2bXbwibOrGzuebzPLmCiV_gz=mDc3Cxf9jyT6BJ6OxPZkUQ@mail.gmail.com>
To: Stanislav Malyshev <smalyshev@gmail.com>
Cc: Marco Pivetta <ocramius@gmail.com>, Dan Ackroyd <danack@basereality.com>, 
	PHP Internals List <internals@lists.php.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: [PHP-DEV] Re: [RFC][VOTE] Add validation functions to filter module
From: yohgaki@ohgaki.net (Yasuo Ohgaki)

Hi Stas,

On Sun, Aug 21, 2016 at 1:39 PM, Stanislav Malyshev <smalyshev@gmail.com> w=
rote:
>
>> It's possible to design web pages/services to "unknown clients", but
>> it's exceptional cases.
>
> No, it's not. Every service exposed to the internet is for unknown client=
s.

Right and wrong.

Every service exposed to the internet is for unknown clients, but we
can impose certain input/output rules defined by web application
developers and/or standards.

>
>> Almost all systems have intended clients. If protocol is HTTP/HTTPS,
>> developers may reject strange data that cannot be right for
>> HTTP/HTTPS. Even higher level than PHP does this. i.e. HTTP servers
>> will rejects malformed and/or prohibited request and terminates
>> execution. Web Application Firewall does more fancy things and
>
> True, because we can have no PHP business logic that can work on HTTPS
> level. We can have PHP business logic that works on PHP level though.
>
>> validate/check, in general.  IMHO, use of WAF is more burden and
>> costly than the input validation that I'm proposing.
>
> I'm not sure where WAF comes in - these are two completely different uses=
.

Not really.

The more application validates inputs, the less WAF rules for web app
are required. There is close relationship between web apps and WAF.

WAF can do more than web application like mitigating protocol/web
server/language vulnerabilities. This differs, but most web app
protection rules can be implemented in web app. Whitelist parameter
validation in WAF is hard, but it's easy in web app because developers
know what parameter should be exactly. e.g. "id" parameter could be
integers or strings like userid, WAF admins have to refer to code to
be precise while app developers know.

>
>> Web application developers have right to define "valid" inputs. ("have
>> right" does not mean "can do anything") PHP script termination for
>> invalid input is just one of terminations. It's nothing special.
>
> You can perfectly well have your app do anything you like - including
> termination - on filter failure. I don't see how it necessitates making
> new set of filters though?
>
>> I think your premise is "Show nice error message for any errors,
>> proceed as normal case".  (Handle invalid/insane data just like mistakes=
)
>>
>> My premise is "Shouldn't show nice messages to attacker, terminate as
>> abnormal case". (Treat them as attack or serious system bug)
>>
>> It's design choice. Either way is possible.
>
> Sure, and it's possible with current filters too. If the check fails,
> and you want to terminate, what prevents you from doing:
>
> if(!filter_var($var, FILTER_VALIDATE_INT) =3D=3D=3D false) { exit(); }
>
> Am I missing some important point here?

There is no string validation filter while string is the most dangerous inp=
ut.

It does not allow multiple rules for an array element with filter_*_array()=
.

Current validation filters are designed for filter and convert. e.g.
FILTER_VALIDATE_BOOL converts empty to FALSE.

Do you think it's nicer to have many lines of

// Assuming log_validation_error_and_exit($key_name, $value)
// is implemented properly.
if(!filter_var($_POST['int'], FILTER_VALIDATE_INT) =3D=3D=3D false) {
log_validation_error_and_exit('int', $_POST['int']); }
if(!filter_var($_POST['bool'], FILTER_VALIDATE_BOOL) =3D=3D=3D false) {
log_validation_error_and_exit('bool', $_POST['bool']); }
if(!filter_var($_POST['float'], FILTER_VALIDATE_FLOAT) =3D=3D=3D false) {
log_validation_error_and_exit('float', $_POST['float']); }
if(!filter_var($_POST['string'],
FILTER_VALIDATE_REGEX,array("options"=3D>array("regexp"=3D>"/^M(.*)/")))
=3D=3D=3D false) { log_validation_error_and_exit('string',
$_POST['string']); }

rather than this?

// Assuming exception handler is set properly.
$post_def =3D array(
   'int' =3D> FILTER_VALIDATE_INT,
   'bool' =3D> FILTER_VALIDTE_BOOL,
   'float' =3D> FILTER_VALIDATE_FLOAT,
   'string' =3D> array(FILTER_VALIDTE_REGEX,
                    array("options"=3D>array("regexp"=3D>"/^M(.*)/")
                  ),
);
filter_require_var_array($_POST, $post_def);

Try to implement "strict validation rules" with current filter,
especially with filter_*_array(). They do not work well as I mentioned
in the RFC.

https://wiki.php.net/rfc/add_validate_functions_to_filter#why_not_compare_f=
ilter_var_array_result
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Why not compare filter_var_array() result?

Following code may seem to work, but it would not.

$ret =3D filter_var_array($arr, $validation_spec);
if ($ret !=3D $arr) {
  die('Input does not validate');
}

One should never compare float equality. (Float string is converted to
float type. Think of huge string value and result of float converted
value comparison.)
They are filter(conversion) functions. e.g. URLs are converted to lowercase=
.
It allows empty input by default and add NULL element.
int/float/bool validation filters trim and convert type. (They cannot
match by =E2=80=9C=3D=3D=E2=80=9D comparison)

For these reasons, comparing original and return(filtered) value is
not suitable for strict input validation.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

It's obvious to me current filter module needs improvements to write
better validation code.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net