Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:100534 Return-Path: Mailing-List: contact internals-help@lists.php.net; run by ezmlm Delivered-To: mailing list internals@lists.php.net Received: (qmail 64697 invoked from network); 12 Sep 2017 04:04:31 -0000 Received: from unknown (HELO lists.php.net) (127.0.0.1) by localhost with SMTP; 12 Sep 2017 04:04:31 -0000 Authentication-Results: pb1.pair.com header.from=php-lists@koalephant.com; sender-id=pass Authentication-Results: pb1.pair.com smtp.mail=php-lists@koalephant.com; spf=pass; sender-id=pass Received-SPF: pass (pb1.pair.com: domain koalephant.com designates 206.123.115.54 as permitted sender) X-PHP-List-Original-Sender: php-lists@koalephant.com X-Host-Fingerprint: 206.123.115.54 mail1.25mail.st Received: from [206.123.115.54] ([206.123.115.54:38394] helo=mail1.25mail.st) by pb1.pair.com (ecelerity 2.1.1.9-wez r(12769M)) with ESMTP id 8B/AE-10715-9CC57B95 for ; Tue, 12 Sep 2017 00:04:27 -0400 Received: from [172.20.10.3] (ppp-223-24-146-240.revip6.asianet.co.th [223.24.146.240]) by mail1.25mail.st (Postfix) with ESMTPSA id 3492760467; Tue, 12 Sep 2017 04:04:12 +0000 (UTC) Message-ID: <15572C6C-4C44-4E76-9B1D-394E663FDBCD@koalephant.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_6C44A3A2-5B96-4796-86BB-8B887A78B3C1" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Tue, 12 Sep 2017 11:04:03 +0700 In-Reply-To: Cc: Tony Marston , "internals@lists.php.net" To: Yasuo Ohgaki References: <2a4491b4-e6f5-4297-beec-363f373a93e6@lsces.co.uk> <3f8be7b1-0e59-21c6-4fe8-8299b2c05645@rhsoft.net> <6ba62d62-f1ab-9e7b-93f0-a1a9238c47a6@lsces.co.uk> <0db9cfa3-2b31-ee41-713c-889b7cc06406@lsces.co.uk> <3C.DD.10715.4E501B95@pb1.pair.com> <93.85.10715.AB3B3B95@pb1.pair.com> <049578E9-4C9A-42D8-B206-8ABAF070E151@koalephant.com> <05A8DB1C-4683-4934-A7DA-C7CD71E6CCB6@koalephant.com> X-Mailer: Apple Mail (2.3273) Subject: Re: [PHP-DEV] A validator module for PHP7 From: php-lists@koalephant.com (Stephen Reay) --Apple-Mail=_6C44A3A2-5B96-4796-86BB-8B887A78B3C1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 12 Sep 2017, at 04:07, Yasuo Ohgaki wrote: >=20 > Stephen, >=20 > On Tue, Sep 12, 2017 at 12:22 AM, Stephen Reay = > wrote: >=20 >> On 11 Sep 2017, at 17:41, Yasuo Ohgaki > wrote: >>=20 >> Hi Stephen, >>=20 >> On Mon, Sep 11, 2017 at 6:37 PM, Stephen Reay = > >> wrote: >>=20 >>> On 11 Sep 2017, at 15:42, Yasuo Ohgaki > wrote: >>>=20 >>> It seems you haven't try to use filter module seriously. >>> It simply does not have enough feature for input validations. >>> e.g. You cannot validate "strings". >>>=20 >>>=20 >>> Yasuo, >>>=20 >>> I=E2=80=99ve asked previously what your proposal actually offers = over the filter >>> functions, and got no response, so please elaborate on this? >>>=20 >>=20 >>=20 >>> Can you show a concrete example that cannot be validated in user = land >>> currently, using the filter functions as a base? >>>=20 >>=20 >> FILTER_VALIDATE_REGEXP is not good enough simply. >> PCRE is known that it is vulnerable to regex DoS still. (as well as >> Oniguruma) >> Users should avoid regex validation whenever it is possible also to = avoid >> various >> risks. >>=20 >> In addition, current filter module does not provide nested array = validation >> array key validation, etc. It's not true validation neither. It does = not >> provide >> simple length, min/max validations. It does non explicit conversions = (i.e. >> trim), etc. >> Length, min/max validation is mandatory validation if you would like = to >> follow >> ISO 27000 requirement. >>=20 >> Regards, >>=20 >> -- >> Yasuo Ohgaki >> yohgaki@ohgaki.net >=20 > So, you still didn=E2=80=99t actually provide an example. I *guess* = you=E2=80=99re talking about character class validation or something = else equally =E2=80=9Csimple=E2=80=9D, because I can=E2=80=99t imagine = what else would be a common enough case that you=E2=80=99d want to have = built-in rules for, and that you wouldn=E2=80=99t internally use RegExp = to test anyway. >=20 > Your request is like "Devil's Proof". Example code that cannot do = things=20 > with existing API cannot exist with meaningful manner. It can be = explained=20 > why it cannot, though. Try what "validate" string validator can do, =20= > Then you'll see. >=20 > $input =3D [ > 'defined_but_should_not_exist' =3D> 'Developer should not allow = unwanted value', > '_invalid_utf8_key_should_not_be_allowed_' =3D> 'Developer should = validate key value as well', > 'utf8_text' =3D> 'Validator should be able to allow UTF-8 and = validate its validity at least', > 'default_must_be_safe' =3D> 'Crackers send all kinds of chars. CNTRL = chars must not be allowed by default', > 'array' =3D> [ > 'complex' =3D> 1, > 'nested' =3D> 'any validation rule should be able to be = applied', > 'array' =3D> 1, > 'key_should_be_validated_also' =3D> 1, > 'array' =3D> [ > 'any_num_of_nesting' =3D> 'is allowed', > ], > ], > 'array_num_elements_must_be_validated' =3D> [ > "a", "b", "c", "d", "e", "f", "and so on", "values must be able = to be validated as user wants", > ], > ]; >=20 > There is no STRING validation filter currently. This fact alone, > it could be said "filter cannot do string validation currently". >=20 > List of problems in current validation filter > - no STRING validator currently > - it allows any inputs by default > - it does not allow multiple rules that allows complex validation = rules for string > - it does not have callback validator > - it does not have key value validation (note: PHP's key could be = binary) > - it does not validate num of elements in array. > - it cannot forbids unwanted elements in array. > - it cannot validate "char encoding". > - it does not enforce white listing. > - and so on >=20 > These are the list that "filter" cannot do. >=20 > Ok so we can=E2=80=99t use filter_var() rules to validate that a = string field is an Alpha or AlphaNum, between 4 and 8 characters long = (technically you could pass mb_strlen() to the INT filter with = {min,max}_range options set to get the length validation, but I=E2=80=99ll= grant you that *is* kind of a crappy workaround right now) >=20 > Why not stop trying to re-invent every single feature already present = in PHP (yes, I=E2=80=99ve been paying attention to all your other = proposals), and just *add* the functionality that=E2=80=99s missing: >=20 > https://wiki.php.net/rfc/add_validate_functions_to_filter = > It's _declined_. You should have supported this RFC if you would like = to add features to filter. > (I'm glad there is a new RFC supporter regardless of occasion) >=20 > I don't mind this result much. > Adding features to "filter" has some of shortcomings mentioned above > even with my proposal. >=20 > A `FILTER_VALIDATE_STRING` filter, with =E2=80=9COptions=E2=80=9D of = `min` =3D> ?int, `max` =3D> ?int and =E2=80=9CFlags=E2=80=9D of = FILTER_FLAG_ALPHA, FILTER_FLAG_NUMERIC (possibly a built in bit mask = =E2=80=9CFILTER_FLAG_ALPHANUMERIC=E2=80=9D ?)=20 >=20 > Simply adding these wouldn't work well as validator because >=20 > - Filter is designed for black listing >=20 > As you may know, all of security standards/guidelines require=20 >=20 > - White listing for validation >=20 > We may change "filter", but it requires BC. > =20 >=20 > Lastly: it may not be the format you personally want, but the filter = extension *does* have the `filter_{input,var}_array` functions. Claiming = something doesn=E2=80=99t exist because it doesn=E2=80=99t work exactly = how you would like it to, makes you seem immature and petty, IMO. >=20 > Discussion is confusing because you ignore this RFC result.=20 > https://wiki.php.net/rfc/add_validate_functions_to_filter = > This RFC proposes filter module improvement while keeping = compatibility. >=20 > I understand your point. This exactly the same reason why I proposed > "improvement" at first, not new extension.=20 >=20 > I don't understand why you insist already failed attempt repeatedly. >=20 > Would you like me to propose previous RFC again? > and implement "ture validation" with filter? > I don't mind implementing it if you would like to update the RFC and = it passes. > I must use "white list" as much as possible. >=20 > Regards, >=20 > P.S. "Filter" module is black listing module. "Validate" is white = listing module. > Even with BC, mixing them would result in confusing FLAGs and codes.=20= > Codes may be cleaned up later, but FLAGs cannot. > We should consider this also. >=20 > -- > Yasuo Ohgaki=20 > yohgaki@ohgaki.net >=20 >=20 I was going to give a lot of detailed replies inline, but I=E2=80=99ve = come to the realisation its pointless with you. You really respond to = what people say, you just use their comments as jumping off points to = re-post your same little rant, ad nauseam. So here=E2=80=99s the summary. Don=E2=80=99t both replying, because I = won=E2=80=99t be reading it. - I never asked for a working code example that is impossible with the = current extension. I asked for a simple example of what you wanted to = achieve. - More than half the =E2=80=9Cissues=E2=80=9D you claim with the filter = extension, are only =E2=80=9Cvalid=E2=80=9D if you agree that it needs = to do complex array structure validation. I do not agree with this. = Userland can iterate an array of rules/input and validate quite easily. - The *actual* issues with the filter extension could be solved by = improving/adding filters. - I already agreed that a string based filter (to test character class = and min/max length, etc) would be a good addition. Continuing to bring = it up when others have acknowledged something doesn=E2=80=99t help your = case, at all. - ACCEPTING that string validation is missing, number/bool/ is still whitelisting. I don=E2=80=99t know how = they=E2=80=99re implemented in C. Maybe that needs improvement. But a = rule saying =E2=80=9Cvalidate that $X is a number between 10 and 100=E2=80= =9D or =E2=80=9Cvalidate that $Y is an email address) is whitelisting. --Apple-Mail=_6C44A3A2-5B96-4796-86BB-8B887A78B3C1--