Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:100556 Return-Path: Mailing-List: contact internals-help@lists.php.net; run by ezmlm Delivered-To: mailing list internals@lists.php.net Received: (qmail 65745 invoked from network); 13 Sep 2017 06:24:10 -0000 Received: from unknown (HELO lists.php.net) (127.0.0.1) by localhost with SMTP; 13 Sep 2017 06:24:10 -0000 Authentication-Results: pb1.pair.com header.from=pthreads@pthreads.org; sender-id=unknown Authentication-Results: pb1.pair.com smtp.mail=pthreads@pthreads.org; spf=permerror; sender-id=unknown Received-SPF: error (pb1.pair.com: domain pthreads.org from 209.85.217.169 cause and error) X-PHP-List-Original-Sender: pthreads@pthreads.org X-Host-Fingerprint: 209.85.217.169 mail-ua0-f169.google.com Received: from [209.85.217.169] ([209.85.217.169:35498] helo=mail-ua0-f169.google.com) by pb1.pair.com (ecelerity 2.1.1.9-wez r(12769M)) with ESMTP id F2/5B-10715-40FC8B95 for ; Wed, 13 Sep 2017 02:24:06 -0400 Received: by mail-ua0-f169.google.com with SMTP id c27so17611576uah.2 for ; Tue, 12 Sep 2017 23:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pthreads-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=yLDLzJ0l++AJKWfT3Toy8CBfQsw7x7ghYwjdHG1U4zs=; b=TaH655geZr8sgIt/GLR+zww9e6cBInAT3ymMm7dk0KsGZr+DzuJc+dqaWBjFfMjADz HIFSvwZDXNoExEEqp6Oz6NexyolcJ3CbDtvWGhJA2IWUxwi9jCK3R36qsKTW7AcGDaf3 G8gIWuuJPe8ELCprw7OuKT237SO34d9zHiD1NLUAIOJvfPfaSer2I84bvZvBtb8GGPX8 5coWkZm2U/glQ0C6sHxLLM3Cg7vXRIHgqinlYoSXByebKqszepJyZ7ExkOqDGUrL0Em9 lGq+VWSRftgQzVmgqpjVUp8UYz/RunuHADMR1I//BjlS3JpCSoj+ztPZ/ghRAlv2FusG 12kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=yLDLzJ0l++AJKWfT3Toy8CBfQsw7x7ghYwjdHG1U4zs=; b=AjDijF9aDyWe9rM7nKzjtxeqHhO9T7zs1HYzdAp6rpKKTi+vARlYY3RK4W3pqJR+aY nAE+NixG2N4yJLCskKhjxPQtycruqhgjd+rQfensMDWkxr2Q2stuURmVetQ7gf7qoE4H 4gzjmt7/cbnA9fZOWut46f2N3W5IARNgqY1EtHtl2dPYqEkXy06zecbk9DAVMQX5mHlq lXOMh4EsH+R/RYhMibn2ChYU+Zrf+FaosV4H6AjzG3Fk1/H6JaHPlEimJumI4uDWHBj4 HG0inPRebzU5UGkhyCT0ZdRHp2znRIeDR3iHTb04pQcjnpGMWC5gmuYbi4tHWGHDz2/L BJfQ== X-Gm-Message-State: AHPjjUiBaIMgbRTchJSeQIlqiJUmxX2v66Qz/wFbwPDqvKO67/oRWg+9 ENVUE/cQZW+4hbuDZ42iFLLcE+j7nswrVd64GJ1URw== X-Google-Smtp-Source: ADKCNb44r3dAggAKOKaseY/P6a3Hu/PaX3BQ1Jj7bV5Tmj+oMrb8vWILxI9VPubrv8CPUjZbiaSIwt5FCimj7p+QbpE= X-Received: by 10.176.82.103 with SMTP id j36mr14782356uaa.156.1505283842038; Tue, 12 Sep 2017 23:24:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.19.174 with HTTP; Tue, 12 Sep 2017 23:24:01 -0700 (PDT) X-Originating-IP: [81.143.223.160] In-Reply-To: References: Date: Wed, 13 Sep 2017 07:24:01 +0100 Message-ID: To: Yasuo Ohgaki Cc: "internals@lists.php.net" , Nikita Popov Content-Type: multipart/alternative; boundary="94eb2c18fc5eb0f2a305590c3484" Subject: Re: [PHP-DEV] Re: hash_hkdf() signature and return value From: pthreads@pthreads.org (Joe Watkins) --94eb2c18fc5eb0f2a305590c3484 Content-Type: text/plain; charset="UTF-8" This proposal was rejected. 6 months has not passed since it was rejected. There will be no vote on these proposals in the near future. Please stop. Joe On Tue, Sep 12, 2017 at 3:52 AM, Yasuo Ohgaki wrote: > Hi all, > > hash_hkdf() discussion was mess, but it came to conclusion finally a while > ago. > Apparently, there are confused readers still. I discovered it in other > thread. > I would like to make it clear again what's wrong in the discussion. > If you think I was wrong about HKDF, you should read this. > > > Nikita and Adnrey's Opinion: > 1. Keys("IKM") must be strong always > (with very limited exception? Usage example is not disclosed) > 2. From 1, "Salt" is pure optional parameter > 3. HKDF is designed for specific purpose > (crypto keys only? I suppose) > > Therefore, "Salt" is the last optional parameter and hash_hkdf() returns > BINARY. > > > I assume Andrey realized his misunderstanding finally and would not repeat > this ridiculous discussion. Andrey, you should realize when you couldn't > show > us any valid usage example. However, I appreciate discussion because I > couldn't > find out where you misunderstood exactly and why you insisted above > without > discussion. > > Anyway, their statements make none of sense with HKDF specifications. > It is easy to accuse "You are wrong" or "You don't understand" regardless > of > validity. It requires a lot of work to make it clear what's really > correct/incorrect. > Those who are still do not understand what the HKDF RFC is about, please > read carefully what the RFC states. > > > > What HKDF RFC says: > > The algorithm: https://tools.ietf.org/html/rfc5869#section-2.2 > ---- > HKDF-Extract(salt, IKM) -> PRK > PRK = HMAC-Hash(salt, IKM) > ---- > From this algorithm alone, HKDF is clearly expecting weak keys("IKM"). > i.e. Remember how hash_hmac() works. > > > About "Salt": https://tools.ietf.org/html/rfc5869#section-3.1 > ---- > HKDF is defined to operate with and without random salt. This is > done to accommodate applications where a salt value is not available. > We stress, however, that the use of salt adds significantly to the > strength of HKDF, ensuring independence between different uses of the > hash function, supporting "source-independent" extraction, and > strengthening the analytical results that back the HKDF design. > ---- > It is clear that salt is made optional only for apps that cannot use salt. > It is clear that HKDF recommends "Salt" regardless of key("IKM") strength. > "Salt" is required for "source-independent" extraction. i.e. Make IKM > entropy/strength irrelevant and derived key stronger. > > > Omitting "Salt": https://tools.ietf.org/html/rfc5869#section-3.3 > ---- > In some applications, the input key material IKM may already be > present as a cryptographically strong key. In this case, one can skip > the > extract part and use IKM directly to key HMAC in the expand step. > ---- > It says "Salt" may be omitted __only when__ keys("IKM") are strong already. > In other words, "Salt" is mandatory for not cryptographically strong > keys("IKM"). > > e.g. $rsa256key = hash_hkdf("sha256", $rsa128key) is misuse, _NOT_ textbook > HKDF usage example as Nikita mentioned. > hash_hkdf("sha256", $rsa128key, 0, '', $random256salt) is correct/designed > usage. > > > > Reason Why Nikita and Andrey misunderstood the RFC: > > They ignored previously explained descriptions and cherry picked irrelevant > statements in the RFC. > > Application of HKDF: https://tools.ietf.org/html/rfc5869#section-4 > ----- > On the other hand, it is anticipated that some applications will not > be able to use HKDF "as-is" due to specific operational requirements, > or will be able to use it but without the full benefits of the > scheme. One significant example is the derivation of cryptographic > keys from a source of low entropy, such as a user's password. The > extract step in HKDF can concentrate existing entropy but cannot > amplify entropy. In the case of password-based KDFs, a main goal is > to slow down dictionary attacks using two ingredients: a salt value, > and the intentional slowing of the key derivation computation. HKDF > naturally accommodates the use of salt; however, a slowing down > mechanism is not part of this specification. > ----- > > Especially this part > > ----- > the derivation of cryptographic > keys from a source of low entropy, such as a user's password. The > extract step in HKDF can concentrate existing entropy but cannot > amplify entropy. > ----- > > It says HKDF cannot be used with low entropy/weak keys("IKM"). This > supports > their opinion. > > However, these are explanations for __"Invalid/Wrong HKDF usage"__ as > "Password > hashing like PBKDF2". These are __irrelevant__ for "Valid/Correct HKDF > usage" at > all. i.e. This basically says "Hey, you shouldn't use HKDF for password > hashing "as-is". > HKDF is not password hashing that requires stretching.". > > Key derivation and password hashing is fundamentally different operation > because "Salt" in password hashing is _non secret_ always by design. > Password hashing requires entropy amplification(stretching), but key > derivation does not require amplification by HMAC/cryptographic hash > characteristics. > > Even with weakest password, password(IKM) is protected and OKM is secure > because HKDF's "Salt" is designed to be "secret" or "non secret" as > explicitly > described in the RFC. Secret salt makes keys are secured. This is obvious > from the algorithm, i.e. Remember how hash_hmac() works again. > > > Lastly, > Application of HKDF: https://tools.ietf.org/html/rfc5869#section-4 > states in the first sentence. > ----- > HKDF is intended for use in a wide variety of KDF applications. > ----- > It is obvious that HKDF is general purpose KDF. The RFC even explains > non KDF usage as CSPRNG. > > > > My Comments: > From the algorithm alone, it is very clear to me that HKDF is expecting > weak keys, and > salt is used for additional entropy/complexity for key security, and salt > should be used > always. > > It was very difficult for me to understand what Nikita and Andrey were > insisting and > referring, because their discussion was very confusing and makes none of > sense > with HKDF RFC.(and HMAC, crypto hash characteristics) > > Current API > - encourages insecure misuse, "Salt" is mandatory always with the > exception > when salt cannot be used. > - extremely insecure, weak keys are vulnerable without "secret Salt" > i.e. Who would use hash_hmac() without keys? > > In addition, it has completely unnecessary API inconsistency with respect > to other > hash functions. > > Nikita and Andrey, don't you have something to say about this mess, do you? > > IMO, you should ask to the list by yourself so that your ridiculous > hash_hkdf() API > will not be kept forever because of your misunderstanding. If I were you, I > would > not leave behind such shameful API that would be laughed at by everyone. > > RMs, if I were you, I will consider fixing the API seriously. Promoting > insecure > _key_ derivations forever is nonsense. > > This would be the last post for this issue, unless there are people who > still don't > understand what HKDF is. > > Regards, > > P.S. > No one really read the internet RFC, this is another issue... > Session security and input data security discussion have similar pattern. > I insists based on standards and/or guidelines, others just ignore > standards/guidelines/security because they don't use/are not familiar with > them. > If it is simply a matter of preference, it would be acceptable. However, > in case of security related issues, this is not a healty technical > discussion > at all. > > > -- > Yasuo Ohgaki > yohgaki@ohgaki.net > > On Wed, Sep 6, 2017 at 10:15 AM, Yasuo Ohgaki wrote: > > > Hi all, > > > > This is the last recommendation for hash_hkdf[1]. In fact, > > this would be the last chance to fix because we'll have 7.2 soon. > > The issue is secure usage and API consistency. > > > > Currently hash_hkdf() has following signature: > > > > hash_hkdf(string $algo , string $ikm [, int $length = 0 > [, > > string $info = '' [, string $salt = '' ]]] ) > > > > These are rationals behind recommendation. There are many, but > > please read on. > > > > === Parameter Order === > > > > HKDF[2] algorithm is: > > 1. General purpose key derivation function as per RFC 5869 > > 2. "$salt" parameter is a "pre-shared _KEY_" in many cases as mentioned > > RFC 5869 > > 3. "$salt" (or preshared key) is very strongly recommended for security > > season as per RFC 5869 > > 4. Supplying salt that the same length of input key does not affect > > performance as per RFC 5969 > > 5. "$info" is what makes HKDF useful, but it's less important as > > described in RFC 5869 > > 6. "$length" is truly an optional parameter for very specific encryption > > algorithm or usage. > > > > Rationale for change: > > 1. Key derivations without secondary key ($salt) does not make sense > when > > secondary key could be available. HKDF is designed for best possible > > key > > security with the key. Not using secondary key ($salt) simply > > downgrades > > key security without no reason. i.e. HKDF performance is the same > > when $salt has the same as hash is set. > > 2. HKDF is based on HMAC. When $info has no use, HMAC would be the best > > choice for it. i.e. $newkey = hash_hmac($ikm, $key); > > 3. It does not make sense violating RFC recommendations for a RFC > > implementation. > > > > From these facts and reasons, $salt, $info and $length parameter order > and > > requirement should be changed from > > > > string $algo , string $ikm [, int $length = 0 [, string $info = '' [, > > string $salt = '' ]]] > > > > to > > > > string $algo , string $ikm , string $salt, string $info = '' [, > > int $length = 0 ] > > Note: Users can set empty string if they really don't need $salt and/or > > $info. > > > > Conclusion: > > This way, users would have better chances to use hash_hkdf() more > securely > > and > > properly. > > > > [1] http://php.net/hash_hkdf > > [2] http://www.faqs.org/rfcs/rfc5869.html > > > > === Return Value and Output Option === > > > > The most common HKDF usage with PHP would be: > > 1. CSRF token generation that is specific to a request with expiration > > time. > > (HEX return value would be appropriate, not BINARY) > > 2. API access token generation that does not transmit "The API Key", but > > derived key by HKDF. It also should have expiration time. > > (HEX return value would be appropriate, not BINARY) > > > > Consistency with other hash_*() functions: > > 1. All of other hash_*() returns HEX string hash value. > > 2. hash_hkdf() is the only one returns BINARY hash value. > > > > Conclusion: > > hash_hkdf() should return HEX by default, not BINARY. > > Optional [, bool $raw_output = false ] should be added just like other > > hash_*(). > > > > > > === Compatibility === > > > > IMHO, current hash_hkdf() should not be added by PHP 7.1.2, but 7.2.0 > > in the first place. The mess could be resolved by 7.2. > > > > Anyway, hash_hkdf() is added 7.1.2. Hopefully not many users are > > using it yet. If we change API with 7.2 release, there would be least > > possible confusions. (We may remove it from 7.1 to avoid confusions, too) > > > > Our choices: > > - Keep the current insecure/inconsistent API forever. > > - Change the API to have secure/consistent API forever. > > > > Conclusion: > > No conclusion for this. There would be conflicting options. > > > > > > I strongly think it is not worth to keep this insecure/inconsistent API > > forever. > > I prefer to change the API to what it should be. > > > > What should we do for this? > > Comments? > > > > > > > > P.S. > > > > Nikita, you've said following during HKDF discussion: > > - HKDF for CSRF tokens/etc does not make sense at all. > > - Current parameter order and return value makes perfect sense > > and has valid/common usages. > > - Everyone one this list, shouldn't listen to me because I'm insane and > > totally misunderstood what the HKDF is. > > > > Phrases are not the exact, but it should be correct enough. Some part > > is my fault, w/o reading mail and/or poor English. I blindly assumed > > you've understand RFC 5869 and possible common usages with PHP. I > > apologized for the confusion and tried to explain why w/o success. > > > > If you still believe what you've said is correct, I don't request you to > > take these back, show us at least one common/reasonable hash_hkdf() > > usage example for current API, and point out what's wrong my > > recommendations > > and rationales above. > > > > If not, I request you to take it back. I respect your contributions much, > > but > > the last one you've said is out of tolerance. > > > > -- > > Yasuo Ohgaki > > yohgaki@ohgaki.net > > > > > --94eb2c18fc5eb0f2a305590c3484--