Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:117046 Return-Path: Delivered-To: mailing list internals@lists.php.net Received: (qmail 14703 invoked from network); 16 Feb 2022 10:06:59 -0000 Received: from unknown (HELO php-smtp4.php.net) (45.112.84.5) by pb1.pair.com with SMTP; 16 Feb 2022 10:06:59 -0000 Received: from php-smtp4.php.net (localhost [127.0.0.1]) by php-smtp4.php.net (Postfix) with ESMTP id 5BED7180088 for ; Wed, 16 Feb 2022 03:24:56 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on php-smtp4.php.net X-Spam-Level: X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-ASN: AS24940 176.9.0.0/16 X-Spam-Virus: No X-Envelope-From: Received: from chrono.xqk7.com (chrono.xqk7.com [176.9.45.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by php-smtp4.php.net (Postfix) with ESMTPS for ; Wed, 16 Feb 2022 03:24:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bastelstu.be; s=mail20171119; t=1645010694; bh=5PJEynWrUVHy6UjMnD1huWxMsfM2RdIln9xdSb33E4Q=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=N9FbcJh2V5HbPWvuap6HtBsWLpKQSCFFk6qi5hHPI8k2y02If4mr/Uf5S3vj63v9u rKG0ZFPxOKHa1P45E1DLyWujHPDwRPQfsBM24NqZV27gEefq8TkQ5/EJzIKeGL7/h9 Tn+fmg7byvsE0e/vHU4dQvPJJXhfXMWqSMLqGQtpz64mYTFVNvs3ns2R/P/WmBMxNj Hm9cARLH39pvy5GFXXpm/Ocq9stqyEHfBHAWywIL31cwkOAXHOX9jvR0hlHZ2oaE++ VWebS5Aw3M2zozpfGjt5JcQg5ASlmRDUBrTN+dCmfSbrZyu+Sat60KIgRKF2RSdvWM BkaUII1aNYZOg== Content-Type: multipart/mixed; boundary="------------jEtVwAAEXQITkXyLrdvFu61L" Message-ID: <5f496cf9-8754-b009-9cb5-b978222b2249@bastelstu.be> Date: Wed, 16 Feb 2022 12:24:51 +0100 MIME-Version: 1.0 Content-Language: en-US To: Go Kudo Cc: internals@lists.php.net References: <41a1b458-4941-f34e-f1b4-e25b3298b80a@bastelstu.be> <553ba7ca-3821-c2d9-f88f-b216013a887b@bastelstu.be> <2c667812-88c8-0b7b-3558-561a1348d0b2@bastelstu.be> In-Reply-To: Subject: Re: [PHP-DEV] [RFC] [Under Discussion] Random Extension 4.0 From: tim@bastelstu.be (=?UTF-8?Q?Tim_D=c3=bcsterhus?=) --------------jEtVwAAEXQITkXyLrdvFu61L Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi On 2/16/22 12:04, Go Kudo wrote: > Thanks for the good idea. I changed the NumberGenerator to Engine and > changed generate() to return a string as suggested. Thanks, I've already seen the updated PR and played around with it. This feels much better now. As a test I implemented xoshiro128++ in pure userland (it being a 32 Bit RNG avoids the signedness issues in PHP userland) and compared it against the reference implementation in C. Find my implementations attached. Both versions give the same results (little endian encoding): $ gcc xoshiro128pp.c ; ./a.out fa3c872c $ sapi/cli/php test_rng.php string(8) "fa3c872c" > The main changes since last time are as follows: > > - The userland implementation can now specify the width of the random > number sequence that can be generated > - Random\Engine::nextByteSize() has been added Is the nextByteSize() method actually required? PHP strings already know their own length. > - Random\Engine::generate() now returns a string I've looked into your C implementation and it appears it still is affected by endianness issues. You can't simply memcpy the raw uintXX_t bytes into the char array. I believe the following should do the trick to for a consistent little endian encoding: bytes[0] = (generated >> 0) & 0xff; bytes[1] = (generated >> 8) & 0xff; bytes[2] = (generated >> 16) & 0xff; bytes[3] = (generated >> 24) & 0xff; The choice of endianness is arbitrary, but it needs to be consistent for every platform. Likewise when converting back to a number from bytes: number = (bytes[0] << 0) | (bytes[1] << 8) (bytes[2] << 16) | (bytes[3] << 24); I believe the same issue exists when initializing the XorShift with a string. > I have not yet come to a final conclusion on whether XorShift128Plus should > be switched to another RNG. For example, what about implementing > XorShift128Plus, but adding Xoshiro256** as well? > That would be fine for me as well. But it might make it harder for the end user to choose an appropriate RNG. Best regards Tim Düsterhus --------------jEtVwAAEXQITkXyLrdvFu61L Content-Type: application/x-php; name="test_rng.php" Content-Disposition: attachment; filename="test_rng.php" Content-Transfer-Encoding: base64 PD9waHAKCnVzZSBSYW5kb21cRW5naW5lOwoKZmluYWwgY2xhc3MgWG9zaGlybzEyOFBQIGlt cGxlbWVudHMgRW5naW5lCnsKICAgIHByaXZhdGUgZnVuY3Rpb24gX19jb25zdHJ1Y3QoCiAg ICAgICAgcHJpdmF0ZSBpbnQgJHMwLAogICAgICAgIHByaXZhdGUgaW50ICRzMSwKICAgICAg ICBwcml2YXRlIGludCAkczIsCiAgICAgICAgcHJpdmF0ZSBpbnQgJHMzCiAgICApIHsKICAg IH0KCiAgICBwdWJsaWMgc3RhdGljIGZ1bmN0aW9uIGZyb21OdW1iZXJzKCRzMCwgJHMxLCAk czIsICRzMykKICAgIHsKICAgICAgICByZXR1cm4gbmV3IHNlbGYoJHMwLCAkczEsICRzMiwg JHMzKTsKICAgIH0KCiAgICBwcml2YXRlIHN0YXRpYyBmdW5jdGlvbiByb3RsKCR4LCAkaykK ICAgIHsKICAgICAgICByZXR1cm4gKCgkeCA8PCAkaykgfCAoJHggPj4gKDMyIC0gJGspKSkg JiAweEZGRkZGRkZGOwogICAgfQoKICAgIHB1YmxpYyBmdW5jdGlvbiBnZW5lcmF0ZSgpOiBz dHJpbmcKICAgIHsKICAgICAgICAkcmVzdWx0ID0gKHNlbGY6OnJvdGwoKCR0aGlzLT5zMCAr ICR0aGlzLT5zMykgJiAweEZGRkZGRkZGLCA3KSArICR0aGlzLT5zMCkgJiAweEZGRkZGRkZG OwoKICAgICAgICAkdCA9ICgkdGhpcy0+czEgPDwgOSkgICYgMHhGRkZGRkZGRjsKCiAgICAg ICAgJHRoaXMtPnMyIF49ICR0aGlzLT5zMDsKICAgICAgICAkdGhpcy0+czMgXj0gJHRoaXMt PnMxOwogICAgICAgICR0aGlzLT5zMSBePSAkdGhpcy0+czI7CiAgICAgICAgJHRoaXMtPnMw IF49ICR0aGlzLT5zMzsKCiAgICAgICAgJHRoaXMtPnMyIF49ICR0OwoKICAgICAgICAkdGhp cy0+czMgPSBzZWxmOjpyb3RsKCR0aGlzLT5zMywgMTEpOwoKICAgICAgICByZXR1cm4gXHBh Y2soJ1YnLCAkcmVzdWx0KTsKICAgIH0KCiAgICBwdWJsaWMgZnVuY3Rpb24gbmV4dEJ5dGVT aXplKCk6IGludAogICAgewogICAgICAgIHJldHVybiA0OwogICAgfQp9CgokZyA9IFhvc2hp cm8xMjhQUDo6ZnJvbU51bWJlcnMoMSwgMiwgMywgNCk7Cgpmb3IgKCRpID0gMDsgJGkgPCAx MDI0MDA7ICRpKyspICRnLT5nZW5lcmF0ZSgpOwp2YXJfZHVtcChcYmluMmhleCgkZy0+Z2Vu ZXJhdGUoKSkpOwoKZXhpdDsK --------------jEtVwAAEXQITkXyLrdvFu61L Content-Type: text/x-csrc; charset=UTF-8; name="xoshiro128pp.c" Content-Disposition: attachment; filename="xoshiro128pp.c" Content-Transfer-Encoding: base64 LyogIFdyaXR0ZW4gaW4gMjAxOSBieSBEYXZpZCBCbGFja21hbiBhbmQgU2ViYXN0aWFubyBW aWduYSAodmlnbmFAYWNtLm9yZykKClRvIHRoZSBleHRlbnQgcG9zc2libGUgdW5kZXIgbGF3 LCB0aGUgYXV0aG9yIGhhcyBkZWRpY2F0ZWQgYWxsIGNvcHlyaWdodAphbmQgcmVsYXRlZCBh bmQgbmVpZ2hib3JpbmcgcmlnaHRzIHRvIHRoaXMgc29mdHdhcmUgdG8gdGhlIHB1YmxpYyBk b21haW4Kd29ybGR3aWRlLiBUaGlzIHNvZnR3YXJlIGlzIGRpc3RyaWJ1dGVkIHdpdGhvdXQg YW55IHdhcnJhbnR5LgoKU2VlIDxodHRwOi8vY3JlYXRpdmVjb21tb25zLm9yZy9wdWJsaWNk b21haW4vemVyby8xLjAvPi4gKi8KCiNpbmNsdWRlIDxzdGRpbnQuaD4KI2luY2x1ZGUgPHN0 ZGlvLmg+CgovKiBUaGlzIGlzIHhvc2hpcm8xMjgrKyAxLjAsIG9uZSBvZiBvdXIgMzItYml0 IGFsbC1wdXJwb3NlLCByb2NrLXNvbGlkCiAgIGdlbmVyYXRvcnMuIEl0IGhhcyBleGNlbGxl bnQgc3BlZWQsIGEgc3RhdGUgc2l6ZSAoMTI4IGJpdHMpIHRoYXQgaXMKICAgbGFyZ2UgZW5v dWdoIGZvciBtaWxkIHBhcmFsbGVsaXNtLCBhbmQgaXQgcGFzc2VzIGFsbCB0ZXN0cyB3ZSBh cmUgYXdhcmUKICAgb2YuCgogICBGb3IgZ2VuZXJhdGluZyBqdXN0IHNpbmdsZS1wcmVjaXNp b24gKGkuZS4sIDMyLWJpdCkgZmxvYXRpbmctcG9pbnQKICAgbnVtYmVycywgeG9zaGlybzEy OCsgaXMgZXZlbiBmYXN0ZXIuCgogICBUaGUgc3RhdGUgbXVzdCBiZSBzZWVkZWQgc28gdGhh dCBpdCBpcyBub3QgZXZlcnl3aGVyZSB6ZXJvLiAqLwoKCnN0YXRpYyBpbmxpbmUgdWludDMy X3Qgcm90bChjb25zdCB1aW50MzJfdCB4LCBpbnQgaykgewoJcmV0dXJuICh4IDw8IGspIHwg KHggPj4gKDMyIC0gaykpOwp9CgoKc3RhdGljIHVpbnQzMl90IHNbNF07Cgp1aW50MzJfdCBu ZXh0KHZvaWQpIHsKCWNvbnN0IHVpbnQzMl90IHJlc3VsdCA9IHJvdGwoc1swXSArIHNbM10s IDcpICsgc1swXTsKCgljb25zdCB1aW50MzJfdCB0ID0gc1sxXSA8PCA5OwoKCXNbMl0gXj0g c1swXTsKCXNbM10gXj0gc1sxXTsKCXNbMV0gXj0gc1syXTsKCXNbMF0gXj0gc1szXTsKCglz WzJdIF49IHQ7CgoJc1szXSA9IHJvdGwoc1szXSwgMTEpOwoKCXJldHVybiByZXN1bHQ7Cn0K CgovKiBUaGlzIGlzIHRoZSBqdW1wIGZ1bmN0aW9uIGZvciB0aGUgZ2VuZXJhdG9yLiBJdCBp cyBlcXVpdmFsZW50CiAgIHRvIDJeNjQgY2FsbHMgdG8gbmV4dCgpOyBpdCBjYW4gYmUgdXNl ZCB0byBnZW5lcmF0ZSAyXjY0CiAgIG5vbi1vdmVybGFwcGluZyBzdWJzZXF1ZW5jZXMgZm9y IHBhcmFsbGVsIGNvbXB1dGF0aW9ucy4gKi8KCnZvaWQganVtcCh2b2lkKSB7CglzdGF0aWMg Y29uc3QgdWludDMyX3QgSlVNUFtdID0geyAweDg3NjQwMDBiLCAweGY1NDJkMmQzLCAweDZm YTAzNWMzLCAweDc3ZjJkYjViIH07CgoJdWludDMyX3QgczAgPSAwOwoJdWludDMyX3QgczEg PSAwOwoJdWludDMyX3QgczIgPSAwOwoJdWludDMyX3QgczMgPSAwOwoJZm9yKGludCBpID0g MDsgaSA8IHNpemVvZiBKVU1QIC8gc2l6ZW9mICpKVU1QOyBpKyspCgkJZm9yKGludCBiID0g MDsgYiA8IDMyOyBiKyspIHsKCQkJaWYgKEpVTVBbaV0gJiBVSU5UMzJfQygxKSA8PCBiKSB7 CgkJCQlzMCBePSBzWzBdOwoJCQkJczEgXj0gc1sxXTsKCQkJCXMyIF49IHNbMl07CgkJCQlz MyBePSBzWzNdOwoJCQl9CgkJCW5leHQoKTsJCgkJfQoJCQoJc1swXSA9IHMwOwoJc1sxXSA9 IHMxOwoJc1syXSA9IHMyOwoJc1szXSA9IHMzOwp9CgoKLyogVGhpcyBpcyB0aGUgbG9uZy1q dW1wIGZ1bmN0aW9uIGZvciB0aGUgZ2VuZXJhdG9yLiBJdCBpcyBlcXVpdmFsZW50IHRvCiAg IDJeOTYgY2FsbHMgdG8gbmV4dCgpOyBpdCBjYW4gYmUgdXNlZCB0byBnZW5lcmF0ZSAyXjMy IHN0YXJ0aW5nIHBvaW50cywKICAgZnJvbSBlYWNoIG9mIHdoaWNoIGp1bXAoKSB3aWxsIGdl bmVyYXRlIDJeMzIgbm9uLW92ZXJsYXBwaW5nCiAgIHN1YnNlcXVlbmNlcyBmb3IgcGFyYWxs ZWwgZGlzdHJpYnV0ZWQgY29tcHV0YXRpb25zLiAqLwoKdm9pZCBsb25nX2p1bXAodm9pZCkg ewoJc3RhdGljIGNvbnN0IHVpbnQzMl90IExPTkdfSlVNUFtdID0geyAweGI1MjM5NTJlLCAw eDBiNmYwOTlmLCAweGNjZjVhMGVmLCAweDFjNTgwNjYyIH07CgoJdWludDMyX3QgczAgPSAw OwoJdWludDMyX3QgczEgPSAwOwoJdWludDMyX3QgczIgPSAwOwoJdWludDMyX3QgczMgPSAw OwoJZm9yKGludCBpID0gMDsgaSA8IHNpemVvZiBMT05HX0pVTVAgLyBzaXplb2YgKkxPTkdf SlVNUDsgaSsrKQoJCWZvcihpbnQgYiA9IDA7IGIgPCAzMjsgYisrKSB7CgkJCWlmIChMT05H X0pVTVBbaV0gJiBVSU5UMzJfQygxKSA8PCBiKSB7CgkJCQlzMCBePSBzWzBdOwoJCQkJczEg Xj0gc1sxXTsKCQkJCXMyIF49IHNbMl07CgkJCQlzMyBePSBzWzNdOwoJCQl9CgkJCW5leHQo KTsJCgkJfQoJCQoJc1swXSA9IHMwOwoJc1sxXSA9IHMxOwoJc1syXSA9IHMyOwoJc1szXSA9 IHMzOwp9CgppbnQKbWFpbih2b2lkKSB7CglzWzBdID0gMTsKCXNbMV0gPSAyOwoJc1syXSA9 IDM7CglzWzNdID0gNDsKCglmb3IgKHNpemVfdCBpID0gMDsgaSA8IDEwMjQwMDsgaSsrKSBu ZXh0KCk7CgoJaW50MzJfdCByZXN1bHQgPSBuZXh0KCk7CgoJcHJpbnRmKCIleCV4JXgleFxu IiwgKChyZXN1bHQgPj4gMCkgJiAweGZmKSwgKChyZXN1bHQgPj4gOCkgJiAweGZmKSwgKChy ZXN1bHQgPj4gMTYpICYgMHhmZiksICgocmVzdWx0ID4+IDI0KSAmIDB4ZmYpKTsKfQo= --------------jEtVwAAEXQITkXyLrdvFu61L--