2022年2月17日(木) 19:25 Tim Düsterhus tim@bastelstu.be:

Hi

The following points have been fixed:

• nextByteSize(): int has been removed from Random\Engine
• If the width of the RNG is statically defined, it will now be used
  preferentially
• Added Xoshiro256StarStar
• Fixed an endianness issue

And updated RFC

https://wiki.php.net/rfc/rng_extension

[...]
This seems to have solved the whole problem. How about it?

Awesome, this is feeling much much better now. As you might've seen I've
made some comments on GitHub regarding implementation bugs.

I have two more conceptional questions:

---

However I believe you did not answer my question regarding the following
and that's something that should be clear in the RFC and documentation:

 <?php

 use Random\Engine\Xoshiro256StarStar;
 use Random\Randomizer;

 $g1 = new Xoshiro256StarStar(1);
 $g2 = clone $g1;
 $r1 = new Randomizer($g1);
 $r2 = new Randomizer($g2);

 var_dump(\bin2hex($r1->getBytes(8))); // string(16) "c510c70f6daff2b3"
 var_dump(\bin2hex($r2->getBytes(4) . $r2->getBytes(4))); //

string(16) "c510c70fea4c3647"

In this example I get 8 bytes from the randomizer. One time by getting
all 8 bytes at once, the second time by getting 4 bytes and then another
4 bytes.

I think that both lines should result in the same output, because in
both cases I am getting 8 bytes, without any other operations that might
affect the engine state in between. As a user I should not be required
to know how the Randomizer works internally (by always getting 8 bytes
from the engine and throwing away unused bytes).

If you disagree and think this is the correct behavior then this should
be documented accordingly in the RFC. If you agree, then this should be
fixed and a testcase be added.

---

This ties into (1): Currently any additional bytes returned by the
engine are silently ignored. Either the bytes should be processed in
full, or an error emitted if the returned bytestring is too long.

Consider the attached test case with a Sha1-based RNG. If I grab 20
bytes (the length of a SHA-1 hash) from the Randomizer then the
'generate()' function will be called 3 times, despite it returning
sufficient bytes on the first attempt. If I want to make sure that no
bytes are wasted, then I need to implement a pretty complex construction
(Sha1_2) to always return exactly 8 bytes.

Best regards
Tim Düsterhus

Hi Tim
Thanks for your continued help!

As for your question, buffering the output can lead to counter-intuitive
behavior in code like the following.

<?php

$engine = new \Random\Engine\Xoshiro256StarStar(1234);
$randomizer = new Randomizer($engine);

// Retrieve only 16 bits (the remaining 48 bits will be buffered in
Randomizer)
$str = $randomizer->getBytes(2);

// Generate a new 64 bits (to waste)
$engine->generate();

// Retrieve 64 bits (first 48 bits from buffer, but last 16 bits newly
generated)
// numerical continuity will be lost.
$str2 = $randomizer->getBytes(8);

However, the Engine will probably not be used by itself very often. I think
buffering should be implemented, but I'd like to solicit opinions on this
in the ML.

I'll implement value buffering first.

Regards
Go Kudo

2022年2月17日(木) 19:25 Tim Düsterhus tim@bastelstu.be:

Hi

The following points have been fixed:

• nextByteSize(): int has been removed from Random\Engine
• If the width of the RNG is statically defined, it will now be used
  preferentially
• Added Xoshiro256StarStar
• Fixed an endianness issue

And updated RFC

https://wiki.php.net/rfc/rng_extension

[...]
This seems to have solved the whole problem. How about it?

Awesome, this is feeling much much better now. As you might've seen I've
made some comments on GitHub regarding implementation bugs.

I have two more conceptional questions:

---

However I believe you did not answer my question regarding the following
and that's something that should be clear in the RFC and documentation:

 <?php

 use Random\Engine\Xoshiro256StarStar;
 use Random\Randomizer;

 $g1 = new Xoshiro256StarStar(1);
 $g2 = clone $g1;
 $r1 = new Randomizer($g1);
 $r2 = new Randomizer($g2);

 var_dump(\bin2hex($r1->getBytes(8))); // string(16) "c510c70f6daff2b3"
 var_dump(\bin2hex($r2->getBytes(4) . $r2->getBytes(4))); //

string(16) "c510c70fea4c3647"

In this example I get 8 bytes from the randomizer. One time by getting
all 8 bytes at once, the second time by getting 4 bytes and then another
4 bytes.

I think that both lines should result in the same output, because in
both cases I am getting 8 bytes, without any other operations that might
affect the engine state in between. As a user I should not be required
to know how the Randomizer works internally (by always getting 8 bytes
from the engine and throwing away unused bytes).

If you disagree and think this is the correct behavior then this should
be documented accordingly in the RFC. If you agree, then this should be
fixed and a testcase be added.

---

This ties into (1): Currently any additional bytes returned by the
engine are silently ignored. Either the bytes should be processed in
full, or an error emitted if the returned bytestring is too long.

Consider the attached test case with a Sha1-based RNG. If I grab 20
bytes (the length of a SHA-1 hash) from the Randomizer then the
'generate()' function will be called 3 times, despite it returning
sufficient bytes on the first attempt. If I want to make sure that no
bytes are wasted, then I need to implement a pretty complex construction
(Sha1_2) to always return exactly 8 bytes.

Best regards
Tim Düsterhus

Hi

Hello.

I have been looking into output buffering, but don't know the right way to
do it. The buffering works fine if all RNG generation widths are static,
but if they are dynamic so complicated.

It is possible to solve this problem by allowing generate() itself to
specify the size it wants, but this would significantly slow down
performance.

I've looked at the sample code, but do you really need support for
Randomizer? Engine::generate() can output dynamic binaries up to 64 bits.
You can use Engine directly, instead of Randomizer::getBytes().

What exactly is the situation where buffering by Randomizer is needed?

Also worried that buffering will cut off random numbers at arbitrary sizes.
It may cause bias in the generated results.

If you don't want to waste the generated values, users can still implement
their own buffering structures into Random\Engine following.

<?php

use Random\Engine;

final class BufferedEngine implements Engine
{
    private Iterator $gen;
    private string $buffer;

    public function __construct(string $seed)
    {
        $this->gen = $this->stream($seed);
        $this->buffer = '';
    }

    private function stream($state)
    {
        while (true) {
            $state = \sha1($state, true);
            for ($i = 0; $i < \strlen($state); $i++) {
                yield $state[$i];
            }
        }
    }

    public function generate(): string
    {
        echo __METHOD__, PHP_EOL;

        $result = "";
        for ($i = 0; $i < 8; $i++) {
            $result .= $this->gen->current();
            $this->gen->next();
        }

        return $result;
    }

    public function getBytes(int $length): string
    {
        $result = '';

        while (\strlen($result) < $length) {
            if ($this->buffer === '') {
                $this->buffer = $this->generate();
            }
            $required = $length - \strlen($result);
            $temp = \substr($this->buffer, 0, $required);
            $this->buffer = \substr($this->buffer, $required);
            $result .= $temp;
        }

        return $result;
    }
}

Buffering mechanism that was implemented in a limited way has been reverted
recently.

Best Regards
Go Kudo