Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:113034
To: Levi Morrison <levi.morrison@datadoghq.com>
CC: "internals@lists.php.net" <internals@lists.php.net>
Thread-Topic: [PHP-DEV] Proposal: Adding a RewindableKeyValueIterator type
 allowing mixed/repeated keys
Thread-Index: AQHW92K4Q7dSQ8CfzUK9YysQDgaLrqpBGVqAgADE/o+AADqmDg==
Date: Sun, 31 Jan 2021 19:13:12 +0000
Message-ID:
 <DM6PR07MB66184BDC43B56468E5F80106F9B79@DM6PR07MB6618.namprd07.prod.outlook.com>
References:
 <DM6PR07MB66184CB5E5A3B466D6883DBDF9B89@DM6PR07MB6618.namprd07.prod.outlook.com>,<CAAKU0unpUmMYUPO7LOECYLJbPZ2nKqYiNXp-oM+LD7JiDjRDTA@mail.gmail.com>,<DM6PR07MB661867B00FFB004338E64012F9B79@DM6PR07MB6618.namprd07.prod.outlook.com>
In-Reply-To:
 <DM6PR07MB661867B00FFB004338E64012F9B79@DM6PR07MB6618.namprd07.prod.outlook.com>
Accept-Language: en-CA, en-US
Content-Language: en-CA
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [gq7swBRLEdEL5keFZD3nJuQytPcWEjo/R3O9KQziHHmGQRzx7KjHXlUvXsJv5GgD]
x-ms-publictraffictype: Email
x-incomingheadercount: 45
x-eopattributedmessage: 0
x-ms-office365-filtering-correlation-id: 2df2003c-08b3-4193-ed38-08d8c61c4151
x-ms-traffictypediagnostic: BN7NAM10HT237:
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 zggfCDWK1VO9ro/gQfSorsMWbJXDQsVK+QMTVMXpLbfMTo77c/Pu5SgduKOvfl3zvfyTGa4XSkUFJzt4zNNZklME+6GGP5m+KRrZiMgWsNHoy10eHABukoPhqPBJTqQp99EhO4A+w9oJczXHTaGibo+EfQMYWILQDQYD/PGhd42ZO6/u3RSH+rjkqLSPu8UsgiXK7MMl16RxD2VbOZ3ehiyOv1tw1ic5j1pQ745clypkylIVNFxM5KSUukuICLm0AXPAaSO770nU+VyW+60yimiVKTNxYHcCXDX0UA0B83cf1jduqs/ov2I4xBhfv1zZViDLf6F6wvcXnJzgLvNjGKrtRqCvNQWTzXfqewcpmYwne8IEftMRsmzF2rca0+lThKZNMDB9kGm0PLshXjjDpEXKL364cpJZmoAJUNpaZRQ=
x-ms-exchange-antispam-messagedata:
 INk3hMGpe9XQyZ/VVt3hl593Ioe/WZuvRu8dEXAwQk09h5eS4IGLZdJjCAX7/HXuZvdpGvOTOe7wqW7XXTqKN+Bp6aCha8W5UfJs7Q/xokd7LkVvHQHqLWURfU/CE6FwSCf3n1fMJ/FREjxvhtJF8aVAbuYnOYS/6ZVjwZjh+ai7Oxo4xEt8qIi7qVOmL7re9oLLT46vKIE13fLjwYhF6A==
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: hotmail.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-AuthSource: BN7NAM10FT012.eop-nam10.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-Network-Message-Id: 2df2003c-08b3-4193-ed38-08d8c61c4151
X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Jan 2021 19:13:12.9057
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7NAM10HT237
Subject: Re: [PHP-DEV] Proposal: Adding a RewindableKeyValueIterator type
 allowing mixed/repeated keys
From: tysonandre775@hotmail.com (tyson andre)

Hi Levi Morrison,=0A=
=0A=
> > > Currently, there don't seem to be any internal classes that can be us=
ed to store a copy of the keys and values of an arbitrary Traversable.=0A=
> > >=0A=
> > > - This would help in eagerly evaluating the result of a generator in =
a memory efficient way that could be exactly stored and reused=0A=
> > >=A0=A0 e.g. `return $this->cachedResults ??=3D new \RewindableKeyValue=
Iterator($this->innerResultsGenerator());`=0A=
> > >=A0=A0 https://externals.io/message/108767#108797=0A=
> > > - This would be useful to exactly represent the keys of sequences wit=
h repeated keys (e.g. `yield 'first'; yield 'second';` implicitly uses the =
key `0` twice.)=0A=
> > > - This would be convenient to have to differentiate between 1, '1', a=
nd true.=0A=
> > > - This would be useful if php were to add internal global functions t=
hat act on iterables and return Traversables with potentially repeated keys=
 based on those iterables,=0A=
> > >=A0=A0=A0 e.g. map(), filter(), take(), flip(), etc=0A=
> > > - If PHP were to add more iterable methods, being able to save an imm=
utable copy of a traversable's result would be useful for end users.=0A=
> > > - If PHP were to add a Map (ordered hash map with null/any scalar/arr=
ays/objects as keys) class type in the future,=0A=
> > >=A0=A0 and that implemented IteratorAggregate, the return type of getI=
terator() would need something like RewindableKeyValueIterator.=0A=
> > > - The lack of a relevant datatype is among the reasons why classes su=
ch as SplObjectStorage are still an Iterator instead of an IteratorAggregat=
e. (and backwards compatibility)=0A=
> > >=0A=
> > > ```=0A=
> > > final class KeyValueSequenceIterator implements Iterator {=0A=
> > >=A0=A0=A0=A0 // loop over all values in $values and store a copy, conv=
erting=0A=
> > >=A0=A0=A0=A0 // references in top-level array values to non-references=
=0A=
> > >=A0=A0=A0=A0 public function __construct(iterable $values) {...}=0A=
> > >=A0=A0=A0=A0 public static function fromKeyValuePairs(iterable $entrie=
s): self {...} // fromKeyValuePairs([[$key1, $value1]])=0A=
> > >=A0=A0=A0=A0 public function rewind(): void {...}=0A=
> > >=A0=A0=A0=A0 public function next(): void {...}=0A=
> > >=A0=A0=A0=A0 public function current(): mixed {...}=0A=
> > >=A0=A0=A0=A0 public function key(): mixed {...}=0A=
> > >=A0=A0=A0=A0 public function valid(): bool {...}=0A=
> > >=A0=A0=A0=A0 // and __debugInfo, __clone(), etc.=0A=
> > > }=0A=
> > > ```=0A=
> >=0A=
> > The names `RewindableKeyValueIterator` and `KeyValueSequenceIterator`=
=0A=
> > are just long-form descriptions of the Iterator API. I don't think we=
=0A=
> > need such long names. The name should focus on what it brings.=0A=
> >=0A=
> > What it brings is a caching iterator around another iterator, that=0A=
> > includes re-windability. The SPL provides a `CachingIterator`, but I=0A=
> > assume it is inadequate somehow (it is basically SPL tradition). Can=0A=
> > you specifically discuss the shortcomings of `CachingIterator` and how=
=0A=
> > you will address them? To that end, have you implemented a=0A=
> > proof-of-concept for the proposed iterator?=0A=
> =0A=
> It caches the results by coercing keys and inserting them into an array, =
which is a different use case.=0A=
> If you rewind a CachingIterator, it rewinds the iterator that it wraps,=
=0A=
> which throws for Generators and other types of iterators.=0A=
> =0A=
> For RewindableKeyValueIterator, it needs to store keys that can't be used=
 as array keys.=0A=
> =0A=
> I was planning to implement a proof of concept if there wasn't widespread=
 opposition=0A=
> and if nobody pointed out the functionality already existed.=0A=
> =0A=
> ```=0A=
> <?php=0A=
> function dump_iterable(iterable $x) {=0A=
> =A0=A0=A0 foreach ($x as $key =3D> $value) {=0A=
> =A0=A0=A0=A0=A0=A0=A0 printf("Key: %s\nValue: %s\n", json_encode($key), j=
son_encode($value));=0A=
> =A0=A0=A0 }=0A=
> }=0A=
> function yields_values(): Generator { yield 0 =3D> 'first'; yield '0' =3D=
> 'second'; }=0A=
> $c =3D new CachingIterator(yields_values(), CachingIterator::FULL_CACHE);=
=0A=
> echo "First CachingIterator iteration\n";=0A=
> dump_iterable($c);=0A=
> var_export($c->getCache()); echo "\n"; // array(0 =3D> 'second') does not=
 represent that data=0A=
> echo "Second CachingIterator iteration\n";=0A=
> // Fatal error: Uncaught Exception: Cannot rewind a generator that was al=
ready run=0A=
> dump_iterable($c);=0A=
> ```=0A=
=0A=
A proof of concept is available at https://github.com/php/php-src/pull/6655=
=0A=
=0A=
The implementation is similar to SplFixedArray, but instead of an array of =
`n` zvals,=0A=
it's an array of `n` pairs of zvals for keys and values.=0A=
=0A=
It's an `Iterator` because IteratorAggregator->getIterator() must eventuall=
y return an Iterator,=0A=
but I'm consider also proposing an IteratorAggregate such as KeyValueSequen=
ce =0A=
that can share the same immutable storage (or copy it).=0A=
=0A=
For example, `someprefix_map(iterable, callable $mapper): KeyValueSequence`=
 could be useful internally to handle=0A=
Traversables that could be repeated or use non-scalar keys.=0A=
=0A=
Thanks,=0A=
- Tyson=0A=