Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:114792
MIME-Version: 1.0
References: <DM6PR07MB66182D44C3AFFAB73D6DED43F98C9@DM6PR07MB6618.namprd07.prod.outlook.com>
 <DM6PR07MB6618D8BF188FB5DFF4BDEDDFF9379@DM6PR07MB6618.namprd07.prod.outlook.com>
 <CAAKU0um72eym04nuenUyS=SFxLb4nOkHDdTO8X0V7v_nFoe+Qg@mail.gmail.com>
In-Reply-To: <CAAKU0um72eym04nuenUyS=SFxLb4nOkHDdTO8X0V7v_nFoe+Qg@mail.gmail.com>
Reply-To: Levi Morrison <levi.morrison@datadoghq.com>
Date: Tue, 8 Jun 2021 22:49:17 -0600
Message-ID: <CAAKU0um1vibiJ8A-=YFKj6SRoODTB9iz5q39GKnwLMd9072E0A@mail.gmail.com>
To: tyson andre <tysonandre775@hotmail.com>
Cc: PHP internals <internals@lists.php.net>
Content-Type: text/plain; charset="UTF-8"
Subject: Re: [PHP-DEV] Re: RFC: CachedIterable (rewindable, allows any
 key&repeating keys)
From: internals@lists.php.net ("Levi Morrison via internals")

On Tue, Jun 8, 2021 at 10:47 PM Levi Morrison
<levi.morrison@datadoghq.com> wrote:
>
> On Tue, Jun 8, 2021 at 6:22 PM tyson andre <tysonandre775@hotmail.com> wrote:
> >
> > Hi internals,
> >
> > > I've created a new RFC https://wiki.php.net/rfc/cachediterable adding CachedIterable,
> > > which eagerly evaluates any iterable and contains an immutable copy of the keys and values of the iterable it was constructed from
> > >
> > > This has the proposed signature:
> > >
> > > ```
> > > final class CachedIterable implements IteratorAggregate, Countable, JsonSerializable
> > > {
> > >     public function __construct(iterable $iterator) {}
> > >     public function getIterator(): InternalIterator {}
> > >     public function count(): int {}
> > >     // [[$key1, $value1], [$key2, $value2]]
> > >     public static function fromPairs(array $pairs): CachedIterable {}
> > >     // [[$key1, $value1], [$key2, $value2]]
> > >     public function toPairs(): array{}
> > >     public function __serialize(): array {}  // [$k1, $v1, $k2, $v2,...]
> > >     public function __unserialize(array $data): void {}
> > >
> > >     // useful for converting iterables back to arrays for further processing
> > >     public function keys(): array {}  // [$k1, $k2, ...]
> > >     public function values(): array {}  // [$v1, $v2, ...]
> > >     // useful to efficiently get offsets at the middle/end of a long iterable
> > >     public function keyAt(int $offset): mixed {}
> > >     public function valueAt(int $offset): mixed {}
> > >
> > >     // '[["key1","value1"],["key2","value2"]]' instead of '{...}'
> > >     public function jsonSerialize(): array {}
> > >     // dynamic properties are forbidden
> > > }
> > > ```
> > >
> > > Currently, PHP does not provide a built-in way to store the state of an arbitrary iterable for reuse later
> > > (when the iterable has arbitrary keys, or when keys might be repeated). It would be useful to do so for many use cases, such as:
> > >
> > > 1. Creating a rewindable copy of a non-rewindable Traversable
> > > 2. Generating an IteratorAggregate from a class still implementing Iterator
> > > 3. In the future, providing internal or userland helpers such as iterable_flip(iterable $input), iterable_take(iterable $input, int $limit),
> > >     iterable_chunk(iterable $input, int $chunk_size), iterable_reverse(), etc (these are not part of the RFC)
> > > 4. Providing memory-efficient random access to both keys and values of arbitrary key-value sequences
> > >
> > > Having this implemented as an internal class would also allow it to be much more efficient than a userland solution
> > > (in terms of time to create, time to iterate over the result, and total memory usage). See https://wiki.php.net/rfc/cachediterable#benchmarks
> > >
> > > After some consideration, this is being created as a standalone RFC, and going in the global namespace:
> > >
> > > - Based on early feedback on https://wiki.php.net/rfc/any_all_on_iterable#straw_poll (on the namespace preferred in previous polls)
> > >   It seems like it's way too early for me to be proposing namespaces in any RFCs for PHP adding to modules that already exist, when there is no consensus.
> > >
> > >   An earlier attempt by others on creating a policy for namespaces in general(https://wiki.php.net/rfc/php_namespace_policy#vote) also did not pass.
> > >
> > >   Having even 40% of voters opposed to introducing a given namespace (in pre-existing modules)
> > >   makes it an impractical choice when RFCs require a 2/3 majority to pass.
> > > - While some may argue that a different namespace might pass,
> > >   https://wiki.php.net/rfc/any_all_on_iterable_straw_poll_namespace#vote had a sharp dropoff in feedback after the 3rd form.
> > >   I don't know how to interpret that - e.g. are unranked namespaces preferred even less than the options that were ranked or just not seen as affecting the final result.
> >
> > A heads up - I will probably start voting on https://wiki.php.net/rfc/cachediterable this weekend after https://wiki.php.net/rfc/cachediterable_straw_poll is finished.
> >
> > Any other feedback on CachedIterable?
> >
> > Thanks,
> > Tyson
> >
> > --
> > PHP Internals - PHP Runtime Development Mailing List
> > To unsubscribe, visit: https://www.php.net/unsub.php
> >
>
> Based on a recent comment you made on GitHub, it seems like
> `CachedIterable` eagerly creates the datastore instead of doing so
> on-demand. Is this correct?

Sorry, yes, that's correct and pointed out in the RFC.

I think that's a significant implementation flaw. I don't see why we'd
balloon memory usage unnecessarily by being eager -- if an operation
needs to fetch more data then it can go ahead and do so.