Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:113157
To: =?iso-8859-2?Q?Alexandru_P=E3tr=E3nescu?= <drealecs@gmail.com>
CC: PHP internals <internals@lists.php.net>
Thread-Topic: [PHP-DEV] RFC: CachedIterable (rewindable, allows any
 key&repeating keys)
Thread-Index: AQHXACLkMdQqZoaVtUy2vSZWXRDmOapUrWsAgAAmA6g=
Date: Fri, 12 Feb 2021 18:19:35 +0000
Message-ID:
 <DM6PR07MB66181F74DF0361A67EE86A1AF98B9@DM6PR07MB6618.namprd07.prod.outlook.com>
References:
 <DM6PR07MB66182D44C3AFFAB73D6DED43F98C9@DM6PR07MB6618.namprd07.prod.outlook.com>,<CAAwdEzAhbArK+UcDGDTSVx_A9Ud8yYY507Qmm_QwYUrTsEGaWg@mail.gmail.com>
In-Reply-To:
 <CAAwdEzAhbArK+UcDGDTSVx_A9Ud8yYY507Qmm_QwYUrTsEGaWg@mail.gmail.com>
Accept-Language: en-CA, en-US
Content-Language: en-CA
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [HsvqUU2eCxkIJy8YSDJuGwIJ7H9I4vFQ]
x-ms-publictraffictype: Email
x-incomingheadercount: 45
x-eopattributedmessage: 0
x-ms-office365-filtering-correlation-id: 11fb02aa-741a-4b15-90c0-08d8cf82c0c8
x-ms-traffictypediagnostic: DM6NAM12HT147:
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 P6ZNWffoq5DzlZC+pZHrqi+7lyKu0dwcLe/z43toUruXlRuMLfgNlP0zKrX3445IBc4UESrHCZX2AA8T1JUqt8C1uNTXQo+eDLVPPkcp4Dd4Nti4GWx3PfG98+qhebrrf/4K7fiIbJIa+1XmxueDHmcFwt7aoep6+rcize7llyMZOVARC6hr4UK/v0BUXLtw+nXagGbwYVY1dIigJ4oiIlpTdVRpPmouSdQmbjBwAh9YWf/rf2+uzlPSEGkDptDMmXW4G7shNpl8SBTqW3kwVVE3NgsEoS+RGcB5EEIhXq0fBu0N6RfA2kNxbRfhfz/1WUAkbsl8H2kiIDGsYxOvTdqPUtjUYZkjP7gv8icjuiNI8Gvtq6ELtZLwT5D/ZjitP+EIa1Yo9w3cpYN/zraxrmQeB6r6FUvM3hbE5VQPKck7gicpjUg4Py8jAQzD7Yc4
x-ms-exchange-antispam-messagedata:
 UOxHBMvbepCsIwN/GGvZ5Q4V6QspIarL1GusAy+EAcxufv2D22XWmCCfun1a2U2kdUj7YyrdWlNHH8jsf1aJIQKz3bRrKfRgEUY3TDxveXAG5bHOvceGMa1zM91iDccyTcl8DlZq9nJGGhrDsgbVJA==
x-ms-exchange-transport-forked: True
Content-Type: multipart/alternative;
	boundary="_000_DM6PR07MB66181F74DF0361A67EE86A1AF98B9DM6PR07MB6618namp_"
MIME-Version: 1.0
X-OriginatorOrg: hotmail.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-AuthSource: DM6NAM12FT015.eop-nam12.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-Network-Message-Id: 11fb02aa-741a-4b15-90c0-08d8cf82c0c8
X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Feb 2021 18:19:35.8983
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6NAM12HT147
Subject: Re: [PHP-DEV] RFC: CachedIterable (rewindable, allows any
 key&repeating keys)
From: tysonandre775@hotmail.com (tyson andre)

--_000_DM6PR07MB66181F74DF0361A67EE86A1AF98B9DM6PR07MB6618namp_
Content-Type: text/plain; charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable

Hi Alex,

> > I've created a new RFC https://wiki.php.net/rfc/cachediterable adding C=
achedIterable,
> > which eagerly evaluates any iterable and contains an immutable copy of =
the keys and values of the iterable it was constructed from
> >
> >
> > Any other feedback unrelated to namespaces?
>
> Hi Tyson,
>
> I needed this feature a few years ago. In that case, the source was a gen=
erator that was slowly generating data while fetching them from a paginated=
 API that had rate limits.
> The result wrapping iterator was used at runtime in multiple (hundreds) o=
ther iterators that were processing elements in various ways (technical ana=
lysis indicator on time series) and after that merged back with some Multip=
leIterator.
>
> Just for reference, this is how the implementation in userland was and I =
was happy with it as a solution:
> https://gist.github.com/drealecs/ad720b51219675a8f278b8534e99d7c7
>
> Not sure if it's useful but I thought I should share it as I noticed you =
mentioned in your example for PolyfillIterator you chose not to use an Iter=
atorAggregate because complexity
> Was wondering how much inefficient this would be compared to the C implem=
entation.

That was for simplicity(shortness) of the RFC for people reading the polyfi=
ll.
I don't expect it to affect CPU timing or memory usage for large arrays in =
the polyfill.

Userland lazy iterable implementations could still benefit from having a Ca=
chedIterable around,
by replacing the lazy IteratorAggregate with a Cached Iterable when the end=
 of iteration was detected.

> Also, the implementation having the ability to be lazy was important and =
I think that should be the case here as well, by design, especially as we a=
re dealing with Generators.

We're dealing with the entire family of iterables, including but not limite=
d to Generators, arrays, user-defined Traversables, etc.

I'd considered that but decided not to include it in the RFC's scope.
If I was designing that, it would be a separate class `LazyCachedIterable`.

Currently, `CachedIterable` has several useful properties:

1. Serialization/Unserializable behavior is predictable - if the object was=
 constructed it can be safely serialized if keys/values can be serialized.
2. Iteration has no side effects (e.g. won't throw)
3. keyAt(int $offset) and so on have predictable behavior, good performance=
, and only one throwable type
4. Memory usage is small - this might also be the case for a LazyIterable d=
epending on implementation choices/constraints.

Adding lazy iteration support would make it no longer have some of those pr=
operties.

While I'd be in favor of that if it was implemented correctly, I don't plan=
 to work on implementing this until I know
if the addition of `CachedIterable` to a large family of iterable classes w=
ould pass.

CachedIterable has some immediate benefits on problems I was actively worki=
ng on, such as:

1. Being able to represent iterable functions such as iterable_reverse()
2. Memory efficiency and time efficiency for iteration
3. Being something internal code could return for getIterator(), etc.

Regards,
Tyson

--_000_DM6PR07MB66181F74DF0361A67EE86A1AF98B9DM6PR07MB6618namp_--