# [Pre-RFC] Lazy iterator_* functions for iterables

- Thread: https://externals.io/message/133181
- Original archive: https://news-web.php.net/php.internals/133181
- Started by pawellwankiewicz@gmail.com on 2026-11-22T11:26:17+00:00
- Messages: 2
- RFC: PHP\iterable\any() and all() on iterables (Declined) — https://externals.io/rfc/any_all_on_iterable.md
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · RFCs at https://externals.io/rfc.md · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #133181 — pawellwankiewicz@gmail.com — 2026-11-22T11:26:17+00:00

URL: https://externals.io/message/133181#133181

Hi internals,

I'm Paweł, and this is my first post here. Before writing an official RFC,
I'd like to gather your feedback on a set of iterator_* functions for
iterables.

THE PROBLEM

Currently, array_* functions only accept arrays. Working with generators,
database cursors, or paged APIs requires calling `iterator_to_array()` first,
which has major drawbacks:

- It loads the entire sequence into memory.
- It prevents early exit (e.g., `array_find()` or `array_any()` could stop at
  the first match).
- It fails to terminate on infinite sources.
- It silently overwrites duplicate keys by default ($preserve_keys = true).

Simply allowing array_* functions to accept iterables wouldn't solve this,
as they would still need to consume the entire input to return an array. We
need lazy equivalents that return an Iterator. As Rowan noted in the pipe
operator thread: "PHP badly needs more native functions for working with
iterators" [1]. With the pipe operator (PHP 8.5) and Partial Function
Application (PHP 8.6), this gap is even more obvious: chaining operations on
an iterable currently means materializing the whole input first, and then
building a full intermediate array at every step.

PROPOSAL

I propose adding 11 functions that mirror existing array_* functions and
pass the exact same arguments to their callbacks:

- Lazy (returning an Iterator): iterator_map, iterator_filter,
  iterator_slice, iterator_keys, iterator_values
- Terminal (returning a value): iterator_reduce, iterator_any, iterator_all,
  iterator_find, iterator_find_key, iterator_first

For example, with $orders being a generator that reads a file or a database
cursor, this reads only as many orders as needed and never holds more than
one in memory:

    $total = $orders
        |> iterator_filter(?, fn (Order $o) => $o->paid)
        |> iterator_map(?, fn (Order $o) => $o->amount)
        |> iterator_slice(?, 0, 100)
        |> iterator_reduce(?, fn ($sum, $x) => $sum + $x, 0);

This set is intentionally closed and introduces no new concepts, leaving all
existing array_* and iterator_* functions unchanged. Since the lazy and
terminal groups are independent, I plan to split the RFC vote into two
parts, just like the array_find RFC did.

KEY DECISIONS (full details: [2])

- iterator_ prefix over a namespace: `iterator_to_array()` and
  `iterator_count()` already form a family and have accepted any iterable
  since PHP 8.2. Per the "Namespaces in bundled extensions" RFC, consistency
  with existing symbols in an extension takes priority.
- Consistent parameter order: the iterable is always first, and the
  callback is last. This works cleanly with PFA: $x |> iterator_map(?, $fn).
- Key preservation: keys are always preserved (like `array_filter()`). If you
  need a re-indexed list, use iterator_values() or
  iterator_to_array($it, false). This includes iterator_slice(), which,
  unlike `array_slice()`, does not renumber integer keys.
- Non-rewindable: lazy functions return an Iterator that cannot be rewound
  once iteration starts, much like a Generator.
- No negative offsets/lengths: iterator_slice() rejects negative $offset
  and $length values, as supporting them would require buffering the entire
  input in memory.

WHY CORE INSTEAD OF USERLAND?

Userland libraries like nikic/iter exist, but they differ in naming,
parameter order, and namespaces. Adding these to core completes the native
iterator_* family and enables zero-dependency pipelines with pipe and PFA.

Compared to a PHP polyfill, lazy functions are faster in C (by 16-35%
without JIT, 11-33% with JIT), while terminal functions are slower (by
6-10% without JIT, 11-26% with JIT): invoking a PHP callback from C has
overhead that the JIT cannot optimize away. This is already true for
 `array_reduce()`, which is 3.4x slower than a foreach loop under JIT. The goal
isn't raw speed over foreach, but standardization, memory efficiency, and
early exit.

INTENTIONAL OMISSIONS

- No iterator_last(): it would have to consume the entire input and would
  never terminate on infinite sources.
- No extra helpers: functions like chunk, zip, chain, or flat_map are left
  for future RFCs.
- No new interfaces: the only new class is final class LazyIterator,
  returned by the lazy functions.
- Null ambiguity: iterator_find(), iterator_find_key(), and iterator_first()
  return null when nothing is found, matching `array_find()`,
  `array_find_key()`, and `array_first()`. iterator_find_key() is additionally
  ambiguous because, unlike an array key, an iterator key can itself be
  null.

HISTORY

The 2021 RFC for any() and all() [3] failed (11:12, with a 2/3 majority
required). The most common reason given in its straw poll was that its
scope was too small. This proposal offers a broader, yet strictly bounded
scope: exact equivalents of existing array_* functions.

CURRENT STATUS

- Polyfill (PHP 8.2+), .phpt tests, and benchmarks are ready [4].
- Native C implementation in ext/spl with tests is ready [5].
- Benchmark (5M NDJSON lines): iterator_reduce() takes ~2.3 s and 11 KB of
  memory, whereas `iterator_to_array()` + `array_reduce()` takes ~3.9 s and
  3.1 GB. For data that is already an array, `array_map()` and
  `array_filter()` remain faster than their lazy counterparts.

Disclosure: I wrote the C implementation, polyfill, tests, and benchmarks
with the help of an AI assistant (Claude). However, all design decisions are
my own. I have thoroughly reviewed all code and verified the tests by
intentionally breaking the code they cover.

QUESTIONS FOR FEEDBACK

1. Should we stick with the iterator_ prefix, or use a namespace
   (Iter\ / iter\)?
2. Is this 11-function scope too small, too large, or just right for a
   single RFC?
3. Is there enough interest to move forward with a formal RFC? If so, could
   someone please grant wiki RFC karma to my account (pawell67)?

Best regards,
Paweł Wankiewicz

[1] https://externals.io/message/126318#126727
[2] https://github.com/pawell67/iterable-functions#design-decisions
[3] https://wiki.php.net/rfc/any_all_on_iterable
[4] https://github.com/pawell67/iterable-functions
[5] https://github.com/pawell67/php-src/compare/master...iterator-functions

---

## #133189 — Morgan — 2026-11-22T20:30:11+00:00

URL: https://externals.io/message/133181#133189 · In reply to: #133181

> Hi internals,
> 
> I propose adding 11 functions that mirror existing array_* functions and
> pass the exact same arguments to their callbacks:
> 
> - Lazy (returning an Iterator): iterator_map, iterator_filter,
>    iterator_slice, iterator_keys, iterator_values
> - Terminal (returning a value): iterator_reduce, iterator_any, iterator_all,
>    iterator_find, iterator_find_key, iterator_first
> 
I like the sound of this (having written my own userland iterator 
library to do these things and a bunch that you defer to later RFCs). 
 From your mention of iterator_to_array and iterator_count I take it 
these new functions would take any iterable as argument (specifically: 
pass them an array directly, instead of wrapping it in an ArrayIterator 
first)?
