# token_get_all() new API

- Thread: https://externals.io/message/72440
- Original archive: https://news-web.php.net/php.internals/72440
- Started by Julien Pauli on 2014-02-10T14:54:38+00:00
- Messages: 3
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #72440 — Julien Pauli — 2014-02-10T14:54:38+00:00

URL: https://externals.io/message/72440#72440

Hi all,

I recently worked on `token_get_all()` implementation to add an extra
argument to the function to show token names directly in the output
instead of having to call `token_name()` on every output (usually what
is done nowadays).

I also added a flag so that the parsed scanner value can become
optionnal in the output, as sometimes one just wants a token stream
and is not interested in dealing with the scanner ouput.

If you have ideas or thoughts ?

https://github.com/jpauli/php-src/compare/token_names

Julien.P

---

## #72441 — Nikita Popov — 2014-02-10T15:29:09+00:00

URL: https://externals.io/message/72440#72441 · In reply to: #72440

> Hi all,
>
> I recently worked on `token_get_all()` implementation to add an extra
> argument to the function to show token names directly in the output
> instead of having to call `token_name()` on every output (usually what
> is done nowadays).
>
> I also added a flag so that the parsed scanner value can become
> optionnal in the output, as sometimes one just wants a token stream
> and is not interested in dealing with the scanner ouput.
>
> If you have ideas or thoughts ?
>
> https://github.com/jpauli/php-src/compare/token_names
>
> Julien.P
>

Do you have some examples for practical uses of these modes? E.g. in what
situation would you need only the token names, but not their content?

What I don't like about this change is that it introduces an additional
llist indirection, which presumably will have the function slower and more
memory heavy.

Nikita

---

## #72442 — Julien Pauli — 2014-02-10T15:38:21+00:00

URL: https://externals.io/message/72440#72442 · In reply to: #72441

> 
> > […]
>
>
> Do you have some examples for practical uses of these modes? E.g. in what
> situation would you need only the token names, but not their content?

Well, having a token stream. T_A + T_B + T_C = something , I don't
care about those ';' and '}'.
They make parsing the array output of `token_get_all()` harder :-p

>
> What I don't like about this change is that it introduces an additional
> llist indirection, which presumably will have the function slower and more
> memory heavy.

Yes, I could use HashTable directly, that's a detail and can be
patched very quickly :-)

Julien