Hi internals,
I've opened voting on https://wiki.php.net/rfc/token_as_object, which adds
a new object-based interface to the tokenizer. Voting closes on 2020-03-20.
Regards,
Nikita
Hello,
I know I’m late, but shouldn't these tokens have a __toString() method?
It’s not listed in rejected features.
--
Côme Chilliet
FusionDirectory - https://www.fusiondirectory.org
On Tue, Mar 10, 2020 at 12:36 PM Côme Chilliet <
come.chilliet@fusiondirectory.org> wrote:
Hello,
I know I’m late, but shouldn't these tokens have a __toString() method?
It’s not listed in rejected features.
Would __toString() return $token->text? I'm not strictly opposed to this,
but I'd be hard pressed to think of a situation where I would prefer
writing (string) $token over $token->text. Generally, the less magic the
better :)
Nikita
Le mardi 10 mars 2020, 15:53:41 CET Nikita Popov a écrit :
Would __toString() return $token->text? I'm not strictly opposed to this,
but I'd be hard pressed to think of a situation where I would prefer
writing (string) $token over $token->text. Generally, the less magic the
better :)
My idea was that since tokens are parsed from a string it should be easy to write a string back with them.
I do not have an obvious usecase but the idea is doing stuff like
if ($token.is('function')) {
echo "public $token";
} else {
echo $token;
}
Am I right to assume that with __toString the same as ->text we’d have implode('', PhpToken::getAll($code)) == $code ?
This would allow:
$tokens = PhpToken::getAll($code);
// Modifications on $tokens
$code = implode('', $tokens);
--
Côme Chilliet
FusionDirectory - https://www.fusiondirectory.org
On Tue, Mar 10, 2020 at 5:19 PM Côme Chilliet <
come.chilliet@fusiondirectory.org> wrote:
Le mardi 10 mars 2020, 15:53:41 CET Nikita Popov a écrit :
Would __toString() return $token->text? I'm not strictly opposed to this,
but I'd be hard pressed to think of a situation where I would prefer
writing (string) $token over $token->text. Generally, the less magic the
better :)My idea was that since tokens are parsed from a string it should be easy
to write a string back with them.
I do not have an obvious usecase but the idea is doing stuff likeif ($token.is('function')) {
echo "public $token";
} else {
echo $token;
}Am I right to assume that with __toString the same as ->text we’d have
implode('', PhpToken::getAll($code)) == $code ?This would allow:
$tokens = PhpToken::getAll($code);
// Modifications on $tokens
$code = implode('', $tokens);
Yes, that would work. It's not much harder without __toString though:
$code = implode(array_column($tokens, 'text'));
Does anyone else have thoughts on whether PhpToken should implement
__toString() or not?
Nikita
On Tue, Mar 10, 2020 at 5:19 PM Côme Chilliet <
come.chilliet@fusiondirectory.org> wrote:Le mardi 10 mars 2020, 15:53:41 CET Nikita Popov a écrit :
Would __toString() return $token->text? I'm not strictly opposed to this,
but I'd be hard pressed to think of a situation where I would prefer
writing (string) $token over $token->text. Generally, the less magic the
better :)My idea was that since tokens are parsed from a string it should be easy
to write a string back with them.
I do not have an obvious usecase but the idea is doing stuff likeif ($token.is('function')) {
echo "public $token";
} else {
echo $token;
}Am I right to assume that with __toString the same as ->text we’d have
implode('', PhpToken::getAll($code)) == $code ?This would allow:
$tokens = PhpToken::getAll($code);
// Modifications on $tokens
$code = implode('', $tokens);Yes, that would work. It's not much harder without __toString though:
$code = implode(array_column($tokens, 'text'));
Does anyone else have thoughts on whether PhpToken should implement
__toString() or not?Nikita
I forget who it was but someone on the list a while back offered a heuristic of "__toString should be used iff there is one and only one logical string representation of a value, because the value is conceptually just a string with extra metadata on it".
That sounds like a really solid heuristic to me. In this case, I think yes, ->text is the only reasonable thing to produce for __toString().
Or perhaps this is a good use case for "you can use a subclass, do that" if there's any good reason to not have it just be ->text.
--Larry Garfield
Le mercredi 11 mars 2020, 10:59:19 CET Larry Garfield a écrit :
I forget who it was but someone on the list a while back offered a heuristic of "__toString should be used iff there is one and only one logical string representation of a value, because the value is conceptually just a string with extra metadata on it".
That sounds like a really solid heuristic to me. In this case, I think yes, ->text is the only reasonable thing to produce for __toString().
Yes that was my reasoning, a token is essentially a substring, so I was expecting __toString to reflect this and allow me to use it as a string.
--
Côme Chilliet
FusionDirectory - https://www.fusiondirectory.org
Hi internals,
I've opened voting on https://wiki.php.net/rfc/token_as_object, which
adds a new object-based interface to the tokenizer. Voting closes on
2020-03-20.
This RFC has been accepted unanimously, with 47 votes in favor.
Based on the additional feedback in this thread, I'll also add a
__toString() method that returns the value of $token->text, unless anyone
has objects to that...
Regards,
Nikita
Hi internals,
I've opened voting on https://wiki.php.net/rfc/token_as_object, which
adds a new object-based interface to the tokenizer. Voting closes on
2020-03-20.This RFC has been accepted unanimously, with 47 votes in favor.
Based on the additional feedback in this thread, I'll also add a
__toString() method that returns the value of $token->text, unless anyone
has objects to that...Regards,
Nikita
As long as __toString() is overridable in subclasses, that seems fine to me.
--Larry Garfield
Hello, internals team!
I would like to propose to rename the PhpToken::getAll() method into the
PhpToken::tokenize(string $source) or PhpToken::lex(string $source) to use
a more meaningful verb word.
After renaming, it will sound more natural: $tokenStream =
PhpToken::tokenize($sourceCode)
Best regards,
Alexander
пн, 23 мар. 2020 г. в 19:04, Larry Garfield larry@garfieldtech.com:
On Fri, Mar 6, 2020 at 10:33 AM Nikita Popov nikita.ppv@gmail.com
wrote:Hi internals,
I've opened voting on https://wiki.php.net/rfc/token_as_object, which
adds a new object-based interface to the tokenizer. Voting closes on
2020-03-20.This RFC has been accepted unanimously, with 47 votes in favor.
Based on the additional feedback in this thread, I'll also add a
__toString() method that returns the value of $token->text, unless anyone
has objects to that...Regards,
NikitaAs long as __toString() is overridable in subclasses, that seems fine to
me.--Larry Garfield