Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:124519
Subject: Re: [PHP-DEV] Request for opinions: bug vs feature - change
 intokenization of yield from
To: internals@lists.php.net
References: <66984FD0.5090805@adviesenzo.nl>
 <AM8P250MB0170FFCA0014FB9EC272DB4FE2AC2@AM8P250MB0170.EURP250.PROD.OUTLOOK.COM>
 <c9736385-94b7-4064-911f-e5fc1df7e2bd@gmx.de>
 <AM8P250MB0170B4C59B4313A565CD029AE2AC2@AM8P250MB0170.EURP250.PROD.OUTLOOK.COM>
 <6699F817.8070806@adviesenzo.nl>
 <9571bb82-9873-4319-9bd1-0361748335be@bastelstu.be>
 <669BDB00.70507@adviesenzo.nl>
 <fa7db246-1577-43b7-a117-c18a73140091@bastelstu.be>
Message-ID: <669BE870.2050908@adviesenzo.nl>
Date: Sat, 20 Jul 2024 18:40:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101
 Thunderbird/38.7.0
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <fa7db246-1577-43b7-a117-c18a73140091@bastelstu.be>
Content-Type: multipart/alternative;
 boundary="------------050401090105090004020902"
From: php-internals_nospam@adviesenzo.nl (Juliette Reinders Folmer)

This is a multi-part message in MIME format.
--------------050401090105090004020902
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

On 20-7-2024 18:04, Tim Düsterhus wrote:
> PHP users have no idea what a token is internally. I'm looking at this 
> from a PHP user perspective. It looks like two keywords, it walks like 
> two keywords and it quacks like two keywords. I find it reasonable for 
> users to consider this as two keywords and not care about how it's 
> implemented internally.
>
>> So, what "feels" natural (whitespace-wise) to one person may not be the
>> same for the next, but comments _within_ tokens is different thing and
>> should in my opinion, not be allowed.
>
> As I've said: I agree that the current situation is unfortunate. But 
> the correct solution is not "disallow comments", but "split the 
> T_YIELD_FROM into T_YIELD T_WHITESPACE T_FROM_FOR_YIELD_FROM".
>

Tim, you're making my point for me. This is *exactly* why the current 
change should be reverted.

I'm not against changing the tokenization of "yield from" and the GH 
ticket thread also contains an alternative proposal for this from Bob 
[1], but like Matthew also said [2]: if that's something we want to do, 
let's have a proper discussion about it and let it go through an RFC and 
be a documented change.

And not, like it is now, an undocumented, random change creating an 
inconsistency in the Tokenizer.

Smile,
Juliette

1: https://github.com/php/php-src/issues/14926#issuecomment-2228855422
2: https://externals.io/message/124462#124515



--------------050401090105090004020902
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">On 20-7-2024 18:04, Tim Düsterhus
      wrote:<br>
    </div>
    <blockquote
      cite="mid:fa7db246-1577-43b7-a117-c18a73140091@bastelstu.be"
      type="cite">PHP users have no idea what a token is internally. I'm
      looking at this from a PHP user perspective. It looks like two
      keywords, it walks like two keywords and it quacks like two
      keywords. I find it reasonable for users to consider this as two
      keywords and not care about how it's implemented internally.
      <br>
      <br>
      <blockquote type="cite">So, what "feels" natural (whitespace-wise)
        to one person may not be the
        <br>
        same for the next, but comments _within_ tokens is different
        thing and
        <br>
        should in my opinion, not be allowed.
        <br>
      </blockquote>
      <br>
      As I've said: I agree that the current situation is unfortunate.
      But the correct solution is not "disallow comments", but "split
      the T_YIELD_FROM into T_YIELD T_WHITESPACE T_FROM_FOR_YIELD_FROM".
      <br>
      <br>
    </blockquote>
    <br>
    Tim, you're making my point for me. This is *exactly* why the
    current change should be reverted.<br>
    <br>
    I'm not against changing the tokenization of "yield from" and the GH
    ticket thread also contains an alternative proposal for this from
    Bob [1], but like Matthew also said [2]: if that's something we want
    to do, let's have a proper discussion about it and let it go through
    an RFC and be a documented change.<br>
    <br>
    And not, like it is now, an undocumented, random change creating an
    inconsistency in the Tokenizer.<br>
    <br>
    Smile,<br>
    Juliette<br>
    <br>
    1:
    <a class="moz-txt-link-freetext" href="https://github.com/php/php-src/issues/14926#issuecomment-2228855422">https://github.com/php/php-src/issues/14926#issuecomment-2228855422</a><br>
    2: <a class="moz-txt-link-freetext" href="https://externals.io/message/124462#124515">https://externals.io/message/124462#124515</a><br>
    <br>
    <br>
  </body>
</html>

--------------050401090105090004020902--