Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:124524
X-Original-To: internals@lists.php.net
Delivered-To: internals@lists.php.net
Received: from php-smtp4.php.net (php-smtp4.php.net [45.112.84.5])
by qa.php.net (Postfix) with ESMTPS id 2EED01A00B7
for <internals@lists.php.net>; Sat, 20 Jul 2024 18:32:04 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=php.net; s=mail;
t=1721500415; bh=kyy00I+TDxI7FJYKu+LvH3BqaFhNAG8uMI4REdR5jIk=;
h=Date:Subject:To:References:From:In-Reply-To:From;
b=A9CwCEy2Im/iC2LGUmcoB+aK2z4dc7DP0Bl2Q1j83wT6Pe/crdbQV6GJtzD1z3ox3
2JTaMjuEcUREbPCX6gA5NFAW3j5eb7oOyFIyZ3vGWjCaf2LwZddTsPZCQMgbEbopw1
XploK3YtgslBOko9h/chwsP7gKbJp1NC/obpAPZ6csJdLn/XX3ntu/eBVmAot6FZxr
J7uvDVpNs9lFagisu7qIGeUCtLFB3fqYBqM3D9vrhZELqVIK43FPZuvLonL/VnRleN
y7POA5icxZuFd0lNsJanzJOPxfJU99Z7r52nrq0qCLWxN6zYq1I/v8Q/nyTk7h14qS
u7qnztqcxckWw==
Received: from php-smtp4.php.net (localhost [127.0.0.1])
by php-smtp4.php.net (Postfix) with ESMTP id 74C3B18005C
for <internals@lists.php.net>; Sat, 20 Jul 2024 18:33:35 +0000 (UTC)
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on php-smtp4.php.net
X-Spam-Level:
X-Spam-Status: No, score=0.2 required=5.0 tests=BAYES_50,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,DMARC_PASS,
FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,
RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS
autolearn=no autolearn_force=no version=4.0.0
X-Spam-Virus: No
X-Envelope-From: <cmbecker69@gmx.de>
Received: from mout.gmx.net (mout.gmx.net [212.227.15.18])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange ECDHE (prime256v1) server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by php-smtp4.php.net (Postfix) with ESMTPS
for <internals@lists.php.net>; Sat, 20 Jul 2024 18:33:34 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.de;
s=s31663417; t=1721500319; x=1722105119; i=cmbecker69@gmx.de;
bh=Pw12iCkQGsWTvoMgSBazmzxEktICng+4OQoJf3CA4dY=;
h=X-UI-Sender-Class:Message-ID:Date:MIME-Version:Subject:To:
References:From:In-Reply-To:Content-Type:
Content-Transfer-Encoding:cc:content-transfer-encoding:
content-type:date:from:message-id:mime-version:reply-to:subject:
to;
b=h3XH/UhivN0zXdYGTHDigFvX82qzomYxMvXaegvQb7iA5CNr0U0/ZWO2Ok9lFgGn
sHPM+jAKIv/1VNYk7TdQ1MZNBF68BPzAmN7WZ5Vzfd1LbB1UiYWFHt/9yHeylgd5k
gr+Xy92RGQpqschv0CXyQr3rILIn5h7Sc+tB8yJQaB7kx+hZe1CoFOVnMAD1yzFfs
PXIq5J+fqBHvb35Dz/jOc9MoRM89so8GDCgyXdEM8NwR/6WbkbYERTyukkLNr5B1L
L3OaTu1h7hdSskSbg3/gyp0bWXD2+qGPNH8xb0GkDfLTufWxZCV3uqbbt064eB17s
afX9SZewnu4K+nXhCg==
X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a
Received: from [192.168.2.130] ([79.251.222.223]) by mail.gmx.net (mrgmx004
[212.227.17.190]) with ESMTPSA (Nemesis) id 1MgvvT-1romhu3JID-00nT9v; Sat, 20
Jul 2024 20:31:59 +0200
Message-ID: <f4925fe4-15fb-432d-9dac-451240c5ea52@gmx.de>
Date: Sat, 20 Jul 2024 20:31:59 +0200
Precedence: bulk
list-help: <mailto:internals+help@lists.php.net
list-unsubscribe: <mailto:internals+unsubscribe@lists.php.net>
list-post: <mailto:internals@lists.php.net>
List-Id: internals.lists.php.net
x-ms-reactions: disallow
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PHP-DEV] Request for opinions: bug vs feature -
changeintokenization of yield from
Content-Language: de-DE
To: =?UTF-8?Q?Tim_D=C3=BCsterhus?= <tim@bastelstu.be>,
Juliette Reinders Folmer <php-internals_nospam@adviesenzo.nl>,
internals@lists.php.net
References: <66984FD0.5090805@adviesenzo.nl>
<AM8P250MB0170FFCA0014FB9EC272DB4FE2AC2@AM8P250MB0170.EURP250.PROD.OUTLOOK.COM>
<c9736385-94b7-4064-911f-e5fc1df7e2bd@gmx.de>
<AM8P250MB0170B4C59B4313A565CD029AE2AC2@AM8P250MB0170.EURP250.PROD.OUTLOOK.COM>
<6699F817.8070806@adviesenzo.nl>
<9571bb82-9873-4319-9bd1-0361748335be@bastelstu.be>
<669BDB00.70507@adviesenzo.nl>
<fa7db246-1577-43b7-a117-c18a73140091@bastelstu.be>
<669BE870.2050908@adviesenzo.nl>
<b7b47e7a-fc20-4774-bc49-fff271386504@bastelstu.be>
In-Reply-To: <b7b47e7a-fc20-4774-bc49-fff271386504@bastelstu.be>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K1:XH1q6NsCU2jU0W/JCXDgnrHGk3NMcuPmSZKlPLkfmF3g85/Vmsm
if1LMxVh+iIlIjerEPTbAKmHiC97+Mbag6wUfaC/VszScuT/wCMhlxCfJHc6fMoL6H9Rvjf
AlD3GQj/PRRzxGPXPpCJAwqf2X6Uo+CkQ7HCauvXjuGb7pwXlEnH4wdhfI1i/mQC4mbPeXU
rJIjAT/0V9HwVWLMpEoew==
UI-OutboundReport: notjunk:1;M01:P0:DlKDx58Ty7E=;kzJosHchuLEZpRz48iKuVTjrJdT
3OA7xFxv7XjDarWiN538FQhh23dEjLULg4iXE4JKCRVZKfhHw3u3+SiVTUScM1WzS/UizxsH6
+AyXXNzj30zpGrgxEiYGcRHaekWe8Yq6OQxkm3fm3eVXedTqoSDXNtEt/FbE3JBpcQcy9hcBB
7RvEapkL8AAiMjhFtsl5s1KQWvt9GAMzXYGNylQATOvcFA5sQ2HyrdhtLf62ppM/DTO4VhnbK
nG7VrXKHcKN0but7B/GmLMFZzlrb6N8QEOpXeYnwO8h0um0hufueqdcvJ6352v1gCM1l98QF8
W56DCcfeA0nthjo4xHcJs98qZBwC0jqQjoHmSbnPg/KyJU6Ps8D6TlPp2EelKzti+5i43okn6
opq6NcZQ4SQTdQ0n8UyV0lcgzT3gGjH8p7PIzhwJYrRlAW2JaSXQ1iXZot9oZHYswY6uJQYex
dxydMEe+tRjPdqsABU+mrqlPjZMenb3rUuwBxEwjjZadjoy1PkKD0Kqg1zSgGPdsxbYEKflRz
LGDUljd/d5TlNqOKc13I/8/6dnRfybN1oQH5oGVTP6pG3gfbiv399RTm9nDkyDTo6B2SSzP/H
1QkQOq4W9dfcp8j2EjOjGZSg3z7Tx/1QH1qP4nfBraoJEvBr1TgAjAes5ywSDonw9ordW5CFH
OGQB4jkXB9/MIb2t3lTu9ARd/5r3Xno/3RU135KfMVgY2uDk8L002dIf1wX1jsDZx/uTW6nI/
WmSZdcwPpxMXJGrNQ15nDkh2atwZuo+1/icoCeTbI2n7IAhsUfM6ug3ShheAxHMnpbKnXpPDI
ao9z8iFYgBDwKRZ0DqtfXAkA==
From: cmbecker69@gmx.de ("Christoph M. Becker")
On 20.07.2024 at 18:51, Tim D=C3=BCsterhus wrote:
>
>> And not, like it is now, an undocumented, random change creating an
>> inconsistency in the Tokenizer.
>
> The tokenizer is doing the right thing: It tokenizes the PHP source
> code. It is absolutely normal that PHP first and second-digit updates
> make changes to the token stream. New tokens are added, old tokens are
> removed, tokens may appear in places where they previously could not
> appear for well-formed PHP programs. Tools working on the token stream
> need to adapt and this change is no different from any other change to
> PHP's syntax in that regard (except that documenting the change was
> forgotten).
If the tokenizer would tokenize a whole file as a single token, would
that also be correct? Of course, I'm exaggerating, but
<https://3v4l.org/qIf2c> doesn't look correct to me =E2=80=93 "yield /* co=
mment
*/ from" shouldn't be a single token.
Cheers,
Christoph