Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:124555 X-Original-To: internals@lists.php.net Delivered-To: internals@lists.php.net Received: from php-smtp4.php.net (php-smtp4.php.net [45.112.84.5]) by qa.php.net (Postfix) with ESMTPS id 030641A00B7 for ; Tue, 23 Jul 2024 10:16:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=php.net; s=mail; t=1721729879; bh=kU/5O055ZWHKBmNI5zF/sIoXFqL2Ox2FAuWRNhUancY=; h=Date:Subject:To:References:From:In-Reply-To:From; b=Oofd9eWid9cgL9f5GpgbMMhiquTe/HMyA9HJhtyHFqoGDslp66Y6xDLKNjW2b5DZE 9SRNQVV06564VIzPOcY/eRYJXQWN9UFc2N97yhatBZd3rEa+6AwTpTmqhrg8q3JuBA ldsJOm6bcOCoVzlyJ9HQPJjnkKfoaDpgOcGFuUduBTHn/mSQ5SFLSPE0BQkp+BsIt8 H+Yuk568UJMmYZINZSFav4/8S1UJQ92R7CXB4ppTVa0YMvoOP72XtaU1/XKiQuZAvT kXURl02oheZSn+HKVcl+903kPOCWKM4iay7ayuBT/88OK/Ui77HK+O6QhWhz2x6rri LB5H4cSPxr9uA== Received: from php-smtp4.php.net (localhost [127.0.0.1]) by php-smtp4.php.net (Postfix) with ESMTP id 820D518002E for ; Tue, 23 Jul 2024 10:17:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on php-smtp4.php.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=BAYES_50,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,DMARC_PASS, FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=4.0.0 X-Spam-Virus: No X-Envelope-From: Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by php-smtp4.php.net (Postfix) with ESMTPS for ; Tue, 23 Jul 2024 10:17:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.de; s=s31663417; t=1721729781; x=1722334581; i=cmbecker69@gmx.de; bh=kU/5O055ZWHKBmNI5zF/sIoXFqL2Ox2FAuWRNhUancY=; h=X-UI-Sender-Class:Message-ID:Date:MIME-Version:Subject:To: References:From:In-Reply-To:Content-Type: Content-Transfer-Encoding:cc:content-transfer-encoding: content-type:date:from:message-id:mime-version:reply-to:subject: to; b=KZAXhHkYo0ERcMakfwWJ20peg8SpaHMb5rK8lTEWI1k/uYlZ0x6LyN2N1hXPyBSR NLu3O5lap/bneIKXG5lKMGcL+AeEWI0U7gO8VHp7PxJQUj53Z3vcDSLnUZZD9KpjT AwZ0JyEQxabv9VKxcp93bCrtlMu8qF/etNpHnYa+M+FEDvTdHDbS8daFM++T5U8bq U6JXBj4264nNqsa0z5qIB1IjAp+U5lirkyhyBC/CmxgwhS+6h4gQT8xeJnAKdnOf8 86kE6w6UM9xnsKzmHQdBHa05norTII+x2UCtlOzTQODLHdT8aVkvkFB6llUXnLLrM LDXYNtSn3ITsZPlE+A== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from [192.168.2.130] ([79.251.205.37]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MgNcz-1roN1m02Ep-00ZPya; Tue, 23 Jul 2024 12:16:21 +0200 Message-ID: Date: Tue, 23 Jul 2024 12:16:20 +0200 Precedence: bulk list-help: list-post: List-Id: internals.lists.php.net x-ms-reactions: disallow MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PHP-DEV] Request for opinions: bug vs feature -changeintokenization of yield from Content-Language: de-DE To: =?UTF-8?Q?Tim_D=C3=BCsterhus?= , Juliette Reinders Folmer , internals@lists.php.net References: <66984FD0.5090805@adviesenzo.nl> <6699F817.8070806@adviesenzo.nl> <9571bb82-9873-4319-9bd1-0361748335be@bastelstu.be> <669BDB00.70507@adviesenzo.nl> <669BE870.2050908@adviesenzo.nl> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:6pLhfssHBu5VCNC+aBEOkFUpj7AueT8BMODQkBHTNs0sBCKBf7R vCan2BG9DFItil6hI+M4pjkrCvzOcyFWGR+nYWv0k2meBgbulLfwv3czAX8uQyKQXWrOnaT zO8RGNX5GGePWHTCAwOtnVfK5ojOZHH7QtqR4GL4Le3KjoPVCFpXPD9AuuptGeZhlIoYeIw 4OI+g4Q11JkKeQUtBBx9w== UI-OutboundReport: notjunk:1;M01:P0:+EnFHK4SqIc=;k91fhy3sk+hYTRdEJ6ueLzsgPZ3 hDXdkJiClImXD8Dd8J5Ayl8SJjhNta2KZgCxOEbOCfpWhM+krTn5Jkvo8kcn4WSm2Fsex9lkw ra/4jC+aVezbpP1BSNnWNzO4L7YqsR80jq6GRZOt8RRDiBypjwb6kAFVK4Qa1LXtraAVN2zdF vPJK8844LB072Vs8h9iK1UBvD/0Pu1DtyTECbU666PHZbTTQiwgCCpneekNHKRONhleka1kYr GugfMWZHP0MIuDgPiAlBdVl02K+qMzvd8ATHR10DAy1hAd+UdxUIOugez4y0FGfBVAXDrKSq5 GDuHpPNV299bWkWIzbz1cXDWvtE9F2EBoXWL0NIHN5Jelp4wvcvSFTwJcyaFPTh+x/g3fyyGm Tf1Y1z/C9uO8GnlvkF2D8zdrQlIha5e+SVuhlGtucOXK5ioAc/xvp/ovh4xzZjCRc/UwM5++3 ACuiaVevZFHuDUyXE1yHXLfHHul+OVnABNg8tbSzqbqo3kxTe8MGsSB5Yf6ZB7DFJRzNso84+ WbnzVT3DrVU3i/JOcg4qDOzkGkJOtmVjM7YaoZcC2ynlA8U+1ImGPhEx6BFNb9K/KiRHsvQxe zqNw6yCZRK1wd9/5jpQuyc7cQeKQAnDv8ADrYshvoE8p6IuhPIhU8DvxmAXzdwjb68rLry2On EVbdsvsFo8gYdpWZZV0HYMeHEFTW+teiejgFr9M0UL/fmKmqDXBdX9SH1Gtypb3S724ouTPbZ ug+JXQ8oMy4w4Er8ctepwFGUPFT1PPVGXwQvGf8madVB5ydmrd9vFAUmffQCt2eqDW2RNxSmL xSrnaRH4FXwPMkeih7cwxxZQ== From: cmbecker69@gmx.de ("Christoph M. Becker") On 21.07.2024 at 13:28, Tim D=C3=BCsterhus wrote: > On 7/20/24 20:31, Christoph M. Becker wrote: > >> If the tokenizer would tokenize a whole file as a single token, would >> that also be correct?=C2=A0 Of course, I'm exaggerating, but > > The function is documented as: > >> token_get_all() parses the given code string into PHP language tokens >> using the Zend engine's lexical scanner. > > So if Zend engine's lexical scanner would tokenize a complete PHP file > as a single token, then the output would be correct, yes. Apparently we cannot even agree that there is a bug, and as such we cannot fix it in PHP 8.3 according to the release process documentation. >> doesn't look correct to me =E2=80=93 "yield /*= comment >> */ from" shouldn't be a single token. > > Looking at this from a user expectation perspective and not a technical > perspective, neither should `yield from` be a single token in: > > =C2=A0=C2=A0=C2=A0 var_dump(token_get_all('