Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:108543 Return-Path: Delivered-To: mailing list internals@lists.php.net Received: (qmail 60075 invoked from network); 13 Feb 2020 18:51:19 -0000 Received: from unknown (HELO php-smtp4.php.net) (45.112.84.5) by pb1.pair.com with SMTP; 13 Feb 2020 18:51:19 -0000 Received: from php-smtp4.php.net (localhost [127.0.0.1]) by php-smtp4.php.net (Postfix) with ESMTP id 261E31804AB for ; Thu, 13 Feb 2020 09:05:47 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on php-smtp4.php.net X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.2 X-Spam-ASN: AS11403 66.111.4.0/24 X-Spam-Virus: No X-Envelope-From: Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by php-smtp4.php.net (Postfix) with ESMTPS for ; Thu, 13 Feb 2020 09:05:46 -0800 (PST) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 2DA8D214CE for ; Thu, 13 Feb 2020 12:05:46 -0500 (EST) Received: from imap26 ([10.202.2.76]) by compute7.internal (MEProxy); Thu, 13 Feb 2020 12:05:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=Gs6TxG cuYDa5poQf3uaNz6NgLsnCyq7aYPMyBe9ZoAw=; b=3SI6PyE48MhCTYN9bcJ7PI LThHRRkbW8JPdj1l2gCv3AWE5NRyVDpphovs8v0eq/vW+1/m3vZ70Ape/fmZDbUn FfTr2IRwfvv4xTJPPAYG0l5+qlDk2s9fnpgDptrUbzVqLZccAajW6th/nRLrTf81 kStRHpJ+Fo0DsFV0mN3fWIIwn7+rX5sB9A5EcHxs26fKEKhMWEsldWKLtyxiSgNB M9SHls8uvB4z9RUHI3E0j3+WbsoVNAeBI9VEjuhqe3NzkCWcWAIjyItcsTgDQprp RJeawxXjbKKEeka/cc2JUqVqDcFX76OUzNUjWURtFpU2m/bsN7NPOq/66BaBHelA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrieekgdeljecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvufgtsehttdertderredtnecuhfhrohhmpedfnfgrrhhr hicuifgrrhhfihgvlhgufdcuoehlrghrrhihsehgrghrfhhivghlughtvggthhdrtghomh eqnecuffhomhgrihhnpehphhhprdhnvghtnecuvehluhhsthgvrhfuihiivgeptdenucfr rghrrghmpehmrghilhhfrhhomheplhgrrhhrhiesghgrrhhfihgvlhguthgvtghhrdgtoh hm X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id EA10814200A2; Thu, 13 Feb 2020 12:05:45 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.7-802-g7a41c81-fmstable-20200203v1 Mime-Version: 1.0 Message-ID: <466bb718-4513-4a87-81e9-295ad3983443@www.fastmail.com> In-Reply-To: References: Date: Thu, 13 Feb 2020 11:05:24 -0600 To: "php internals" Content-Type: text/plain Subject: Re: [PHP-DEV] [RFC] token_get_all() TOKEN_AS_OBJECT mode From: larry@garfieldtech.com ("Larry Garfield") On Thu, Feb 13, 2020, at 3:47 AM, Nikita Popov wrote: > Hi internals, > > This has been discussed a while ago already, now as a proper proposal: > https://wiki.php.net/rfc/token_as_object > > tl;dr is that it allows you to get token_get_all() output as an array of > PhpToken objects. This reduces memory usage, improves performance, makes > code more uniform and readable... What's not to like? > > An open question is whether (at least to start with) PhpToken should be > just a data container, or whether we want to add some helper methods to it. > If this generates too much bikeshed, I'll drop methods from the proposal. > > Regards, > Nikita I love everything about this. 1) I would agree with Nicolas that a static constructor would be better. I don't know about polyfilling it, but it's definitely more self-descriptive. 2) I'm skeptical about the methods. I can see them being useful, but also being bikeshed material. For instance, if you're doing annotation parsing then docblocks are not ignorable. They're what you're actually looking for. Two possible additions, feel free to ignore if they're too complicated: 1) Should it return an array of token objects, or a lazy iterable? If I'm only interested in certain types (eg, doc strings, classes, etc.) then a lazy iterable would allow me to string some filter and map operations on to it and use even less memory overall, since the whole tree is not in memory at once. 2) Rather than provide bikesheddable methods, would it be feasible to take a queue from PDO and let users specify a subclass of PhpToken to fetch into? That way the properties are always there, but a user can attach whatever methods make sense for them. IMO the laziness would be more valuable, since a struct class can be operated on by an external function just as easily as a method, especially if using a functional pipeline. --Larry Garfield