Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:108579 Return-Path: Delivered-To: mailing list internals@lists.php.net Received: (qmail 53146 invoked from network); 14 Feb 2020 16:27:01 -0000 Received: from unknown (HELO php-smtp4.php.net) (45.112.84.5) by pb1.pair.com with SMTP; 14 Feb 2020 16:27:01 -0000 Received: from php-smtp4.php.net (localhost [127.0.0.1]) by php-smtp4.php.net (Postfix) with ESMTP id 76D741804D7 for ; Fri, 14 Feb 2020 06:41:41 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on php-smtp4.php.net X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-ASN: AS15169 209.85.128.0/17 X-Spam-Virus: No X-Envelope-From: Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by php-smtp4.php.net (Postfix) with ESMTPS for ; Fri, 14 Feb 2020 06:41:40 -0800 (PST) Received: by mail-lf1-f53.google.com with SMTP id n25so6938200lfl.0 for ; Fri, 14 Feb 2020 06:41:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WeUxgk/4JCzpawDSWxSWa+B3zjichzgOMnBV2q/rDTI=; b=UIrDjp4tj0dZgEhwgP9UkWQl+pSiYd0qjaB8H27jUQ51bZIcULs+yO6VXHlwFAVwBB pe2+mDLxEAziIFyBP5lMkqwe/TPaMhwPlShBf6walNLe+tBWfq6ND6DCGfZVL0mnCAG6 j399BOpth+rWajsJzbsaTKI6efVdJTMI4bL2PQQy1ZCmzjPrLei7qYbw7kUCH97hqp9E SvDCt4o/0stszsYRZXbN5J8Of8MgrFeV5MJQ99gHLhVtidhD6YWnirn9j1IR4TlxQ4G3 uC90oW3e9jpv6uhbXXMg0SU6WZOfE8GTEoSkck7okZPzyLAcoX0qEnaCUKsbDM1FtQwo R8vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WeUxgk/4JCzpawDSWxSWa+B3zjichzgOMnBV2q/rDTI=; b=hzAnJ4LlH78Seg1fqPHU24kaBWk7JopEbPX0ssrt0s2obAs0S9meywxrPM+ofSzU2w Gm/wG8SnT3Fm3FmHfYktlV0ord6IY0yKQxVdeUlDJ6VGUhwjupPded9UkHvbEQ+f20Sr ueJtOAjSC/uvyEf5lIs/ahHhJ5ZzdM4WpQI3gElHsPFoFtzkKaV4AwaTMwaNGsoW/a++ m+q6Hqxvkjr5NjbQJJccEOk5ntgpwi4MnN034d4ZX+jKng/PlFNlVRoeuZeiE4Xp+Tw5 YCqvv3Ls0o4QciQ0CxV90XuZjJOiWibSCZS3iEhgIkBMFAAAAdgS21n08TrzYAcUf5gQ j9jA== X-Gm-Message-State: APjAAAW0RRdEQuffYiZokIANW3XAPFNQtrziTg9Em2j0d9pciqLQMk0h tdJUyeNDLqEPEJUtRyx9O+fEIEvTYn2AQuJ6Fno= X-Google-Smtp-Source: APXvYqymHPfsNFqz3HWorY6PJMizdM459H6Pc+nDydfS10EzR8yBzcNfvSMtLIc2+1CTVlADs2YvzQBi5x3/YJgYW3s= X-Received: by 2002:ac2:43c6:: with SMTP id u6mr1876122lfl.191.1581691297780; Fri, 14 Feb 2020 06:41:37 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: Date: Fri, 14 Feb 2020 15:41:21 +0100 Message-ID: To: Marco Pivetta Cc: Sara Golemon , PHP internals Content-Type: multipart/alternative; boundary="000000000000f2d998059e8a335e" Subject: Re: [PHP-DEV] [RFC] token_get_all() TOKEN_AS_OBJECT mode From: nikita.ppv@gmail.com (Nikita Popov) --000000000000f2d998059e8a335e Content-Type: text/plain; charset="UTF-8" On Fri, Feb 14, 2020 at 2:44 PM Marco Pivetta wrote: > On Fri, Feb 14, 2020 at 2:38 PM Sara Golemon wrote: > >> Thanks for picking it up, and I agree with your response to Larry. As >> nice >> as it would be to lazy iterate, the scanner is just in NO shape to >> tolerate >> reentering userspace and potentially reinvoking the scanner before the >> first run through is done. >> > > If this is the current state, maybe it would suffice to declare the return > type as `iterable`, and return a strict (fully populated) structure in a > first implementation, later to be changed to an iterator, if applicable? > > Marco Pivetta > The fact that it returns an array is an important part of the contract. If an iterator variant is added in the future, it should be added as a separate method. I don't want to be writing $tokens = PhpToken::getAll($code); if (!is_array($tokens)) { $tokens = iterator_to_array($tokens); } to convert this to the right type. And I also don't want my usage to be implicitly "upgraded" to an iterator in the future: The iterator will always be less efficient, and I don't want to be forced to use it if I need an array anyway. Nikita --000000000000f2d998059e8a335e--