Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:124115 X-Original-To: internals@lists.php.net Delivered-To: internals@lists.php.net Received: from php-smtp4.php.net (php-smtp4.php.net [45.112.84.5]) by qa.php.net (Postfix) with ESMTPS id 866CE1A009C for ; Sun, 30 Jun 2024 20:28:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=php.net; s=mail; t=1719779407; bh=VKfATErA5b9qIFvRBZKNUKdZfLnCCO0ZcIek5vjkbqI=; h=References:In-Reply-To:From:Date:Subject:To:From; b=F6beKYQvxzIh2EurZd+D3VLPnt4OEdC0QohxsU33HPSniujn60GB7BpOhesYz6HpS upS5ovWrdj/ZBIsmnfs6UJ2nJz4vYEklqGgumybqEVqVZlLcGogMhAXzlHT65x46TL z2uSKJTQCj9hx8KwSAX6SDrSzD5JmIOtp46WRdi4cWyJyRUGprpYYJTKYRcL8WNAG9 REhTpKCxDGTju791wNHVoYXArw9YRYeMrS2r2QbiYKyk1LCi2vev1dZbpI99kjFXnj +O8RE6nSl+8f4COeHZJSrTnU600wWtacafI3fU66CbiL4d6dFx2FJ01gvAUaaIqhOM jfYW8czAnzsoA== Received: from php-smtp4.php.net (localhost [127.0.0.1]) by php-smtp4.php.net (Postfix) with ESMTP id 57262180003 for ; Sun, 30 Jun 2024 20:30:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on php-smtp4.php.net X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_50,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,DMARC_PASS,FREEMAIL_FROM, HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=4.0.0 X-Spam-Virus: Error (Cannot connect to unix socket '/var/run/clamav/clamd.ctl': connect: Connection refused) X-Envelope-From: Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by php-smtp4.php.net (Postfix) with ESMTPS for ; Sun, 30 Jun 2024 20:30:05 +0000 (UTC) Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-43fdb797ee2so12514701cf.3 for ; Sun, 30 Jun 2024 13:28:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719779325; x=1720384125; darn=lists.php.net; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=1p9Yrbynql9qlz/I94VgE3qjdvHyi2EzaDpPddoiRuM=; b=fPfu/xjS1ckOqP32FXxfnnXRSLNWWLE+RoZuZAe2oT2tFXF5XgstjPptjqeYhZng+J vyBHPaFc6oQmS6HcJm0YyXqGNDfb2yYSyKIJ7AiVHeCG9xcz4pJpABMU5UXm9o6XhNv5 jj+uBgwfEw8WZMzNR76vRZ4OaVDu+d+HBPmzFKydfq5i+3DqJ0q1xHsxNCopg824qotE m68yr2EqdvshfATH4LZhQKoyQ4sd2QKZWaUABY28FyZqz6Sc6Q6X0NDqJRSvFDoXMiB0 ql21igVU7p6Sy44hJIrD6EiIxFO3iyfgyiSgo4FY0N8IvnPpZ6kmxn8boPbfpBswgjFc dfIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719779325; x=1720384125; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1p9Yrbynql9qlz/I94VgE3qjdvHyi2EzaDpPddoiRuM=; b=Y4Cc9ZG82k86lWCRA4CIB7p2AmAeD6iR6It+Q1zeIkGQ+lQqvIAm/Z/46ZuiGdSSpD b3YXmJRfoTTI7/R2zS3YZ9gKyHsIcbqwZWIojf5ZiiZzHOdAMLpXKySk7pDvf7fzHid1 +XcsSSEpM9Ing3MWpXDngQn6gZQB7XCN6IjI3K/tysAd7DftgNUQVxmB2OGN+MiYCre4 EyfejlESwdeSsbael5ocyu2ZkfW34PU2D35itODk4BfeEraUDB5DoKw1dCxzfq/Kjm3Q jkbuWmgobk4ZrsBu3DqVlA3iRrPXmHJCqbi7PJ6Er9PEJXq4RU5b2JItzOuEOvcWgdZ+ E6yA== X-Gm-Message-State: AOJu0YwvOtzc7pvrkVBV/1b0HUxJljqwmIHYWKLZ5iUZvosAWb6GPkM3 xnFb8PDZzIMV3YzrNJY12y4BP/haqtWbfAmMUim8+KDPuHa6Goj4zCSVDLSyTTQx1zGIlVA99JY KtY0CWnmICY1MH6cQpomJ5hMOqGosfjQV X-Google-Smtp-Source: AGHT+IF26z0yT7AWiqmFH3XbK0HO2Pthc5W5GXMhBP8POJkyj8bH0C8r2Rai6vQ3jydGRrUqKKRF3E9q9RmAUQ8T8WE= X-Received: by 2002:a05:6214:f2d:b0:6b5:4146:c4d8 with SMTP id 6a1803df08f44-6b5b70a859cmr48332106d6.1.1719779324498; Sun, 30 Jun 2024 13:28:44 -0700 (PDT) Precedence: bulk list-help: list-post: List-Id: internals.lists.php.net MIME-Version: 1.0 References: <1917CF7C-26D8-4DBE-B05C-5AA650AC6C9F@rwec.co.uk> <551cd5b0-1c00-4818-a9ca-97f6b7e8c3dc@app.fastmail.com> <39B496F8-062E-4848-9B3B-529BE8D3415A@newclarity.net> <856F4F70-DC81-4098-82DD-5F6D47CDF3F0@newclarity.net> In-Reply-To: Date: Sun, 30 Jun 2024 16:28:32 -0400 Message-ID: Subject: [PHP-DEV] Iteration III: Packages (was Re: [PHP-DEV] [Initial Feedback] PHP User Modules - An Adaptation of ES6 from JavaScript) To: PHP internals Content-Type: multipart/alternative; boundary="000000000000ba9c12061c215377" From: tendoaki@gmail.com (Michael Morris) --000000000000ba9c12061c215377 Content-Type: text/plain; charset="UTF-8" So let's take another crack at this based on all the points raised in the thread. This should also underline why I don't consider this an RFC - I am iterating until we arrive at something that may be refinable into an RFC. And I say we because without the aid of those in this conversation I would not have arrived at what will follow. Before I continue I would like to apologize for being somewhat irritable. We're all here because we enjoy using this language and want to see it improved and prevent bad changes. Opinions will differ on this and in the heat of the moment of arguing a point things can get borderline. Returning to a point I made earlier, Composer isn't used on Wordpress. I went over to the Wordpress discussion list and read over why, because that discussion provides clues to what kind of package management may be adoptable. I think the largest point is that Wordpress should be usable without ever resorting to using the command line. Yes, it does have a command line tool - wp-cli - and it is powerful, but using it as an administrator of a Wordpress site is not required. The largest block to composer's inclusion in Wordpress is the inability to run multiple versions of a module. Yes, it's a mess when this happens, but if you're an end user, you just want your plugins to work. If one plugin that no one has updated in a year that you're using is consuming version 2 of a package, you're gonna be annoyed at best if the module stops working when you install a new plugin that is using version 3 of the same package and has a BC break in it. Composer can't resolve this easily. There are WordPress plugins that use composer - I have a couple in the website I'm working on. But they accomplish the inclusion of composer by redistributing the packages, and using a utility called brianhenryie/strauss to monkey type the entire included package into the plugin, changing the namespace of the entire package to something different. The approach works, but it's ugly. In any event, the plugin that results from this carries a copy of the code from packagist rather than sourcing the code from packagist. -- IMPORT -- The import statement is for bringing in packages. It needs to be able to deal with: * Extensions - the existing and oldest of packages for PHP * PECL Extensions * Phar Packages * Composer Packages * PHP Modules - this is the new module system that has dominated the conversation, but in this iteration it's going to be broken away from import to some degree in this iteration. Today we'll look just at composer. Now import needs to load packages in a manner that allows different versions to be run concurrently. A PHP application such as Wordpress should be distributable without needing to use the command line. That is, if WordPress leverages this in any way, they don't have to give up their famous 10 minute quick install. Some terms here to keep myself from getting lost (let alone anyone trying to read this). * APPLICATION - This is the overall application - WordPress, Drupal, BobbysFirstFramework, etc. - that is doing the import. This code is on the root scope. * ROOT SCOPE - This is where the global variables and the namespaces as we know them exist. Contrast this with * PACKAGE SCOPE - Each package brought in with import gets its own package scope. This is a distinct behavior from Include/Require. I think each package scope will need to be on its own request thread, but this is an implementation detail I can't speak to with any authority. The goal is whatever happens in a package stays in the package. If two different packages want to define /foo(), they can. When a package is imported the parser will look for a `.php.mod` file at the root of the package. Among other things, this file details what type of package it is and where to mount it by default in the namespace of the ROOT SCOPE. So, GIVEN a package with this .php.mod file package MyModule WHEN I issue this import in an application import "MyModule"; THEN I should be able to access a method in that module with this code \MyModule\foo(); Aliasing is an option - `import "MyModule" as BeerModule` will make the methods accessible in the root with \BeerModule\foo(); Unlike require/include import is sensitive to the namespace it is in for mounting. So namespace Trees; import "MyModule"; MyModule\foo(); // works \Trees\MyModule\foo(); // needed from another namespace. That said, with aliasing an absolute namespace for the module can be assigned. namespace Trees; import "MyModule" as \MyModule; MyModule\foo(); // works if my understanding of existing namespace resolution rules is correct. \MyModule\foo(); // also works. Now, with that in place, let's crack a tougher nut - handling a composer package. By default composer is designed to set up an autoloader, then resolve symbol references as they come up. This works until you have two packages that want the same symbol reference - which will most frequently occur with incompatible versions of the same package. So our puzzle here is how to allow composer to do its thing without rewriting it. We'll deal with admittedly the hardest case first - importing a package whose maintainers have taken no action to make it compatible with this new system. import "composer://packagist.org/packages/twig/twig#v3.10.3" as TemplateEngine The reason for that alias and not "Twig" is because the mounting point comes before the internal namespace of the file. This is unavoidable with this scheme The URL there is "loader://package_url". PHP by default will know what the composer loader is. It will look to see if the user has globally installed composer already and use that, otherwise it will locally install composer for the project, initialize it, download the package and have composer resolve the package ending in setting up an autoloader that is only invoked within that package. Application configuration can make a lot of this go away. So let's step away from the import statement itself to look at that. -- APPLICATIONS -- Applications can configure how they store their packages, but in the absence of such PHP will use some logical default behaviors. We've already looked at one, the loading of a composer package, but we have a long ugly import as a result. Most PHP applications have a single point of entry, and part of that is the establishment of a cwd (current working directory). When PHP loads a file it will look for a `.php-packages` directory in the current working directory and if one doesn't exist it will make one the first time the import statement is invoked (so code not using import will not have this directory created). It is here that the package downloads land. We can also choose to go ahead and make this directory ourselves and place `.php.mod` in that directory. Let's look at what one might look like for Drupal, which already uses composer. package Drupal; php: 10 registry packagist.org/packages composer imports ( phar://getcomposer.org/composer.phar ) init ( composer install ) require ( ./vendor/autoload.php ) Now, composer is a known and popular quantity, so the imports, init and require directives can probably be baked into PHP, but if they ever change - or if a competitor to composer shows up like yarn did to npm then there needs to be a way to set it up. Also, for the moment I'm using go.mod's format because it feels the cleanest. The exact format of this file - whether it's yaml, json, toml, .ini or whatever else, is a discussion for another day. Key in on the type of information that needs to be relayed here, not how it's relayed. Importantly, because this .php.mod file is at the top level of the application's .php-packages directory it affects the behavior of the ROOT SCOPE. The php directive gives the minimum php version for the application. The registry directive sets the registry to packagist and sets the loader for that registry to composer. Multiple registries can have separate loaders. The imports directive loads composer using the default phar loader. We use an absolute path because we don't have a phar registry. This particular call could be baked in due to composer's popularity. The init directive runs the first time the application runs, just before any file in the application is parsed. The .php.sum file will bookmark the last time the init has run and there will likely need to be a mechanism to force it's rerun. The require directive requires these files once before starting the application's first file. Assume we move our existing composer.json file into .php-packages, what then? We gain the following: * Composer install will be ran for us, without using the cli. * The autoloader will be set up for us without explicitly requiring it anywhere. * We can install an alternate version of packages into their own package scope import "twig/twig#v2.5" as OldTwig; Or again, if we are in a namespace we can import to that namespace. I'll stop here cause this is a healthy chunk to absorb and I just spent 4 hours thinking on this as I wrote it out and I'm tired. If this was in the language today, Wordpress plugins could start using it without fear of mucking each other up or messing with Wordpress core. But that's a discussion for later. --000000000000ba9c12061c215377 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
So let's take another crack at this based on all the p= oints raised in the thread. This should also underline why I don't cons= ider this an RFC - I am iterating until we arrive at something that may be = refinable into an RFC. And I say we because without the aid of those in thi= s conversation I would not have arrived at what will follow.

=
Before I continue I would like to apologize for being somewhat irritab= le. We're all here because we enjoy using this language and want to see= it improved and prevent bad changes. Opinions will differ on this and in t= he heat of the moment of arguing a point things can get borderline.


Returning to a point I made earlier, Compo= ser isn't used on Wordpress.=C2=A0 I went over to the Wordpress discuss= ion list and read over why, because that discussion provides clues to what = kind of package management may be adoptable. I think the largest point is t= hat Wordpress should be usable without ever resorting to using the command = line. Yes, it does have a command line tool - wp-cli - and it is powerful, = but using it as an administrator of a Wordpress site is not required.
=

The largest block to composer's inclusion in Wordpr= ess is the inability to run multiple versions of a module. Yes, it's a = mess when this happens, but if you're an end user, you just want your p= lugins to work.=C2=A0 If one plugin that no one has updated in a year that = you're using is consuming version 2 of a package, you're gonna be a= nnoyed at best if the module stops working when you install a new plugin th= at is using version 3 of the same package and has a BC break in it.=C2=A0 C= omposer can't resolve this easily.

There are W= ordPress plugins that use composer - I have a couple in the website I'm= working on. But they accomplish the inclusion of composer by redistributin= g the packages, and using a utility called=C2=A0brianhenryie/strauss to mon= key type the entire included package into the plugin, changing the namespac= e of the entire package to something different. The approach works, but it&= #39;s ugly.=C2=A0 In any event, the plugin that results from this carries a= copy of the code from packagist rather than sourcing the code from packagi= st.


-- IMPORT --

The import statement is for bringing in packages.=C2=A0 It needs to = be able to deal with:

* Extensions - the existing = and oldest of packages for PHP
* PECL Extensions
* Phar= Packages
* Composer Packages
* PHP Modules - this is t= he new module system that has dominated the conversation, but in this itera= tion it's going to be broken away from import to some degree in this it= eration.

Today we'll look just at composer.

Now import needs to load packages in a manner that a= llows different versions to be run concurrently. A PHP application such as = Wordpress should be distributable without needing to use the command line. = That is, if WordPress leverages this in any way, they don't have to giv= e up their famous 10 minute quick install.

Some te= rms here to keep myself from getting lost (let alone anyone trying to read = this).

* APPLICATION - This is the overall applica= tion - WordPress, Drupal, BobbysFirstFramework, etc.=C2=A0- that is doing t= he import. This code is on the root scope.
* ROOT SCOPE - This is= where the global variables and the namespaces as we know them exist.=C2=A0= Contrast this with
* PACKAGE SCOPE - Each package brought in wit= h import gets its own package scope. This is a distinct behavior from Inclu= de/Require. I think each package scope will need to be on its own request t= hread, but this is an implementation detail I can't speak to with any a= uthority. The goal is whatever happens in a package stays in the package.= =C2=A0 If two different packages want to define /foo(), they can.

When a package is imported the parser will look for a `.php= .mod` file at the root of the package. Among other things, this file detail= s what type of package it is and where to mount it by default in the namesp= ace of the ROOT SCOPE. So,=C2=A0

GIVEN a package with this .p= hp.mod file

package MyModule

WHEN I issue this import i= n an application

imp= ort "MyModule";
THEN I should be able to access a method in that module = with this code

\MyMo= dule\foo();
Aliasing is an option - `import "MyModule" as B= eerModule` will make the methods accessible in the root with \BeerModule\fo= o();

Unlike require/include import is sensitive to= the namespace it is in for mounting.=C2=A0 So

namespace Trees;
import "MyModule";

MyModule\f= oo(); // works

\Trees\MyModule\foo(); // needed from another namespa= ce.

That said, with alia= sing an absolute namespace for the module can be assigned.

namespace Trees;
import "MyModule" as \MyModule;

MyModule\foo(); // works if my understanding of existing nam= espace resolution rules is correct.
\MyModule\foo(); // also works.

Now, with that in place, let= 's crack a tougher nut - handling a composer package. By default compos= er is designed to set up an autoloader, then resolve symbol references as t= hey come up. This works until you have two packages that want the same symb= ol reference - which will most frequently occur with incompatible versions = of the same package.=C2=A0 So our puzzle here is how to allow composer to d= o its thing without rewriting it.=C2=A0 We'll deal with admittedly the = hardest case first - importing a package whose maintainers have taken no ac= tion to make it compatible with this new system.
= =C2=A0
import "composer://= packagist.org/p= ackages/twig/twig#v3.10.3&= quot; as TemplateEngine

The reason for that alias and not "= ;Twig" is because the mounting point comes before the internal namespa= ce of the file. This is unavoidable with this scheme

The URL there is "loader://package_url". PHP by default will k= now what the composer loader is. It will look to see if the user has global= ly installed composer already and use that, otherwise it will locally insta= ll composer for the project, initialize it, download the package and have c= omposer resolve the package ending in setting up an autoloader that is only= invoked within that package.

Application configur= ation can make a lot of this go away.=C2=A0 So let's step away from the= import statement itself to look at that.


-- APPLICATIONS --
Applications can configure how they sto= re their packages, but in the absence of such PHP will use some logical def= ault behaviors. We've already looked at one, the loading of a composer = package, but we have a long ugly import as a result.=C2=A0

Most PHP applications have a single point of entry, and part of th= at is the establishment of a cwd (current working directory). When PHP load= s a file it will look for a `.php-packages` directory in the current workin= g directory and if one doesn't exist it will make one the first time th= e import statement is invoked (so code not using import will not have this = directory created). It is here that the package downloads=C2=A0land.=C2=A0 = We can also choose to go ahead and make this directory ourselves and place = `.php.mod` in that directory. Let's look at what one might look like fo= r Drupal, which already uses composer.
<= br>
package Drupal;
<= div>
php: 10

r= egistry packagist.org/packages composer

imports (=C2=A0 phar://getcomposer= .org/composer.phar
)

init (
=C2=A0 composer install
)
require (
=C2=A0./vendor/autoload.php
)

Now, composer is a=C2=A0known=C2=A0and popular quantity,=C2= =A0so the imports, init and require directives can probably be baked into P= HP, but if they ever change - or if a competitor to composer shows up like = yarn did to npm then there needs to be a way to set it up.

Also, for the moment I'm using go.mod's format because it = feels the=C2=A0cleanest.=C2=A0 The exact format of this file - whether it&#= 39;s yaml, json, toml, .ini or whatever else, is a discussion for another d= ay. Key in on the type of information that needs to be relayed here, not ho= w it's relayed.

Importantly, because this .php= .mod file is at the top level of the application's .php-packages direct= ory it affects the behavior of the ROOT SCOPE.=C2=A0=C2=A0

The php directive gives the minimum php version for the applicatio= n.

The registry directive sets the registry to pac= kagist and sets the loader for that registry to composer. Multiple registri= es can have separate loaders.

The imports directiv= e loads composer using the default phar loader. We use an absolute path bec= ause we don't have a phar registry. This particular call could be baked= in due to composer's popularity.

The init dir= ective runs the first time the application runs, just before any file in th= e application is parsed.=C2=A0 The .php.sum file will bookmark the last tim= e the init has run and there will likely need to be a mechanism to force it= 's rerun.

The require directive requires these= files once before starting the application's first file.
Assume we move our existing composer.json file into .php-packag= es, what then?=C2=A0 We gain the following:
* Composer install wi= ll be ran for us, without using the cli.
* The autoloader will be= set up=C2=A0for us without explicitly requiring it anywhere.
* W= e can install an alternate version of packages into their own package scope=

import "twig/t= wig#v2.5" as OldTwig;

Or again, if we are in a namespace we can import to that namespace= .

I'll stop here cause this is a healthy chunk= to absorb and I just spent 4 hours thinking on this as I wrote it out and = I'm tired. If this was in the language today, Wordpress plugins could s= tart using it without fear of mucking each other up or messing with Wordpre= ss core. But that's a discussion for later.

--000000000000ba9c12061c215377--