Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:124116 X-Original-To: internals@lists.php.net Delivered-To: internals@lists.php.net Received: from php-smtp4.php.net (php-smtp4.php.net [45.112.84.5]) by qa.php.net (Postfix) with ESMTPS id 801D31A009C for ; Sun, 30 Jun 2024 21:19:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=php.net; s=mail; t=1719782448; bh=rbEGvFaUinkgc+0zFSnEesYgTKsIYSVaQC24ii3RYS4=; h=In-Reply-To:References:Date:From:To:Subject:From; b=oEdtJA3oBt3UwZ5So0Sma5jhyo5S+OgDoCm9q6dm5GAEN1+5OeTyuFoaB17KchvDt u3V9SyyQS/lVN+eB2KNw1o0yPnkn0qv2P+O0qitNXCXMFvhMFju/XSaS64pt+z18Zg mV9sU9OQGvWiDXGzZoJqClwtswSG5Q//vZ5ywwoJcWOKBrDrayAUGq/3JueXoZdzJO Qjc3UpZNT0i50hOU2xbxlBbb9K9CcCCtuaX8rP46JmiKFPk3dOg7FLQ0bGC0xap3ET jYkj/+T9oahaGkDgQN5vvbESfK+HrsvWHUOrOAT6bgs3kNrGnXFT+BvglW4ji+tpO/ uAQE+GzuM9bAw== Received: from php-smtp4.php.net (localhost [127.0.0.1]) by php-smtp4.php.net (Postfix) with ESMTP id E159F18060E for ; Sun, 30 Jun 2024 21:20:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on php-smtp4.php.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=BAYES_50,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,DMARC_MISSING,HTML_MESSAGE, RCVD_IN_DNSWL_LOW,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=4.0.0 X-Spam-Virus: Error (Cannot connect to unix socket '/var/run/clamav/clamd.ctl': connect: Connection refused) X-Envelope-From: Received: from fhigh4-smtp.messagingengine.com (fhigh4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by php-smtp4.php.net (Postfix) with ESMTPS for ; Sun, 30 Jun 2024 21:20:46 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id BBAD7114017B for ; Sun, 30 Jun 2024 17:19:25 -0400 (EDT) Received: from imap49 ([10.202.2.99]) by compute1.internal (MEProxy); Sun, 30 Jun 2024 17:19:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bottled.codes; h=cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1719782365; x=1719868765; bh=WeZM8Bzkk3 4jqyGdoOS60ZOaGQhzpX4UPvdvBabkaXg=; b=DiTDj03cMR3UQ9hLORTu4ockSW 2ou5GKEzBe5eBjwSK89s6Dwi9CNr/jjr4VpkTrqC8dNJKmZ8xhOkLOI8fZVfM9Vd tjsjsRuud/oX8tJbpS8WCwPUBHE8wW5Be0TMBxmwGs3dRpid6OdZs8IYjiMXDt3M CN8ZveoZHunG64tEv3nHCIdbS3DK/CyddS2Te+WZgnDAnu/j5lkDNMnaPCKzB+Yt y9hjgngKkktnzlLmDh/VAWqXuGl5QfXQ4jAX9YE60lKhMHHE+Li79AIP/S5ph7AA Z7sT9QNT6pmHVaBVq47p/Ru8g7kNONYQCMhbxqj/aGyJL4omkauNJHGr0SPQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1719782365; x=1719868765; bh=WeZM8Bzkk34jqyGdoOS60ZOaGQhz pX4UPvdvBabkaXg=; b=YDjcstopNy2Oa9sKdplaKt/JYwPxfuIprunhwPCYEQ50 JI32/Jn+FDvi8kPjxIbI7l4CyG9E52gq7saHqSPcjNfrR2gVHPabQFYzHdT/C61R nsgUljazkaOFhssPf+UIXbRcgmRo3LoaowqpLdipk/Lx/Dj/6VwN54T0USKx3s7R nyYgs7mL5Kb9fyehWaG/rWf3unCm95btYrzdS63oOCu3aXMgKUimDpzQ0Ez1bc82 4vzVqrBD1wrvneQeH6LXW9psYCownquXQEDkFacaW+6nPLpcEYw3SzutI/3dR4FK Ki0UGF3v44GVoJFBuBt5bYrpLPJ6T1r1ebL/Zxsqrg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddruddugdduieduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsegrtd erreerreejnecuhfhrohhmpedftfhosgcunfgrnhguvghrshdfuceorhhosgessghothht lhgvugdrtghouggvsheqnecuggftrfgrthhtvghrnhepkeeffeetffegffeftdetueeuvd eiuedvjeffvdegueevlefhfeeujeeijeekhfegnecuffhomhgrihhnpehprggtkhgrghhi shhtrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homheprhhosgessghothhtlhgvugdrtghouggvsh X-ME-Proxy: Feedback-ID: ifab94697:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 4623E15A0092; Sun, 30 Jun 2024 17:19:25 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.11.0-alpha0-538-g1508afaa2-fm-20240616.001-g1508afaa Precedence: bulk list-help: list-post: List-Id: internals.lists.php.net MIME-Version: 1.0 Message-ID: <9f3da67f-5512-4643-89cc-05fc0981a791@app.fastmail.com> In-Reply-To: References: <1917CF7C-26D8-4DBE-B05C-5AA650AC6C9F@rwec.co.uk> <551cd5b0-1c00-4818-a9ca-97f6b7e8c3dc@app.fastmail.com> <39B496F8-062E-4848-9B3B-529BE8D3415A@newclarity.net> <856F4F70-DC81-4098-82DD-5F6D47CDF3F0@newclarity.net> Date: Sun, 30 Jun 2024 23:18:20 +0200 To: internals@lists.php.net Subject: Re: [PHP-DEV] Iteration III: Packages (was Re: [PHP-DEV] [Initial Feedback] PHP User Modules - An Adaptation of ES6 from JavaScript) Content-Type: multipart/alternative; boundary=28ebcaa9f4b347508c0be00c7b6cbc26 From: rob@bottled.codes ("Rob Landers") --28ebcaa9f4b347508c0be00c7b6cbc26 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable On Sun, Jun 30, 2024, at 22:28, Michael Morris wrote: > So let's take another crack at this based on all the points raised in = the thread. This should also underline why I don't consider this an RFC = - I am iterating until we arrive at something that may be refinable into= an RFC. And I say we because without the aid of those in this conversat= ion I would not have arrived at what will follow. >=20 > Before I continue I would like to apologize for being somewhat irritab= le. We're all here because we enjoy using this language and want to see = it improved and prevent bad changes. Opinions will differ on this and in= the heat of the moment of arguing a point things can get borderline. >=20 >=20 > Returning to a point I made earlier, Composer isn't used on Wordpress.= I went over to the Wordpress discussion list and read over why, becaus= e that discussion provides clues to what kind of package management may = be adoptable. I think the largest point is that Wordpress should be usab= le without ever resorting to using the command line. Yes, it does have a= command line tool - wp-cli - and it is powerful, but using it as an adm= inistrator of a Wordpress site is not required. >=20 > The largest block to composer's inclusion in Wordpress is the inabilit= y to run multiple versions of a module. Yes, it's a mess when this happe= ns, but if you're an end user, you just want your plugins to work. If o= ne plugin that no one has updated in a year that you're using is consumi= ng version 2 of a package, you're gonna be annoyed at best if the module= stops working when you install a new plugin that is using version 3 of = the same package and has a BC break in it. Composer can't resolve this = easily. >=20 > There are WordPress plugins that use composer - I have a couple in the= website I'm working on. But they accomplish the inclusion of composer b= y redistributing the packages, and using a utility called brianhenryie/s= trauss to monkey type the entire included package into the plugin, chang= ing the namespace of the entire package to something different. The appr= oach works, but it's ugly. In any event, the plugin that results from t= his carries a copy of the code from packagist rather than sourcing the c= ode from packagist. >=20 >=20 > -- IMPORT -- >=20 > The import statement is for bringing in packages. It needs to be able= to deal with: >=20 > * Extensions - the existing and oldest of packages for PHP > * PECL Extensions > * Phar Packages > * Composer Packages > * PHP Modules - this is the new module system that has dominated the c= onversation, but in this iteration it's going to be broken away from imp= ort to some degree in this iteration. >=20 > Today we'll look just at composer. >=20 > Now import needs to load packages in a manner that allows different ve= rsions to be run concurrently. A PHP application such as Wordpress shoul= d be distributable without needing to use the command line. That is, if = WordPress leverages this in any way, they don't have to give up their fa= mous 10 minute quick install. >=20 > Some terms here to keep myself from getting lost (let alone anyone try= ing to read this). >=20 > * APPLICATION - This is the overall application - WordPress, Drupal, B= obbysFirstFramework, etc. - that is doing the import. This code is on th= e root scope. > * ROOT SCOPE - This is where the global variables and the namespaces a= s we know them exist. Contrast this with > * PACKAGE SCOPE - Each package brought in with import gets its own pac= kage scope. This is a distinct behavior from Include/Require. I think ea= ch package scope will need to be on its own request thread, but this is = an implementation detail I can't speak to with any authority. The goal i= s whatever happens in a package stays in the package. If two different = packages want to define /foo(), they can. >=20 > When a package is imported the parser will look for a `.php.mod` file = at the root of the package. Among other things, this file details what t= ype of package it is and where to mount it by default in the namespace o= f the ROOT SCOPE. So,=20 >=20 >> GIVEN a package with this .php.mod file >>=20 >>> package MyModule >>=20 >> WHEN I issue this import in an application >>=20 >>> import "MyModule"; >>=20 >> THEN I should be able to access a method in that module with this code >>=20 >>> \MyModule\foo(); >=20 > Aliasing is an option - `import "MyModule" as BeerModule` will make th= e methods accessible in the root with \BeerModule\foo(); Just to challenge you a bit here. The language already has `use` for thi= s. Does `use` stay or is it replaced with `import` and why not just chan= ge the meaning of `use` in a package context? >=20 > Unlike require/include import is sensitive to the namespace it is in f= or mounting. So >=20 >> namespace Trees; >> import "MyModule"; >>=20 >> MyModule\foo(); // works >>=20 >> \Trees\MyModule\foo(); // needed from another namespace. >=20 > That said, with aliasing an absolute namespace for the module can be a= ssigned. >=20 >> namespace Trees; >> import "MyModule" as \MyModule; >>=20 >> MyModule\foo(); // works if my understanding of existing namespace re= solution rules is correct. >> \MyModule\foo(); // also works. >=20 > Now, with that in place, let's crack a tougher nut - handling a compos= er package. By default composer is designed to set up an autoloader, the= n resolve symbol references as they come up. This works until you have t= wo packages that want the same symbol reference - which will most freque= ntly occur with incompatible versions of the same package. So our puzzl= e here is how to allow composer to do its thing without rewriting it. W= e'll deal with admittedly the hardest case first - importing a package w= hose maintainers have taken no action to make it compatible with this ne= w system. >> =20 >> import "composer://packagist.org/packages/twig/twig#v3.10.3" as Templ= ateEngine >=20 > The reason for that alias and not "Twig" is because the mounting point= comes before the internal namespace of the file. This is unavoidable wi= th this scheme >=20 > The URL there is "loader://package_url". PHP by default will know what= the composer loader is. It will look to see if the user has globally in= stalled composer already and use that, otherwise it will locally install= composer for the project, initialize it, download the package and have = composer resolve the package ending in setting up an autoloader that is = only invoked within that package. Adding this after I just wrote the below book. It started out simple eno= ugh and it drives me crazy when people crash my proposal with a counter-= proposal. So, by all means, take it with a grain of salt ... I kinda wen= t overboard thinking about it. I think composer and friends are a moot point. If we go a bespoke way fo= r everything, we end up with a mess. What about creating "hooks" that th= ings like composer can "register" an installer at? For example, we could= define a "WELL_KNOWN/installers/composer/hooks.json" (I'm gonna steal a= bunch of ideas from kubernetes from here on), where WELL_KNOWN is some = engine-specific directory (like where the php.ini file is). Basically, a= ny installer can register an installer by creating a directory in WELL_K= NOWN/installers of which an installer might look like the following for = composer: WELL_KNOWN/ installers/ composer/ hooks.json composer.phar cache and hooks.json could have a schema of something like: { "name": "composer", "version": "4.5", "executable": "./composer.phar", "scheme": "composer", "command": "install-package-module" } Then the engine can scan these directories and read each hooks.json. The= n when it gets to your example above, it sees the scheme "composer" in t= he URL, looks for an installer with that name, and calls the executable = with some arguments (the command, the "URL" aka the package, current dir= ectory, etc). So, it might call composer with something like: /WELL_KNOWN/installers/composer/composer.phar install-package-module pac= kagist.org/packages/twig/twig#v3.10.3 /app/public The command is expected to dump the file to stdout -- which PHP then pip= es to wherever it is supposed to go (as you mention below). At this point, the user may not know a single thing about composer or ho= w it works, or anything, really. As far as they are concerned, they said= they wanted an import and they got one. However, there still exists a b= unch of composer-specific scripts, etc. In this case, we make new WELL_K= NOWN types. For example, we can have "script-runners" that can be regist= ered so you can just do "php run composer test" and it will run `compose= r test` for you. If you want something shorter, you can just add `alias = composer=3D"php run composer"` to your shell and bob's your uncle.=20 This is basically how kubernetes handles networking, disks, etc. so that= the entire thing is completely swappable and extendable. So, we know th= e idea is sound and "just works," we just need to customize it for php. =E2=80=94 Rob --28ebcaa9f4b347508c0be00c7b6cbc26 Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable

=
On Sun, Jun 30, 2024, at 22:28, Michael Morris wrote:
=
So let's take another crack at this based on all the points raised in= the thread. This should also underline why I don't consider this an RFC= - I am iterating until we arrive at something that may be refinable int= o an RFC. And I say we because without the aid of those in this conversa= tion I would not have arrived at what will follow.

Before I continue I would like to apologize for being somewhat i= rritable. We're all here because we enjoy using this language and want t= o see it improved and prevent bad changes. Opinions will differ on this = and in the heat of the moment of arguing a point things can get borderli= ne.


Returning to a point I m= ade earlier, Composer isn't used on Wordpress.  I went over to the = Wordpress discussion list and read over why, because that discussion pro= vides clues to what kind of package management may be adoptable. I think= the largest point is that Wordpress should be usable without ever resor= ting to using the command line. Yes, it does have a command line tool - = wp-cli - and it is powerful, but using it as an administrator of a Wordp= ress site is not required.

The largest bloc= k to composer's inclusion in Wordpress is the inability to run multiple = versions of a module. Yes, it's a mess when this happens, but if you're = an end user, you just want your plugins to work.  If one plugin tha= t no one has updated in a year that you're using is consuming version 2 = of a package, you're gonna be annoyed at best if the module stops workin= g when you install a new plugin that is using version 3 of the same pack= age and has a BC break in it.  Composer can't resolve this easily.<= br>

There are WordPress plugins that use compos= er - I have a couple in the website I'm working on. But they accomplish = the inclusion of composer by redistributing the packages, and using a ut= ility called brianhenryie/strauss to monkey type the entire include= d package into the plugin, changing the namespace of the entire package = to something different. The approach works, but it's ugly.  In any = event, the plugin that results from this carries a copy of the code from= packagist rather than sourcing the code from packagist.
<= br>

-- IMPORT --

T= he import statement is for bringing in packages.  It needs to be ab= le to deal with:

* Extensions - the existin= g and oldest of packages for PHP
* PECL Extensions
* Phar Packages
* Composer Packages
*= PHP Modules - this is the new module system that has dominated the conv= ersation, but in this iteration it's going to be broken away from import= to some degree in this iteration.

Today we= 'll look just at composer.

Now import needs= to load packages in a manner that allows different versions to be run c= oncurrently. A PHP application such as Wordpress should be distributable= without needing to use the command line. That is, if WordPress leverage= s this in any way, they don't have to give up their famous 10 minute qui= ck install.

Some terms here to keep myself = from getting lost (let alone anyone trying to read this).
=
* APPLICATION - This is the overall application - WordPre= ss, Drupal, BobbysFirstFramework, etc. - that is doing the import. = This code is on the root scope.
* ROOT SCOPE - This is whe= re the global variables and the namespaces as we know them exist.  = Contrast this with
* PACKAGE SCOPE - Each package brought = in with import gets its own package scope. This is a distinct behavior f= rom Include/Require. I think each package scope will need to be on its o= wn request thread, but this is an implementation detail I can't speak to= with any authority. The goal is whatever happens in a package stays in = the package.  If two different packages want to define /foo(), they= can.

When a package is imported the parser= will look for a `.php.mod` file at the root of the package. Among other= things, this file details what type of package it is and where to mount= it by default in the namespace of the ROOT SCOPE. So, 

GIVEN a package with this = .php.mod file

<= div>package MyModu= le

WHEN I issue t= his import in an application

import "MyModule";
<= div>
THEN I should be able to access a method in tha= t module with this code

\MyModule\foo();
=

Aliasing is an option - `import "MyModule= " as BeerModule` will make the methods accessible in the root with \Beer= Module\foo();

Just to ch= allenge you a bit here. The language already has `use` for this. Does `u= se` stay or is it replaced with `import` and why not just change the mea= ning of `use` in a package context?


= Unlike require/include import is sensitive to the namespace it is in for= mounting.  So

names= pace Trees;
import "MyModule= ";

MyModule\foo(); // works
\Trees\MyModule\foo(); // needed from another namespace.

That said, with aliasing an ab= solute namespace for the module can be assigned.

namespace Trees;
<= blockquote style=3D"margin-top:0px;margin-right:0px;margin-bottom:0px;ma= rgin-left:40px;border-top-width:initial;border-right-width:initial;borde= r-bottom-width:initial;border-left-width:initial;border-top-style:none;b= order-right-style:none;border-bottom-style:none;border-left-style:none;b= order-top-color:initial;border-right-color:initial;border-bottom-color:i= nitial;border-left-color:initial;border-image-source:initial;border-imag= e-slice:initial;border-image-width:initial;border-image-outset:initial;b= order-image-repeat:initial;padding-top:0px;padding-right:0px;padding-bot= tom:0px;padding-left:0px;">
import "MyModule" as \MyModule;

MyModule\foo(); // works if my understanding of existin= g namespace resolution rules is correct.
\MyModule\foo(); // also wor= ks.

Now, with that in= place, let's crack a tougher nut - handling a composer package. By defa= ult composer is designed to set up an autoloader, then resolve symbol re= ferences as they come up. This works until you have two packages that wa= nt the same symbol reference - which will most frequently occur with inc= ompatible versions of the same package.  So our puzzle here is how = to allow composer to do its thing without rewriting it.  We'll deal= with admittedly the hardest case first - importing a package whose main= tainers have taken no action to make it compatible with this new system.=
 
import "composer://packagist.org/packages/twig/twig#v3.10.3" as TemplateEngine

The reason for that alias and n= ot "Twig" is because the mounting point comes before the internal namesp= ace of the file. This is unavoidable with this scheme

=
The URL there is "loader://package_url". PHP by default will = know what the composer loader is. It will look to see if the user has gl= obally installed composer already and use that, otherwise it will locall= y install composer for the project, initialize it, download the package = and have composer resolve the package ending in setting up an autoloader= that is only invoked within that package.
<= div>
Adding this after I just wrote the below book. It sta= rted out simple enough and it drives me crazy when people crash my propo= sal with a counter-proposal. So, by all means, take it with a grain of s= alt ... I kinda went overboard thinking about it.

I think composer and friends are a moot point. If we go a bespoke way= for everything, we end up with a mess. What about creating "hooks" that= things like composer can "register" an installer at? For example, we co= uld define a "WELL_KNOWN/installers/composer/hooks.json" (I'm gonna stea= l a bunch of ideas from kubernetes from here on), where WELL_KNOWN is so= me engine-specific directory (like where the php.ini file is). Basically= , any installer can register an installer by creating a directory in WEL= L_KNOWN/installers of which an installer might look like the following f= or composer:

WELL_KNOWN/
&nbs= p; installers/
    composer/
 = ;     hooks.json
      composer.p= har
      cache

and hooks.json could have a schema of something like:
{
  "name": "composer",
&n= bsp; "version": "4.5",
  "executable": "./composer.ph= ar",
  "scheme": "composer",
  "co= mmand": "install-package-module"
}

Then the engine can scan these directories and read each hooks.jso= n. Then when it gets to your example above, it sees the scheme "composer= " in the URL, looks for an installer with that name, and calls the execu= table with some arguments (the command, the "URL" aka the package, curre= nt directory, etc). So, it might call composer with something like:
<= /div>

/WELL_KNOWN/installers/composer/composer.phar i= nstall-package-module packagist.org/packages/twig/twig#v3.10.3 /app/pu= blic

The command is expected to dump the fi= le to stdout -- which PHP then pipes to wherever it is supposed to go (a= s you mention below).

At this point, the us= er may not know a single thing about composer or how it works, or anythi= ng, really. As far as they are concerned, they said they wanted an impor= t and they got one. However, there still exists a bunch of composer-spec= ific scripts, etc. In this case, we make new WELL_KNOWN types. For examp= le, we can have "script-runners" that can be registered so you can just = do "php run composer test" and it will run `composer test` for you. If y= ou want something shorter, you can just add `alias composer=3D"php run c= omposer"` to your shell and bob's your uncle. 

This is basically how kubernetes handles networking, disks, etc= . so that the entire thing is completely swappable and extendable. So, w= e know the idea is sound and "just works," we just need to customize it = for php.

=E2=80=94 Rob<= br>
--28ebcaa9f4b347508c0be00c7b6cbc26--