Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:124312
Feedback-ID: i8414410d:Fastmail
User-Agent: Cyrus-JMAP/3.11.0-alpha0-568-g843fbadbe-fm-20240701.003-g843fbadb
Precedence: bulk
MIME-Version: 1.0
Message-ID: <7b40e925-d642-4cf4-83f8-f903a9964362@app.fastmail.com>
In-Reply-To: <D412D0D3-AD94-4B06-A302-AC7215271367@newclarity.net>
References: <09559430-4477-4516-8D78-6F4071E1AA6C@newclarity.net>
 <0182F3D6-F464-477F-9029-A2D0A8B50C71@koalephant.com>
 <GV1PR01MB10526D0B220ED0AC9D1C17473C2DD2@GV1PR01MB10526.eurprd01.prod.exchangelabs.com>
 <1AFD7AAE-8BEA-460D-88A8-15BB3D30A775@koalephant.com>
 <CAJp_myXHSHD6+76gyOs=d+cfVgwCmpuhKMtYcVBM6b-i3Bf0WQ@mail.gmail.com>
 <CAEUnE0dObGkb+jUZrVhNRg0JHaM5Kt9d93k3p_0D=jX7kxmYAg@mail.gmail.com>
 <e42cd46b-d98b-4567-a854-03ca54d4dbcf@app.fastmail.com>
 <CAEUnE0cQA-QrmDmAjpBm=sXkE58Y4ODbNwPYY=QBJV46Q8nR9g@mail.gmail.com>
 <CAEUnE0eEjsBXmz8AEVG1SKE=i08-9dit1ze1K0mWrdyQt4okqQ@mail.gmail.com>
 <7B633CC7-C768-4852-A4D0-B252A04F7DE1@newclarity.net>
 <CAEUnE0fFfggyvtNGMytWATG7xUrwfjtXp5F0Sk9PvXo-1Rf+_g@mail.gmail.com>
 <0E11F373-99DC-496E-9BBC-2F8688B9F66A@newclarity.net>
 <CAEUnE0fMGzYimJr71VSFr44FQPBzszfd7FbScfwYx794GZMsrQ@mail.gmail.com>
 <4F720A7A-B7DD-4B31-B0C9-6907419B53A5@newclarity.net>
 <CAEUnE0cf2ekFTsQphH8GjzZ4Yds2DXwxS_DT37UZ4reMHg=D+Q@mail.gmail.com>
 <D412D0D3-AD94-4B06-A302-AC7215271367@newclarity.net>
Date: Tue, 09 Jul 2024 12:15:41 -0500
To: "php internals" <internals@lists.php.net>
Subject: Re: [PHP-DEV] [PHP-Dev] Versioned Packagers (Iteration IV)
Content-Type: text/plain;charset=utf-8
Content-Transfer-Encoding: quoted-printable
From: larry@garfieldtech.com ("Larry Garfield")

On Sat, Jul 6, 2024, at 1:12 AM, Mike Schinkel wrote:

> <epiphany>
>
> Reading this however caused me to ponder things certain people has sai=
d=20
> recently =E2=80=94 and many people have said for years on this list =E2=
=80=94 and I=20
> think I am recognizing something that I have always known but never pu=
t=20
> the pieces together before.
>
> Many (most?) people on PHP Internals view WordPress coding standards a=
s=20
> bad and some even view addressing WordPress developers needs as bad fo=
r=20
> PHP. And in general I concur that those people are reasonably justifie=
d=20
> in their belief WordPress' coding standards are not the standards that=20
> PHP developer who want to do professional level software engineering=20
> should aspire.=20
>
> And since many (most?)* *PHP Internals members generally do not=20
> experience the issues that WordPress developers have they do not=20
> recognize that they are issues; IOW, *"out of sight, out of mind." *
>
> I also think some list members tend to dismiss WordPress developers=20
> pains as unimportant and/or think that addressing those pains have wil=
l=20
> harm* *PHP.=20
>
> (BTW, I recently had a dialog off-list with someone who wrote in an=20
> email that *"Wordpress is an exception, but nobody these days treats=20
> WordPress as a valid example to do anything. It is an ancient piece of=20
> legacy code that has no bearing on modern situation and it's their=20
> problem to deal with." *So I am not just erecting a straw man here.)
>
> But I think what most may not consciously recognize is that* WordPress=20
> is a different type of web app* than an app build using Symfony or=20
> Laravel and deployed by its developers, or by some other professional=20
> developer.=20
>
> WordPress differs from the apps many *(most?)* developers on PHP=20
> Internals work with in the following way:
>
> WordPress =3D *User-managed app*
> Most =3D *Developer-managed apps*
>
> In a* Developer-Managed app* developers choose which 3rd party=20
> functionality will be incorporated into their sites whereas with a=20
> *User-managed app* users choose which 3rd party functionality will be=20
> incorporated into their site. And that is the KEY difference.=20
>
> So I am wondering if we can get people on this PHP Internals list who=20
> dismiss the needs of WordPress developer BECAUSE it is WordPress to=20
> recognize that User-Managed apps ARE a class of PHP applications have=20
> needs that **deserve** to be addressed? =20
> *
> *
> Two (2)* unmet needs of User-Managed apps *that *"standard" *PHP=20
> currently does not address come to mind:
>
> User-managed apps needs to be able to handle both:
> *
> *
> 1. *User-added add-ons* *("plugins" in WordPress, "modules" in Drupal)=20
> *that have conflicting dependencies, and
>
> 2. *Add-on directory structures *that do not follow a PSR-4 directory =
hierarchy.
>
> As for #2, even if those apps could rearchitect their existing=20
> directory structure they cannot realistically be expected to do with=20
> because of the huge BC issues their users would experience.=20
>
> And newly created User-managed apps may still find that a PSR-4=20
> directory structure is not in the best interest of their project or=20
> their users. To elaborate, PSR-4 generally assumes that ALL code goes=20
> into ONE hierarchy and that any and all code that will be autoload get=
s=20
> placed in that hierarchy.
>
> But with add-ons it makes a lot more sense to have the entire add-on=20
> contained in its own add-on directory. This is exactly where PSR-4=20
> breaks down with respect to User-managed apps.
>
> Sure, you can have multiple PSR-4 autoloader root directories, but tha=
t=20
> does not scale well to websites with a large number of add-ons as many=20
> WordPress sites I worked on used. Some had over 100 plugins.  With a=20
> hierarchy of autoloader maps that Michael Morris is proposing WordPres=
s=20
> could collect up all the maps and create one map every time a plugin i=
s=20
> added, updated or deleted.
>
> </epiphany>

I am going to jump in here on this point specifically, because it seems =
to be a mix of genuinely insightful observation (though not unique) and =
uninformed FUD.

Some context: I haven't seriously used Wordpress in, ever.  However, I w=
as a Drupal lead developer for many years, and wrote, among other things=
, Drupal's DBAL, Drupal's first autoloader, Drupal's PSR-3 implementatio=
n, was involved in Drupal's file organization guidelines for Drupal 8+ (=
when Drupal adopted a PSR-0/4 autoloader), and led the Drupal 8 "Moderni=
ze all the things" effort.  So I do have some non-trivial experience in =
this area.

First, you're correct that there is an architectural difference between =
"projects that assume the owner has CLI access" and those that do not.  =
You are also correct that most of the Internals crowd comes from the for=
mer. =20

However, I don't think it's fair to say that's why Internals folks "dism=
iss" Wordpress generally.  We dismiss Wordpress generally because

1. WP actively harms the PHP community by encouraging the use of ancient=
 PHP versions with known security issues.
2. WP's code base actively avoids using what have been considered known =
best-practices (in either type of application) for 15 years.
3. WP's core team actively avoids being involved in Internals to collabo=
rate on how to make the language better for them.  In fact, they've made=
 it very clear that PHP is a legacy implementation detail and Node/clien=
t-side JS is where their focus is.  The only WP-affiliated person I can =
even think of that has been a semi-regular Internals contributor is Juli=
ette (whose participation I very much welcome).

That said, it was recently pointed out to me that Automattic is the top =
contributor to the PHP Foundation (https://opencollective.com/phpfoundat=
ion), which is very much appreciated and nothing to sneeze at.

And yes, I fully agree that any module/package/thing needs to take into =
account the needs of both types of projects.  As Rowan has repeated, tha=
t means keeping the impact of any changes minimal, so that the Composer =
ecosystem and WP ecosystem can TYPO3 ecosystem can build their own tooli=
ng on top of it.

You'll note I did not list Drupal there.  That's because modern Drupal i=
s composer-based, and has been for many years.  I was the one that pushe=
d hard for adopting Composer, its autoloader, and PSR-4 for Drupal 8 in =
the first place.  While much of the transition happened after I left the=
 project, the groundwork is over a decade old.  Composer is the preferre=
d way to use Drupal, and to install Drupal modules.

So the line is not as hard between those two models as you might think.

> PSR-4 generally assumes that ALL code goes=20
> into ONE hierarchy and that any and all code that will be autoload get=
s=20
> placed in that hierarchy.

This is flatly untrue, and belies a considerable ignorance about how PSR=
-4 and Composer work.

PHP supports multiple autoloader callbacks, and has for over 15 years.  =
You absolutely can register multiple if you'd like, using whatever logic=
 you like. PHP will call each one in turn until the class is loaded.

All PSR-4 does is specify a directory structure that makes a common auto=
loader stupidly simple to write.  It's just a few lines long.  But you c=
an already do any logic you like for an autoloader.  PHP doesn't care.

However, nothing precludes you from registering multiple autoloaders, al=
l using PSR-4, all using a different path root.  That has been trivially=
 simple to do since 2009.  (OK, it was PSR-0 at the time, but the implic=
ations here are the same.)  So your statement above about "all code goes=
 into one hierarchy" is simply flat out false.

Of course, as you note, registering lots of separate autoloaders has a p=
erformance impact.  That is true.  Which is why I don't think anyone act=
ually does that.

Composer, for instance, registers a single autoloader only.  That autolo=
ader internally tracks many dozens of PSR-4 roots (one for each package,=
 sometimes two per package), as well as files that will get force-loaded=
 when the autoloader is registered, plus generated classmaps.

Using class maps, you can put a hundred classes in one file and composer=
 can handle that *today*.  That has always been possible.  That no one d=
oes so is a sign that there's little reason to do so in most cases.

In fact, if you use an optimized/dumped autoloader, then Composer simply=
 builds an internal giant lookup table of what class maps to what file. =
 PSR-4 is then *completely irrelevant* at runtime.  It's already one gia=
nt O(1) lookup map.  That can be done *today*.  That *is* done today.

But what about systems like Drupal, that don't put code in `/vendor/`?  =
Drupal ties directly into Composer via its API,and has done so for a dec=
ade.  Drupal, a "user-managed application" as you describe it, has Compo=
ser baked in at a core level.

It looks like the integration has evolved considerably since I was last =
involved, but have a look at:

https://git.drupalcode.org/project/drupal/-/tree/11.x/composer

As of when I last looked at it (around 2016 or so), Drupal registers its=
 module code roots with Composer directly, and then Composer takes over =
from there and integrates Drupal's code into its own indexes.  There is =
still only one single autoloader registered with PHP.  This is entirely =
fine.

I would encourage you to do your research before speaking pseudo-authori=
tatively on this topic, as you clearly are mis-stating both the problem =
and the tools involved today.

What Drupal does not do is address the "different dependency version" qu=
estion.  And neither does Wordpress.  Or TYPO3.  Or any other project.  =
Because that's a core PHP limitation. =20

In Python, every module is really just an object with a big dictionary o=
f the functions/classes it has.  When you "import" a symbol from another=
 module, the engine is doing little more than `$this['foo'] &=3D $that['=
foo']`.  That's a core part of how the language works.  (I'm not sure of=
 Javascript's details, but I suspect from using it that it's similar.)

PHP works very very differently.  PHP has a single global list of symbol=
s.  (Well, two, for classes and functions.)  Namespaces are just syntax =
sugar over very-long-names, nothing more.  There is no "local symbol tab=
le," so having different local symbol tables point to different code blo=
cks using the same name is not even conceivable.

If you want to change that, and give PHP multiple local symbol tables, t=
hen autoloading... is utterly irrelevant.  The question there is "how ca=
n we introduce local symbol tables in the engine without requiring 10 mi=
llion developers to rewrite the file header of 1 billion PHP files acros=
s the world?"  Honestly, I'm not convinced its even possible. Someone wi=
th more engine knowledge than I could be able to find away, maybe, but I=
 am skeptical.  If it's even possible, I suspect it would be an absurdly=
 large amount of work and necessarily include many hard BC breaks.

If you'd like to prove me wrong, go for it.  But that's the problem to a=
ddress.  Debating file paths is about four steps down the line before it=
's even relevant.  And even then... if you can't make a PSR-4-organized =
package (of which there are several hundred thousand) slot into that new=
 model comfortably with zero effort on the part of the package author, i=
t's doomed.

So please, spare us the ill-informed descriptions of how you think autol=
oaders work, when you have demonstrated you do not know how they work. =20

Spare us the litany of complaints about PSR-4 when you have demonstrated=
 you don't know what PSR-4 says. =20

Spare us the gnashing of teeth about how hard it is to use a Wordpress p=
lugin that hasn't been updated in 10 years with a modern plugin because =
the former is still using a 10 year old abandoned version of some librar=
y, when that's not PHP's problem, that's a Wordpress maintenance problem.

If you want to move this effort forward, here's your todo list:

1. Do some research in the engine to determine if local symbol tables ar=
e even possible without rewriting the engine.
2. Work through the highly complex logic of handling three layer overlap=
ping transitive dependencies in a diamond pattern with conflicting versi=
on requirements.
3. Investigate the performance impact of maintaining multiple versions o=
f the same code in memory at once, when the order they get loaded will v=
ary by request.
4. Think through how you'd support *both* composer-based and "user manag=
ed" applications with such a model, especially projects that are already=
 architecturally a decade out of date (like Wordpress).

When you have a proven that it's even possible to have multiple local sy=
mbol tables, we can talk.  Until then, please spare us.

--Larry Garfield