Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:124295
Precedence: bulk
MIME-Version: 1.0
References: <09559430-4477-4516-8D78-6F4071E1AA6C@newclarity.net>
 <0182F3D6-F464-477F-9029-A2D0A8B50C71@koalephant.com> <GV1PR01MB10526D0B220ED0AC9D1C17473C2DD2@GV1PR01MB10526.eurprd01.prod.exchangelabs.com>
 <1AFD7AAE-8BEA-460D-88A8-15BB3D30A775@koalephant.com> <CAJp_myXHSHD6+76gyOs=d+cfVgwCmpuhKMtYcVBM6b-i3Bf0WQ@mail.gmail.com>
 <CAEUnE0dObGkb+jUZrVhNRg0JHaM5Kt9d93k3p_0D=jX7kxmYAg@mail.gmail.com>
 <e42cd46b-d98b-4567-a854-03ca54d4dbcf@app.fastmail.com> <CAEUnE0cQA-QrmDmAjpBm=sXkE58Y4ODbNwPYY=QBJV46Q8nR9g@mail.gmail.com>
 <CAEUnE0eEjsBXmz8AEVG1SKE=i08-9dit1ze1K0mWrdyQt4okqQ@mail.gmail.com>
 <CAMrTa2EvD3qcTL5SoXwAJ6kT5NTu0QLSQjjJRa0z-esUMVxj6w@mail.gmail.com> <1BE6A849-A2A9-4E17-9C11-5099EF74F5C0@rwec.co.uk>
In-Reply-To: <1BE6A849-A2A9-4E17-9C11-5099EF74F5C0@rwec.co.uk>
Date: Mon, 8 Jul 2024 13:54:26 -0700
Message-ID: <CAMrTa2HsB8KBWzbRxTMFBcMqR4NBCFN8uxBgZYACJWrEtm==gA@mail.gmail.com>
Subject: Re: [PHP-DEV] [PHP-Dev] Versioned Packagers (Iteration IV)
To: "Rowan Tommins [IMSoP]" <imsop.php@rwec.co.uk>
Cc: internals@lists.php.net
Content-Type: multipart/alternative; boundary="0000000000000a7d32061cc29ffe"
From: jordan.ledoux@gmail.com (Jordan LeDoux)

--0000000000000a7d32061cc29ffe
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Jul 8, 2024 at 2:42=E2=80=AFAM Rowan Tommins [IMSoP] <imsop.php@rwe=
c.co.uk>
wrote:

>
>
> On 8 July 2024 04:25:45 CEST, Jordan LeDoux <jordan.ledoux@gmail.com>
> wrote:
> >I think it's strange that this discussion has driven deep down the tange=
nt
> >of versioning...
> [...]
> >Things like separating global scope between importer and importee, manag=
ed
> >visibility of symbols and exports from modules/packages, allowing for
> >separate autoloaders for things which are called or included via an
> import,
> >etc. Those are the things that the language itself can do.
> >
> >All this other stuff feels like a distraction.
>
> I agree. I wrote most of the below a couple of days ago, but I don't thin=
k
> it posted correctly, so apologies if some people see it twice:
>
> Autoloading is just a way to load files later, by the engine telling you
> when a class is first needed. PHP does not, and should not, make any
> assumptions about how files are laid out on disk; an autoloader doesn't
> actually need to load any files at all, and if it does, it uses the same
> include or require statements which have been in PHP for decades.
>
> Likewise, installing packages and defining version schemes is a completel=
y
> separate problem space that can probably be served by a few small tweaks =
to
> Composer once the language provides the underlying functionality.
>
> The core of the problem you seem to want to solve is this: if you have tw=
o
> files foo_1.php and foo_2.php, which both define a class \Acme\Foo, how d=
o
> you load both of them, so that you end up with two differently named
> classes?
>
> In JS, that's easy, because functions and object constructors (and
> "classes") exist as objects you can pass around as variables, they don't
> need to know their own name. In PHP, everything is based on the idea that
> functions and classes are identified by name. You can rewrite the name in
> the class declaration, and in direct references to it, but what about cod=
e
> using ::class, or constructing a name and using "new $name", and so on? H=
ow
> will tools using static analysis or reflection handle the renaming - e.g.
> how does DI autowiring work if names are in some sense dynamic?
>
> You've also got to work out what to do with transitive dependencies - if =
I
> "import 'foo_1.php' as MyFoo", but Foo in turn has "import 'guzzle_2.php'
> as MyGuzzle", what namespace do all Guzzle's classes get rewritten into?
> What about dependencies that are specifically intended to bridge between
> packages, like PSR-7 RequestInterface?
>
> My advice: start with the assumption that something has already installed
> all the files you need into an arbitrary directory structure, and somethi=
ng
> is going to generate a bunch of statements to load them. What happens nex=
t,
> in the language itself, to make them live side by side without breaking? =
If
> we get a solid solution to that (which I'm skeptical of), we can discuss
> how Composer, or the WordPress plugin installer, would generate whatever
> include/import/alias/rewrite statements we end up creating.
>
> Regards,
> --
> Rowan Tommins
> [IMSoP]
> Rowan Tommins
> [IMSoP]
>

I think it could be done somewhat simply (relative to the other things that
have been discussed) if the engine reserved a specific namespace for
imported symbols internally. Something like:

`\__Imported\MyImportStatement`

Where the `\__Imported` namespace is reserved and throws a parser error if
it occurs in code anywhere, and `MyImportStatement` corresponds to an
application importing the code using something like `import MyPackage as
MyImportStatement;`

Then, all symbols which are loaded into the global space as a result of the
import are actually rewritten into the hidden namespace the engine actually
uses under the hood, and any uses from the import statement in the
application code which has the import would reference the symbols in the
prefixed namespace.

This would not be trivial however. The engine code which supports this
would need to keep track of a kind of "context" for each file, based on
what namespace the file was included from. For instance, if an autoload
occurs inside the package that was loaded into `MyImportStatement`, the
engine would need to be aware that the code being executed is defined in
that namespace, REGARDLESS of whether it was a class, function, or
statement, and load ALL symbols that are created as a result into the
rewritten namespace. It would also need to translate in the other direction
for `use` statements inside the package, since it would not know ahead of
time what rewritten namespace it would actually be loaded in.

However, this is the simplest solution I see that doesn't involve writing a
second PHP engine just for this sort of thing.

Jordan

PS: For those unaware, for each "symbol" (something that has a unique
referenceable name in the code, roughly), there is at least one name that
refers to ONLY that thing internally. (I'm fairly certain that there are NO
situations where one name can refer to two things at all, but I am not
enough of an expert in the C code to be completely certain about this, and
it's entirely possible this is in fact a niche common thing that I've never
encountered before). When something is namespaced, the entire namespace in
the engine is prefixed to the "name" of the thing when it is created. So a
function `foo` in the namespace `Bar` has the name "\Bar\foo". Any time you
use it as just "foo", the engine because of context knows to put "Bar" in
front of it before looking up its definition to execute it.

The global symbol space are the items which have nothing prepended, and if
a namespace was inaccessible because the parser errored on `use`,
`namespace`, `new`, and other similar statements that used a part of that
namespace like I outlined, the result would be that for the engine it would
treat all of the code as if it were one application that it knows some
extremely complex namespace replacement rules for (because that's what it
actually would be), but to PHP devs it would act almost like sandboxes
where code from one area cannot access or affect other areas.

This would create some edge cases, like how `global` behaves, or how any of
the superglobal variables could be used, etc. But those are probably easier
to nail down than writing a different engine or running a second process
and setting up some messaging between the two. Though that might be the
more "correct" way to handle something like this. However, I could be wrong
about the difficulty of this, as I've never attempted that kind of change
in the Zend engine before.

--0000000000000a7d32061cc29ffe
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Mon, Jul 8, 2024 at 2:42=E2=80=AFA=
M Rowan Tommins [IMSoP] &lt;<a href=3D"mailto:imsop.php@rwec.co.uk">imsop.p=
hp@rwec.co.uk</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" sty=
le=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddi=
ng-left:1ex"><br>
<br>
On 8 July 2024 04:25:45 CEST, Jordan LeDoux &lt;<a href=3D"mailto:jordan.le=
doux@gmail.com" target=3D"_blank">jordan.ledoux@gmail.com</a>&gt; wrote:<br=
>
&gt;I think it&#39;s strange that this discussion has driven deep down the =
tangent<br>
&gt;of versioning...<br>
[...]<br>
&gt;Things like separating global scope between importer and importee, mana=
ged<br>
&gt;visibility of symbols and exports from modules/packages, allowing for<b=
r>
&gt;separate autoloaders for things which are called or included via an imp=
ort,<br>
&gt;etc. Those are the things that the language itself can do.<br>
&gt;<br>
&gt;All this other stuff feels like a distraction.<br>
<br>
I agree. I wrote most of the below a couple of days ago, but I don&#39;t th=
ink it posted correctly, so apologies if some people see it twice: <br>
<br>
Autoloading is just a way to load files later, by the engine telling you wh=
en a class is first needed. PHP does not, and should not, make any assumpti=
ons about how files are laid out on disk; an autoloader doesn&#39;t actuall=
y need to load any files at all, and if it does, it uses the same include o=
r require statements which have been in PHP for decades.<br>
<br>
Likewise, installing packages and defining version schemes is a completely =
separate problem space that can probably be served by a few small tweaks to=
 Composer once the language provides the underlying functionality.<br>
<br>
The core of the problem you seem to want to solve is this: if you have two =
files foo_1.php and foo_2.php, which both define a class \Acme\Foo, how do =
you load both of them, so that you end up with two differently named classe=
s? <br>
<br>
In JS, that&#39;s easy, because functions and object constructors (and &quo=
t;classes&quot;) exist as objects you can pass around as variables, they do=
n&#39;t need to know their own name. In PHP, everything is based on the ide=
a that functions and classes are identified by name. You can rewrite the na=
me in the class declaration, and in direct references to it, but what about=
 code using ::class, or constructing a name and using &quot;new $name&quot;=
, and so on? How will tools using static analysis or reflection handle the =
renaming - e.g. how does DI autowiring work if names are in some sense dyna=
mic?<br>
<br>
You&#39;ve also got to work out what to do with transitive dependencies - i=
f I &quot;import &#39;foo_1.php&#39; as MyFoo&quot;, but Foo in turn has &q=
uot;import &#39;guzzle_2.php&#39; as MyGuzzle&quot;, what namespace do all =
Guzzle&#39;s classes get rewritten into? What about dependencies that are s=
pecifically intended to bridge between packages, like PSR-7 RequestInterfac=
e?<br>
<br>
My advice: start with the assumption that something has already installed a=
ll the files you need into an arbitrary directory structure, and something =
is going to generate a bunch of statements to load them. What happens next,=
 in the language itself, to make them live side by side without breaking? I=
f we get a solid solution to that (which I&#39;m skeptical of), we can disc=
uss how Composer, or the WordPress plugin installer, would generate whateve=
r include/import/alias/rewrite statements we end up creating.<br>
<br>
Regards,<br>
-- <br>
Rowan Tommins<br>
[IMSoP]<br>
Rowan Tommins<br>
[IMSoP]<br></blockquote><div><br></div><div>I think it could be done somewh=
at simply (relative to the other things that have been discussed) if the en=
gine reserved a specific namespace for imported symbols internally. Somethi=
ng like:<br><br></div><div>`\__Imported\MyImportStatement`</div><div><br></=
div><div>Where the `\__Imported` namespace is reserved and throws a parser =
error if it occurs in code anywhere, and `MyImportStatement` corresponds to=
 an application importing the code using something like `import MyPackage a=
s MyImportStatement;`</div><div><br></div><div>Then, all symbols which are =
loaded into the global space as a result of the import are actually rewritt=
en into the hidden namespace the engine actually uses under the hood, and a=
ny uses from the import statement in the application code which has the imp=
ort would reference the symbols in the prefixed namespace.</div><div><br></=
div><div>This would not be trivial however. The engine code which supports =
this would need to keep track of a kind of &quot;context&quot; for each fil=
e, based on what namespace the file was included from. For instance, if an =
autoload occurs inside the package that was loaded into `MyImportStatement`=
, the engine would need to be aware that the code being executed is defined=
 in that namespace, REGARDLESS of whether it was a class, function, or stat=
ement, and load ALL symbols that are created as a result into the rewritten=
 namespace. It would also need to translate in the other direction for `use=
` statements inside the package, since it would not know ahead of time what=
 rewritten namespace it would actually be loaded in.</div><div><br></div><d=
iv>However, this is the simplest solution I see that doesn&#39;t involve wr=
iting a second PHP engine just for this sort of thing.</div><div><br></div>=
<div>Jordan</div><div><br></div><div>PS: For those unaware, for each &quot;=
symbol&quot; (something that has a unique referenceable name in the code, r=
oughly), there is at least one name that refers to ONLY that thing internal=
ly. (I&#39;m fairly certain that there are NO situations where one name can=
 refer to two things at all, but I am not enough of an expert in the C code=
 to be completely certain about this, and it&#39;s entirely possible this i=
s in fact a niche common thing that I&#39;ve never encountered before). Whe=
n something is namespaced, the entire namespace in the engine is prefixed t=
o the &quot;name&quot; of the thing when it is created. So a function `foo`=
 in the namespace `Bar` has the name &quot;\Bar\foo&quot;. Any time you use=
 it as just &quot;foo&quot;, the engine because of context knows to put &qu=
ot;Bar&quot; in front of it before looking up its definition to execute it.=
</div><div><br></div><div>The global symbol space are the items which have =
nothing prepended, and if a namespace was inaccessible because the parser e=
rrored on `use`, `namespace`, `new`, and other similar statements that used=
 a part of that namespace like I outlined, the result would be that for the=
 engine it would treat all of the code as if it were one application that i=
t knows some extremely complex namespace replacement rules for (because tha=
t&#39;s what it actually would be), but to PHP devs it would act almost lik=
e sandboxes where code from one area cannot access or affect other areas.</=
div><div><br></div><div>This would create some edge cases, like how `global=
` behaves, or how any of the superglobal variables could be used, etc. But =
those are probably easier to nail down than writing a different engine or r=
unning a second process and setting up some messaging between the two. Thou=
gh that might be the more &quot;correct&quot; way to handle something like =
this. However, I could be wrong about the difficulty of this, as I&#39;ve n=
ever attempted that kind of change in the Zend engine before.<br></div></di=
v></div>

--0000000000000a7d32061cc29ffe--