Hi there!
Within the PHP Foundation, we have been talking for a while as what to
do with PECL, and its website.
The code is old, and hard to maintain. And the database is full of
mojibake. It is also an outdated method of installing things, especially
because userland code is so much easier to handle through Composer.
Through the Sovereign Tech Fund (https://www.sovereigntechfund.de/) the
Foundation has acquired some funding to improve this situation.
Hence, to start of replacing PECL with something applicable for this
decade, we started working on a requirements document:
https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo
In this first stage, we would like to invite you in commenting on the
document (either inline, or here).
Please keep in mind that this is a requirements document, and should not
contain either design or implementation details. Once we have thought
about these details as well, this will be turned into an RFC.
cheers,
Derick
--
https://derickrethans.nl | https://xdebug.org | https://dram.io
Author of Xdebug. Like it? Consider supporting me: https://xdebug.org/support
Host of PHP Internals News: https://phpinternals.news
mastodon: @derickr@phpc.social @xdebug@phpc.social
twitter: @derickr and @xdebug
Hi Derick,
Hi there!
Within the PHP Foundation, we have been talking for a while as what to
do with PECL, and its website.The code is old, and hard to maintain. And the database is full of
mojibake. It is also an outdated method of installing things, especially
because userland code is so much easier to handle through Composer.Through the Sovereign Tech Fund (https://www.sovereigntechfund.de/) the
Foundation has acquired some funding to improve this situation.Hence, to start of replacing PECL with something applicable for this
decade, we started working on a requirements document:https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo
In this first stage, we would like to invite you in commenting on the
document (either inline, or here).Please keep in mind that this is a requirements document, and should not
contain either design or implementation details. Once we have thought
about these details as well, this will be turned into an RFC.
You may mention https://github.com/FriendsOfPHP/pickle
which supports pecl.php.net pear format, composer like support, and
conversion to composer format.
I did a website as well as a test bed and we had a long with the composer
team what would be the best way to integrate it. That's something that
became possible with composer 2.x, which now widely used. Similarly we
discussed the idea to create a packagist dedicated to extensions.
That would be, imo, the best option to date.
The current code may need some updates for php8 but nothing difficult or
requiring big changes.
best,
Pierre
You may mention https://github.com/FriendsOfPHP/pickle
which supports pecl.php.net pear format, composer like support, and
conversion to composer format.I did a website as well as a test bed and we had a long with the composer
team what would be the best way to integrate it. That's something that
became possible with composer 2.x, which now widely used. Similarly we
discussed the idea to create a packagist dedicated to extensions.That would be, imo, the best option to date.
The current code may need some updates for php8 but nothing difficult or
requiring big changes.
there was also a draft rfc while we were thinking to propose to bundle it.
But we then decided to follow the composer's packagist philosophy instead.
You may mention https://github.com/FriendsOfPHP/pickle
which supports pecl.php.net pear format, composer like support, and
conversion to composer format.I did a website as well as a test bed and we had a long with the composer
team what would be the best way to integrate it. That's something that
became possible with composer 2.x, which now widely used. Similarly we
discussed the idea to create a packagist dedicated to extensions.That would be, imo, the best option to date.
The current code may need some updates for php8 but nothing difficult or
requiring big changes.there was also a draft rfc while we were thinking to propose to bundle it.
But we then decided to follow the composer's packagist philosophy instead.
Hi Derick,
Thanks for creating the google doc for us, it's really easy to contribute
now, and I think I'll use this same approach for php.net/docs modernization
approach.
I came here to write about Pierre's Pickle project, but he replied whilst I
was mid-writing this email :-)
I'll add comments onto the Google Doc.
Many thanks,
Paul
You may mention https://github.com/FriendsOfPHP/pickle
which supports pecl.php.net pear format, composer like support, and
conversion to composer format.I did a website as well as a test bed and we had a long with the
composer
team what would be the best way to integrate it. That's something that
became possible with composer 2.x, which now widely used. Similarly we
discussed the idea to create a packagist dedicated to extensions.That would be, imo, the best option to date.
The current code may need some updates for php8 but nothing difficult or
requiring big changes.there was also a draft rfc while we were thinking to propose to bundle it.
But we then decided to follow the composer's packagist philosophy instead.Hi Derick,
Thanks for creating the google doc for us, it's really easy to contribute
now, and I think I'll use this same approach for php.net/docs
modernization approach.I came here to write about Pierre's Pickle project, but he replied whilst
I was mid-writing this email :-)I'll add comments onto the Google Doc.
Many thanks,
Paul
Hi Derick,
I've added my comments, for now, in the first part of the document.
As for the latter points in the document, such as json structures and so
on, I think it might make sense to complete the Pickle evaluation, as a
first phase thing, because the way Pickle is designed may influence how
packages are structured.
I'm very enthusiastic to hear how this topic evolves :-)
Have a great day <3
Many thanks,
Paul
The code is old, and hard to maintain. And the database is full of
mojibake. It is also an outdated method of installing things, especially
because userland code is so much easier to handle through Composer.
Thanks for working on this.
I would say that one of the reasons userland code is easier is because
composer packages are local and unprivileged, whereas extensions are
global and are typically require root to install. That problem is not
going to go away. For community-supplied packages we're mostly using
Ondrej Sury's Debian packages. For our own packages (LuaSandbox,
Excimer) we're building debs from git tags without reference to PECL.
So PECL for us is part of the release procedure just as a courtesy to
the downstream community.
In terms of security, there should be no way to get root by uploading
a malicious PECL package. With debs, I'm not sure if that separation
is rock-solid, but at least it exists aspirationally. During build, we
run code from the package as non-root. During installation, as root we
install generated files into locations which hopefully won't be
executed as root. To achieve this separation with PECL, there would
need to be some sort of system-aware privilege separation inside the
PECL installer. At least it should fork and drop root privileges.
Signed tags only help if you trust the package developer. I don't want
to have to trust anyone.
The PHP version dependency is a tricky part of the spec. When we make
a release, we have no way to know how forwards-compatible it is. At
some point, a PHP core change will break the extension. The proposed
structure suggests that we would have to release a new version of the
extension just to say that it's broken. Then later when we make a fix,
we would release again with that information.
My motivation to maintain a historical list of supported version
ranges is very limited. I just want to declare, for the code I'm
actually releasing, the earliest supported PHP version and maybe the
latest confirmed PHP version.
Have you considered keeping the support matrix in the registry
database, instead of in pecl.json? Then it can be updated with new
build/test information after release.
-- Tim Starling
Hello Tim,
Have you considered keeping the support matrix in the registry
database, instead of in pecl.json? Then it can be updated with new
build/test information after release.
We do it for windows, used by pickle, you can see one part here:
https://windows.php.net/downloads/php-sdk/deps/dllmapping.json
The other parts being part of each extension.
It could be achieved similarly for linux package managers, maybe
maintained by them directly.
A few years back, I made some tests on windows and linux using
https://conan.io/ and https://vcpkg.io/, they both work on most
platforms PHP supports and also have cmake integration (php build
using cmake has been worked done, very nicely :).
Having maintained these parts for many years, dealt with 3rd parties
(developers of librairies, distro packages maintainers, etc) as well
as dealt with many end users (php developers), I must say I would
absolutely not start to define our own things but rely as much as
possible on the well working existing solutions.
On the same topic, npecl's requirements (why not a RFC/proposal btw?)
are kind of out of sync with what I have seen and still see. F.e.,
doing it outside the composer ecosystem will most likely create a
niche and not a wide adoption.There have been a lot of efforts put
into similar topics since 5.3 and a lot of learning, it may be a good
thing to learn from them. There are also tools now available to give
developers out of box tools to define whatever environments they may
need and switch between them within a click or command line (Laravel
community created a few good ones, mac for the biggest one, and other
platforms), having them rely on php's platform for the core and non
core extensions (including their respective deps) will be amazingly
helpful.
Cheers,
Pierre
@pierrejoye | http://www.libgd.org
The code is old, and hard to maintain. And the database is full of
mojibake. It is also an outdated method of installing things,
especially because userland code is so much easier to handle through
Composer.Thanks for working on this.
I would say that one of the reasons userland code is easier is because
composer packages are local and unprivileged, whereas extensions are
global and are typically require root to install. That problem is not
going to go away. For community-supplied packages we're mostly using
Ondrej Sury's Debian packages. For our own packages (LuaSandbox,
Excimer) we're building debs from git tags without reference to PECL.
So PECL for us is part of the release procedure just as a courtesy to
the downstream community.
From what I can see, once you drop a "new-pecl.json" file (or whatever
we'll end up calling it) into the repo at
https://github.com/wikimedia/mediawiki-php-luasandbox it should
automatically become compatible with the new PECL. You shouldn't have to
deal with making actual packages and uploading them to the PECL website.
I'm going to hope that that is an improvement for you?
In terms of security, there should be no way to get root by uploading
a malicious PECL package. With debs, I'm not sure if that separation
is rock-solid, but at least it exists aspirationally. During build, we
run code from the package as non-root. During installation, as root we
install generated files into locations which hopefully won't be
executed as root. To achieve this separation with PECL, there would
need to be some sort of system-aware privilege separation inside the
PECL installer. At least it should fork and drop root privileges.
Yes, that is a good recommendation, and something that should definitely
go into the design document.
Signed tags only help if you trust the package developer. I don't want
to have to trust anyone.
That's true. I think signatures make more sense when a package is
authored by a known organisation, such as wikimedia or MongoDB. Where
the signatures therefore include these organisation's domain names.
The PHP version dependency is a tricky part of the spec. When we make
a release, we have no way to know how forwards-compatible it is. At
some point, a PHP core change will break the extension. The proposed
structure suggests that we would have to release a new version of the
extension just to say that it's broken. Then later when we make a fix,
we would release again with that information.
I don't think you'll have to make a new release, as I was hoping to
access the meta data file in the main/master branch.
I solve this for Xdebug by always setting a hard upper limit. Better
safe than sorry.
My motivation to maintain a historical list of supported version
ranges is very limited. I just want to declare, for the code I'm
actually releasing, the earliest supported PHP version and maybe the
latest confirmed PHP version.Have you considered keeping the support matrix in the registry
database, instead of in pecl.json? Then it can be updated with new
build/test information after release.
Yeah, I understand. And I feel the same. But if there is no registry (a
goal), then there is no way to find out which of your extension version
support a specific PHP version, without checking every tag in your
repository. That's not going to be fast.
cheers,
Derick
Yeah, I understand. And I feel the same. But if there is no registry (a
goal), then there is no way to find out which of your extension version
support a specific PHP version, without checking every tag in your
repository. That's not going to be fast.cheers,
Derick
See, I think I disagree on this point. Having a registry is table-stakes for package managers these days. Composer, npm/yarn, whatever Python does this week, Cargo, Bundle, Maven/Gradle... a packager isn't complete without a registry. The one exception is Go, and its approach is... just a mess, frankly.
Whether PECLtng piggybacks on Packagist, self-hosts using something like Satis, or does a home-grown thing, I don't think a neo-PECL without a registry is a viable and sustainable way forward. That needs to be in the initial design.
--Larry Garfield
On Mon, Dec 11, 2023, 11:21 PM Larry Garfield larry@garfieldtech.com
wrote:
Yeah, I understand. And I feel the same. But if there is no registry (a
goal), then there is no way to find out which of your extension version
support a specific PHP version, without checking every tag in your
repository. That's not going to be fast.cheers,
DerickSee, I think I disagree on this point. Having a registry is table-stakes
for package managers these days. Composer, npm/yarn, whatever Python does
this week, Cargo, Bundle, Maven/Gradle... a packager isn't complete without
a registry. The one exception is Go, and its approach is... just a mess,
frankly.Whether PECLtng piggybacks on Packagist, self-hosts using something like
Satis, or does a home-grown thing, I don't think a neo-PECL without a
registry is a viable and sustainable way forward. That needs to be in the
initial design.
hi Larry,
you are fully right here.
On top of the already mentioned points, it brings visibility to extension
packages, which is a key factor to adoption at large.
It also helps for an automatic builds.
A service is also a must for automatic releases/triggers builds. You can
see this service with a github tag hook on pickleweb.
It is also quite a good thing to automate dependencies management. It could
be relatively straightforward to update them per distros, or windows (see
my earlier post, already managed using windows. php.net data).
Also without the services (implemented but no public packagist for
extensions yet), I think pickle already does 95% of it (including local
windows builds). Adding signature is relatively simple.
Important note, during pickle development, the number 1 request was to be
fully integrated with composer. it was not possible until composer v2,
plugin and custom tags. Given that composer is the only used package
manager , it should be the way to go. To create something from scratch
without taking these points into accounts can only lead to a sub optimal
solution.
There are also development environments tools out there which could provide
relatively easily per project extensions management (either cli or pools).
PHP is not my primary platform anymore, business wise, however I am happy
to help if desired. I can only, with a diplomatic emphasis, suggest to the
author to understand where the php ecosystem is (or was 3 years or more
ago).
best,
Pierre
Le 05/12/2023 à 14:45, Derick Rethans a écrit :
https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo
In this first stage, we would like to invite you in commenting on the
document (either inline, or here).
As inline requires a Google account...
priority (optional): The installation "priority".
1/ I don't like this name which is ambiguous
higher value seems to give higher priority
which lower are loaded first
This prefix, used in various linux distro,
is really a "load order", needed when using --enable-rtld-now
So I proprose "load order" instead
2/ values
Default value can be 40 or max(requirement) + 10
In Fedora where we use this for years we have
(IIRC debian have different practice)
0-19 for zend_extension
10 for opcache
15 for xdebug
20-39 for php-src extension
20 for most
30 if requirement (ex: pdo drivers)
40-99 for other (pecl...)
40 for igbinary, msgpack
50 for redis which use igbinary, msgpack
Remi
Le 05/12/2023 à 14:45, Derick Rethans a écrit :
https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo
In this first stage, we would like to invite you in commenting on the
document (either inline, or here).As inline requires a Google account...
priority (optional): The installation "priority".
1/ I don't like this name which is ambiguous
higher value seems to give higher priority
which lower are loaded firstThis prefix, used in various linux distro,
is really a "load order", needed when using --enable-rtld-nowSo I proprose "load order" instead
We can quibble over the name :-)
I picked something short. Debian calls it
'priority' too in their ini files. For example:
; configuration for php common module
; priority=20
extension=calendar.so
2/ values
Default value can be 40 or max(requirement) + 10
In Fedora where we use this for years we have
(IIRC debian have different practice)0-19 for zend_extension
10 for opcache
15 for xdebug
That's not really a good default. At least Xdebug should be
near the end of the loading order.
20-39 for php-src extension
20 for most
30 if requirement (ex: pdo drivers)
40-99 for other (pecl...)
40 for igbinary, msgpack
50 for redis which use igbinary, msgpack
I am happy to pick a different default. I had picked what Debian seemed
to have (20).
I don't think it matters much though, as distribitions can decide to
change them in their binary packages regardless.
cheers,
Derick
Le 05/12/2023 à 14:45, Derick Rethans a écrit :
https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo
In this first stage, we would like to invite you in commenting on the
document (either inline, or here).As inline requires a Google account...
priority (optional): The installation "priority".
1/ I don't like this name which is ambiguous
higher value seems to give higher priority
which lower are loaded firstThis prefix, used in various linux distro,
is really a "load order", needed when using --enable-rtld-nowSo I proprose "load order" instead
We can quibble over the name :-)
I picked something short. Debian calls it
'priority' too in their ini files. For example:; configuration for php common module ; priority=20 extension=calendar.so
2/ values
Default value can be 40 or max(requirement) + 10
In Fedora where we use this for years we have
(IIRC debian have different practice)0-19 for zend_extension
10 for opcache
15 for xdebugThat's not really a good default. At least Xdebug should be
near the end of the loading order.20-39 for php-src extension
20 for most
30 if requirement (ex: pdo drivers)
40-99 for other (pecl...)
40 for igbinary, msgpack
50 for redis which use igbinary, msgpackI am happy to pick a different default. I had picked what Debian seemed
to have (20).I don't think it matters much though, as distribitions can decide to
change them in their binary packages regardless.cheers,
Derick--
To unsubscribe, visit: https://www.php.net/unsub.php
Hello,
I don't know if you've seen this prior art:
https://github.com/mlocati/docker-php-extension-installer
It's extremely convenient for Docker images, and usually has the
ability to apply appropriate patches (such as the memcached + PHP
8.1-ish issue a couple of years ago) to get things to install. It
would be amazing if the new pecl could be as easy to use as this
wrapper/tool.
Robert Landers
Software Engineer
Utrecht NL