Hi!
We bundle an unmodified libsqlite3 for at least two years. Since then
all updates go into any dev, alpha and beta releases, while security
patches (usually backports from libsqlite3) go into stable branches.
ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has no
specific version requirement; there is a check for sqlite3_open()[3]
which I believe is available since 3.0.0.
So is there any particular reason why we still bundle libsqlite3?
[1]
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/sqlite3/config0.m4#L37-L47
[2] https://sqlite.org/oldnews.html#2007_01_04
[3]
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/pdo_sqlite/config.m4#L55-L58
--
Christoph M. Becker
Hi
Den man. 17. sep. 2018 kl. 15.16 skrev Christoph M. Becker cmbecker69@gmx.de:
Hi!
We bundle an unmodified libsqlite3 for at least two years. Since then
all updates go into any dev, alpha and beta releases, while security
patches (usually backports from libsqlite3) go into stable branches.ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has no
specific version requirement; there is a check for sqlite3_open()[3]
which I believe is available since 3.0.0.So is there any particular reason why we still bundle libsqlite3?
[1]
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/sqlite3/config0.m4#L37-L47
[2] https://sqlite.org/oldnews.html#2007_01_04
[3]
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/pdo_sqlite/config.m4#L55-L58
I don't think there is any paticular reason for bundling it anymore,
at least not from the Windows side of things where we should be easily
able to adapt the build system to link to it instead of building the
bundled version. (Anatol cc'ed)
--
regards,
Kalle Sommer Nielsen
kalle@php.net
So is there any particular reason why we still bundle libsqlite3?
An issue is that users often don't control the version being used and
SQLite adds features to it's SQL dialect from time to time. At the
moment, with the bundled version you can be quite certain that a PHP
version maps to a SQLite version, so when migrating to a different host
you're going to have the same feature set available. (Or is that not
true since Linux distros don't like using bundled libs, but prefer
system installed shared libs so they just upgrade that on security
issues?)
Most other libraries don't impact in regards to features as much as
newer library features would often have to be exported to userspace
explicitly.
johannes
So is there any particular reason why we still bundle libsqlite3?
An issue is that users often don't control the version being used and
SQLite adds features to it's SQL dialect from time to time. At the
moment, with the bundled version you can be quite certain that a PHP
version maps to a SQLite version, so when migrating to a different host
you're going to have the same feature set available. (Or is that not
true since Linux distros don't like using bundled libs, but prefer
system installed shared libs so they just upgrade that on security
issues?)
I suppose that (most) distros use shared system libs instead of our
bundled stuff.
--
Christoph M. Becker
On Mon, Sep 17, 2018 at 3:16 PM Christoph M. Becker cmbecker69@gmx.de
wrote:
Hi!
We bundle an unmodified libsqlite3 for at least two years. Since then
all updates go into any dev, alpha and beta releases, while security
patches (usually backports from libsqlite3) go into stable branches.ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has no
specific version requirement; there is a check for sqlite3_open()[3]
which I believe is available since 3.0.0.So is there any particular reason why we still bundle libsqlite3?
[1]
<
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/sqlite3/config0.m4#L37-L47[2] https://sqlite.org/oldnews.html#2007_01_04
[3]
<
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/pdo_sqlite/config.m4#L55-L58
+1 on unbundling libsqlite. If we a) don't patch a library and b) it is
widely available in distros, then we have no business bundling the library.
Bundled libraries are extra maintenance work for us and a security risk for
users (we are slower at updating bundled libraries than distros).
Three more libraries that we (as far as I know) bundle without modification
are:
- libzip: Unbundling was already planned for 7.3 but held up due to some
CI issues. Next try at https://github.com/php/php-src/pull/3532. - oniguruma
- pcre
I'm not sure how widely available libonig and libpcre2 are.
Nikita
Hi,
-----Original Message-----
From: Nikita Popov nikita.ppv@gmail.com
Sent: Monday, September 17, 2018 7:07 PM
To: Christoph Becker cmbecker69@gmx.de
Cc: PHP internals internals@lists.php.net
Subject: Re: [PHP-DEV] Unbundle libsqlite3?On Mon, Sep 17, 2018 at 3:16 PM Christoph M. Becker cmbecker69@gmx.de
wrote:Hi!
We bundle an unmodified libsqlite3 for at least two years. Since then
all updates go into any dev, alpha and beta releases, while security
patches (usually backports from libsqlite3) go into stable branches.ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has
no specific version requirement; there is a check for
sqlite3_open()[3] which I believe is available since 3.0.0.So is there any particular reason why we still bundle libsqlite3?
[1]
<
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/sqlite3/config0.m
4#L37-L47[2] https://sqlite.org/oldnews.html#2007_01_04
[3]
<
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/pdo_sqlite/config
.m4#L55-L58+1 on unbundling libsqlite. If we a) don't patch a library and b) it is
widely available in distros, then we have no business bundling the library.
Bundled libraries are extra maintenance work for us and a security risk for users
(we are slower at updating bundled libraries than distros).Three more libraries that we (as far as I know) bundle without modification
are:
- libzip: Unbundling was already planned for 7.3 but held up due to some CI
issues. Next try at https://github.com/php/php-src/pull/3532.- oniguruma
- pcre
I'm not sure how widely available libonig and libpcre2 are.
For SQLite3 it should be fine.
For PCRE2 - a case we require the bundled lib badly is for debugging and Valgrind integration for JIT. Sure some debug symbols can be available from a distribution, but I never tried it that way and it might differ from one Linux to another especially regarding Valgrind. Non-Linux environments like BSD based are even not to mention. An alternative to the distribution supplied builds might be to always have custom Valgrind enabled builds of libpcre2 just for the PHP development, whereby different versions might collide with some system provided libraries. IMO it is more convenient for us to keep the bundled PCRE libs for this reasons.
Some libonig versions seem to be available on even old stable Debian and current Fedora, but further targets need to be investigated.
Windows builds for all these should be doable. I'd need to check closer for PCRE2 and Oniguruma, but SQLite3 is for sure good.
Regards
Anatol
Le 17/09/2018 à 15:16, Christoph M. Becker a écrit :
Hi!
We bundle an unmodified libsqlite3 for at least two years. Since then
all updates go into any dev, alpha and beta releases, while security
patches (usually backports from libsqlite3) go into stable branches.ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has no
specific version requirement; there is a check for sqlite3_open()[3]
which I believe is available since 3.0.0.So is there any particular reason why we still bundle libsqlite3?
+1 to drop the bundled library
Remi
On Mon, 17 Sep 2018 15:16:13 +0200 / "Christoph M. Becker"
cmbecker69@gmx.de said :
Hi!
We bundle an unmodified libsqlite3 for at least two years. Since then
all updates go into any dev, alpha and beta releases, while security
patches (usually backports from libsqlite3) go into stable branches.ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has
no specific version requirement; there is a check for
sqlite3_open()[3] which I believe is available since 3.0.0.
I believe that SQLite3Stmt::readOnly requires SQLite 3.7.4 released in
2010, bundled since PHP 5.3.6 (!).
There is also a PR patch (from me) that should make it in PHP 7.4 that
implements this feature in pdo_sqlite too.
I believe this is the most recent feature that we are using. But it
should return false if the SQLite version installed is older than that
(IMHO it should throw an exception instead, but that's the current
behaviour anyway).
As for bundling or not, I'm on Devuan and it is using the system
version of SQLite as far as I know so I don't see the need to bundle it
with PHP, but I don't have a good view of whether it might be needed on
other platforms.
On Mon, 17 Sep 2018 15:16:13 +0200 / "Christoph M. Becker"
cmbecker69@gmx.de said :ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has
no specific version requirement; there is a check for
sqlite3_open()[3] which I believe is available since 3.0.0.I believe that SQLite3Stmt::readOnly requires SQLite 3.7.4 released in
2010, bundled since PHP 5.3.6 (!).
Ah, indeed[1]!
There is also a PR patch (from me) that should make it in PHP 7.4 that
implements this feature in pdo_sqlite too.
PR #2698[2] seems to require an RFC. IMHO, it's best to present RFCs
and voting on them as early as possible (instead of rushing them at the
last moment). :)
I believe this is the most recent feature that we are using. But it
should return false if the SQLite version installed is older than that
(IMHO it should throw an exception instead, but that's the current
behaviour anyway).
I agree that an exception would be better. A sensible alternative might
be to not even include the method for older libsqlite3. Either way, a
PR targeting master would be welcome.
[1]
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/sqlite3/sqlite3.c#L1393-L1397
[2] https://github.com/php/php-src/pull/2698
--
Christoph M. Becker
On Wed, 19 Sep 2018 19:41:11 +0200 / "Christoph M. Becker"
cmbecker69@gmx.de said :
PR #2698[2] seems to require an RFC. IMHO, it's best to present RFCs
and voting on them as early as possible (instead of rushing them at
the last moment). :)
Yeah there was an RFC and it was voted no:
https://wiki.php.net/rfc/implement_sqlite_openblob_in_pdo
Discussion was that current PDO way of handling DB-specific features is
not the best. Unfortunately no one volunteered to change that fact, and
as my patch wasn't approved PDO_SQLite is still behind SQLite3 and
missing features.
I was actually talking about https://github.com/php/php-src/pull/2760
which was approved just after feature freeze of PHP 7.3.
I agree that an exception would be better. A sensible alternative
might be to not even include the method for older libsqlite3. Either
way, a PR targeting master would be welcome.
Yeah unfortunately due to the result of the previous RFC I stopped my
efforts trying to bring up pdo_sqlite and SQLite3 up to date. I have
very little free time, so spending it for useless patches is not the
best use of it, sorry.
On Wed, 19 Sep 2018 19:41:11 +0200 / "Christoph M. Becker"
cmbecker69@gmx.de said :PR #2698[2] seems to require an RFC. IMHO, it's best to present RFCs
and voting on them as early as possible (instead of rushing them at
the last moment). :)Yeah there was an RFC and it was voted no:
https://wiki.php.net/rfc/implement_sqlite_openblob_in_pdo
Ah, thanks! Since the RFC has been declined, I've closed the PR.
Discussion was that current PDO way of handling DB-specific features is
not the best. Unfortunately no one volunteered to change that fact, and
as my patch wasn't approved PDO_SQLite is still behind SQLite3 and
missing features.
It's unfortunate, but unless someone invests the time to come up with
a solution, things will likely not change.
I was actually talking about https://github.com/php/php-src/pull/2760
which was approved just after feature freeze of PHP 7.3.
Ah, thanks. I've update NEWS and UPGRADING accordingly[1].
[1]
http://git.php.net/?p=php-src.git;a=commit;h=7b9b398daad24d8829df487131bdf5d266e34f74
--
Christoph M. Becker
We bundle an unmodified libsqlite3 for at least two years. Since then
all updates go into any dev, alpha and beta releases, while security
patches (usually backports from libsqlite3) go into stable branches.ext/sqlite3 requires libsqlite ≥ 3.3.9[1] which has been released on
2007-01-04[2] (i.e. more than eleven years ago!). ext/pdo_sqlite has no
specific version requirement; there is a check for sqlite3_open()[3]
which I believe is available since 3.0.0.So is there any particular reason why we still bundle libsqlite3?
[1]
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/sqlite3/config0.m4#L37-L47
[2] https://sqlite.org/oldnews.html#2007_01_04
[3]
https://github.com/php/php-src/blob/php-7.3.0RC1/ext/pdo_sqlite/config.m4#L55-L58
I have submitted https://github.com/php/php-src/pull/3548.
--
Christoph M. Becker