Hi everyone,
I would like to provide an update regarding the git.php.net security
incident. To briefly summarize the most important information:
- We no longer believe the git.php.net server has been compromised.
However, it is possible that the master.php.net user database leaked. - master.php.net has been migrated to a new system main.php.net.
- All php.net passwords have been reset. Go to
https://main.php.net/forgot.php to set a new password. - git.php.net and svn.php.net are both read-only now, but will remain
available for the time being.
The following is a more detailed explanation of what happened and which
actions were taken.
When the first malicious commit was made under Rasmus' name, my initial
reaction was to revert the change and revoke commit access for Rasmus'
account, on the assumption that this was an individual account compromise.
In hindsight, this action didn't really make sense, because there was (at
the time) no reason to believe that the push occurred through Rasmus'
account in particular. Any account with access to the php-src repository
could have performed the push under a false name.
When the second malicious commit was made under my own name, I reviewed the
logs of our gitolite installation, in order to determine which account was
actually used to perform the push. However, while all adjacent commits were
accounted for, no git-receive-pack entries for the two malicious commits
were present, which means that these two commits bypassed the gitolite
infrastructure entirely. This was interpreted as likely evidence of a
server compromise.
Shortly after that, we made the decision to discontinue git.php.net and
make GitHub our primary repository host instead. Retaining our own Git
infrastructure would have required setting up a new git.php.net server
after determining the root cause of the compromise. This would take a lot
of time and disrupt PHP development in the meantime. A basic migration to
GitHub could be performed much more quickly, as most repositories were
already mirrored there. At this point, a lot of development was already
going through GitHub anyway, and our own Git infrastructure was mostly a
security liability and complication to the development workflow, so it was
not a hard decision to make the switch.
Something I was not aware of at the time is that git.php.net
(intentionally) supported pushing changes not only via SSH (using the
gitolite infrastructure and public key cryptography), but also via HTTPS.
The latter did not use gitolite, and instead used git-http-backend behind
Apache2 Digest authentication against the master.php.net user database. I'm
not sure why password-based authentication was supported in the first
place, as it is much less secure than pubkey authentication.
Based on access logs, we can determine that the commits were indeed pushed
using HTTPS and password-based authentication. An excerpt of relevant log
entries is shown below:
[redacted] - rasmus@[redacted] [27/Mar/2021:19:19:23 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - rasmus@[redacted] [27/Mar/2021:19:19:28 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - rasmus [27/Mar/2021:20:56:51 -0700] "GET
/push/php-src.git/info/refs?service=git-receive-pack HTTP/1.1" 200 125315
[redacted] - rasmus [27/Mar/2021:20:58:13 -0700] "POST
/push/php-src.git/git-receive-pack HTTP/1.1" 200 1080
[redacted] - nikita.ppv [28/Mar/2021:09:09:15 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - nikita.ppv [28/Mar/2021:09:09:18 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - nikitappv [28/Mar/2021:09:09:35 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - nikitappv [28/Mar/2021:09:09:36 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - nikita [28/Mar/2021:09:09:50 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - nikita [28/Mar/2021:09:09:53 -0700] "GET
/push/php-src.git/info/refs?service=git-upload-pack HTTP/1.1" 401 941
[redacted] - nikic [28/Mar/2021:09:11:31 -0700] "GET
/push/php-src.git/info/refs?service=git-receive-pack HTTP/1.1" 401 941
[redacted] - nikic [28/Mar/2021:09:11:31 -0700] "GET
/push/php-src.git/info/refs?service=git-receive-pack HTTP/1.1" 401 941
[redacted] - nikic [28/Mar/2021:09:13:28 -0700] "GET
/push/php-src.git/info/refs?service=git-receive-pack HTTP/1.1" 200 123263
[redacted] - nikic [28/Mar/2021:09:13:39 -0700] "POST
/push/php-src.git/git-receive-pack HTTP/1.1" 200 1079
It is notable that the attacker only makes a few guesses at usernames, and
successfully authenticates once the correct username has been found. While
we don't have any specific evidence for this, a possible explanation is
that the user database of master.php.net has been leaked, although it is
unclear why the attacker would need to guess usernames in that case.
The master.php.net system, which is used for authentication and various
management tasks, was running very old code on a very old operating system
/ PHP version, so some kind of vulnerability would not be terribly
surprising. We have made a number of changes to increase the security of
this system:
- master.php.net was migrated to a new system (running PHP 8) and renamed
to main.php.net at the same time. Among other things, the new system
supports TLS 1.2, which means you should no longer see TLS version warnings
when accessing this site. - The implementation has been moved towards using parameterized queries,
to be more confident that SQL injections cannot occur. - Passwords are now stored using bcrypt.
- Existing passwords were reset (use main.php.net/forgot.php to generate a
new one).
Previously, passwords were stored in a format compatible with HTTP Digest
authentication (essentially a plain md5 hash), which was required for HTTP
authentication on git.php.net and svn.php.net. As git.php.net has been made
read-only as a result of this incident, we decided to make svn.php.net
read-only as well, and thus remove the need to store passwords in an
insecure format. Only a small handful of PECL extensions were still using
the SVN server. The following SVN repositories had semi-recent activity and
have been migrated to GitHub:
https://github.com/php/pecl-authentication-krb5
https://github.com/php/pecl-caching-varnish
https://github.com/php/pecl-database-dbase
https://github.com/php/pecl-datetime-hrtime
https://github.com/php/pecl-datetime-timezonedb
https://github.com/php/pecl-math-trader
https://github.com/php/pecl-networking-geoip
https://github.com/php/pecl-processing-rrd
https://github.com/php/pecl-system-expect
https://github.com/php/pecl-text-pdflib
https://github.com/php/pecl-tools-svn
https://github.com/php/pecl-xml-xmldiff
Please contact me at nikic@php.net if additional migrations or permission
adjustments are needed. Please also report any issues you encounter with
main.php.net -- it's likely that some things broke during the migration.
Regards,
Nikita
Nikita Popov in php.internals (Tue, 6 Apr 2021 20:28:03 +0200):
Something I was not aware of at the time is that git.php.net
(intentionally) supported pushing changes not only via SSH (using the
gitolite infrastructure and public key cryptography), but also via HTTPS.
The latter did not use gitolite, and instead used git-http-backend behind
Apache2 Digest authentication against the master.php.net user database. I'm
not sure why password-based authentication was supported in the first
place, as it is much less secure than pubkey authentication.
Password-based authentication on Github is deprecated for some time now
and will be disabled on August 13, 2021. See the timeline in
https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/
Jan
Nikita Popov in php.internals (Tue, 6 Apr 2021 20:28:03 +0200):
- Existing passwords were reset (use main.php.net/forgot.php to generate a
new one).
The Wiki has another password reset link:
https://wiki.php.net/start?do=resendpwd
It sends a new password (in plain text), but that password still fails.
I had to reset my password at https://main.php.net/forgot.php to be able
to login to the Wiki and, for instance, vote for RFC's.
Jan
Nikita Popov in php.internals (Tue, 6 Apr 2021 20:28:03 +0200):
- Existing passwords were reset (use main.php.net/forgot.php to generate a
new one).The Wiki has another password reset link:
https://wiki.php.net/start?do=resendpwdIt sends a new password (in plain text), but that password still fails.
I had to reset my password at https://main.php.net/forgot.php to be able
to login to the Wiki and, for instance, vote for RFC's.
The resendpwd link is for Wiki user accounts, i.e. for users who don't
have a php.net account, only.
--
Christoph M. Becker
"Christoph M. Becker" in php.internals (Mon, 12 Apr 2021 14:23:33 +0200):
Nikita Popov in php.internals (Tue, 6 Apr 2021 20:28:03 +0200):
- Existing passwords were reset (use main.php.net/forgot.php to generate a
new one).The Wiki has another password reset link:
https://wiki.php.net/start?do=resendpwdIt sends a new password (in plain text), but that password still fails.
I had to reset my password at https://main.php.net/forgot.php to be able
to login to the Wiki and, for instance, vote for RFC's.The resendpwd link is for Wiki user accounts, i.e. for users who don't
have a php.net account, only.
It is confusing, to say the least. You follow a RFC link and notice that
your password does not work any more. So you follow the link after
'Forgotten your password? Get a new one', follow the actions and get a new
password. You try the password and it still does not work. There is no
reference to https://main.php.net/forgot.php anywhere on the login page
for RFC's and the like.
Jan