# bugs.php.net downtime

- Thread: https://externals.io/message/102899
- Original archive: https://news-web.php.net/php.internals/102899
- Started by Rasmus Lerdorf on 2018-07-17T20:38:07+00:00
- Messages: 24
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #102899 — Rasmus Lerdorf — 2018-07-17T20:38:07+00:00

URL: https://externals.io/message/102899#102899

I need to move bugs.php.net to another server sometime today. I won't be
able to do it without a little bit of downtime as I have to stop new
activity on the existing box, copy the DB over and then point DNS to the
new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable for
much longer than that.

Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5
to 7.2 on faster hardware.

-Rasmus

---

## #102911 — marius adrian popa — 2018-07-19T08:35:11+00:00

URL: https://externals.io/message/102899#102911 · In reply to: #102899

Hello Rasmus,

I can't access previous patches

https://bugs.php.net/patch-display.php?bug_id=62300&patch=0&revision=latest



> I need to move bugs.php.net to another server sometime today. I won't be
> able to do it without a little bit of downtime as I have to stop new
> activity on the existing box, copy the DB over and then point DNS to the
> new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable for
> much longer than that.
>
> Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5
> to 7.2 on faster hardware.
>
> -Rasmus

---

## #102919 — Niklas Keller — 2018-07-19T20:35:24+00:00

URL: https://externals.io/message/102899#102919 · In reply to: #102911

Hey Rasmus

I just found this bug: https://bugs.php.net/bug.php?id=76553

Has this bug been like that before the migration, too? Or did
something go wrong?

Regards, Niklas
Am Do., 19. Juli 2018 um 10:35 Uhr schrieb marius adrian popa
<mapopa@gmail.com>:
>
> Hello Rasmus,
>
> I can't access previous patches
>
> https://bugs.php.net/patch-display.php?bug_id=62300&patch=0&revision=latest
>
> 
>
> > […]

---

## #102921 — Hoffman — 2018-07-19T20:45:57+00:00

URL: https://externals.io/message/102899#102921 · In reply to: #102919

> Hey Rasmus
> 
> I just found this bug: https://bugs.php.net/bug.php?id=76553
> 
> Has this bug been like that before the migration, too? Or did
> something go wrong?

No, those used to be Unicode characters from the cyrillic block.


https://web.archive.org/web/20180719203931/https://webcache.googleusercontent.com/search%3Fq=cache:https://bugs.php.net/bug.php%3Fid=76553

(https://tinyurl.com/it-used-to-look-like-this)

--
Zach Hoffman

---

## #102924 — Yasuo Ohgaki — 2018-07-19T21:51:01+00:00

URL: https://externals.io/message/102899#102924 · In reply to: #102899

Hi,



> Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5
> to 7.2 on faster hardware.
>

It feels faster now. Nice work.

Anyway, it seemed UTF-8 char is not handled well.
I tried to send reply with UTF-8 chars

にほんご文字列

for

https://bugs.php.net/bug.php?id=76635

and response is empty page with 500 code.

----
Request URL: https://bugs.php.net/bug.php?id=76635&edit=1
Request Method: POST
Status Code: 500 Internal Server Error
Remote Address: 206.189.200.141:443
Referrer Policy: no-referrer-when-downgrade
----

Invalid UTF-8 data stored in bug db can result in broken pages.
How about convert invalid data into URL encoded data or something else?

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

---

## #102926 — Rasmus Lerdorf — 2018-07-21T00:52:10+00:00

URL: https://externals.io/message/102899#102926 · In reply to: #102911

It looks like the MDB2 -> PDO migration messed this up. There is a weird
MDB2 feature fetchAll() feature that returns the first field as the
associative key or something like that. Need to rewrite the code to not
rely on that. The MDB2 query returns:

array(2) {
  ["fix-71630.patch"]=>
  array(1) {
    [0]=>
    array(2) {
      [0]=>
      string(10) "1522170171"
      [1]=>
      string(11) "cmb@php.net"
    }
  }
  ["zero-data.patch"]=>
  array(1) {
    [0]=>
    array(2) {
      [0]=>
      string(10) "1521729404"
      [1]=>
      string(11) "cmb@php.net"
    }
  }
}

And the PDO query gives:

array(2) {
  [0]=>
  array(3) {
    ["patch"]=>
    string(15) "fix-71630.patch"
    ["revision"]=>
    int(1522170171)
    ["developer"]=>
    string(11) "cmb@php.net"
  }
  [1]=>
  array(3) {
    ["patch"]=>
    string(15) "zero-data.patch"
    ["revision"]=>
    int(1521729404)
    ["developer"]=>
    string(11) "cmb@php.net"
  }
}

I will find some time to fix this in the next day or two unless someone
beats me to it.

-Rasmus

On Thu, Jul 19, 2018 at 4:35 AM, marius adrian popa <mapopa@gmail.com>
wrote:

> Hello Rasmus,
>
> I can't access previous patches
>
> https://bugs.php.net/patch-display.php?bug_id=62300&
> patch=0&revision=latest
>
> On Tue, Jul 17, 2018 at 11:38 PM, Rasmus Lerdorf <rasmus@lerdorf.com>
> wrote:
>
> > […]

---

## #102927 — Rasmus Lerdorf — 2018-07-21T01:12:17+00:00

URL: https://externals.io/message/102899#102927 · In reply to: #102921

On Thu, Jul 19, 2018 at 4:45 PM, Hoffman, Zachary Robert <zrhoffman@ku.edu>
wrote:

> 
> > […]
>
> No, those used to be Unicode characters from the cyrillic block.


 This appears to be database-related. Something got messed up with
encodings on the mysql dump/import from MySQL 5.1.73 into MariaDB 10.1.26:

mysql> select sdesc from bugdb where id=76553;
+----------------------------------------------------------------------------------+
| sdesc
       |
+----------------------------------------------------------------------------------+
| Имя переменной может содержать управляющие |
+----------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553;
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| sdesc

             |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Ð˜Ð¼Ñ Ð¿ÐµÑ€ÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ Ð¼Ð¾Ð¶ÐµÑ‚ Ñ Ð¾Ð´ÐµÑ€Ð¶Ð°Ñ‚ÑŒ ÑƒÐ¿Ñ€Ð°Ð²Ð»Ñ
ÑŽÑ‰Ð¸Ðµ
                |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

The dumped table schema from MySQL has:

DROP TABLE IF EXISTS `bugdb`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `bugdb` (
  `id` int(8) NOT NULL AUTO_INCREMENT,
  `package_name` varchar(80) CHARACTER SET latin1 DEFAULT NULL,
  `bug_type` varchar(32) CHARACTER SET latin1 NOT NULL DEFAULT 'Bug',
  `email` varchar(40) CHARACTER SET latin1 NOT NULL DEFAULT '',
  `reporter_name` varchar(80) CHARACTER SET latin1 DEFAULT '',
  `sdesc` varchar(80) CHARACTER SET latin1 NOT NULL DEFAULT '',
  `ldesc` text CHARACTER SET latin1 NOT NULL,
  `php_version` varchar(100) CHARACTER SET latin1 DEFAULT NULL,
  `php_os` varchar(32) CHARACTER SET latin1 DEFAULT NULL,
  `status` varchar(16) CHARACTER SET latin1 DEFAULT NULL,
  `ts1` datetime DEFAULT NULL,
  `ts2` datetime DEFAULT NULL,
  `assign` varchar(20) CHARACTER SET latin1 DEFAULT NULL,
  `passwd` varchar(64) CHARACTER SET latin1 DEFAULT NULL,
  `registered` tinyint(1) NOT NULL DEFAULT '0',
  `block_user_comment` char(1) DEFAULT 'N',
  `cve_id` varchar(15) DEFAULT NULL,
  `private` char(1) DEFAULT 'N',
  `visitor_ip` int(10) unsigned NOT NULL,
  PRIMARY KEY (`id`),
  KEY `php_version` (`php_version`(1)),
  KEY `status` (`status`),
  KEY `package_name` (`package_name`),
  FULLTEXT KEY `email` (`email`,`sdesc`,`ldesc`)
) ENGINE=MyISAM AUTO_INCREMENT=76637 DEFAULT CHARSET=utf8 PACK_KEYS=1;
/*!40101 SET character_set_client = @saved_cs_client */;

When I dump it from MariaDB I see:

DROP TABLE IF EXISTS `bugdb`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `bugdb` (
  `id` int(8) NOT NULL AUTO_INCREMENT,
  `package_name` varchar(80) CHARACTER SET latin1 DEFAULT NULL,
  `bug_type` varchar(32) CHARACTER SET latin1 NOT NULL DEFAULT 'Bug',
  `email` varchar(40) CHARACTER SET latin1 NOT NULL DEFAULT '',
  `reporter_name` varchar(80) CHARACTER SET latin1 DEFAULT '',
  `sdesc` varchar(80) CHARACTER SET latin1 NOT NULL DEFAULT '',
  `ldesc` text CHARACTER SET latin1 NOT NULL,
  `php_version` varchar(100) CHARACTER SET latin1 DEFAULT NULL,
  `php_os` varchar(32) CHARACTER SET latin1 DEFAULT NULL,
  `status` varchar(16) CHARACTER SET latin1 DEFAULT NULL,
  `ts1` datetime DEFAULT NULL,
  `ts2` datetime DEFAULT NULL,
  `assign` varchar(20) CHARACTER SET latin1 DEFAULT NULL,
  `passwd` varchar(64) CHARACTER SET latin1 DEFAULT NULL,
  `registered` tinyint(1) NOT NULL DEFAULT '0',
  `block_user_comment` char(1) DEFAULT 'N',
  `cve_id` varchar(15) DEFAULT NULL,
  `private` char(1) DEFAULT 'N',
  `visitor_ip` int(10) unsigned NOT NULL,
  PRIMARY KEY (`id`),
  KEY `php_version` (`php_version`(1)),
  KEY `status` (`status`),
  KEY `package_name` (`package_name`),
  FULLTEXT KEY `email` (`email`,`sdesc`,`ldesc`)
) ENGINE=MyISAM AUTO_INCREMENT=76650 DEFAULT CHARSET=utf8 PACK_KEYS=1;
/*!40101 SET character_set_client = @saved_cs_client */;

Other than the autoincrement they are identical. I normally use utf8mb4,
but I figured I would play it safe and copy it over verbatim. I guess it
wasn't safe.
I'll do some research, but ideas welcome.

-Rasmus

---

## #102928 — Nikita Popov — 2018-07-21T08:46:39+00:00

URL: https://externals.io/message/102899#102928 · In reply to: #102899

> I need to move bugs.php.net to another server sometime today. I won't be
> able to do it without a little bit of downtime as I have to stop new
> activity on the existing box, copy the DB over and then point DNS to the
> new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable for
> much longer than that.
>
> Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5
> to 7.2 on faster hardware.
>
> -Rasmus
>

I'm getting a WSOD when trying to add a comment to a bug while logged in.
Not sure what's different for me, as it seems to work for other @php.net
users.

Nikita

---

## #102929 — Christoph M. Becker — 2018-07-21T09:41:45+00:00

URL: https://externals.io/message/102899#102929 · In reply to: #102928

> 
> 
> > […]
> 
> I'm getting a WSOD when trying to add a comment to a bug while logged in.
> Not sure what's different for me, as it seems to work for other @php.net
> users.

Same for me; it doesn't matter whether I'm logged in as cmb@ or just
commenting as user.  Interestingly, altough I get the 500, I can change
the meta information (status, summary, etc.), but no comment is added.

-- 
Christoph M. Becker

---

## #102930 — Rasmus Lerdorf — 2018-07-21T09:51:54+00:00

URL: https://externals.io/message/102899#102930 · In reply to: #102929

That one should be fixed. It was actually posting the comments, but getting
an error after. If you reload the bug you see it.

On Sat, Jul 21, 2018 at 5:41 AM, Christoph M. Becker <cmbecker69@gmx.de>
wrote:

> 
>
> > […]
> wrote:
> > […]
> be
> > […]
> for
> > […]
> 5.5
> > […]
>
> Same for me; it doesn't matter whether I'm logged in as cmb@ or just
> commenting as user.  Interestingly, altough I get the 500, I can change
> the meta information (status, summary, etc.), but no comment is added.
>
> --
> Christoph M. Becker

---

## #102931 — Nikita Popov — 2018-07-21T10:04:20+00:00

URL: https://externals.io/message/102899#102931 · In reply to: #102930

Still doesn't work for me. I get a white screen and the comment does not
appear even after I reload the page.

Nikita



> That one should be fixed. It was actually posting the comments, but
> getting an error after. If you reload the bug you see it.
>
> On Sat, Jul 21, 2018 at 5:41 AM, Christoph M. Becker <cmbecker69@gmx.de>
> wrote:
>
> > […]

---

## #102932 — Christoph M. Becker — 2018-07-21T10:06:33+00:00

URL: https://externals.io/message/102899#102932 · In reply to: #102930

> That one should be fixed. It was actually posting the comments, but getting
> an error after. If you reload the bug you see it.

I've commented multiple times on bug 76637, but none of the comments are
visible.  I also commented on bug 76642 and successfully changed a typo
in the title (summary), but the “meta info changed” comment is not there
either.

> On Sat, Jul 21, 2018 at 5:41 AM, Christoph M. Becker <cmbecker69@gmx.de>
> wrote:
> 
> > […]

---

## #102933 — Rasmus Lerdorf — 2018-07-21T10:10:10+00:00

URL: https://externals.io/message/102899#102933 · In reply to: #102932

Works fine for me on https://bugs.php.net/bug.php?id=76635&thanks=1

On Sat, Jul 21, 2018 at 6:06 AM, Christoph M. Becker <cmbecker69@gmx.de>
wrote:

> 
>
> > […]
> getting
> > […]
>
> I've commented multiple times on bug 76637, but none of the comments are
> visible.  I also commented on bug 76642 and successfully changed a typo
> in the title (summary), but the “meta info changed” comment is not there
> either.
>
> > […]
> the
> > […]
> in.
> > […]
> php.net
> > […]

---

## #102934 — Christoph M. Becker — 2018-07-21T10:18:29+00:00

URL: https://externals.io/message/102899#102934 · In reply to: #102933

> Works fine for me on https://bugs.php.net/bug.php?id=76635&thanks=1

I just commented “Testing”, but the comment isn't there.  Don't the
error logs give some useful hints?

> On Sat, Jul 21, 2018 at 6:06 AM, Christoph M. Becker <cmbecker69@gmx.de>
> wrote:
> 
> > […]

---

## #102935 — Rasmus Lerdorf — 2018-07-21T10:20:38+00:00

URL: https://externals.io/message/102899#102935 · In reply to: #102934

Yeah, working on it. Amazingly it looks like it is an ipv6 issue. The old
bugs didn't have an ipv6 address, but the new one does and we store the ip
of the commenter using the MySQL function inet_aton() and not inet6_aton().
Need to change that, but also alter the table to make the column fit an
ipv6 addr.

On Sat, Jul 21, 2018 at 6:18 AM, Christoph M. Becker <cmbecker69@gmx.de>
wrote:

> 
>
> > […]
>
> I just commented “Testing”, but the comment isn't there.  Don't the
> error logs give some useful hints?
>
> > […]
> cmbecker69@gmx.de>
> > […]
> won't
> > […]
> unavailable
> > […]
> PHP
> > […]
> change
> > […]

---

## #102936 — Rasmus Lerdorf — 2018-07-21T10:53:03+00:00

URL: https://externals.io/message/102899#102936 · In reply to: #102935

By the way, the following users have ssh accounts on bugs.php.net:
bjori  derick  felipe  nikic  rasmus  sas  scottmac  tyrael

the error log is in /srv/bugs.php.net/logs/error.log
and the code is here: git@git.php.net:/web/bugs.git

Feel free to log in and fix stuff as you find more issues. Right now the
patch handling is still broken along with the latin1 columns that need to
be converted to utf8.

-Rasmus

---

## #102938 — Christoph M. Becker — 2018-07-21T11:16:33+00:00

URL: https://externals.io/message/102899#102938 · In reply to: #102935

> Yeah, working on it. Amazingly it looks like it is an ipv6 issue. The old
> bugs didn't have an ipv6 address, but the new one does and we store the ip
> of the commenter using the MySQL function inet_aton() and not inet6_aton().
> Need to change that, but also alter the table to make the column fit an
> ipv6 addr.

Works fine for me again.  Thanks, Rasmus!

> On Sat, Jul 21, 2018 at 6:18 AM, Christoph M. Becker <cmbecker69@gmx.de>
> wrote:
> 
> > […]

---

## #102939 — Rasmus Lerdorf — 2018-07-21T12:30:15+00:00

URL: https://externals.io/message/102899#102939 · In reply to: #102927

For future reference, here is what I did to fix the encoding problem:

MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553;
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| sdesc

             |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Ð˜Ð¼Ñ Ð¿ÐµÑ€ÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ Ð¼Ð¾Ð¶ÐµÑ‚ Ñ Ð¾Ð´ÐµÑ€Ð¶Ð°Ñ‚ÑŒ ÑƒÐ¿Ñ€Ð°Ð²Ð»Ñ
ÑŽÑ‰Ð¸Ðµ
                |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MariaDB [phpbugsdb]> alter table bugdb drop index email;
Query OK, 76298 rows affected (0.85 sec)
Records: 76298  Duplicates: 0  Warnings: 0

MariaDB [phpbugsdb]> alter table bugdb modify sdesc varbinary(80) NOT NULL
DEFAULT '', modify ldesc binary NOT NULL, modify email varbinary(40) NOT
 `NULL` DEFAULT '';
Query OK, 76298 rows affected, 65535 warnings (0.65 sec)
Records: 76298  Duplicates: 0  Warnings: 76091

MariaDB [phpbugsdb]> alter table bugdb modify sdesc varchar(80) CHARACTER
SET utf8mb4 NOT NULL DEFAULT '', modify ldesc text CHARACTER SET utf8mb4
NOT NULL, modify email varchar(40) CHARACTER SET utf8mb4 NOT NULL DEFAULT
'';
Query OK, 76298 rows affected, 127 warnings (0.57 sec)
Records: 76298  Duplicates: 0  Warnings: 127

MariaDB [phpbugsdb]> alter table bugdb add FULLTEXT INDEX `email`
(`email`,`sdesc`,`ldesc`);
Query OK, 76298 rows affected (1.56 sec)
Records: 76298  Duplicates: 0  Warnings: 0

MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553;
+----------------------------------------------------------------------------------+
| sdesc
       |
+----------------------------------------------------------------------------------+
| Имя переменной может содержать управляющие
        |
+----------------------------------------------------------------------------------+
1 row in set (0.00 sec)

The trick was to convert the columns to binary first. When I went straight
from latin1 to utf8 I got the utf8 equivalent of the latin1 characters. By
telling it that the data was actually binary first, it converted from
binary to utf8 which appears to have worked. There were some warnings,
which I assume are invalid utf8 byte sequences somewhere.

-Rasmus

---

## #102940 — Rasmus Lerdorf — 2018-07-21T12:31:53+00:00

URL: https://externals.io/message/102899#102940 · In reply to: #102939

Uh, ok, something obviously went wrong there. Checking.



> For future reference, here is what I did to fix the encoding problem:
>
> MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553;
> +-----------------------------------------------------------
> ------------------------------------------------------------
> ---------------------------------------------+
> | sdesc
>
>                |
> +-----------------------------------------------------------
> ------------------------------------------------------------
> ---------------------------------------------+
> | Ð˜Ð¼Ñ Ð¿ÐµÑ€ÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ Ð¼Ð¾Ð¶ÐµÑ‚ Ñ Ð¾Ð´ÐµÑ€Ð¶Ð°Ñ‚ÑŒ ÑƒÐ¿Ñ€Ð°Ð²Ð»Ñ
> ÑŽÑ‰Ð¸Ðµ
>                 |
> +-----------------------------------------------------------
> ------------------------------------------------------------
> ---------------------------------------------+
> 1 row in set (0.00 sec)
>
> MariaDB [phpbugsdb]> alter table bugdb drop index email;
> Query OK, 76298 rows affected (0.85 sec)
> Records: 76298  Duplicates: 0  Warnings: 0
>
> MariaDB [phpbugsdb]> alter table bugdb modify sdesc varbinary(80) NOT NULL
> DEFAULT '', modify ldesc binary NOT NULL, modify email varbinary(40) NOT
> `NULL` DEFAULT '';
> Query OK, 76298 rows affected, 65535 warnings (0.65 sec)
> Records: 76298  Duplicates: 0  Warnings: 76091
>
> MariaDB [phpbugsdb]> alter table bugdb modify sdesc varchar(80) CHARACTER
> SET utf8mb4 NOT NULL DEFAULT '', modify ldesc text CHARACTER SET utf8mb4
> NOT NULL, modify email varchar(40) CHARACTER SET utf8mb4 NOT NULL DEFAULT
> '';
> Query OK, 76298 rows affected, 127 warnings (0.57 sec)
> Records: 76298  Duplicates: 0  Warnings: 127
>
> MariaDB [phpbugsdb]> alter table bugdb add FULLTEXT INDEX `email`
> (`email`,`sdesc`,`ldesc`);
> Query OK, 76298 rows affected (1.56 sec)
> Records: 76298  Duplicates: 0  Warnings: 0
>
> MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553;
> +-----------------------------------------------------------
> -----------------------+
> | sdesc
>          |
> +-----------------------------------------------------------
> -----------------------+
> | Имя переменной может содержать управляющие
>         |
> +-----------------------------------------------------------
> -----------------------+
> 1 row in set (0.00 sec)
>
> The trick was to convert the columns to binary first. When I went straight
> from latin1 to utf8 I got the utf8 equivalent of the latin1 characters. By
> telling it that the data was actually binary first, it converted from
> binary to utf8 which appears to have worked. There were some warnings,
> which I assume are invalid utf8 byte sequences somewhere.
>
> -Rasmus

---

## #102941 — Rasmus Lerdorf — 2018-07-21T12:43:29+00:00

URL: https://externals.io/message/102899#102941 · In reply to: #102940

Ok, should be fixed now. https://bugs.php.net/bug.php?id=76553 looks good.
But I think between my backup and the conversion I lost a couple of bug
comments from this morning. Sorry about that.

-Rasmus



> Uh, ok, something obviously went wrong there. Checking.
>
> On Sat, Jul 21, 2018 at 8:30 AM, Rasmus Lerdorf <rasmus@lerdorf.com>
> wrote:
>
> > […]

---

## #102943 — Yasuo Ohgaki — 2018-07-21T22:01:26+00:00

URL: https://externals.io/message/102899#102943 · In reply to: #102927

> Other than the autoincrement they are identical. I normally use utf8mb4,
> but I figured I would play it safe and copy it over verbatim. I guess it
> wasn't safe.
>

Right. There are risks.

For example, encoding like SJIS contains \ as a part of valid char.
When encoding is mixed, escape could be disabled and injections are
possible.

Even when UTF-8 is used, mixed invalid encoding handling can break security
measures. e.g. Invalid UTF-8 encoding that is missing the last multibyte
byte.

When santaization is required, programmers have two choices.
 - Remove all bytes specified by MSB of UTF-8 first byte. i.e. Consume the
last byte.
 - Remove only bytes that are invalid as UTF-8. i.e Leave last ASCII char,
for example

If these designs are mixed, encoding attack is possible also.

DoS by invalid char is trivial. Current web browsers can refuse to render
entire page
that has badly broken encoding.

The only good countermeasure against encoding attack is encoding validation
with
Fail Fast principle. i.e. Validate encoding at application software's outer
most trust
boundary.

I'll do some research, but ideas welcome.
>

IMO, all data should be converted to valid UTF-8 encoding as we use
UTF-8 as bugs.php.net encoding. Replace invalid date to "?" or something
else.
Some data will be lost, however valid char encoding is mandatory for
correct data
handling as described above.

In order to replace invalid char to "?" (or something else),
 `mb_convert_encoding()`
can be used. "mbstring.substitute_character" INI is for specifying
replacing char.
Default is none, so it removes invalid data by default.

If you would like to keep original data, number of detected invalid chars
are recorded
and can be retrieved by `mb_get_info()`'s array. "illegal_chars" is "Total
number of
illegal chars in the script's lifetime". By checking this, invalid char
existence can be
checked.
(Alternatively, simply comparing original and converted data works also.)

You might want to count number of all illegal chars in the db before
converting
data. "illegal_chars" is handy for this.

Old data may be added to converted data by using base64 if it's necessary.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

---

## #103033 — mail@pmmaga.net — 2018-08-03T17:16:50+00:00

URL: https://externals.io/message/102899#103033 · In reply to: #102899

Hi Rasmus,

I'm not able to attach a PR to a bug report, however I'm not sure if this
wasn't already happening before the migration. When submitting the form, I
get a 500.

Thanks,
Pedro



> I need to move bugs.php.net to another server sometime today. I won't be
> able to do it without a little bit of downtime as I have to stop new
> activity on the existing box, copy the DB over and then point DNS to the
> new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable for
> much longer than that.
>
> Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5
> to 7.2 on faster hardware.
>
> -Rasmus

---

## #103034 — Gabriel Caruso — 2018-08-03T17:44:26+00:00

URL: https://externals.io/message/102899#103034 · In reply to: #103033

>
> Hi Rasmus,
>
> I'm not able to attach a PR to a bug report, however I'm not sure if this
> wasn't already happening before the migration. When submitting the form, I
> get a 500.
>
> Thanks,
> Pedro
>
> 
>
> > […]
> for
> > […]
> 5.5
> > […]
>

Pedro

This was previously reported as a bug https://bugs.php.net/bug.php?id=76079.

Thanks
-- 
Gabriel Caruso

---

## #103036 — mail@pmmaga.net — 2018-08-03T18:19:58+00:00

URL: https://externals.io/message/102899#103036 · In reply to: #103034

On Fri, Aug 3, 2018 at 6:44 PM Gabriel Caruso <carusogabriel34@gmail.com>
wrote:

> Pedro
>
> This was previously reported as a bug
> https://bugs.php.net/bug.php?id=76079.
>

I knew I had read about it somewhere! There it is. Thanks!
