Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:115676
To: internals@lists.php.net
References: <CAJmy8YHcaOBs2wqxE0QMpmS9EHLCm+Cxy-tnEUVmuwDQCC43JQ@mail.gmail.com>
Message-ID: <eae5ee77-cc15-79b9-4e4a-08107308c3a0@gmail.com>
Date: Mon, 9 Aug 2021 21:14:17 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <CAJmy8YHcaOBs2wqxE0QMpmS9EHLCm+Cxy-tnEUVmuwDQCC43JQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
Subject: Re: [PHP-DEV] mb_check_encoding slow performance?
From: rowan.collins@gmail.com (Rowan Tommins)

On 07/08/2021 18:57, Hans Henrik Bergan wrote:
> can someone shed some light on this? why does mb_check_encoding seem to be
> so much slower than the alternatives?
> benchmark code+results is here https://stackoverflow.com/a/68690757/1067003


Hi Hans,

Since you ran the test on PHP 7.4, the relevant implementation is here: 
https://heap.space/xref/PHP-7.4/ext/mbstring/mbstring.c?r=0cafd53d#php_mb_check_encoding_impl

As you can maybe see, it takes a rather "brute force" approach: it runs 
the entire string through a conversion routine, and then checks (among 
other things) that the output is identical to the input. That makes it 
scale horribly with string length, with no optimization for returning 
false early.

The good news is that Alex Dowad has been doing a lot of work to improve 
ext/mbstring recently, and landed a completely new implementation for 
mb_check_encoding a few months ago: 
https://github.com/php/php-src/commit/be1a2155 although it was then 
changed slightly by later cleanup: 
https://github.com/php/php-src/commit/3e7acf90

That was too late for PHP 8.0, so I compiled an up to date git checkout, 
and ran your benchmark (with 100_000 iterations instead of 1_000_000; I 
guess my PC's a lot slower than yours!)

PHP 7.4:
mbstring: 57000 / 57100 / 56200
PCRE: 1500 / 1200 / 12400

PHP 8.1 beta:
mbstring: 35600 / 1200 / 36700
PCRE: 1400 / 1200 / 12100

So, mbstring now detects a failure at the start of the string as quickly 
as PCRE does, because the new algorithm has an early return, but is 
still slower than PCRE when it has to check the whole string.

Looking at the PCRE source, I think the relevant code is this: 
https://vcs.pcre.org/pcre2/code/trunk/src/pcre2_valid_utf.c?view=markup

It has the advantage of only handling a handful of encodings, and only 
needing to do a few operations on them. The main problem ext/mbstring 
has is that it supports a lot of operations, on a lot of different 
encodings, so it's still reusing a general purpose "convert and filter" 
algorithm.


Regards,

-- 
Rowan Tommins
[IMSoP]