Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:92298
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain thefsb.org designates 173.203.187.99 as permitted sender)
To: Sara Golemon <pollita@php.net>, PHP internals <internals@lists.php.net>
References: <CAESVnVouJP7Cad-AgoT4_nGSDXE6sejE9tGY5EOJitaH7JKS8A@mail.gmail.com>
 <alpine.DEB.2.20.1604051004480.4094@whisky.home.derickrethans.nl>
 <57050CAB.1040302@php.net> <570BD2A2.4040504@gmail.com>
 <CAESVnVqYevVYpPbaLP8rqbQDFi7vdDJHMeRv6jaAHV3GXqVEDg@mail.gmail.com>
Message-ID: <570F9075.4010205@thefsb.org>
Date: Thu, 14 Apr 2016 08:43:33 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0)
 Gecko/20100101 Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <CAESVnVqYevVYpPbaLP8rqbQDFi7vdDJHMeRv6jaAHV3GXqVEDg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] IntlCharsetDetector
From: fsb@thefsb.org (Tom Worster)

On 4/11/16 6:11 PM, Sara Golemon wrote:
> On Mon, Apr 11, 2016 at 9:36 AM, Stanislav Malyshev <smalyshev@gmail.com> wrote:
>> The point is even imperfect detection may be useful in certain
>> circumstances, and detector being part of ICU hints that people find it
>> useful enough to spend time implementing and supporting it. We should
>> not ignore that.
>>
> Well, Stas, your informal thumbs up to the idea means enough to me to
> at least formalize it into an RFC even though I was previously feeling
> negative on it.
>
> I may yet vote no on my own RFC after the discussion period, but as
> you say it's worth considering the fact that someone thought it
> reasonable enough to actually build into ICU...

The general problem is impossible. If you constrain the question, for 
example as Stas says by knowing the language and choosing between a 
given set of codes, then you may have success. And I'm sure I'm not 
alone in sometimes using a simple heuristic to choose between cp1252 and 
utf8.

But this does not logically imply that ICU CharsetDetector is a suitable 
solution in such cases or that it's a good API or a decent 
implementation. Or that PHP should expose it. An SO chat doesn't 
necessarily count as a feature request.

I'd rather people engineered real solutions specific to their 
requirements than resort to any of the failed attempts to solve the 
general problem.

Tom