Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:92199
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.192.173 as permitted sender)
To: internals@lists.php.net
References: <CAESVnVouJP7Cad-AgoT4_nGSDXE6sejE9tGY5EOJitaH7JKS8A@mail.gmail.com>
 <alpine.DEB.2.20.1604051004480.4094@whisky.home.derickrethans.nl>
 <57050CAB.1040302@php.net>
Message-ID: <570BD2A2.4040504@gmail.com>
Date: Mon, 11 Apr 2016 11:36:50 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0)
 Gecko/20100101 Thunderbird/38.7.1
MIME-Version: 1.0
In-Reply-To: <57050CAB.1040302@php.net>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] IntlCharsetDetector
From: smalyshev@gmail.com (Stanislav Malyshev)

Hi!

>> As you say, it doesn't work properly. As a matter of fact, guessing 
>> charsets, like timezones, is not possible. You need to know which 
>> charset something is in. If not, you need to address *that* problem.

It is true that you can not detect charsets with 100% accuracy. It is,
however, also true that many charsets can be distinguished with enough
accuracy to make it useful, especially if you know the set of charsets
you are dealing with. E.g., Russian had about 5 commonly used encodings
before everybody started to use UTF-8, and several exotic ones. Being
able to detect at least the major ones while dealing with a
heterogeneous library of Russian-language texts is a great help. There
may be other cases like this.

The point is even imperfect detection may be useful in certain
circumstances, and detector being part of ICU hints that people find it
useful enough to spend time implementing and supporting it. We should
not ignore that.

-- 
Stas Malyshev
smalyshev@gmail.com