Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:93647
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.161.173 as permitted sender)
To: Sara Golemon <pollita@php.net>
References: <CALYT1Yo97rG=onQPoVC8Mt9qhScdDfk9KYuKAmKyzK=7U3TfcA@mail.gmail.com>
 <e21002a7-5d21-212c-e344-2ea545bc5c3b@gmail.com>
 <CAESVnVrx6MwE0dktVDgSt6B+cTtOTTS_qeE4CwqFbTKKUEuqNw@mail.gmail.com>
Cc: Sammy Kaye Powers <me@sammyk.me>, PHP Internals <internals@lists.php.net>
Message-ID: <7b80be21-b397-c40c-8e80-ff4da4f97634@gmail.com>
Date: Mon, 30 May 2016 19:18:29 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:45.0)
 Gecko/20100101 Thunderbird/45.1.0
MIME-Version: 1.0
In-Reply-To: <CAESVnVrx6MwE0dktVDgSt6B+cTtOTTS_qeE4CwqFbTKKUEuqNw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] PHP's handling of BOM (byte order mark)
From: smalyshev@gmail.com (Stanislav Malyshev)

Hi!

> In fact, the idea of stripping content from a script file isn't
> without precedent.  Shebang lines are routinely removed from
> cli/cgi/fpm, and if you want to properly output it, you need to do so

True, because in the context of CLI we know what is expected - a CLI
script which can start with #!. It is very unlikely that we'd have a
template run directly as CLI script and we would have this template
starting with #! which we want to output. But we lack such context in a
generic script - namely, the context that would tell us if it's safe to
drop the BOM.

> So can we apply the same to the BOM?  There's the obvious BC danger of
> files which might depend on this behavior (declaring their encoding
> via BOM, which happens to be the same as the script encoding).

Given that BOM in script files is mostly useless, and BOM in UTF-8 is
useless and not recommended for use either, I don't see why we need to.

In general, I don't think BOM is a real issue worth messing with the
lexer. Surely, from time to time somebody would use weird editor which
produces BOMs, like editing PHP scripts in Word. Surely, they'd have
weird effects that would force them to spend 5 minutes googling and
fixing it. I don't think it is the reason to spend day-persons of our
collective time to find a fix to this very niche problem and risk
potential BC issues.

If it is really becoming an issue, we could probably make the lexer
treat BOM+<? the same as <?, but I'm not convinced it is a serious
enough issue.

> So how about declare statement?
> 
> {U+FEFF}<?php
>   declare(strip_bom=true);

That presumes you know there's BOM in the beginning of your file. If so,
why don't you just delete it instead of typing a long declare directive?
If you don't know it, you'd be forced to add it to every (non-template)
file in your codebase - which sounds a bit excessive.

-- 
Stas Malyshev
smalyshev@gmail.com