Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:93644
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: softfail (pb1.pair.com: domain golemon.com does not designate 209.85.214.51 as permitted sender)
MIME-Version: 1.0
Sender: php@golemon.com
In-Reply-To: <e21002a7-5d21-212c-e344-2ea545bc5c3b@gmail.com>
References: <CALYT1Yo97rG=onQPoVC8Mt9qhScdDfk9KYuKAmKyzK=7U3TfcA@mail.gmail.com>
	<e21002a7-5d21-212c-e344-2ea545bc5c3b@gmail.com>
Date: Mon, 30 May 2016 18:57:19 -0700
Message-ID: <CAESVnVrx6MwE0dktVDgSt6B+cTtOTTS_qeE4CwqFbTKKUEuqNw@mail.gmail.com>
To: Stanislav Malyshev <smalyshev@gmail.com>
Cc: Sammy Kaye Powers <me@sammyk.me>, PHP Internals <internals@lists.php.net>
Content-Type: text/plain; charset=UTF-8
Subject: Re: [PHP-DEV] PHP's handling of BOM (byte order mark)
From: pollita@php.net (Sara Golemon)

On Mon, May 30, 2016 at 5:40 PM, Stanislav Malyshev <smalyshev@gmail.com> wrote:
>> BOM's should not be treated as characters and should not be sent to
>> the output. Is there any reason this should be considered the expected
>> behavior?
>
> The reason would be PHP does not know where surrounding output ends and
> the code starts, beyond <?php. That means if there is some stuff in the
> file before <?php, it would be output - and it's an intended behavior,
> and so will happen with BOM too. Particular sequence of bytes being BOM
> and whether it is desired or not depends on context, but PHP engine does
> not have this context. Remember that pure HTML page is also a valid PHP
> file.
>
I'm with Sammy on the principle that being able to have a BOM in a
given file is important to any non-ascii code development.  Though we
can argue whether that's good or even necessary, I honestly don't know
how prevalent non-english coding is among PHP developers.

In fact, the idea of stripping content from a script file isn't
without precedent.  Shebang lines are routinely removed from
cli/cgi/fpm, and if you want to properly output it, you need to do so
in a coded echo statement. (The stripping only applies to a literal,
non-scripting line in the file, not dynamic output).

So can we apply the same to the BOM?  There's the obvious BC danger of
files which might depend on this behavior (declaring their encoding
via BOM, which happens to be the same as the script encoding).

So how about declare statement?

{U+FEFF}<?php
  declare(strip_bom=true);

code(); code(); code();

It's got the advantage of being per-file (a view template might
actually want the BOM included, while some business logic piece
doesn't, for example.  It's a compile-time strip, so it has no runtime
cost.  It's non-surprising, since it's stated in every file for which
the BOM strip is intentional.

-Sara