[Concept] declare(strict_identifiers=1)

Hey Juris,

To prevent errors? I must admit I don't rly understand all the terms.
I assume it implies identifiers should be more visible/readable, right?

Not readability -- unambiguity. Three concrete things, no jargon:

1. Two identifiers that look identical on screen are currently two
      different identifiers. $x and $x followed by U+00A0 NO-BREAK SPACE
      are separate variables. So are "a" plus a combining diaeresis and the
      single character U+00E4; both display as a-umlaut.

2. Text that is not valid UTF-8 at all is currently a valid identifier.
      ${"\xFF\xFE"} = 1; compiles.

3. Nobody can state what a PHP identifier is except by copying the byte
      class out of the scanner. The manual does exactly that, and so does
      PHP-Parser.

The declare says: in this file, an identifier is well-formed UTF-8, is a
Unicode identifier in the sense of UAX #31 (the Unicode annex that
defines this for programming languages), and is written in one canonical
spelling. Anything else is a compile error.

Why would anything break if it's per-file?

Nothing breaks. That was bad wording on my part.

The survey answers a different question: if a maintainer adds the declare
to a file they already have, does it still compile? That is adoption
friction, not breakage. It is also a calibration check -- a rule that
rejected a lot of legitimate existing code would be the wrong rule, and I
wanted to know that before proposing it rather than after.

Do I understand it correctly that by adding that declare to 168604 you
would uncover a single risky identifier? Not that convincing...

That number is the cost, not the benefit, and I should have separated the
two more clearly.

The single Packagist finding is what would stop compiling: symfony/cache
declares a class whose entire name is the single byte 0xA9. That is the
whole measured adoption cost across the 250 most-installed packages.

What the rule catches can only be measured where non-ASCII identifiers
actually occur, which is not in the top Packagist packages. In the GitHub
corpus, 33 of 136 non-ASCII identifiers fail the rule and 11 contain an
invisible character. One is live code: the Alipay OpenAPI SDK has

    $chrtext<U+00A0> = null;
    // ...
    openssl_public_encrypt($block, $chrtext<U+00A0>, $res);

with a no-break space inside the name. It works only because the typo is
consistent throughout the function. Anyone who types $chrtext normally
gets a different variable, passed by reference, that stays null. Five
vendored copies across four unrelated projects in my sample.

But you are right that these are small numbers, and I would rather say so
than dress them up. The case does not rest on the bug count.

Would it be fair to say that the same constraints can be enforced by
linters/cs tooling?

Partly yes, and I will concede that plainly: UTF-8 validity, NFC and
UAX #31 conformance are all statically checkable. My survey tool is
exactly such a linter, written in PHP, and it is in the repository.

Three things it cannot do.

It cannot define the language. "What is a valid PHP identifier" currently
has no answer other than "whatever bytes the scanner happened to accept",
which is why the manual, PHP-Parser, every IDE and every static analyser
separately reverse-engineer the same byte class. A declare makes it a
versioned, testable statement.

It does not travel with the code. A declare is in the source file; a lint
configuration is in someone's toolchain. The file keeps its guarantee
after composer install, and the rule also covers generated and eval'd
code that never reaches a linter.

And it cannot touch semantics. Case-insensitive symbol lookup folds ASCII
only, so Stra<U+00DF>e and STRA<U+00DF>E are the same class while
Stra<U+1E9E>e is not, and Strasse and STRASSE are. No linter can fix
that, and I do not think it can sensibly be fixed before there is a
definition of what an identifier is. I deliberately kept case folding out
of this proposal, but that is the thing underneath it.

The same objection would apply to strict_types -- static analysers check
types, so why does the engine need a per-file declaration? I do not think
the answer there was "it doesn't", though I accept the parallel is not
exact, since strict_types changes runtime behaviour and this does not.

This is the weakest point of the proposal and you found it on the first
reading. If the list's view is that specifying the rule and leaving
enforcement to tooling is the right scope, that is a smaller and possibly
better RFC, and I would rather establish that now than after writing the
patch.

Thanks for the questions.

Regards,
Luca

What problem would this restriction solve?
Is it about an code smuggling attack vector using code obfuscation with
indistinguishable Unicode sequences?

No, and I would rather decline that framing than borrow it.

I did look for it. Across both corpora -- 168,604 files -- there is not
one identifier mixing Latin with Cyrillic or Greek, and not one that maps
to a plausible ASCII identifier under homoglyph substitution. The
proposal does not reject confusables and the draft says so explicitly.
UAX #31 takes the same position: for programming languages, spoofing is
better addressed by higher-level diagnostics than in the lexer.

What it does solve, ordered by how much I think each is actually worth:

1. PHP has no definition of an identifier. The only answer to "what is a
   valid PHP identifier" is "whatever bytes the scanner accepted", which
   is why the manual, PHP-Parser, every IDE and every static analyser
   each copy out the same byte class. That is a language-definition gap,
   not a bug report.

2. Identifiers that render identically are different identifiers. A
   no-break space or a decomposed umlaut inside a name is invisible in
   every editor. Real, but rare: 11 instances in 168,604 files.

3. Case-insensitive lookup folds ASCII only. Stra<U+00DF>e and
   STRA<U+00DF>E are the same class; Stra<U+1E9E>e is not, and Strasse is
   not. That rule is coherent only if identifiers are ASCII.

Taken alone, (2) is thin. Juris made the same point in the other
subthread and he is right; I am not going to inflate it.

Which leaves the question I think has to be answered before this is worth
either of our time:

PHP accepts non-ASCII identifiers today, and not by decision. The manual
says outright that PHP "doesn't support Unicode variable names" and that
they work because of how UTF-8 happens to encode. So either they are a
supported feature, in which case they need a definition and (3) is an
inconsistency in the engine -- or they are not, in which case that ought
to be a decision somebody made rather than a note explaining an accident,
and I will drop this.

I would rather have a clear answer than a favourable one, because
anything else I might propose in this area depends on it.

If your -1 rests on "not supported, and we are not going to build on it",
I would sooner have that now than after writing a patch. It is a
legitimate answer and it settles the question.

Regards,
Luca