Hi Internals
The main gist:
I suggest to introduce an official type-checker / static analyser /
preprocessor / transpiler for PHP, somewhat similar to TypeScript in
the JavaScript world, which will allow to introduce the features
otherwise impossible for the PHP world, at the cost of including an
additional transpilation step.
The reasoning:
You all know there are a lot of problems with the language, especially
large gaps with the current type system, which have large community
demand to overcome, but still cannot be solved due to the paradigm
chosen by PHP. And the paradigm is to be statically typed (to some
extent), but dynamically checked language, which limits the number of
available options due to performance reasons.
The introduction of static types have gradually eliminated the need of
using external means of communicating the types (phpdoc @param and
@return annotations), replacing them with native language constructs,
making the language more expressive and enterprise-ready. However,
there are a lot of things still missing, like types of arrays,
including lists, hash-maps and structure-like arrays, and of course
generic classes and functions. These are still covered nowadays with
external means like phpdoc annotations and static analysers / type
checkers to validate them. This leaves PHP in the odd mixture of
native types and externally validated annotations via comments.
Paradigm of a dynamically checked language also leaves out the problem
of type-checking before running the code. Even though the types are
defined statically, you still have to physically run the code, with
all possible execution paths, to catch a type error. To avoid that you
still would have to use an external typechecker / static analyser,
even if PHP could cover 100% of type demand natively.
Python solves this problem nicely I think. It is dynamically typed,
but it allows a special syntax for static types and also has a
separate static type checker. I think this is really a good approach
and it would be great if PHP followed this path for the beginning, but
we have what we have already.
There were attempts to create preprocessors for PHP (like PHP++ IIRC),
which allowed features like generics, but they did not gain enough
popularity, weren't baked by large enough companies, didn't make their
way to major IDEs support. I believe the only solution is to make an
official one.
Therefore, I suggest the following thing:
- Introduce a new #declare directive which will mark the file as
requiring pre-processing. - Let PHP introduce special features that only work in such
"pre-processing required" files, such as typed lists and generics. - Implement a static type-checker in PHP that will verify those typed
lists and generics are indeed used in a valid way. - Make the PHP preprocessor output the "normal" PHP file somewhere,
which has list types and generics erased. I think something like that
was suggested earlier many times.
As an alternative to a new #declare feature + validation +
transpilation in "plain old PHP", the Python way of just ignoring
statically checked stuff could be applied. This approach though looks
odd to me in PHP world, because having some stuff checked in runtime,
and some only statically may confuse developers a lot.
Basically, this will split PHP in two languages, one being superset of
other, but checked statically. I realize this might be much of a
paradigm shift, it may require lots of work if accepted (I can only
dream of it though), but still I want to share my thoughts with you
and propose something for consideration.
Regards,
Illia / someniatko
I'd like to also emphasize that the concept of compilation and static
type-checking / analysis is not foreign to the PHP community. Major
frameworks like Symfony already have a concept of compiling e.g. a DI
container, and static analyzers like Psalm, PHPStan and others are
actively used.
I'd like to also emphasize that the concept of compilation and static
type-checking / analysis is not foreign to the PHP community. Major
frameworks like Symfony already have a concept of compiling e.g. a DI
container, and static analyzers like Psalm, PHPStan and others are
actively used.--
To unsubscribe, visit: https://www.php.net/unsub.php
I think the idea overall is pretty solid and could get a few community
supporters. But the devil is in the details.
-
Making it official is a trade-off. It's the highest chance of widespread
adoption and the heaviest burden on PHP Internals. -
Who's going to fund it, develop it, maintain it? It's a significant
project and a heavy burden. -
What's the MVP? How do we measure usefulness, adoption and maintenance
complexity? -
Is it an aborttable idea? If things go south and can't be supported
anymore, how bad for PHP adopters would it be? -
How would voting on the spec be like? Complete years-work-hours spec or
small incremental functionalities? -
Is versioning tightly coupled with PHP version or does it have its own
version support system targeting multiple php versions at the same time?
Overall, I think PHP has been constantly losing popularity in favor of
Typescript and a transpiler could create an exciting solution for some of
PHP's problems, but it does not seem like a pet project.
Hi,
These all are valid concerns! I cannot speak about management and cost
issues though, as I am not an active PHP dev team member.
- What's the MVP? How do we measure usefulness, adoption and maintenance complexity?
As a shot in the dark, the first step could be implementation of typed
lists in my opinion. Do not implement full-blown static analyser nor
the type checker for each possible case, just only check the usage of
typed lists within the pre-processed files. Assume that whatever comes
from the non-preprocessed ones is correct. Make only simple naive
assumptions for all internal array_ functions as well as basic array
operations. If there is some runtime magic going on with constructing
the list, just fallback to infering the type as a simple array. For
such cases, allow a special attribute which will tell the type checker
this variable / property etc will indeed contain the typed list, like
a @var annotation for IDEs / static analysers nowadays.
The preprocessor / type checker should not aim to 100% guarantee the
correct types will be in the runtime: TypeScript does this neither.
Rowan has mentioned Dart language in a parallel thread, which makes
most of the checks statically, but for some cases the compiler still
adds runtime checks. Rowan warned that PHP might have a lot of such
cases possible though. But I still think this also might be a good
idea for MVP.
- Is it an aborttable idea? If things go south and can't be supported anymore, how bad for PHP adopters would it be?
I don't think any of PHP new features that go live are abortable. This
affects not only my suggestion, but also everything what has been done
for PHP in past. Once a feature is in the language and released to the
public, you cannot revert it. There must be a long multi-year process
preceded with a deprecation at least. A concept of experimental
features was suggested earlier, but this is completely different
topic, so I believe we should not go into that rabbit hole here.
Overall, I consider this question orthogonal (or irrelevant if you
prefer) to the original topic, let's take as an axiom that there is no
way back.
- How would voting on the spec be like? Complete years-work-hours spec or small incremental functionalities?
Small incremental functionalities. Gradually more stuff could be
proven statically, more complex cases covered, and while they are not,
an attribute / static assertion could be used. Maybe some of such
assertions could be allowed to be preserved in runtime, at the cost of
some performance penalty, IDK. But I believe we should leave such
choice to developers, whether they want the checks in runtime as well.
- Is versioning tightly coupled with PHP version or does it have its own version support system targeting multiple php versions at the same time?
I strongly believe it must go hand in hand with the PHP version, and
not as a separate tool, because I see it as a feature of a PHP
language itself.
Overall, I think PHP has been constantly losing popularity in favor of Typescript and a transpiler could create an exciting solution for some of PHP's problems, but it does not seem like a pet project.
Sure, it should be considered seriously and requires some upfront
planning. Though, well, PHP itself started as a pet project :D
I suggest to introduce an official type-checker / static analyser /
preprocessor / transpiler for PHP, somewhat similar to TypeScript in
the JavaScript world, which will allow to introduce the features
otherwise impossible for the PHP world, at the cost of including an
additional transpilation step.
This is something I have thought about a lot over the years, and in some
ways, it is very appealing; but it does have some problems.
One of the biggest is the concept of "type erasure": in a pre-processed
or transpiled language, features like Reflection are crippled by not
knowing the full information present in the source code. More
importantly, dynamic language features - of which PHP has many - also
have no access to the type information. That means that a List<Dog> can
sometimes end up with a Cat in it at run-time, because a dynamic call
doesn't know that's a problem.
I think it's interesting to look around at different approaches to this
elsewhere:
You mention Python, which I think has probably the worst approach: it
has official syntax which appears to convey type information, but
doesn't actually provide any guarantees if you forget to run an external
tool. Outside of that tool, the type information is just another type of
comment, just less clearly so than the annotations understood by tools
like PhpStan and Psalm in today's PHP.
Another approach is TypeScript, which is essentially a complete language
in its own right, with JavaScript as a compilation target. As I
understand it, you can call JS code from inside a TS project (and vice
versa, once the TS is transpiled), but that won't always be seamless, so
the two languages will tend to end up with separate ecosystems of
libraries and tooling. Implementing a new language also takes time, and
ultimately money - in this case, Microsoft's money.
A similar project, closer to PHP's heart, is Hack; the main difference
being that rather than transpiling to PHP, it initially aimed to compile
both languages to a common bytecode, to be executed in a shared VM.
However, it turns out that as well as adding features on top of PHP,
it was necessary for Hack to remove features of PHP that weren't
compatible with a static typing system, and eventually the
interoperability was dropped altogether. It also has money behind it, in
this case from Facebook.
Dart, which was written completely from scratch (the money this time is
Google's), has a type system checked mostly at compile-time, but with
certain run-time assertions inserted where the compiler can't prove
soundness. The approach seems appealing, but PHP would probably end up
with a lot of such cases - all the features that the Hack team decided
to remove - leading to poor performance and confusing developer experience.
I don't have a conclusion to this e-mail, I just thought I'd throw out
these thoughts.
Regards,
--
Rowan Tommins
[IMSoP]
2023-02-02 15:19 GMT+01:00, someniatko someniatko@gmail.com:
Hi Internals
The main gist:
I suggest to introduce an official type-checker / static analyser /
preprocessor / transpiler for PHP, somewhat similar to TypeScript in
the JavaScript world, which will allow to introduce the features
otherwise impossible for the PHP world, at the cost of including an
additional transpilation step.The reasoning:
You all know there are a lot of problems with the language, especially
large gaps with the current type system, which have large community
demand to overcome, but still cannot be solved due to the paradigm
chosen by PHP. And the paradigm is to be statically typed (to some
extent), but dynamically checked language, which limits the number of
available options due to performance reasons.The introduction of static types have gradually eliminated the need of
using external means of communicating the types (phpdoc @param and
@return annotations), replacing them with native language constructs,
making the language more expressive and enterprise-ready. However,
there are a lot of things still missing, like types of arrays,
including lists, hash-maps and structure-like arrays, and of course
generic classes and functions. These are still covered nowadays with
external means like phpdoc annotations and static analysers / type
checkers to validate them. This leaves PHP in the odd mixture of
native types and externally validated annotations via comments.Paradigm of a dynamically checked language also leaves out the problem
of type-checking before running the code. Even though the types are
defined statically, you still have to physically run the code, with
all possible execution paths, to catch a type error. To avoid that you
still would have to use an external typechecker / static analyser,
even if PHP could cover 100% of type demand natively.Python solves this problem nicely I think. It is dynamically typed,
but it allows a special syntax for static types and also has a
separate static type checker. I think this is really a good approach
and it would be great if PHP followed this path for the beginning, but
we have what we have already.There were attempts to create preprocessors for PHP (like PHP++ IIRC),
which allowed features like generics, but they did not gain enough
popularity, weren't baked by large enough companies, didn't make their
way to major IDEs support. I believe the only solution is to make an
official one.Therefore, I suggest the following thing:
- Introduce a new #declare directive which will mark the file as
requiring pre-processing.- Let PHP introduce special features that only work in such
"pre-processing required" files, such as typed lists and generics.- Implement a static type-checker in PHP that will verify those typed
lists and generics are indeed used in a valid way.
No need to waste resources implementing a new one when we already have
two very competent already: Psalm and Phpstan? Or why is those not
enough for your use-case?
"The greatest enemy to a perfect solution is a good enough solution
already in place." ;)
Olle
2023-02-02 15:19 GMT+01:00, someniatko someniatko@gmail.com:
Hi Internals
The main gist:
I suggest to introduce an official type-checker / static analyser /
preprocessor / transpiler for PHP, somewhat similar to TypeScript in
the JavaScript world, which will allow to introduce the features
otherwise impossible for the PHP world, at the cost of including an
additional transpilation step.The reasoning:
You all know there are a lot of problems with the language, especially
large gaps with the current type system, which have large community
demand to overcome, but still cannot be solved due to the paradigm
chosen by PHP. And the paradigm is to be statically typed (to some
extent), but dynamically checked language, which limits the number of
available options due to performance reasons.The introduction of static types have gradually eliminated the need of
using external means of communicating the types (phpdoc @param and
@return annotations), replacing them with native language constructs,
making the language more expressive and enterprise-ready. However,
there are a lot of things still missing, like types of arrays,
including lists, hash-maps and structure-like arrays, and of course
generic classes and functions. These are still covered nowadays with
external means like phpdoc annotations and static analysers / type
checkers to validate them. This leaves PHP in the odd mixture of
native types and externally validated annotations via comments.Paradigm of a dynamically checked language also leaves out the problem
of type-checking before running the code. Even though the types are
defined statically, you still have to physically run the code, with
all possible execution paths, to catch a type error. To avoid that you
still would have to use an external typechecker / static analyser,
even if PHP could cover 100% of type demand natively.Python solves this problem nicely I think. It is dynamically typed,
but it allows a special syntax for static types and also has a
separate static type checker. I think this is really a good approach
and it would be great if PHP followed this path for the beginning, but
we have what we have already.There were attempts to create preprocessors for PHP (like PHP++ IIRC),
which allowed features like generics, but they did not gain enough
popularity, weren't baked by large enough companies, didn't make their
way to major IDEs support. I believe the only solution is to make an
official one.Therefore, I suggest the following thing:
- Introduce a new #declare directive which will mark the file as
requiring pre-processing.- Let PHP introduce special features that only work in such
"pre-processing required" files, such as typed lists and generics.- Implement a static type-checker in PHP that will verify those typed
lists and generics are indeed used in a valid way.No need to waste resources implementing a new one when we already have
two very competent already: Psalm and Phpstan? Or why is those not
enough for your use-case?"The greatest enemy to a perfect solution is a good enough solution
already in place." ;)Olle
--
To unsubscribe, visit: https://www.php.net/unsub.php
Why not follow the steps of other projects that have been created
independently from the core and,
at some point, gained enough support/traction that were incorporated into
it?
2023-02-03 17:38 GMT+01:00, Flávio Heleno flaviohbatista@gmail.com:
On Fri, Feb 3, 2023 at 6:19 AM Olle Härstedt olleharstedt@gmail.com
wrote:2023-02-02 15:19 GMT+01:00, someniatko someniatko@gmail.com:
Hi Internals
The main gist:
I suggest to introduce an official type-checker / static analyser /
preprocessor / transpiler for PHP, somewhat similar to TypeScript in
the JavaScript world, which will allow to introduce the features
otherwise impossible for the PHP world, at the cost of including an
additional transpilation step.The reasoning:
You all know there are a lot of problems with the language, especially
large gaps with the current type system, which have large community
demand to overcome, but still cannot be solved due to the paradigm
chosen by PHP. And the paradigm is to be statically typed (to some
extent), but dynamically checked language, which limits the number of
available options due to performance reasons.The introduction of static types have gradually eliminated the need of
using external means of communicating the types (phpdoc @param and
@return annotations), replacing them with native language constructs,
making the language more expressive and enterprise-ready. However,
there are a lot of things still missing, like types of arrays,
including lists, hash-maps and structure-like arrays, and of course
generic classes and functions. These are still covered nowadays with
external means like phpdoc annotations and static analysers / type
checkers to validate them. This leaves PHP in the odd mixture of
native types and externally validated annotations via comments.Paradigm of a dynamically checked language also leaves out the problem
of type-checking before running the code. Even though the types are
defined statically, you still have to physically run the code, with
all possible execution paths, to catch a type error. To avoid that you
still would have to use an external typechecker / static analyser,
even if PHP could cover 100% of type demand natively.Python solves this problem nicely I think. It is dynamically typed,
but it allows a special syntax for static types and also has a
separate static type checker. I think this is really a good approach
and it would be great if PHP followed this path for the beginning, but
we have what we have already.There were attempts to create preprocessors for PHP (like PHP++ IIRC),
which allowed features like generics, but they did not gain enough
popularity, weren't baked by large enough companies, didn't make their
way to major IDEs support. I believe the only solution is to make an
official one.Therefore, I suggest the following thing:
- Introduce a new #declare directive which will mark the file as
requiring pre-processing.- Let PHP introduce special features that only work in such
"pre-processing required" files, such as typed lists and generics.- Implement a static type-checker in PHP that will verify those typed
lists and generics are indeed used in a valid way.No need to waste resources implementing a new one when we already have
two very competent already: Psalm and Phpstan? Or why is those not
enough for your use-case?"The greatest enemy to a perfect solution is a good enough solution
already in place." ;)Olle
--
To unsubscribe, visit: https://www.php.net/unsub.php
Why not follow the steps of other projects that have been created
independently from the core and,
at some point, gained enough support/traction that were incorporated into
it?
You'd have to rewrite Psalm or Phpstan in C if you want it to be part
of php-src. Or write a completely new project.
Olle
You'd have to rewrite Psalm or Phpstan in C if you want it to be part
of php-src. Or write a completely new project.
I don't think that's necessarily true. There are already tools like run-tests written in PHP and shipped with php-src, PECL is written in PHP, and there have even been suggestions that rewriting some included functions in PHP would be a good long-term goal.
The far larger concerns are governance, resources, and release cycles.
Regards,
--
Rowan Tommins
[IMSoP]
Hi,
why not using the existing Typescript programming language and transpile
it to PHP? :)
There are community examples for doing that:
"ts2php"
https://www.npmjs.com/package/ts2php
Maybe creating an own PHP transpiler for Typescript written in pure PHP
(community friendly) would be an alternative of creating a complete new
high ievel language developed in C-programming langage.
Best regards,
Thomas
Am 03.02.2023 20:17 schrieb Rowan Tommins:
On 3 February 2023 18:30:00 GMT, "Olle Härstedt"
olleharstedt@gmail.com wrote:You'd have to rewrite Psalm or Phpstan in C if you want it to be part
of php-src. Or write a completely new project.I don't think that's necessarily true. There are already tools like
run-tests written in PHP and shipped with php-src, PECL is written in
PHP, and there have even been suggestions that rewriting some included
functions in PHP would be a good long-term goal.The far larger concerns are governance, resources, and release cycles.
Regards,
--
Rowan Tommins
[IMSoP]
Hi,
why not using the existing Typescript programming language and transpile it to PHP? :)
There are community examples for doing that:
"ts2php"
https://www.npmjs.com/package/ts2php
I can think of a few reasons.
Firstly, TypeScript is a language designed with a very specific purpose: to build a statically typed language on top of the existing functionality of JavaScript. Although some of the decisions that follow from that would also aid compatibility with PHP, some would lead to serious mismatches - for example, JavaScript has an entirely different object model, with prototypes rather than classes [1], and methods more closely resembling closures bound to the object.
Secondly, designing the language is only half the battle, probably less. It requires expertise to do well, but the bulk of the time is spent implementing that design. It can be deceptively easy to implement a proof of concept that can run some carefully chosen sample code, but actually making something production ready is a much bigger effort. It doesn't surprise me at all that the project you linked had some initial beta releases, and then went quiet for several years.
Thirdly, if this approach made sense, TypeScript wouldn't need to exist - Microsoft could have written tooling to translate C# to JavaScript instead. The reason languages like TypeScript are created is specifically to bridge existing code and existing knowledge from an existing language. That might mean running in the same run-time as the older language, or just being similar enough to allow a supervised one-off conversion. For PHP, that's the thinking behind Hack (from HHVM), Zephir (from the Phalcon framework), and more recently Ara (https://ara-lang.io/).
To succeed, a new language needs a rare combination of committed resources (which generally means commercial backing to pay multiple full time developers), plus a critical mass of users. Zephir lacked both; Hack had backing inside Facebook, but didn't gain traction outside it; I wish Saif luck with Ara, but I'm not holding my breath.
[1] Yes, there's now a "class" keyword, but it's just syntactic sugar; the fundamental inheritance model still consists of prototype chains.
--
Rowan Tommins
[IMSoP]