# generating code from AST - Thread: https://externals.io/message/98410 - Original archive: https://news-web.php.net/php.internals/98410 - Started by Adam Baratz on 2017-03-06T21:37:05+00:00 - Messages: 10 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #98410 — Adam Baratz — 2017-03-06T21:37:05+00:00 URL: https://externals.io/message/98410#98410 I'm exploring how to automate some basic kinds of refactor operations. One approach I'm considering: - Generate an AST - Rearrange it as needed - Turn it back into userland code Is this something anyone's explored? Thanks, Adam --- ## #98412 — Sara Golemon — 2017-03-06T21:40:45+00:00 URL: https://externals.io/message/98410#98412 · In reply to: #98410 > I'm exploring how to automate some basic kinds of refactor operations. One > approach I'm considering: > > - Generate an AST > - Rearrange it as needed > - Turn it back into userland code > > Is this something anyone's explored? > The engine's internals actually have an ast export method which does this alread (it's used by the assert message generator). astkit makes use of this in an extension to do much of what you describe: https://github.com/sgolemon/astkit -Sara --- ## #98413 — Sara Golemon — 2017-03-06T21:42:43+00:00 URL: https://externals.io/message/98410#98413 · In reply to: #98412 > > > […] > The engine's internals actually have an ast export method which does > this alread (it's used by the assert message generator). astkit > makes use of this in an extension to do much of what you describe: > https://github.com/sgolemon/astkit > Hrmmm, I seem to have not updated the README.md, but it is in there: https://github.com/sgolemon/astkit/blob/master/astkit-node.c#L204-L214 -Sara --- ## #98415 — Sebastian Bergmann — 2017-03-07T05:54:59+00:00 URL: https://externals.io/message/98410#98415 · In reply to: #98412 Am 06.03.2017 um 22:40 schrieb Sara Golemon: > https://github.com/sgolemon/astkit There's also https://github.com/nikic/php-ast. But wouldn't it be great if PHP 7.2 shipped with a built-in, enabled-by-default extension for AST operations? --- ## #98416 — David Walker — 2017-03-07T06:01:43+00:00 URL: https://externals.io/message/98410#98416 · In reply to: #98415 > Am 06.03.2017 um 22:40 schrieb Sara Golemon: > > […] > > There's also https://github.com/nikic/php-ast. > > But wouldn't it be great if PHP 7.2 shipped with a built-in, > enabled-by-default extension for AST operations? I'd be happy if this were to go forward. When/if there are changes to AST, having a tool like these in core would require them to be updated at the same time. Would this concept also want to extend to a VLD-esque extension as well for the same reasons? I'd be receptive for abilities to expose the core internal ast/ops to userland more easily as a core extension However I'm sure there are valid reasons why VLD (or similar) have not been included; and would wonder if those would also end up applying to the AST -- Dave --- ## #98417 — Sebastian Bergmann — 2017-03-07T06:04:12+00:00 URL: https://externals.io/message/98410#98417 · In reply to: #98416 Am 07.03.2017 um 07:01 schrieb David Walker: > When/if there are changes to AST, having a tool like these in core would require them to be updated at the same time. Exactly. > Would this concept also want to extend to a VLD-esque extension as well for > the same reasons? I'd be receptive for abilities to expose the core > internal ast/ops to userland more easily as a core extension Having a bytecode disassembler in core would also be nice, yes. --- ## #98418 — Derick Rethans — 2017-03-07T10:33:36+00:00 URL: https://externals.io/message/98410#98418 · In reply to: #98417 > Am 07.03.2017 um 07:01 schrieb David Walker: > > > […] > > Exactly. > > > […] > > Having a bytecode disassembler in core would also be nice, yes. Because installing an extension is too hard? It sounds like reinventing the wheel. cheers, Derick --- ## #98419 — Rowan Collins — 2017-03-07T19:51:25+00:00 URL: https://externals.io/message/98410#98419 · In reply to: #98418 > > […] > Because installing an extension is too hard? It sounds like reinventing > the wheel. Not necessarily reinventing it, just pre-attaching it to the carriage ;) By which I mean, there's no reason "bundling a VLD-esque extension" couldn't mean "bundling VLD". But as Dave said: > However I'm sure there are valid reasons why VLD (or similar) have not been > included; and would wonder if those would also end up applying to the AST Basically, we're back to the age-old discussion of exactly what are the pros and cons of "bundling" an extension, given the way PHP is packaged by vendors in practice. Regards, -- Rowan Collins [IMSoP] --- ## #98421 — Sebastian Bergmann — 2017-03-08T08:49:56+00:00 URL: https://externals.io/message/98410#98421 · In reply to: #98418 Am 07.03.2017 um 11:33 schrieb Derick Rethans: > Because installing an extension is too hard? No. To ensure that userland functionality that is based on compiler internals (token stream, abstract syntax tree, bytecode) does not fall out of sync with the compiler. We already have an extension that is built-in and enabled-by-default for the token stream. Why not have one for the abstract syntax tree and bytecode representations? --- ## #98423 — Nikita Nefedov — 2017-03-08T12:05:13+00:00 URL: https://externals.io/message/98410#98423 · In reply to: #98421 > > > Am 07.03.2017 um 11:33 schrieb Derick Rethans: > > […] > > No. To ensure that userland functionality that is based on compiler > internals (token stream, abstract syntax tree, bytecode) does not fall out > of sync with the compiler. > > We already have an extension that is built-in and enabled-by-default for > the token stream. Why not have one for the abstract syntax tree and > bytecode representations? To be honest, that’s exactly what we might want to avoid. Last time someone wanted to do actual improvements to our lexing they were stopped because changing tokens would mean breaking code that relies on `token_get_all()` which is a core extension user land function. It might be though that it’s only the problem with lexing and it should not be a problem for AST because it is, ahem, abstract.