# DOMXPath / XSLTProcessor function callbacks - Thread: https://externals.io/message/121286 - Original archive: https://news-web.php.net/php.internals/121286 - Started by Niels Dossche on 2023-10-12T22:39:32+00:00 - Messages: 11 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #121286 — Niels Dossche — 2023-10-12T22:39:32+00:00 URL: https://externals.io/message/121286#121286 Hi internals I'm looking to extend the functionality of calling PHP functions from within the DOMXPath or XSLTProcessor classes. In case you're unfamiliar here's a quick rundown. The DOMXPath class allows you to execute XPath queries on a DOM tree to lookup certain nodes satisfying a filter. PHP allows the user to execute function callbacks within these. For example (from the manual): $xpath->query('//book[php:functionString("substr", title, 0, 3) = "PHP"]'); This will read the title element's text content, call substr on it, and then compare the output against "PHP". You can not only call builtin functions, but also user functions. To be able to call PHP functions, you need to use DOMXPath::registerPhpFunctions() (https://www.php.net/manual/en/domxpath.registerphpfunctions.php). You either pass in `NULL` to allow all functions, or pass in which function names are allowed to be called. Similarly, XSLTProcessor has the same registerPhpFunctions() method. For XSLT it's mostly used for performing arbitrary manipulations on input data. Normally the output of the function is put into the resulting document. So what's the problem? The current system doesn't allow you to call closures or object methods. There are tricks you can do with global variables and global functions to try to work around this, but that's quite cumbersome. There are two feature requests for this on the old bugtracker: - https://bugs.php.net/bug.php?id=38595 - https://bugs.php.net/bug.php?id=49567 It's not hard to implement support for this, the question is just what API we should go with. Based on what I've read, there are at least two obvious options: OPTION 1) Extend registerPHPFunctions() such that you can pass in callables ``` // Adapted from https://bugs.php.net/bug.php?id=38595 $xslt->registerPHPFunctions(array( 'functionblah', // Like we used to 'func2' => fn ($x) => ..., 'func3' => array($obj, 'method'), // etc )); ``` Example: Using php:function("func3") inside XPath/XSLT in this case will result in calling method on $obj. Similarly func2 will call the closure, and functionblah in the snippet just allowlists calling functionblah. It's a backwards compatible solution and a natural extension to the current method. It may be hard to discover this feature compared to having a new API though. Furthermore, once you pass in function names to registerPHPFunctions(), you're restricting what can be called. For example: imagine you want to call both `ucfirst()` and $obj->method(), so you pass in an entry like func3 in the above example. Now you have to pass in ucfirst to registerPHPFunctions() too, because registerPHPFunctions() acts as an allowlist. May be a bit inconvenient. OPTION 2) Add new methods to register / unregister callables This may be the cleaner way to go about it on first sight, but there's a potential BC break when new methods clash in user-defined subclasses. Question here is: what about the interaction with registerPHPFunction? What if both registerPHPFunction() and the register method add something with the same name? What if registerPHPFunction() didn't allowlist a function but the register method added it, may be a bit confusing for users. The interaction may be surprising. Please let me know your thoughts. Cheers Niels --- ## #121306 — tim@bastelstu.be — 2023-10-14T10:30:01+00:00 URL: https://externals.io/message/121286#121306 · In reply to: #121286 Hi > Please let me know your thoughts. What does calling ->registerPHPFunctions() do when it's called more than once? Will the existing allow-list be overwritten or amended? i.e. $xpath->registerPHPFunctions([ 'strtoupper', ]); $xpath->registerPHPFunctions([ 'ucfirst', ]); will I be able to: (a) Call `ucfirst()`, but not `strtoupper()` (b) Call both Best regards Tim Düsterhus --- ## #121308 — Niels Dossche — 2023-10-14T11:54:59+00:00 URL: https://externals.io/message/121286#121308 · In reply to: #121306 Hi Tim > Hi > > > > […] > > What does calling ->registerPHPFunctions() do when it's called more than once? Will the existing allow-list be overwritten or amended? > > i.e. > > $xpath->registerPHPFunctions([ >   'strtoupper', > ]); > $xpath->registerPHPFunctions([ >   'ucfirst', > ]); > > will I be able to: > > (a) Call `ucfirst()`, but not `strtoupper()` > (b) Call both You can call both, it's additive. > > Best regards > Tim Düsterhus Cheers Niels --- ## #121309 — tim@bastelstu.be — 2023-10-14T12:42:18+00:00 URL: https://externals.io/message/121286#121309 · In reply to: #121308 Hi > > […] > > You can call both, it's additive. > Okay, for your suggestion (1) the following would work then? // Register all native PHP functions. $xpath->registerPHPFunctions(null); // Register additional callables. $xpath->registerPHPFunctions([ 'staticMethod' => ClassWith::staticMethod(...), 'classWithMagicInvoke' => new ClassWithMagicInvoke(), ]); I would not find that usage to be too unreasonable. Especially since allow-listing *all* of PHP's global functions would be pretty questionable (looking at eval() and all functions with side-effects). Best regards Tim Düsterhus --- ## #121310 — Niels Dossche — 2023-10-14T15:14:20+00:00 URL: https://externals.io/message/121286#121310 · In reply to: #121309 Hi Tim > Hi > > > > […] > > Okay, for your suggestion (1) the following would work then? > > // Register all native PHP functions. > $xpath->registerPHPFunctions(null); > > // Register additional callables. > $xpath->registerPHPFunctions([ >   'staticMethod' => ClassWith::staticMethod(...), >   'classWithMagicInvoke' => new ClassWithMagicInvoke(), > ]); Yes this would work. > > I would not find that usage to be too unreasonable. Especially since allow-listing *all* of PHP's global functions would be pretty questionable (looking at eval() and all functions with side-effects). That's a fair argument, allowlisting all would indeed be not very defensive programming. Thanks for your input. > > Best regards > Tim Düsterhus Cheers Niels --- ## #121321 — Marc — 2023-10-15T10:53:39+00:00 URL: https://externals.io/message/121286#121321 · In reply to: #121308 Hi, > Hi Tim > > > > […] > You can call both, it's additive. As it's not clear what these functions do wouldn't it make more sense to introduce new / more clear functions and deprecate the old? Like: ->addXPathCallable(string $alias, callable $fn); ->getXPathCallables(): array ->removeXPathCallable(string $alias); And is it really necessary to be able to register all functions? Also what would happen in this case with functions included later? Would they also be callable or only the functions known at the point of registering? > > […] > Cheers > Niels Best, Marc --- ## #121323 — Niels Dossche — 2023-10-15T12:04:53+00:00 URL: https://externals.io/message/121286#121323 · In reply to: #121321 Hi Marc > Hi, > > > > […] > > As it's not clear what these functions do wouldn't it make more sense to introduce new / more clear functions and deprecate the old? > > Like: > > ->addXPathCallable(string $alias, callable $fn); > ->getXPathCallables(): array > ->removeXPathCallable(string $alias); This would be the cleaner option. The question here is how they would interact with each other and the existing registerPhpFunctions(): - addXPathCallable() when the alias already exists: I guess throw a ValueError - remove and get should likely also act on the functions registered via registerPhpFunctions() - What should happen when we call registerPhpFunctions() followed by addXPathCallable("foo", function ($...) {...}); Probably both global functions and foo should be callable. But what should happen then when there is also a global function foo, which one should be called? Probably the explicitly registered callable? What's currently also weird is that in the following snippet: ``` $xpath->registerPhpFunctions(); $xpath->registerPhpFunctions("ucfirst"); ``` then only `ucfirst()` would be callable, which is surprising especially considering that calling $xpath->registerPhpFunctions("otherfunction") afterwards will make both ucfirst and otherfunction callable. > > And is it really necessary to be able to register all functions? Also what would happen in this case with functions included later? Would they also be callable or only the functions known at the point of registering? > For the first question: unlikely. For the second question: they would also be callable. > > […] > > > Best, > Marc > Cheers Niels --- ## #121329 — Frederik Bosch — 2023-10-15T19:37:41+00:00 URL: https://externals.io/message/121286#121329 · In reply to: #121286 Dear Niels, First of all, thanks for all your hard work already on the DOM and SimpleXML extensions. I have been following your work in PHP-SRC, great! I am the author of this XSL 2.0 Transpiler in PHP package (https://github.com/genkgo/xsl). It is indeed possible to use workarounds for closures or object methods. I am relying on them in my package. My suggestion for the future would be to add the following method. public XSLTProcessor::registerFunctionsNS(string $namespace, array|ArrayAccess $functions): void Then a user can register functions like this. $xsltProcessorOrDomXpath->registerFunctionsNS('urn:my.namespace', array('upper-case', 'strtoupper', 'pow' => fn ($a, $b) => $a[0]->textContent ** $b[0]->textContent, 'other' => [$obj, 'method']); The registered functions should use the same methodology as php:function(). Hence, string casting of arguments is something the library user should do. I would leave registerPHPFunctions as is, and maybe discourage it in favor of the method above. What if both are called? I think it would be most clear if the registerFunctionsNS method throws InvalidArgumentException when http://php.net/xsl or http://php.net/xpath is passed as namespace. Cheers, Frederik > I'm looking to extend the functionality of calling PHP functions from within the DOMXPath or XSLTProcessor classes. > > In case you're unfamiliar here's a quick rundown. > The DOMXPath class allows you to execute XPath queries on a DOM tree to lookup certain nodes satisfying a filter. > PHP allows the user to execute function callbacks within these. For example (from the manual): > $xpath->query('//book[php:functionString("substr", title, 0, 3) = "PHP"]'); > This will read the title element's text content, call substr on it, and then compare the output against "PHP". > You can not only call builtin functions, but also user functions. > > To be able to call PHP functions, you need to use DOMXPath::registerPhpFunctions() (https://www.php.net/manual/en/domxpath.registerphpfunctions.php). > You either pass in `NULL` to allow all functions, or pass in which function names are allowed to be called. > > Similarly, XSLTProcessor has the same registerPhpFunctions() method. > For XSLT it's mostly used for performing arbitrary manipulations on input data. > Normally the output of the function is put into the resulting document. > > > So what's the problem? > The current system doesn't allow you to call closures or object methods. > There are tricks you can do with global variables and global functions to try to work around this, but that's quite cumbersome. > > There are two feature requests for this on the old bugtracker: > - https://bugs.php.net/bug.php?id=38595 > - https://bugs.php.net/bug.php?id=49567 > > It's not hard to implement support for this, the question is just what API we should go with. > Based on what I've read, there are at least two obvious options: > > > OPTION 1) Extend registerPHPFunctions() such that you can pass in callables > > ``` > // Adapted from https://bugs.php.net/bug.php?id=38595 > $xslt->registerPHPFunctions(array( > 'functionblah', // Like we used to > 'func2' => fn ($x) => ..., > 'func3' => array($obj, 'method'), // etc > )); > ``` > > Example: Using php:function("func3") inside XPath/XSLT in this case will result in calling method on $obj. > Similarly func2 will call the closure, and functionblah in the snippet just allowlists calling functionblah. > > It's a backwards compatible solution and a natural extension to the current method. > It may be hard to discover this feature compared to having a new API though. > > Furthermore, once you pass in function names to registerPHPFunctions(), you're restricting what can be called. > For example: imagine you want to call both `ucfirst()` and $obj->method(), so you pass in an entry like func3 in the above example. > Now you have to pass in ucfirst to registerPHPFunctions() too, because registerPHPFunctions() acts as an allowlist. May be a bit inconvenient. > > > OPTION 2) Add new methods to register / unregister callables > > This may be the cleaner way to go about it on first sight, but there's a potential BC break when new methods clash in user-defined subclasses. > > Question here is: what about the interaction with registerPHPFunction? > What if both registerPHPFunction() and the register method add something with the same name? > What if registerPHPFunction() didn't allowlist a function but the register method added it, may be a bit confusing for users. > The interaction may be surprising. > > > > Please let me know your thoughts. > > Cheers > Niels --- ## #121340 — Niels Dossche — 2023-10-16T21:19:02+00:00 URL: https://externals.io/message/121286#121340 · In reply to: #121329 Hi Frederik Sorry for the resend... I accidentally replied to you only without including the list the first time. > Dear Niels, > > First of all, thanks for all your hard work already on the DOM and SimpleXML extensions. I have been following your work in PHP-SRC, great! I am the author of this XSL 2.0 Transpiler in PHP package (https://github.com/genkgo/xsl). It is indeed possible to use workarounds for closures or object methods. I am relying on them in my package. > > My suggestion for the future would be to add the following method. > > public XSLTProcessor::registerFunctionsNS(string $namespace, array|ArrayAccess $functions): void > > Then a user can register functions like this. > > $xsltProcessorOrDomXpath->registerFunctionsNS('urn:my.namespace', array('upper-case', 'strtoupper', 'pow' => fn ($a, $b) => $a[0]->textContent ** $b[0]->textContent, 'other' => [$obj, 'method']); > Interesting suggestion. So you want to be able to use something like `my.namespace:function(...)` as I understand it. I'm not sure that adding this is that much more beneficial though (complexity vs benefit trade-off). I assume this is motivated by the fact that you can then use third party libraries while having to worry less about name clashes? Let's say we add non-namespace `registerFunction(string $name, callable $callback): void`, you can then still use a convention of using a prefix, thus _kinda_ achieving the same. In any case, this is going to be hard to support in combination with the underlying library (libxslt). That's because the function namespace registration is process-wide, so this cannot be changed at runtime and certainly not for ZTS SAPIs. > The registered functions should use the same methodology as php:function(). Hence, string casting of arguments is something the library user should do. I would leave registerPHPFunctions as is, and maybe discourage it in favor of the method above. What if both are called? I think it would be most clear if the registerFunctionsNS method throws InvalidArgumentException when http://php.net/xsl or http://php.net/xpath is passed as namespace. > > Cheers, > Frederik Cheers Niels > > > > > […] --- ## #121699 — Frederik Bosch — 2023-11-16T19:34:01+00:00 URL: https://externals.io/message/121286#121699 · In reply to: #121340 Hi Niels, > Sorry for the resend... I accidentally replied to you only without including the list the first time. > > > > […] > Interesting suggestion. So you want to be able to use something like `my.namespace:function(...)` as I understand it. Sorry I missed these messages completely. Unfortunately internals ends up in my spam. Sorry that I did not reply. What I want to use is this. $processor->registerFunctionNS(     'urn:my.ns',     [         'round-up' => fn ($arg1) => ceil($arg1),         'round-down' => fn ($arg1) => floor($arg1)     ] );         While it follows the logic of XSL setParameter which also allows a namespace and DOM setAttributeNS, createElementNS et al methods, it also allows you to keep registerPhpFunctions method as is. Because basically, it can then become shorthand for registerFunctionNS (simplified): $processor->registerFunctionNS(     'http://php.net/xsl',     [         'functionString' => fn ($func, ...$args) => call_user_func_array($func, $args),         'function' => fn ($func, ...$args) => call_user_func_array($func, $args)     ] ); > I'm not sure that adding this is that much more beneficial though (complexity vs benefit trade-off). > I assume this is motivated by the fact that you can then use third party libraries while having to worry less about name clashes? It allows me to use namespaces as intended. > Let's say we add non-namespace `registerFunction(string $name, callable $callback): void`, you can then still use a convention of using a prefix, thus _kinda_ achieving the same. I don't understand this one. Can I use a prefix without namespace? > > In any case, this is going to be hard to support in combination with the underlying library (libxslt). > That's because the function namespace registration is process-wide, so this cannot be changed at runtime and certainly not for ZTS SAPIs. So namespaces http://php.net/xsl and http://php.net/xpath are already process-wide, right? I do not see why more custom namespaces then would be a problem, but I have very little knowledge of libxslt library. > > > […] > Cheers > Niels > > > […] Again, whatever solution you choose, I am happy with all your contributions. Cheers, Frederik --- ## #121716 — Niels Dossche — 2023-11-19T18:26:15+00:00 URL: https://externals.io/message/121286#121716 · In reply to: #121699 Hi Frederik > Hi Niels, > > > > […] > > Sorry I missed these messages completely. Unfortunately internals ends up in my spam. Sorry that I did not reply. What I want to use is this. > > $processor->registerFunctionNS( >     'urn:my.ns', >     [ >         'round-up' => fn ($arg1) => ceil($arg1), >         'round-down' => fn ($arg1) => floor($arg1) >     ] > ); > > >     >     > > > While it follows the logic of XSL setParameter which also allows a namespace and DOM setAttributeNS, createElementNS et al methods, it also allows you to keep registerPhpFunctions method as is. Because basically, it can then become shorthand for registerFunctionNS (simplified): > > $processor->registerFunctionNS( >     'http://php.net/xsl', >     [ >         'functionString' => fn ($func, ...$args) => call_user_func_array($func, $args), >         'function' => fn ($func, ...$args) => call_user_func_array($func, $args) >     ] > ); > Ah okay I get now what you mean. This would be a further nice-to-have as well. And as I'm updating the API anyway I think it's reasonable to add this to the RFC. I'm currently busy with something else but I'll be able to work again on this towards the end of the month probably. > > […] > It allows me to use namespaces as intended. > > […] > I don't understand this one. Can I use a prefix without namespace? No, this was just me misunderstanding :) > > […] > So namespaces http://php.net/xsl and http://php.net/xpath are already process-wide, right? I do not see why more custom namespaces then would be a problem, but I have very little knowledge of libxslt library. When I wrote this I only found APIs for process-wide registrations. This would be bad in threaded webserver contexts as request data leaks between requests. I looked again and I now found the APIs that we can use for request-wide registrations instead of process-wide, which doesn't have that problem. > > […] Kind regards Niels > > […] > > Again, whatever solution you choose, I am happy with all your contributions. > > Cheers, > Frederik