Hello internals@,
I have an (exotic) case where I need to be able to get the
filedescriptor from a previously opened stream (via stream_socket_pair)
to hand over to the child process, which needs to reexecute.
I saw that the php:// wrapper supports php://fd/X syntax, so I reckon
there should be a way to extract the information as well from a stream
as well. Unfortunately I don't seem to be able to find it.
Can someone point me in the right direction?
Sincerely,
Martijn van Duren
Hi Martijn
2017-07-06 11:12 GMT+02:00 Martijn van Duren php@list.imperialat.at:
Hello internals@,
I have an (exotic) case where I need to be able to get the
filedescriptor from a previously opened stream (via stream_socket_pair)
to hand over to the child process, which needs to reexecute.I saw that the php:// wrapper supports php://fd/X syntax, so I reckon
there should be a way to extract the information as well from a stream
as well. Unfortunately I don't seem to be able to find it.Can someone point me in the right direction?
It seems like this is not exposed at all by PHP.
I have CC'ed Sara as one of the old school streams persons, maybe she
has some insights why it is not exposed or if it is worth exposing
fileno() in 7.2+
--
regards,
Kalle Sommer Nielsen
kalle@php.net
2017-07-06 11:12 GMT+02:00 Martijn van Duren php@list.imperialat.at:
I have an (exotic) case where I need to be able to get the
filedescriptor from a previously opened stream (via stream_socket_pair)
to hand over to the child process, which needs to reexecute.
"hand over to a child process"
Child as in pcntl_fork()
? Or a subprocess via exec()
et. al.?
For the former, the variable should (I think) carry over.
For the latter, the stream number is going to be a meaningless value
to the subprocess. ((e.g. stdin (fd=0) for the parent is a different
source than stdin for the child, despite being the same number)).
I'm not sure of your exact use case, but you might be able to use
proc_open to spawn the child and supply the socket in the descriptor
spec which will (I think, iirc) bind the internal socket to a new pipe
between the parent and child. Though it's possible you'll need to
maintain a proxy loop in the parent.
40% of the above is guesswork and conjecture, but maybe it'll spawn
the right question.
-Sara
On Thu, Jul 6, 2017 at 10:53 AM, Kalle Sommer Nielsen kalle@php.net
wrote:2017-07-06 11:12 GMT+02:00 Martijn van Duren php@list.imperialat.at:
I have an (exotic) case where I need to be able to get the
filedescriptor from a previously opened stream (via stream_socket_pair)
to hand over to the child process, which needs to reexecute."hand over to a child process"
Child as inpcntl_fork()
? Or a subprocess viaexec()
et. al.?For the former, the variable should (I think) carry over.
For the latter, the stream number is going to be a meaningless value
to the subprocess. ((e.g. stdin (fd=0) for the parent is a different
source than stdin for the child, despite being the same number)).I'm not sure of your exact use case, but you might be able to use
proc_open to spawn the child and supply the socket in the descriptor
spec which will (I think, iirc) bind the internal socket to a new pipe
between the parent and child. Though it's possible you'll need to
maintain a proxy loop in the parent.40% of the above is guesswork and conjecture, but maybe it'll spawn
the right question.
There's a (I think undocumented) feature that allow transferring sockets to
other processes with the socket extension.
https://github.com/amphp/aerys/commit/40fae01e4f5f82570a0bc3cb8ebbf1fee36dbb0b
Bob implemented that, so maybe he can help you.
But what's your exact use case?
Regards, Niklas
On Thu, Jul 6, 2017 at 10:53 AM, Kalle Sommer Nielsen kalle@php.net
wrote:2017-07-06 11:12 GMT+02:00 Martijn van Duren php@list.imperialat.at:
I have an (exotic) case where I need to be able to get the
filedescriptor from a previously opened stream (via stream_socket_pair)
to hand over to the child process, which needs to reexecute."hand over to a child process"
Child as inpcntl_fork()
? Or a subprocess viaexec()
et. al.?
The latter.
For the former, the variable should (I think) carry over.
For the latter, the stream number is going to be a meaningless value
to the subprocess. ((e.g. stdin (fd=0) for the parent is a different
source than stdin for the child, despite being the same number)).
Not entirely, when I use stream_socket_pair I get 2 filedescriptors
and two resources with possible 2 different IDs.
E.g. fd 5 and 6 while the resources have id 8 and 9.
When I do the fork I close one end in the parent and one end in the
child. If I exec into another program I want to be able to say:
we can communicate over this fd, so that the child program can hook
into that specific file descriptor.
I'm not sure of your exact use case, but you might be able to use
proc_open to spawn the child and supply the socket in the descriptor
spec which will (I think, iirc) bind the internal socket to a new pipe
between the parent and child. Though it's possible you'll need to
maintain a proxy loop in the parent.
My use case is to set up a cluster of processes each with their own task
and permission sets. These processes need to be able to communicate with
each other over socket pairs, not just between the parent and the child,
but also between children.
Although the proc_open does solve some problems, it doesn't solve them
all. E.g. I can't create socket pairs between children via this options,
and socketpairs opened to the proc_open would be left dangling.
Hence I want to set up all my socket pairs, pcntl_fork the child
process, close the socket pair ends that are not owned by the child
and execute the new process and specify the fds via commandline
arguments.
For processes that execute a new PHP process I would also prefer to wrap
an existing fd in a stream (similar to fdopen in C), but I can live with
the dup-ing of the file-descriptor in fopen(php://fd/N, X).
So the end result would (somewhat) look like:
+------+
|master|
+------+
3 7
/
/
4 8
+------+ +------+
|child1|5-----------6|child2|
+------+ +------+
instead of:
4568
+------+
|master|
+------+
3 7
/
/
4 8
+------+ +------+
|child1|5-----------6|child2|
+------+ +------+
3678 3457
40% of the above is guesswork and conjecture, but maybe it'll spawn
the right question.There's a (I think undocumented) feature that allow transferring sockets to
other processes with the socket extension.https://github.com/amphp/aerys/commit/40fae01e4f5f82570a0bc3cb8ebbf1fee36dbb0b
That's a big help on another project I have, so thank you for that hint.
I've actually been looking for that option for quite some time.
Bob implemented that, so maybe he can help you.
But what's your exact use case?
Regards, Niklas
On Fri, Jul 7, 2017 at 10:04 AM, Martijn van Duren
php@list.imperialat.at wrote:
My use case is to set up a cluster of processes each with their own task
and permission sets. These processes need to be able to communicate with
each other over socket pairs, not just between the parent and the child,
but also between children.
Sounds like you're describing unix domain sockets (or NamedPipes if
you're on Windows).
-Sara
On Fri, Jul 7, 2017 at 10:04 AM, Martijn van Duren
php@list.imperialat.at wrote:My use case is to set up a cluster of processes each with their own task
and permission sets. These processes need to be able to communicate with
each other over socket pairs, not just between the parent and the child,
but also between children.Sounds like you're describing unix domain sockets (or NamedPipes if
you're on Windows).-Sara
Yes, although tcp/ip are also possible with stream_socket_pair.
The point is, regardless of unix - or tcp sockets, they still have an
underlying file descriptor, which I need to specify to the child, so
that they can start using it. Ergo, I need to find a way to export this
from the stream into PHP.
To put it in code:
<?php
$sp = stream_socket_pair(STREAM_PF_UNIX, STREAM_SOCK_STREAM, 0);
echo (int) $sp[0]."\n";
?>
This returns 4 (the object id), instead of 3 (the fd).
If I could get the fd, I could turn this into something like:
<?php
$sp = stream_socket_pair(STREAM_PF_UNIX, STREAM_SOCK_STREAM, 0);
switch (pcntl_fork()) {
case -1:
exit(255);
case 0:
fclose($sp[1]);
pcntl_exec("/usr/local/libexec/whatever", ["-i", (int) $sp[0]]);
default:
fclose($sp[0]);
do_parent_code();
}
?>
Where whatever can be anything from a PHP-script to a C-compiled binary.
On Fri, Jul 7, 2017 at 11:04 AM, Martijn van Duren
php@list.imperialat.at wrote:
Yes, although tcp/ip are also possible with stream_socket_pair.
nit; socket_pair sockets are not AF_INET, they're just sockets.
The point is, regardless of unix - or tcp sockets, they still have an
underlying file descriptor, which I need to specify to the child, so
that they can start using it. Ergo, I need to find a way to export this
from the stream into PHP.
You misunderstand, I'm not saying to create a unix socket and try to
pass that open pair to the child. I'm saying listen on a unix domain
socket on the file system and tell the children what the path is.
They can all individually then open streams to the parent using the
path.
Psuedo code:
// parent
$mysocket = 'unix://tmp/parent.sock';
$server = stream_socket_server($mysocket);
spawn_child('child_proc --sock='.escapeshellarg($mysocket));
$child = stream_socket_accept($server);
var_dump(fgets($child)); // string (8) "Hi mom!\n"
// child
$parentssocket = parseCliArg('sock');
$stream = stream_socket_client($parentssocket);
fwrite($stream, "Hi mom!\n");
Then the children can each report their own listening sockets to the
parent and the parent can provide a lookup for siblings to find each
other and allow them to open sockets to each other.
Trying to pass a meaningless number to the child that it can't do
anything with isn't going to work.
-Sara
On 07/07/17 18:03, Sara Golemon wrote
The point is, regardless of unix - or tcp sockets, they still have an
underlying file descriptor, which I need to specify to the child, so
that they can start using it. Ergo, I need to find a way to export this
from the stream into PHP.You misunderstand, I'm not saying to create a unix socket and try to
pass that open pair to the child. I'm saying listen on a unix domain
socket on the file system and tell the children what the path is.
They can all individually then open streams to the parent using the
path.
That would be an extremely ugly solution. You would create overhead for 1,
and 2, you leave open the possibility that an arbitrary application
(bound to the filesystem permissions) allows to connect to it during
the time the socket is listening.
See [0] for what can go wrong with a similar solutions.
Psuedo code:
// parent
$mysocket = 'unix://tmp/parent.sock';
$server = stream_socket_server($mysocket);
spawn_child('child_proc --sock='.escapeshellarg($mysocket));
$child = stream_socket_accept($server);
var_dump(fgets($child)); // string (8) "Hi mom!\n"// child
$parentssocket = parseCliArg('sock');
$stream = stream_socket_client($parentssocket);
fwrite($stream, "Hi mom!\n");Then the children can each report their own listening sockets to the
parent and the parent can provide a lookup for siblings to find each
other and allow them to open sockets to each other.Trying to pass a meaningless number to the child that it can't do
anything with isn't going to work.
That's the whole point. They're not meaningless numbers, they're the
actual references that the kernel exposes to the process for file
access:
$ cd /tmp
$ cat test.php
<?php
$sp = stream_socket_pair(STREAM_PF_UNIX, STREAM_SOCK_STREAM, 0);
switch(pcntl_fork()) {
case -1:
exit(255);
case 0:
fclose($sp[1]);
// 3 is known because of known open files in default shell
pcntl_exec("/tmp/test", ["-i", "3"]);
default:
fclose($sp[0]);
fwrite($sp[1], "hello world");
}
?>
$ cat test.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int
main(int argc, char *argv[]) {
extern char *optarg;
int ch;
int fd = -1;
int readbytes;
char buf[sizeof("hello world")];
while ((ch = getopt(argc, argv, "i:")) != -1) {
switch(ch) {
case 'i':
fd = atoi(optarg);
break;
default:
fprintf(stderr, "wrong parameter\n");
exit(1);
}
}
readbytes = read(fd, buf, sizeof(buf));
buf[readbytes] = '\0';
printf("read: %s\n", buf);
}
$ gcc ./test.c -o test && php ./test.php
read: hello world
$ cat test.c
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/socket.h>
int
main(int argc, char *argv[]) {
int sp[2];
char *fd;
extern int errno;
socketpair(AF_UNIX, SOCK_STREAM, 0, sp);
switch(fork()) {
case -1:
exit(1);
case 0:
close(sp[1]);
asprintf(&fd, "%d", sp[0]);
execl("/usr/bin/env", "/usr/bin/env", "php", "/tmp/test2.php", "-i", fd, NULL);
default:
close(sp[0]);
write(sp[1], "hello world", sizeof("hello world"));
}
}
$ cat test2.php
<?php
$opts = getopt("i:");
$fd = null;
foreach ($opts as $opt => $arg) {
switch($opt) {
case "i":
$fd = fopen("php://fd/" . $arg, "r");
break;
default:
trigger_error("wrong parameter", E_USER_ERROR);
}
}
$buf = fread($fd, 11);
echo "read: " . $buf . "\n";
$ gcc ./test2.c -o ./test2 && ./test2
read: hello world
Don't mind the crummy C here, it's for illustration purposes only.
As you can see, these "meaningless numbers" allow me to connect two
applications written in two completely different languages without
setting up any external listening system.
The problem here is that for test.php I had to hardcode the value
3 because I don't know how to retrieve it from $sp[0]. This only
works in a default shell where only stdin, stdout and stderr
(fd0, fd1 and fd2) are set. The kernel allocates the next free fd,
which happens to have number 3.
As soon as I would execute it like:
php ./test.php < ./test.c
it would fail, because fd3 would be taken by a handle to test.c
and $sp[0] would be allocated on 4.
-Sara
On Fri, Jul 7, 2017 at 4:06 PM, Martijn van Duren
php@list.imperialat.at wrote:
blah blah blah
Ignoring the brokenness of your design, the original ask: Exposing the
underlying FD from a file stream? Sure. No particular objection to
it, and the stream::set_option interface has the right mechanism to
surface it in a general way. Find someone to RFC it, and I don't see
why it won't show up in PHP 7.3. What you're able to do with it is
your problem.
-Sara
On Fri, Jul 7, 2017 at 4:06 PM, Martijn van Duren
php@list.imperialat.at wrote:blah blah blah
Ignoring the brokenness of your design, the original ask: Exposing the
underlying FD from a file stream? Sure. No particular objection to
it, and the stream::set_option interface has the right mechanism to
surface it in a general way. Find someone to RFC it, and I don't see
why it won't show up in PHP 7.3. What you're able to do with it is
your problem.
Here's a diff: https://github.com/php/php-src/compare/master...sgolemon:expose.fd
-Sara
On Fri, Jul 7, 2017 at 4:06 PM, Martijn van Duren
php@list.imperialat.at wrote:blah blah blah
Ignoring the brokenness of your design, the original ask: Exposing the
underlying FD from a file stream? Sure. No particular objection to
it, and the stream::set_option interface has the right mechanism to
surface it in a general way. Find someone to RFC it, and I don't see
why it won't show up in PHP 7.3. What you're able to do with it is
your problem.Here's a diff: https://github.com/php/php-src/compare/master...sgolemon:expose.fd
-Sara
Thank you.
As per https://wiki.php.net/rfc/howto I created the account martijn.
I'd like the Karma to turn this patch into an RFC.
On Sat, Jul 8, 2017 at 8:49 AM, Martijn van Duren php@list.imperialat.at
wrote:
On Fri, Jul 7, 2017 at 4:06 PM, Martijn van Duren
php@list.imperialat.at wrote:blah blah blah
Ignoring the brokenness of your design, the original ask: Exposing the
underlying FD from a file stream? Sure. No particular objection to
it, and the stream::set_option interface has the right mechanism to
surface it in a general way. Find someone to RFC it, and I don't see
why it won't show up in PHP 7.3. What you're able to do with it is
your problem.Here's a diff: https://github.com/php/php-src/compare/master...sgolemon:
expose.fd-Sara
Thank you.
As per https://wiki.php.net/rfc/howto I created the account martijn.
I'd like the Karma to turn this patch into an RFC.--
I've granted you with rfc karma on the wiki.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
On Thu, Jul 6, 2017 at 11:12 AM, Martijn van Duren php@list.imperialat.at
wrote:
Hello internals@,
I have an (exotic) case where I need to be able to get the
filedescriptor from a previously opened stream (via stream_socket_pair)
to hand over to the child process, which needs to reexecute.I saw that the php:// wrapper supports php://fd/X syntax, so I reckon
there should be a way to extract the information as well from a stream
as well. Unfortunately I don't seem to be able to find it.Can someone point me in the right direction?
Sincerely,
Martijn van Duren
--
hi,
would you need something like fildes_fileno() from
https://github.com/Tyrael/php-fildes ?
I never managed to put that up for pecl, not sure if it even compiles
anymore with recent php versions
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
Hello Ferenc,
Hello internals@, I have an (exotic) case where I need to be able to get the filedescriptor from a previously opened stream (via stream_socket_pair) to hand over to the child process, which needs to reexecute. I saw that the php:// wrapper supports php://fd/X syntax, so I reckon there should be a way to extract the information as well from a stream as well. Unfortunately I don't seem to be able to find it. Can someone point me in the right direction? Sincerely, Martijn van Duren --
hi,
would you need something like fildes_fileno() from https://github.com/Tyrael/php-fildes ?
I never managed to put that up for pecl, not sure if it even compiles anymore with recent php versions
Thanks for the hint, but I'd like to refrain from external
libraries as much as I can, especially for something simple as
a simple number inside a data structure.
I might also achieve something like this by eio from pecl, through
e.g. eio_dup2, but it's just unrealistic overkill to ask for a full
extension installation for just 1 one trivial piece of functionality.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
On Fri, Jul 7, 2017 at 10:14 PM, Martijn van Duren php@list.imperialat.at
wrote:
Hello Ferenc,
On Thu, Jul 6, 2017 at 11:12 AM, Martijn van Duren <
php@list.imperialat.at mailto:php@list.imperialat.at> wrote:Hello internals@, I have an (exotic) case where I need to be able to get the filedescriptor from a previously opened stream (via
stream_socket_pair)
to hand over to the child process, which needs to reexecute. I saw that the php:// wrapper supports php://fd/X syntax, so I reckon there should be a way to extract the information as well from a
stream
as well. Unfortunately I don't seem to be able to find it. Can someone point me in the right direction? Sincerely, Martijn van Duren --
hi,
would you need something like fildes_fileno() from
https://github.com/Tyrael/php-fildes ?
I never managed to put that up for pecl, not sure if it even compiles
anymore with recent php versionsThanks for the hint, but I'd like to refrain from external
libraries as much as I can, especially for something simple as
a simple number inside a data structure.I might also achieve something like this by eio from pecl, through
e.g. eio_dup2, but it's just unrealistic overkill to ask for a full
extension installation for just 1 one trivial piece of functionality.--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
sure, I just thought that you are asking for a solution and not proposing a
new feature/function to the core.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu