# Gearman support for php-fpm? - Thread: https://externals.io/message/48723 - Original archive: https://news-web.php.net/php.internals/48723 - Started by Rasmus Lerdorf on 2010-06-10T14:56:00+00:00 - Messages: 8 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #48723 — Rasmus Lerdorf — 2010-06-10T14:56:00+00:00 URL: https://externals.io/message/48723#48723 Has anybody thought about adding Gearman support to the fpm sapi? Managing Gearman workers is pretty much identical to managing fastcgi workers, so it doesn't seem like much of a stretch. -Rasmus --- ## #48724 — Reilly — 2010-06-10T14:59:56+00:00 URL: https://externals.io/message/48723#48724 · In reply to: #48723 Rasmus, I can work on it if no one has it already. Warm Regards, Patrick > Has anybody thought about adding Gearman support to the fpm sapi? > Managing Gearman workers is pretty much identical to managing fastcgi > workers, so it doesn't seem like much of a stretch. > > -Rasmus --- ## #48725 — Brian Moon — 2010-06-10T15:20:52+00:00 URL: https://externals.io/message/48723#48725 · In reply to: #48723 The big difference is that different processes do different jobs in Gearman. All PHP workers in fpm just run what ever code is handed to them. How do you handle that? Brian. -------- http://brian.moonspot.net/ > Has anybody thought about adding Gearman support to the fpm sapi? > Managing Gearman workers is pretty much identical to managing fastcgi > workers, so it doesn't seem like much of a stretch. > > -Rasmus --- ## #48727 — Rasmus Lerdorf — 2010-06-10T15:46:22+00:00 URL: https://externals.io/message/48723#48727 > > > […] > Threading ;) ? Definitely not. Gearman is actually the answer to a lot of problems where people think they need threads. As far as different workers doing different jobs, I don't see the issue here. fpm doesn't need to know the details. All it has to know is that it needs to maintain pools of php processes. Each pool would correspond to a Gearman job. The workers can take care of registering themselves with gearmand. All fpm has to do is start up and maintain N processes per worker pool. [gearman] worker1.path = /path/to/worker1.php worker1.pm = dynamic worker1.pm.max_children = 10 worker1.pm.start_servers = 5 worker2.path = /path/to/worker2.php worker2.pm = dynamic worker2.pm.max_children = 4 worker2.pm.start_servers = 2 ... or however you do multiple nested conf blocks in the fpm .ini file. In fact, this may not need to be gearman-specific. We could just call it cli job management or something like that if we left it completely up to the workers to register with the gearmand. Alternatively, we could make fpm Gearman-aware and register the workers from there, but I am not sure I see much of an advantage to that other than making the worker code slightly simpler. -Rasmus --- ## #48728 — Tjerk Anne Meesters — 2010-06-10T15:53:48+00:00 URL: https://externals.io/message/48723#48728 · In reply to: #48725 All "fpm" workers would be called using the same function name (e.g. "runphp"). This allows you to reuse a gearmand instance for other workers too. The advantages of fpm vs gearman would be mostly performance (it uses ping and requires an underlying layer on top of tcp/ip), followed by dynamic spawning and "crash recovery". Think there are more but I can't access the website from over here ;) > The big difference is that different processes do different jobs in > Gearman. All PHP workers in fpm just run what ever code is handed to > them. How do you handle that? > > > Brian. > -------- > http://brian.moonspot.net/ > > > > […] > > -- > > > > -- -- Tjerk --- ## #48729 — Rasmus Lerdorf — 2010-06-10T16:00:07+00:00 URL: https://externals.io/message/48723#48729 · In reply to: #48728 > All "fpm" workers would be called using the same function name (e.g. > "runphp"). This allows you to reuse a gearmand instance for other > workers too. I don't think we would need to require that. Obviously you could build your Gearman worker such that it registers "runphp" and has a big switch in it to run various types of jobs and fpm would just need to manage a single pool of workers. But, assuming fpm could manage multiple pools, I don't see the problem with having different types of workers. -Rasmus --- ## #48730 — Brian Moon — 2010-06-10T16:31:12+00:00 URL: https://externals.io/message/48723#48730 · In reply to: #48727 > [gearman] > worker1.path = /path/to/worker1.php > worker1.pm = dynamic > worker1.pm.max_children = 10 > worker1.pm.start_servers = 5 > > worker2.path = /path/to/worker2.php > worker2.pm = dynamic > worker2.pm.max_children = 4 > worker2.pm.start_servers = 2 So, this is a proposed new entry in the ini format? I am a bit ignorant about fpm, I admit. This would require a good bit of fiddling for our gearman set up to work with it. We run X workers that know how to do all jobs available and then 1 + n (defined by ini file) that only do one job for each available job. This gives us good coverage with controlled process load. I guess it would just require a lot more configuration than we have now. For example, could we pass parameters to the script? Having a separate script plus a separate class for every function quickly makes a lot of files. I will simply stay tuned and see where it goes. FWIW, this is my GearmanManager that basically does this already via PHP+pcntl. http://github.com/brianlmoon/GearmanManager. It can not dynamically ramp up more workers like FPM can. But, I would never run it that way anyhow. I like my processes under control. Brian. --- ## #48731 — Rasmus Lerdorf — 2010-06-10T16:37:44+00:00 URL: https://externals.io/message/48723#48731 · In reply to: #48730 > > […] > > So, this is a proposed new entry in the ini format? I am a bit ignorant > about fpm, I admit. This would require a good bit of fiddling for our > gearman set up to work with it. We run X workers that know how to do all > jobs available and then 1 + n (defined by ini file) that only do one job > for each available job. This gives us good coverage with controlled > process load. I guess it would just require a lot more configuration > than we have now. For example, could we pass parameters to the script? > Having a separate script plus a separate class for every function > quickly makes a lot of files. > > I will simply stay tuned and see where it goes. > > FWIW, this is my GearmanManager that basically does this already via > PHP+pcntl. http://github.com/brianlmoon/GearmanManager. It can not > dynamically ramp up more workers like FPM can. But, I would never run it > that way anyhow. I like my processes under control. Yes, I know, I have seen it. Process management and signal handling like that really isn't something that should be done in PHP as far as I am concerned. And FPM already does all of this and more. It just needs a slight tweak to be able to manage any sort of php process. -Rasmus