Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:88682
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain zend.com designates 209.85.213.51 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <029301d0ff6c$a1cc8370$e5658a50$@belski.net>
References: <CAMbn9ZEHXT7SPCo1ai4FpVfjR7X+vPe+uEOKwY13VDdJR4g6+g@mail.gmail.com>
	<CA+9eiLsu2K9v-89w8RbHvyd=yH+Nu_tQLnzucRWhr1CzdhGciA@mail.gmail.com>
	<CAMbn9ZEKG0c=xaDiN5hzxC2H+rvL2pszUh3-+D+n+QXvopejbQ@mail.gmail.com>
	<CA+9eiLtHFFXFqOA18VJGKULgqHqjx=krPWQxBrSnb-V2MogQ1w@mail.gmail.com>
	<CA+9eiLt=qF8yy0rPYqUH3ZdrNRUvkCo94gbR685SB5NRD2wacQ@mail.gmail.com>
	<CAMbn9ZHcZ3FZHyGDKMwjAWBKW4Wum84q9ShUJ+ES8eqaK+Oyvw@mail.gmail.com>
	<CA+9eiLtwyayxzbTT+0Fi7JnB9WPc7SkUFH+r3vOW49bdjRCUJw@mail.gmail.com>
	<CAEZPtU5YKUOeNT0G3=8fyy3g0Fug1ZpMkSE_th35YKM9azamEA@mail.gmail.com>
	<CAMbn9ZGq0+VgSjfSsoozKat5WKjVLLUpN8LWSqS81Hym2LybVQ@mail.gmail.com>
	<CA+9eiLt=WZfvA=i+rCCs27LewBS+gCbvLAGbVHOFe_37=r+j_Q@mail.gmail.com>
	<024e01d0fc89$31005990$93010cb0$@belski.net>
	<CA+9eiLuM6ozhVMU_=YoVDnm=5EbW=VDEUJ8=acffWu_-BQfg+w@mail.gmail.com>
	<01e101d0ff3f$8cc0c310$a6424930$@belski.net>
	<CA+9eiLsL3+rBFap6xQXtPx3gvkEWB_mcRYk8cE7awHGK9gfBEg@mail.gmail.com>
	<029301d0ff6c$a1cc8370$e5658a50$@belski.net>
Date: Mon, 5 Oct 2015 16:30:39 +0300
Message-ID: <CA+9eiLv7oS8ea7++BaKSM1aCjGgD=oJHB31Y_8+ia4etZ5ZSAA@mail.gmail.com>
To: Anatol Belski <anatol.php@belski.net>
Cc: Matt Ficken <themattficken@gmail.com>, Pierre Joye <pierre.php@gmail.com>, 
	Laruence <laruence@php.net>, PHP Internals <internals@lists.php.net>, 
	"dmitry@php.net" <dmitry@php.net>
Content-Type: multipart/alternative; boundary=001a1143eb1af32b4505215b8432
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
From: dmitry@zend.com (Dmitry Stogov)

--001a1143eb1af32b4505215b8432
Content-Type: text/plain; charset=UTF-8

On Mon, Oct 5, 2015 at 3:51 PM, Anatol Belski <anatol.php@belski.net> wrote:

>
>
> > -----Original Message-----
> > From: Dmitry Stogov [mailto:dmitry@zend.com]
> > Sent: Monday, October 5, 2015 1:08 PM
> > To: Anatol Belski <anatol.php@belski.net>
> > Cc: Matt Ficken <themattficken@gmail.com>; Pierre Joye
> > <pierre.php@gmail.com>; Laruence <laruence@php.net>; PHP Internals
> > <internals@lists.php.net>; dmitry@php.net
> > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> >
> > On Mon, Oct 5, 2015 at 10:29 AM, Anatol Belski <anatol.php@belski.net>
> > wrote:
> >
> > > Hi Dmitry,
> > >
> > > > -----Original Message-----
> > > > From: Dmitry Stogov [mailto:dmitry@zend.com]
> > > > Sent: Friday, October 2, 2015 9:34 AM
> > > > To: Anatol Belski <anatol.php@belski.net>
> > > > Cc: Matt Ficken <themattficken@gmail.com>; Pierre Joye
> > > > <pierre.php@gmail.com>; Anatoliy Belsky <ab@php.net>; Laruence
> > > > <laruence@php.net>; PHP Internals <internals@lists.php.net>;
> > > dmitry@php.net
> > > > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> > > >
> > > > > Dmitry, I'd have a question to this
> > > > > > Also. if we can't map SHM into desired address space, we may map
> > > > > > it in
> > > > > any
> > > > > > other address and copy data into the process memory similar to
> > > > > file-cache.
> > > > > In randomized memory layout, even the base were available and
> > > > > OpenFileMapping were worky, some parts of that memory were already
> > > > taken.
> > > > > How exactly it could be done, could you please give a couple of
> > > > > pointers to this?
> > > >
> > > >
> > > > If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
> > > > MapViewOfFileEx(..., NULL).
> > > >
> > > >
> > > > > Would the file cache be always required then?
> > > > >
> > > >
> > > > This is not necessary, but depends on implementation of course.
> > > >
> > > Thanks for the advice. I was playing with this over these days. There
> > > are two usual cases where it possibly fails when reattaching ATM
> > >
> > > -
> > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
> > > in32.c#L151
> > > - the saved address is available but is not suitable
> > > -
> > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
> > > in32.c#L159
> > > - the actual MapViewOfFileEx case
> > >
> > > An unrelated plain C test shows, that MapViewOfFileEx can possibly
> > > fail when called second time, too. Even with NULL or with another
> > > address as base. But even if it could map at a different base - the
> > > internal structures will probably have invalid addresses.
> >
> >
> > right. we might need different code for zend_accle_hash access or convert
> > corresponding structures to PIC.
> > For opcdeos "invalid address" don't matter because we will copy them into
> > process memory and fix (like in file-cache).
> >
> Ah, I have to study the file cache code then. But generally it sounds not
> like something that can be done offhand. I've also thought about other
> things like interned strings (maybe something else), not sure they're
> stored with the cache.
>
> >
> > > So it looks like there's indeed no profit to do any retry once the
> > > actualy base address needed was failed to reattach to.
> > >
> > > IMHO the scenario that were usable - in case it fails to reattach to
> > > the exact address, it has to switch to heap. Just for one request, it
> > > should get a heap allocated segment and then invalidate all the cache.
> > > That way we fulfill the following
> > >
> > > - the request is served for sure
> > > - the scripts are not cached indeed, so no divergence with the actual
> > > real cache
> > >
> > > A heap fallback memory handler would be probably quite easy to
> implement.
> > > What do you think?
> > >
> > > Apropos with the heap - it also looks like when PHP is used as module
> > > under mpm_winnt, all the cache could use heap instead of SHM. In that
> > > case, there is only one Apache process serving with many threads.
> > > Except one would want to share that cache outside Apache, using heap
> > > there could be a much simpler solution.
> > >
> >
> > Heap based cache makes the same problems. It increase the memory usage
> and
> > doesn't provide cache consistency.
> > Just fall-back to file-cache may be better.
> Do you think something like this would suffice as file cache fallback
> https://gist.github.com/weltling/224001a468f04de13693  ? Though it'd
> still diverge from the "main" cache.
>

I think we should enable file-cache automatically, but we can set
ZCG(accel_directives).file_cache_only
if the file_cache already enabled.


>
> Actually in such case all the processes should switch to file cache?


No. Only the processes that weren't be able to attach to SHM.


> Just not sure how they all would negotiate that when no SHM is available
> (probably through files, or a separate shared chunk).
>

yeah. Processes that use file-cache-only won't be able to negotiate SHM
cache. :(


>
> >
> > Actually, when I implemented file-cache I had a though about different
> storage
> > back-ends (e.g. sqlite or memcache).
> > This abstraction might make support for "improperly mapped SHM" quite
> easy.
> >
> Yeah, that could be useful. Maybe also module based so one could even
> supply them separately, then an arbitrary storage mechanism could be
> provided. Fe like I've mentioned for mpm_winnt -if there's no intention to
> share the cache outside Apache, just one heap for all could be much simpler
> to avoid all that reattach mechanics.
>

Does mpm_winnt use pure ZTS without processes?

Thanks. Dmitry.


>
> Thanks.
>
> Anatol
>
>

--001a1143eb1af32b4505215b8432--