Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:88693
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain zend.com designates 209.85.213.44 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <02ca01d0ff83$f0d58c70$d280a550$@belski.net>
References: <CAMbn9ZEHXT7SPCo1ai4FpVfjR7X+vPe+uEOKwY13VDdJR4g6+g@mail.gmail.com>
	<CA+9eiLsu2K9v-89w8RbHvyd=yH+Nu_tQLnzucRWhr1CzdhGciA@mail.gmail.com>
	<CAMbn9ZEKG0c=xaDiN5hzxC2H+rvL2pszUh3-+D+n+QXvopejbQ@mail.gmail.com>
	<CA+9eiLtHFFXFqOA18VJGKULgqHqjx=krPWQxBrSnb-V2MogQ1w@mail.gmail.com>
	<CA+9eiLt=qF8yy0rPYqUH3ZdrNRUvkCo94gbR685SB5NRD2wacQ@mail.gmail.com>
	<CAMbn9ZHcZ3FZHyGDKMwjAWBKW4Wum84q9ShUJ+ES8eqaK+Oyvw@mail.gmail.com>
	<CA+9eiLtwyayxzbTT+0Fi7JnB9WPc7SkUFH+r3vOW49bdjRCUJw@mail.gmail.com>
	<CAEZPtU5YKUOeNT0G3=8fyy3g0Fug1ZpMkSE_th35YKM9azamEA@mail.gmail.com>
	<CAMbn9ZGq0+VgSjfSsoozKat5WKjVLLUpN8LWSqS81Hym2LybVQ@mail.gmail.com>
	<CA+9eiLt=WZfvA=i+rCCs27LewBS+gCbvLAGbVHOFe_37=r+j_Q@mail.gmail.com>
	<024e01d0fc89$31005990$93010cb0$@belski.net>
	<CA+9eiLuM6ozhVMU_=YoVDnm=5EbW=VDEUJ8=acffWu_-BQfg+w@mail.gmail.com>
	<01e101d0ff3f$8cc0c310$a6424930$@belski.net>
	<CA+9eiLsL3+rBFap6xQXtPx3gvkEWB_mcRYk8cE7awHGK9gfBEg@mail.gmail.com>
	<029301d0ff6c$a1cc8370$e5658a50$@belski.net>
	<CA+9eiLv7oS8ea7++BaKSM1aCjGgD=oJHB31Y_8+ia4etZ5ZSAA@mail.gmail.com>
	<02ca01d0ff83$f0d58c70$d280a550$@belski.net>
Date: Tue, 6 Oct 2015 11:00:34 +0300
Message-ID: <CA+9eiLs1S=Vm_g04eq9LLzsDB7T3LhpRWFXZThzC-_ziPPrQJA@mail.gmail.com>
To: Anatol Belski <anatol.php@belski.net>
Cc: Matt Ficken <themattficken@gmail.com>, Pierre Joye <pierre.php@gmail.com>, 
	Laruence <laruence@php.net>, PHP Internals <internals@lists.php.net>, 
	"dmitry@php.net" <dmitry@php.net>
Content-Type: multipart/alternative; boundary=001a11431e7650fe9805216b06bd
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
From: dmitry@zend.com (Dmitry Stogov)

--001a11431e7650fe9805216b06bd
Content-Type: text/plain; charset=UTF-8

On Mon, Oct 5, 2015 at 6:38 PM, Anatol Belski <anatol.php@belski.net> wrote:

>
>
> > -----Original Message-----
> > From: Dmitry Stogov [mailto:dmitry@zend.com]
> > Sent: Monday, October 5, 2015 3:31 PM
> > To: Anatol Belski <anatol.php@belski.net>
> > Cc: Matt Ficken <themattficken@gmail.com>; Pierre Joye
> > <pierre.php@gmail.com>; Laruence <laruence@php.net>; PHP Internals
> > <internals@lists.php.net>; dmitry@php.net
> > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> >
> > > > > > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> > > > > >
> > > > > > > Dmitry, I'd have a question to this
> > > > > > > > Also. if we can't map SHM into desired address space, we may
> > > > > > > > map it in
> > > > > > > any
> > > > > > > > other address and copy data into the process memory similar
> > > > > > > > to
> > > > > > > file-cache.
> > > > > > > In randomized memory layout, even the base were available and
> > > > > > > OpenFileMapping were worky, some parts of that memory were
> > > > > > > already
> > > > > > taken.
> > > > > > > How exactly it could be done, could you please give a couple
> > > > > > > of pointers to this?
> > > > > >
> > > > > >
> > > > > > If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
> > > > > > MapViewOfFileEx(..., NULL).
> > > > > >
> > > > > >
> > > > > > > Would the file cache be always required then?
> > > > > > >
> > > > > >
> > > > > > This is not necessary, but depends on implementation of course.
> > > > > >
> > > > > Thanks for the advice. I was playing with this over these days.
> > > > > There are two usual cases where it possibly fails when reattaching
> > > > > ATM
> > > > >
> > > > > -
> > > > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_all
> > > > > oc_w
> > > > > in32.c#L151
> > > > > - the saved address is available but is not suitable
> > > > > -
> > > > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_all
> > > > > oc_w
> > > > > in32.c#L159
> > > > > - the actual MapViewOfFileEx case
> > > > >
> > > > > An unrelated plain C test shows, that MapViewOfFileEx can possibly
> > > > > fail when called second time, too. Even with NULL or with another
> > > > > address as base. But even if it could map at a different base -
> > > > > the internal structures will probably have invalid addresses.
> > > >
> > > >
> > > > right. we might need different code for zend_accle_hash access or
> > > > convert corresponding structures to PIC.
> > > > For opcdeos "invalid address" don't matter because we will copy them
> > > > into process memory and fix (like in file-cache).
> > > >
> > > Ah, I have to study the file cache code then. But generally it sounds
> > > not like something that can be done offhand. I've also thought about
> > > other things like interned strings (maybe something else), not sure
> > > they're stored with the cache.
> > >
> > > >
> > > > > So it looks like there's indeed no profit to do any retry once the
> > > > > actualy base address needed was failed to reattach to.
> > > > >
> > > > > IMHO the scenario that were usable - in case it fails to reattach
> > > > > to the exact address, it has to switch to heap. Just for one
> > > > > request, it should get a heap allocated segment and then
> invalidate all the
> > cache.
> > > > > That way we fulfill the following
> > > > >
> > > > > - the request is served for sure
> > > > > - the scripts are not cached indeed, so no divergence with the
> > > > > actual real cache
> > > > >
> > > > > A heap fallback memory handler would be probably quite easy to
> > > implement.
> > > > > What do you think?
> > > > >
> > > > > Apropos with the heap - it also looks like when PHP is used as
> > > > > module under mpm_winnt, all the cache could use heap instead of
> > > > > SHM. In that case, there is only one Apache process serving with
> many
> > threads.
> > > > > Except one would want to share that cache outside Apache, using
> > > > > heap there could be a much simpler solution.
> > > > >
> > > >
> > > > Heap based cache makes the same problems. It increase the memory
> > > > usage
> > > and
> > > > doesn't provide cache consistency.
> > > > Just fall-back to file-cache may be better.
> > > Do you think something like this would suffice as file cache fallback
> > > https://gist.github.com/weltling/224001a468f04de13693  ? Though it'd
> > > still diverge from the "main" cache.
> > >
> >
> > I think we should enable file-cache automatically, but we can set
> > ZCG(accel_directives).file_cache_only
> > if the file_cache already enabled.
> >
> I've revoked the approach
> https://gist.github.com/weltling/69bd1e47dc15273edde5 , also added
> enforcement per request (was missing in the previous version). Or did you
> mean "we should NOT enable file cache automatically"? Can be easy changed
> ofc. IMHO one can enforce automatically, careful programmers do check error
> logs :)
>


I wouldn't enable file cache automatically, but this really not an
implementation problem.

+	if (NULL != ZCG(accel_directives).file_cache) {
+		ZCG(accel_directives).file_cache_only = 1;
+	}



>
> >
> > >
> > > Actually in such case all the processes should switch to file cache?
> >
> >
> > No. Only the processes that weren't be able to attach to SHM.
> >
> >
> > > Just not sure how they all would negotiate that when no SHM is
> > > available (probably through files, or a separate shared chunk).
> > >
> >
> > yeah. Processes that use file-cache-only won't be able to negotiate SHM
> cache.
> > :(
> >
> ACK, so basically it is the same principle Matt suggested with sidestep
> cache. I could imagine synchronizing all the processes through another
> shared segment. Not the big one where all the cache is handled, but just a
> couple of bytes that wouldn't require to be attached to the same address.
> That would allow to signal other file-cache-only processes for cache reset,
> etc.
>

This idea should work. Mapping of small portion into the same address space
shouldn't be required.

Thanks. Dmitry.


>
> >
> > >
> > > >
> > > > Actually, when I implemented file-cache I had a though about
> > > > different
> > > storage
> > > > back-ends (e.g. sqlite or memcache).
> > > > This abstraction might make support for "improperly mapped SHM"
> > > > quite
> > > easy.
> > > >
> > > Yeah, that could be useful. Maybe also module based so one could even
> > > supply them separately, then an arbitrary storage mechanism could be
> > > provided. Fe like I've mentioned for mpm_winnt -if there's no
> > > intention to share the cache outside Apache, just one heap for all
> > > could be much simpler to avoid all that reattach mechanics.
> > >
> >
> > Does mpm_winnt use pure ZTS without processes?
> >
> The master process starts only one child (also documented here
> http://httpd.apache.org/docs/2.4/mod/mpm_winnt.html). IMHO even If one
> would decide supporting virtual hosts (not supported by the current
> solution anyway), that were done relatively simple creating private heaps
> inside the same child process.
>
> Thanks
>
> Anatol
>
>

--001a11431e7650fe9805216b06bd--