Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:43833
Return-Path: <shire@php.net>
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Delivered-To: mailing list internals@lists.php.net
Received: (qmail 92845 invoked from network); 4 May 2009 15:51:34 -0000
Received: from unknown (HELO lists.php.net) (127.0.0.1)
  by localhost with SMTP; 4 May 2009 15:51:34 -0000
Authentication-Results: pb1.pair.com smtp.mail=shire@php.net; spf=unknown; sender-id=unknown
Authentication-Results: pb1.pair.com header.from=shire@php.net; sender-id=unknown
Received-SPF: unknown (pb1.pair.com: domain php.net does not designate 208.43.138.18 as permitted sender)
X-PHP-List-Original-Sender: shire@php.net
X-Host-Fingerprint: 208.43.138.18 sizzo.org Linux 2.6
Received: from [208.43.138.18] ([208.43.138.18:38091] helo=sizzo.org)
	by pb1.pair.com (ecelerity 2.1.1.9-wez r(12769M)) with ESMTP
	id AB/E6-57065-50F0FF94 for <internals@lists.php.net>; Mon, 04 May 2009 11:51:34 -0400
Received: from shirebook.tekrat.linksys (unknown [75.101.56.2])
	(using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits))
	(No client certificate requested)
	by sizzo.org (Postfix) with ESMTPSA id A5FE1CBE617;
	Mon,  4 May 2009 08:51:30 -0700 (PDT)
Message-ID: <49FF0EFF.4020109@php.net>
Date: Mon, 04 May 2009 08:51:27 -0700
Reply-To: shire@php.net
User-Agent: Postbox 1.0b11 (Macintosh/2009041623)
MIME-Version: 1.0
To: lbarnaud@php.net
CC: Matt Wilmas <php_lists@realplain.com>, internals@lists.php.net, 
 Nuno Lopes <nlopess@php.net>,
 Lukas Kahwe Smith <mls@pooteeweet.org>, Dmitry Stogov <dmitry@zend.com>
References: <6604D94D40FD465F992144110B075BB5@pc1>	 <E17F0EFAC0F847C994EBDA6D020E52BF@PC3EE1F19287>	 <9D5D4CBF-5CB1-47EC-81F4-59E3C48EEEEF@pooteeweet.org>	 <C8E7CB69316B42D1B8BB5ED78408088D@PC3EE1F19287>	 <B06CC8E0A66F4959B3BFC4AA0698223D@pc1> <49FE9AE7.4000008@php.net> <b6f28ec20905040440w2cfaf934qfa001e3689c566b7@mail.gmail.com>
In-Reply-To: <b6f28ec20905040440w2cfaf934qfa001e3689c566b7@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] [PATCH] Scanner "diet" with fixes, etc.
From: shire@php.net (shire)

Arnaud Le Blanc wrote:
> Hi,
> On Mon, May 4, 2009 at 9:36 AM, shire<shire@php.net>  wrote:
>> Regarding the ZEND_MMAP_AHEAD issue and the temp. fix that Dmitry put in we
>> need to find a solution to that, perhaps I can play with that this week too
>> as I think I'm seeing some related issues in my testing of 5.3.  Essentially
>> we abuse ZEND_MMAP_AHEAD by adding it to the file size and passing it to the
>> mmap call which isn't at all valid and only really works up to PAGESIZE.  We
>> could possibly use YYFILL to re-allocate more space as necessary past the
>> end of file to fix this.
>
> I was thinking of doing something like that with YYFILL too. However
> there is a bunch of pointers to take in to account and to update (e.g.
> yy_marker, yy_text, etc).
>

Yeah, I'm pretty sure that's how most of the example re2c code is setup:

#define YYFILL(n) {cursor = fill(s, cursor);}

uchar *fill(Scanner *s, uchar *cursor){
     if(!s->eof){
   unint cnt = s->lim - s->tok;
   uchar *buf = malloc((cnt + 1)*sizeof(uchar));
   memcpy(buf, s->tok, cnt);
   cursor = &buf[cursor - s->tok];
   s->pos = &buf[s->pos - s->tok];
   s->ptr = &buf[s->ptr - s->tok];
   s->lim = &buf[cnt];
   s->eof = s->lim; *(s->eof)++ = '\n';
   s->tok = buf;
     }
     return cursor;
}



-shire