# HALT Patch

- Thread: https://externals.io/message/15375
- Original archive: https://news-web.php.net/php.internals/15375
- Started by Ilia Alshanetsky on 2005-03-11T19:36:16+00:00
- Messages: 20
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #15375 — Ilia Alshanetsky — 2005-03-11T19:36:16+00:00

URL: https://externals.io/message/15375#15375

The attach patch implements a special "token" <?php HALT; ?> that can be 
used to stop the Zend lexical parser from parsing any data after this 
token.

The idea behind this patch is to allow tucking on of any data (binary 
and otherwise) to the PHP script without having to encode it. It also 
saves on memory/cpu that normally be used/needed to parse the data. 
Memory is particularly important point, since many systems run with 
default memory limit (8 megs) and would easily hit it when a script had 
a 2-4 megs of data tacked on.

The patch would be extremely helpful for application distribution where 
the entire install/upgrade script can be just a single script.

The syntax chosen for the patch was design specifically in such a way as 
to not cause parser errors in older PHPs without the support for this 
feature.

Ilia

---

## #15377 — John Coggeshall — 2005-03-11T20:22:56+00:00

URL: https://externals.io/message/15375#15377 · In reply to: #15375

We've been talking about this on IRC for awhile now and I think it's a
nice patch to minimize memory usage when you want to create these sorts
of "bundled" PHP/data hybrid scripts.

John


> The attach patch implements a special "token" <?php HALT; ?> that can be 
> used to stop the Zend lexical parser from parsing any data after this 
> token.
> 
> The idea behind this patch is to allow tucking on of any data (binary 
> and otherwise) to the PHP script without having to encode it. It also 
> saves on memory/cpu that normally be used/needed to parse the data. 
> Memory is particularly important point, since many systems run with 
> default memory limit (8 megs) and would easily hit it when a script had 
> a 2-4 megs of data tacked on.
> 
> The patch would be extremely helpful for application distribution where 
> the entire install/upgrade script can be just a single script.
> 
> The syntax chosen for the patch was design specifically in such a way as 
> to not cause parser errors in older PHPs without the support for this 
> feature.
> 
> Ilia
> 
> ______________________________________________________________________
> Index: Zend/zend_language_scanner.l
> ===================================================================
> RCS file: /repository/ZendEngine2/zend_language_scanner.l,v
> retrieving revision 1.124
> diff -u -a -p -r1.124 zend_language_scanner.l
> --- Zend/zend_language_scanner.l	7 Mar 2005 16:48:49 -0000	1.124
> +++ Zend/zend_language_scanner.l	11 Mar 2005 19:30:47 -0000
> @@ -1342,6 +1342,10 @@ NEWLINE ("\r"|"\n"|"\r\n")
>  	return T_INLINE_HTML;
>  }
>  
> +<INITIAL>"<?php"{WHITESPACE}"HALT;"{WHITESPACE}*"?>" {
> +	yyterminate();
> +}
> +
>  <INITIAL>"<?"|"<script"{WHITESPACE}+"language"{WHITESPACE}*"="{WHITESPACE}*("php"|"\"php\""|"\'php\'"){WHITESPACE}*">" {
>  	HANDLE_NEWLINES(yytext, yyleng);
>  	if (CG(short_tags) || yyleng>2) { /* yyleng>2 means it's not <? but <script> */
> 
> 
> ______________________________________________________________________

---

## #15376 — Sara Golemon — 2005-03-11T20:23:21+00:00

URL: https://externals.io/message/15375#15376 · In reply to: #15375

> The attach patch implements a special "token" <?php HALT; ?> that can be
> used to stop the Zend lexical parser from parsing any data after this
> token.
>
+1

---

## #15379 — Daniel Convissor — 2005-03-11T21:22:24+00:00

URL: https://externals.io/message/15375#15379 · In reply to: #15375

Hi Ilia:


> The attach patch implements a special "token" <?php HALT; ?> that can be 
> used to stop the Zend lexical parser from parsing any data after this 
> token.

Would the data after that point be sent directly to STDOUT?  Or would the 
present script somehow be able to use it?

Thanks,

--Dan

-- 
 T H E   A N A L Y S I S   A N D   S O L U T I O N S   C O M P A N Y
            data intensive web and database programming
                http://www.AnalysisAndSolutions.com/
 4015 7th Ave #4, Brooklyn NY 11232  v: 718-854-0335 f: 718-854-0409

---

## #15380 — Ilia Alshanetsky — 2005-03-11T21:27:53+00:00

URL: https://externals.io/message/15375#15380 · In reply to: #15379

> Would the data after that point be sent directly to STDOUT?  Or would the 
> present script somehow be able to use it?

The data will not be parsed or output. When you need it you would make 
the script open itself and read the (binary) data dump from the end of 
the file and use it various creative ways.

Ilia

---

## #15381 — Daniel Convissor — 2005-03-11T21:37:10+00:00

URL: https://externals.io/message/15375#15381 · In reply to: #15380

Hi Ilia:



> The data will not be parsed or output. When you need it you would make 
> the script open itself and read the (binary) data dump from the end of 
> the file and use it various creative ways.

Interesting.  I'm wondering about the security implications of this.  
This makes it very easy to use PHP as a means to propogate all sorts of 
nasty things.  Well, people could even do that today in one script by 
setting a variable to a base64 encoded string then decoding it.  None the 
less, putting binary data in PHP scripts gives me pause.

--Dan

-- 
 T H E   A N A L Y S I S   A N D   S O L U T I O N S   C O M P A N Y
            data intensive web and database programming
                http://www.AnalysisAndSolutions.com/
 4015 7th Ave #4, Brooklyn NY 11232  v: 718-854-0335 f: 718-854-0409

---

## #15382 — Ilia Alshanetsky — 2005-03-11T21:43:21+00:00

URL: https://externals.io/message/15375#15382 · In reply to: #15381

Daniel Convissor wrote:
> Interesting.  I'm wondering about the security implications of this.  
> This makes it very easy to use PHP as a means to propogate all sorts of 
> nasty things. 

You can already use PHP to propagate all sorts of nasty things, nothing 
changes in this respect.

> Well, people could even do that today in one script by 
> setting a variable to a base64 encoded string then decoding it.

Sure, BUT this approach makes the final file approximately %30 larger 
and any time it is executed this "data dump" as I like to refer to it 
will be parsed and stored in memory. Which makes this approach highly 
inefficient.

Ilia

---

## #15383 — Wez Furlong — 2005-03-11T21:46:12+00:00

URL: https://externals.io/message/15375#15383 · In reply to: #15375

If you name the token __HALT_PHP_PARSER__ instead (or something
equally unlikely to be used by accident and unambiguous in meaning),
you'll get a +1 from me :-)

--Wez.



> The attach patch implements a special "token" <?php HALT; ?> that can be
> used to stop the Zend lexical parser from parsing any data after this
> token.
> 
> The idea behind this patch is to allow tucking on of any data (binary
> and otherwise) to the PHP script without having to encode it. It also
> saves on memory/cpu that normally be used/needed to parse the data.
> Memory is particularly important point, since many systems run with
> default memory limit (8 megs) and would easily hit it when a script had
> a 2-4 megs of data tacked on.
> 
> The patch would be extremely helpful for application distribution where
> the entire install/upgrade script can be just a single script.
> 
> The syntax chosen for the patch was design specifically in such a way as
> to not cause parser errors in older PHPs without the support for this
> feature.
> 
> Ilia
> 
> 
> Index: Zend/zend_language_scanner.l
> ===================================================================
> RCS file: /repository/ZendEngine2/zend_language_scanner.l,v
> retrieving revision 1.124
> diff -u -a -p -r1.124 zend_language_scanner.l
> --- Zend/zend_language_scanner.l        7 Mar 2005 16:48:49 -0000       1.124
> +++ Zend/zend_language_scanner.l        11 Mar 2005 19:30:47 -0000
> @@ -1342,6 +1342,10 @@ NEWLINE ("\r"|"\n"|"\r\n")
>         return T_INLINE_HTML;
>  }
> 
> +<INITIAL>"<?php"{WHITESPACE}"HALT;"{WHITESPACE}*"?>" {
> +       yyterminate();
> +}
> +
>  <INITIAL>"<?"|"<script"{WHITESPACE}+"language"{WHITESPACE}*"="{WHITESPACE}*("php"|"\"php\""|"\'php\'"){WHITESPACE}*">" {
>         HANDLE_NEWLINES(yytext, yyleng);
>         if (CG(short_tags) || yyleng>2) { /* yyleng>2 means it's not <? but <script> */

---

## #15384 — John Coggeshall — 2005-03-11T21:49:50+00:00

URL: https://externals.io/message/15375#15384 · In reply to: #15381

> Interesting.  I'm wondering about the security implications of this.  
> This makes it very easy to use PHP as a means to propogate all sorts of 
> nasty things.  Well, people could even do that today in one script by 
> setting a variable to a base64 encoded string then decoding it.  None the 
> less, putting binary data in PHP scripts gives me pause.

There is no issue here. You can throw binary data at the end of a PHP
script as it is now:

<?php 

   /* stuff */
   exit;
?>
<binary data here>


 -- the halt token only makes it so PHP doesn't waste time processing
something that doesn't need to be processed. 

John

---

## #15385 — Ilia Alshanetsky — 2005-03-11T22:05:48+00:00

URL: https://externals.io/message/15375#15385 · In reply to: #15383

Wez Furlong wrote:
> If you name the token __HALT_PHP_PARSER__ instead (or something
> equally unlikely to be used by accident and unambiguous in meaning),
> you'll get a +1 from me :-)

I aim to please ;-) Here is the revised patch that makes the token a bit 
  clearer and also increases the strictness of the parser. For those of 
you wondering how would you quickly locate the end of script and start 
of data here are some solutions:

1) while (fgets($fp) != '<?php __HALT_PHP_PARSER__; ?>'));
2) Assuming the author knows the maximum length of the actual code, they 
could read X bytes and then using strpos locate the position of the HALT 
token and start reading data dump from there.
         $fp = fopen(__FILE__, "r");
         $halt_token = "<?php "."__HALT_PHP_PARSER__"."; ?>";
         $pos = strpos(fread($fp, 10000), $halt_token);
         fseek($fp, $pos + strlen($halt_token));

Ilia

---

## #15386 — Sara Golemon — 2005-03-11T22:11:41+00:00

URL: https://externals.io/message/15375#15386 · In reply to: #15385

> 1) while (fgets($fp) != '<?php __HALT_PHP_PARSER__; ?>'));
>
Just a nudge about this code approach.
  #1 It doesn't detect EOF
  #2 It forgets about the newline returned by `fgets()`
  #3 It doesn't allow the HALT; to float within a line between other content 
(not that you'd do that anyway but...

while (($tmp = fgets($fp)) && (strpos($tmp, '<?php __HALT_PHP_PARSER__; ?>') 
!== false));

---

## #15387 — Daniel Convissor — 2005-03-11T22:13:45+00:00

URL: https://externals.io/message/15375#15387 · In reply to: #15384

> There is no issue here. You can throw binary data at the end of a PHP
> script as it is now:
> 
> <?php 
> 
>    /* stuff */
>    exit;
> ?>
> <binary data here>

True enough.

Thanks,

--Dan

-- 
 T H E   A N A L Y S I S   A N D   S O L U T I O N S   C O M P A N Y
            data intensive web and database programming
                http://www.AnalysisAndSolutions.com/
 4015 7th Ave #4, Brooklyn NY 11232  v: 718-854-0335 f: 718-854-0409

---

## #15388 — Daniel Convissor — 2005-03-11T22:14:40+00:00

URL: https://externals.io/message/15375#15388 · In reply to: #15382

> Daniel Convissor wrote:
> > […]
>
> You can already use PHP to propagate all sorts of nasty things, nothing
> changes in this respect.
>
> > […]
>
> Sure, BUT this approach makes the final file approximately %30 larger
... snip...

No doubt.  My second point was more to diminish my initial assertion, not
to diminish the validity of your patch.

Thanks,

--Dan

-- 
 T H E   A N A L Y S I S   A N D   S O L U T I O N S   C O M P A N Y
            data intensive web and database programming
                http://www.AnalysisAndSolutions.com/
 4015 7th Ave #4, Brooklyn NY 11232  v: 718-854-0335 f: 718-854-0409

---

## #15389 — Ilia Alshanetsky — 2005-03-11T22:30:30+00:00

URL: https://externals.io/message/15375#15389 · In reply to: #15386

Sara Golemon wrote:
> Just a nudge about this code approach.
>   #1 It doesn't detect EOF

Well, if you encounter EOF before the HALT tag, it means as a developer 
of the script you are trying to break your own code ;-)

>   #2 It forgets about the newline returned by fgets(

True, I suppose strncmp could be used here, since newline is not always 
constant.

>   #3 It doesn't allow the HALT; to float within a line between other content 
> (not that you'd do that anyway but...

Going back to #1 ;-).

Ilia

---

## #15392 — Moriyoshi Koizumi — 2005-03-11T23:54:40+00:00

URL: https://externals.io/message/15375#15392 · In reply to: #15385

Hi,

I modified your patch so it can capture the position where the
supposed data begins into the constant __HALT_PHP_PARSER__.

There may be a problem with my patch if more than one require()'d /
include()'d script contain __HALT_PHP_PARSER__, but it'd be
quite handy if such an issue is resolved.

<?php
$fp = fopen(__FILE__, 'rb');
fseek($fp, __HALT_PHP_PARSER__, SEEK_SET);
fpassthru($fp);
?>
<?php __HALT_PHP_PARSER__; ?>
abc
def

Moriyoshi

---

## #15396 — Christian Schneider — 2005-03-12T11:58:51+00:00

URL: https://externals.io/message/15375#15396 · In reply to: #15392

Moriyoshi Koizumi wrote:
> I modified your patch so it can capture the position where the
> supposed data begins into the constant __HALT_PHP_PARSER__.

Sounds like a good idea to me as all the manual work to guess the start 
of the data look a bit kludgy to me.

> <?php
> $fp = fopen(__FILE__, 'rb');
> fseek($fp, __HALT_PHP_PARSER__, SEEK_SET);
> fpassthru($fp);
> ?>
> <?php __HALT_PHP_PARSER__; ?>
> abc
> def

Hmm.. I was wondering if we should go one step further and also provide 
a stream to the data. If the most common usage of this construct is 
going to be reading the data after __HALT_PHP_PARSER__ then we should 
optimize for that case. Something like

<?php
$data = gzdeflate(file_get_contents("php://data"));
?>
<?php __DATA__; ?>
...

- Chris

---

## #15397 — John Coggeshall — 2005-03-12T12:19:13+00:00

URL: https://externals.io/message/15375#15397 · In reply to: #15396

I think the lexer patch is good enough. 

John


> Moriyoshi Koizumi wrote:
> > […]
> 
> Sounds like a good idea to me as all the manual work to guess the start 
> of the data look a bit kludgy to me.
> 
> > […]
> 
> Hmm.. I was wondering if we should go one step further and also provide 
> a stream to the data. If the most common usage of this construct is 
> going to be reading the data after __HALT_PHP_PARSER__ then we should 
> optimize for that case. Something like
> 
> <?php
> $data = gzdeflate(file_get_contents("php://data"));
> ?>
> <?php __DATA__; ?>
> ...
> 
> - Chris

---

## #15405 — Sara Golemon — 2005-03-12T17:26:18+00:00

URL: https://externals.io/message/15375#15405 · In reply to: #15396

> Hmm.. I was wondering if we should go one step further and also provide a 
> stream to the data. If the most common usage of this construct is going to 
> be reading the data after __HALT_PHP_PARSER__ then we should optimize for 
> that case. Something like
>
> <?php
> $data = gzdeflate(file_get_contents("php://data"));
> ?>
> <?php __DATA__; ?>
> ...
I think at that point this progresses from "unobtrusive lexer enhancement" 
into the realm of "magic voodoo fluffiness".

---

## #15402 — Ilia Alshanetsky — 2005-03-12T18:20:46+00:00

URL: https://externals.io/message/15375#15402 · In reply to: #15392

Moriyoshi Koizumi wrote:
> Hi,
> 
> I modified your patch so it can capture the position where the
> supposed data begins into the constant __HALT_PHP_PARSER__.
> 
> There may be a problem with my patch if more than one require()'d /
> include()'d script contain __HALT_PHP_PARSER__, but it'd be
> quite handy if such an issue is resolved.

+1 seems like a good idea.

Ilia

---

## #15404 — Ilia Alshanetsky — 2005-03-12T18:27:19+00:00

URL: https://externals.io/message/15375#15404 · In reply to: #15396

> Hmm.. I was wondering if we should go one step further and also provide 
> a stream to the data. If the most common usage of this construct is 
> going to be reading the data after __HALT_PHP_PARSER__ then we should 
> optimize for that case. Something like
> 
> <?php
> $data = gzdeflate(file_get_contents("php://data"));
> ?>
> <?php __DATA__; ?>
> ...

I would suggest keeping such implementation in the user-land code rather 
then the core. It would be fairly trivial to implement a streams wrapper 
to do this and it can internally handle various decoding schema like 
decompression etc...

Ilia