Hi guys,
Just wanted to know if those issues are resolved?
I rather not have a flow of changes in this area just before the next
RC2.
As a matter of fact this is the main thing that needs to get done
before RC2.
Other things are listed here:
http://wiki.php.net/todo/php53#rc2
I would not mind being able to release RC Thursday next week. Which
would mean commit freeze starting Monday next week. So how are things
looking?
regards,
Lukas Kahwe Smith
mls@pooteeweet.org
Hi guys,
Just wanted to know if those issues are resolved?
I rather not have a flow of changes in this area just before the next RC2.
As a matter of fact this is the main thing that needs to get done before
RC2.
Other things are listed here:
http://wiki.php.net/todo/php53#rc2I would not mind being able to release RC Thursday next week. Which would
mean commit freeze starting Monday next week. So how are things looking?
No chance from my side to get a commit freeze on next Monday. It is
way too short sorry.
Cheers,
Pierre
On Tue, Mar 31, 2009 at 12:09 AM, Lukas Kahwe Smith <mls@pooteeweet.org
wrote:
Hi guys,Just wanted to know if those issues are resolved?
I rather not have a flow of changes in this area just before the
next RC2.
As a matter of fact this is the main thing that needs to get done
before
RC2.
Other things are listed here:
http://wiki.php.net/todo/php53#rc2I would not mind being able to release RC Thursday next week. Which
would
mean commit freeze starting Monday next week. So how are things
looking?No chance from my side to get a commit freeze on next Monday. It is
way too short sorry.
Ok, then I would appreciate estimates for each item that cannot be
completed until then.
As soon as you can give them ...
regards,
Lukas Kahwe Smith
mls@pooteeweet.org
On Tue, Mar 31, 2009 at 12:09 AM, Lukas Kahwe Smith <mls@pooteeweet.org
wrote:
Hi guys,Just wanted to know if those issues are resolved?
I rather not have a flow of changes in this area just before the
next RC2.
As a matter of fact this is the main thing that needs to get done
before
RC2.
Other things are listed here:
http://wiki.php.net/todo/php53#rc2I would not mind being able to release RC Thursday next week.
Which would
mean commit freeze starting Monday next week. So how are things
looking?No chance from my side to get a commit freeze on next Monday. It is
way too short sorry.Ok, then I would appreciate estimates for each item that cannot be
completed until then.
As soon as you can give them ...
So nobody is able to give me any feedback, so that we can plan RC2? I
must say this is kind of irritating, but so it goes.
regards,
Lukas Kahwe Smith
mls@pooteeweet.org
Hi Lukas,
(I see you CC'd Christian, but I don't think he's involved with this...)
----- Original Message -----
From: "Lukas Kahwe Smith"
Sent: Saturday, April 04, 2009
On Tue, Mar 31, 2009 at 12:09 AM, Lukas Kahwe Smith <mls@pooteeweet.org
wrote:
Hi guys,Just wanted to know if those issues are resolved?
I rather not have a flow of changes in this area just before the next
RC2.
As a matter of fact this is the main thing that needs to get done
before
RC2.
Other things are listed here:
http://wiki.php.net/todo/php53#rc2I would not mind being able to release RC Thursday next week. Which
would
mean commit freeze starting Monday next week. So how are things
looking?No chance from my side to get a commit freeze on next Monday. It is
way too short sorry.Ok, then I would appreciate estimates for each item that cannot be
completed until then.
As soon as you can give them ...So nobody is able to give me any feedback, so that we can plan RC2? I
must say this is kind of irritating, but so it goes.
Sorry if I was irritating (in addition to my noise about this number stuff).
:-) When you sent the first message of this thread last week, I was just
starting to look at changing the conversion to something I'd be more
satisfied with than the patch I sent. So I was waiting to see what to
report before replying. Then Thursday I replied to Dmitry with some of
those details, instead of replying to you... Then I was going to reply to
this last message, asking for feedback, a few hours after you sent it, but
didn't get time. Glad I didn't though (not to irritate you more, ha),
because yesterday after testing things further with my more "more complete
and precise" conversion method, I'm thinking differently after noting
behavior patterns -- on Windows and [firsts for me] 32- and 64-bit Linux.
(Probably a good thing, because what I had, while only 70-ish lines (with
duplication) looked like overkill, with more "magic" than Dmitry commented
on with the other version!)
So, whether that made any sense to anyone, what does this mean? Well, now I
think the best proposal will be to use the same conversion as 5.2 and
earlier, with only a small change to address the issue Zoe reported in Bug
#42868, which caused the 5.3 change. This would keep/restore the old
overflow behavior that most people have always been getting with PHP by
having conversion behave the same on a platform where 5.2 didn't work the
"usual" way -- like the Mac in Zoe's report. It's very simple and would
look almost exactly like 5.2's code. :-) Since nobody has tried to define
what exactly the behavior should be, after analyzing everything I can
gather, I think it's a good, simple solution to keep things consistent,
within the limitations I've observed across platforms...
Still thinking about one thing, but I'll send an updated patch in a day or
two I guess.
Oh, other than that about the RC2 subject, I was also going to propose some
scanner changes like I mentioned to Brian on the 26th. Assuming it works
like in my head, it would basically be a big "diet" for the scanner, making
string/comment handling smaller and simpler, while addressing the cause (for
strings/comments at least) of Bug #46817 (tokenizer misses last single-line
comment) which is wrongly marked Closed and DONE on the wiki. Before RC1,
Brian reverted his changes that, I guess, had fixed it.
regards,
Lukas Kahwe Smith
mls@pooteeweet.org
- Matt
Matt Wilmas wrote:
Oh, other than that about the RC2 subject, I was also going to propose
some scanner changes like I mentioned to Brian on the 26th. Assuming it
works like in my head, it would basically be a big "diet" for the
scanner, making string/comment handling smaller and simpler, while
addressing the cause (for strings/comments at least) of Bug #46817
(tokenizer misses last single-line comment) which is wrongly marked
Closed and DONE on the wiki. Before RC1, Brian reverted his changes
that, I guess, had fixed it.
It is? I just tested this again to be sure something wasn't missed and it looks like it's working to me. Which test case where you using?
-shire
Hi Brian,
----- Original Message -----
From: "shire"
Sent: Monday, April 06, 2009
Matt Wilmas wrote:
Oh, other than that about the RC2 subject, I was also going to propose
some scanner changes like I mentioned to Brian on the 26th. Assuming it
works like in my head, it would basically be a big "diet" for the
scanner, making string/comment handling smaller and simpler, while
addressing the cause (for strings/comments at least) of Bug #46817
(tokenizer misses last single-line comment) which is wrongly marked
Closed and DONE on the wiki. Before RC1, Brian reverted his changes
that, I guess, had fixed it.It is? I just tested this again to be sure something wasn't missed and it
looks like it's working to me. Which test case where you using?
Yep, 5.3's snapshot self-compiled from a couple days ago on Windows (not
that that should matter). (I'm not regenerating it with re2c, which also
shouldn't matter; using the existing .c file. I haven't touched the scanner
stuff in a long time (yet) to regen.) Scanner of course hasn't changed
since then.
Test case is the one in the bug report. :-) Last token is not the comment,
but whitespace. No problem on 5.2.9. (From the scanner code, and current
re2c behavior, I don't see why the behavior would be different than what I'm
getting.)
Also, the unterminated comment Warning is still missing with "<?php /* blah
" like it's been since the re2c change (except maybe for the time your fix
was applied). My changes would clean this up of course, unless you do
something first.
-shire
- Matt
Hey Matt,
Matt Wilmas wrote:
Yep, 5.3's snapshot self-compiled from a couple days ago on Windows (not
that that should matter). (I'm not regenerating it with re2c, which also
shouldn't matter; using the existing .c file. I haven't touched the
scanner stuff in a long time (yet) to regen.) Scanner of course hasn't
changed since then.
Here's what I'm currently doing (more or less with some changed paths):
$ echo $PHPCVS
:pserver:shire@cvs.php.net:/repository
$ cvs -d $PHPCVS co -r PHP_5_3 php5
...
$ cd php5
$ ./buildconf && ../config.nice && make
...
$ cat ../bug46817-1.php
<?php
var_dump(token_get_all(file_get_contents('../bug46817-2.php')));
$ cat ../bug46817-2.php
<?php
// this comment and trailing blank contain windows CR+LF
$ ./sapi/cli/php ../bug46817-1.php
array(3) {
[0]=>
array(3) {
[0]=>
int(368)
[1]=>
string(6) "<?php
"
[2]=>
int(1)
}
[1]=>
array(3) {
[0]=>
int(366)
[1]=>
" string(57) "// this comment and trailing blank contain windows CR+LF
[2]=>
int(2)
}
[2]=>
array(3) {
[0]=>
int(371)
[1]=>
string(3) "
"
[2]=>
int(2)
}
}
The newlines look like this in the second file:
<?php$
// this comment and trailing blank contain windows CR+LF^M$
^M$
Unfortunately I can't test on a windows build, perhaps you could re-test or share your reproduction that fails as this seems to work for me unless I'm of course missing some difference.
Test case is the one in the bug report. :-) Last token is not the
comment, but whitespace.
There are two reproductions in the bug report ;-)
Also, the unterminated comment Warning is still missing with "<?php /*
blah " like it's been since the re2c change (except maybe for the time
your fix was applied). My changes would clean this up of course, unless
you do something first.
I think fixing this would be great as well as the other highlighter test that was changed. I would just prefer that the scanner handle these rather than us implementing what is essentially a hand-written scanner within the lexer file.
-shire
Hi again Brian,
----- Original Message -----
From: "shire"
Sent: Monday, April 06, 2009
Hey Matt,
Matt Wilmas wrote:
Yep, 5.3's snapshot self-compiled from a couple days ago on Windows (not
that that should matter). (I'm not regenerating it with re2c, which also
shouldn't matter; using the existing .c file. I haven't touched the
scanner stuff in a long time (yet) to regen.) Scanner of course hasn't
changed since then.Here's what I'm currently doing (more or less with some changed paths):
[...]
[1]=>
array(3) {
[0]=>
int(366)
[1]=>
" string(57) "// this comment and trailing blank contain windows CR+LF
[2]=>
int(2)
}
As a side note, I just noticed that the full Windows newline (\r\n, CR+LF)
isn't getting taken with the comment (\n included in WHITESPACE after), as
*nix's \n does. See the " before string(57)? Because the CR is resetting
the line I guess, without going to the next. It's this rule:
<ST_ONE_LINE_COMMENT>[^\n\r?%>]*{ANY_CHAR} {
that's only matching the \r before returning T_COMMENT. Simple enough to
fix as well, but I hadn't spotted that one before until I was trying to see
why that quote was out-of-place. :-) (This isn't new in 5.3 though...)
[2]=>
array(3) {
[0]=>
int(371)
[1]=>
string(3) ""
[2]=>
int(2)
}
}The newlines look like this in the second file:
<?php$
// this comment and trailing blank contain windows CR+LF^M$
^M$Unfortunately I can't test on a windows build, perhaps you could re-test
or share your reproduction that fails as this seems to work for me unless
I'm of course missing some difference.Test case is the one in the bug report. :-) Last token is not the
comment, but whitespace.There are two reproductions in the bug report ;-)
Oops, forgot about the second one -- I meant the first in the initial
report. The part I'm talking about is: "It only seems to occur if there
isn't a newline behind the comment." So the easiest way to see is simply:
var_dump(token_get_all('<?php // test'));
array(1) {
[0]=>
array(3) {
[0]=>
int(368)
[1]=>
string(6) "<?php "
[2]=>
int(1)
}
}
Also, the unterminated comment Warning is still missing with "<?php /*
blah " like it's been since the re2c change (except maybe for the time
your fix was applied). My changes would clean this up of course, unless
you do something first.I think fixing this would be great as well as the other highlighter test
that was changed. I would just prefer that the scanner handle these
rather than us implementing what is essentially a hand-written scanner
within the lexer file.
Yeah, I remember you said that last time. :-) But like the inline HTML
scanner part you mentioned then, if it's pretty simple to implement
manually, I thought it seemed logical (I don't know if that stuff was
possible with how flex worked; it was only after seeing the HTML scanning
that I thought, "Ah.") The regex would've generated more code, and probably
wouldn't make much difference for readability...? (I still wonder if it
wasn't used because it wouldn't work with the re2c issues otherwise.) With
the string, etc. scanning, my regular expressions are pretty complicated, to
match stuff that isn't very complicated, which generates a LOT of code, and
probably aren't that readable or easy to understand, even with the comments.
Well anyway, if I do something I'll send it along for analysis!
-shire
- Matt
Hi,
@Matt: thx for getting back to me on this.
In general all the technical details are way over my head. What
worries me is that we are going back and forth with these changes. It
seems there is a pretty large "trail and error" part to these changes.
So for the most part at this stage I do not care all that much about
fixing things that were broken in 5.2 and when in doubt I would rather
stick with behavior form 5.2, than continue messing around with a
better approach for 5.3. At this stage the best service we can do our
users is to release 5.3 ASAP.
So please everybody working on these changes, keep the above in mind.
As soon as you can give me some more details on when all of this will
be completed I would appreciate it, at which stage I will make a
release plan for RC2.
I know that there are still some issues on the windows side. I am
waiting for details on that, but do note that RC2 will not wait for
windows todo items that are not spelled out at this point. even then
it will be decided on a case by case basis .. again .. if something
was broken in 5.2 and will remain broken in 5.3 .. so be it .. we have
enough improvements for a couple of minor releases at this point, that
are waiting to be released with 5.3.
Aside from this nobody raised their hands, so I will not wait for any
thing else. I would also appreciate it if people would refrain from
committing anything but clear bug fixes at this point. Anything with
any level of a BC break should be brought to the list first. Keep in
mind that HEAD is for development.
regards,
Lukas