# Support negative indexes for arrays and strings - Thread: https://externals.io/message/62606 - Original archive: https://news-web.php.net/php.internals/62606 - Started by Rasmus Schultz on 2012-08-31T13:14:10+00:00 - Messages: 12 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #62606 — Rasmus Schultz — 2012-08-31T13:14:10+00:00 URL: https://externals.io/message/62606#62606 Having thought about this for a while, I think this is a bad idea - here's why: $array = array(1001, 1002, 1003, 1004); $number = $array[-1]; // => 1004 $number[-1] = 1005; $number = $array[-1]; // => ???? Obviously, the last statement must return 1005, since otherwise that value would be inaccessible. On the other hand, to know whether you have to know whether your array is currently a hash or not - since array index now works completely differently depending on whether your array is currently a hash or an array. This is not something you can currently even test for - and not something you should really need to know or care about in the first place. Having to know this information in certain cases would defeat the purpose of having a single collection-type in the first place: it really *should* work the same, all the time. Just my $0.02... --- ## #62607 — Ferenc Kovacs — 2012-08-31T13:24:37+00:00 URL: https://externals.io/message/62606#62607 · In reply to: #62606 > Having thought about this for a while, I think this is a bad idea - here's > why: > > $array = array(1001, 1002, 1003, 1004); > > $number = $array[-1]; // => 1004 > > $number[-1] = 1005; > > $number = $array[-1]; // => ???? > > Obviously, the last statement must return 1005, since otherwise that > value would be inaccessible. > > maybe you wanted to write $array[-1] = 1005; instead of $number[-1] = 1005; ? otherwise I can't see the problem that you are mentioning. could you clarify? -- Ferenc Kovács @Tyr43l - http://tyrael.hu --- ## #62613 — Rasmus Schultz — 2012-08-31T17:11:01+00:00 URL: https://externals.io/message/62606#62613 · In reply to: #62607 Yes, typo! sorry. $array = array(1001, 1002, 1003, 1004); $number = $array[-1]; // => 1004 $array[-1] = 1005; $number = $array[-1]; // => ???? Looking at the resulting code, I would like to point out also that it's extremely misleading... because $array[-1] references two completely different elements depending on whether you're reading or writing the value... unless $array[-1] = 1005 would actually overwrite the last element in the array - in which case it gets even more dangerous, as well as breaking backwards compatibility... > > > > > […] > > maybe you wanted to write > $array[-1] = 1005; > instead of > $number[-1] = 1005; > ? > > otherwise I can't see the problem that you are mentioning. > could you clarify? > > -- > Ferenc Kovács > @Tyr43l - http://tyrael.hu --- ## #62617 — Tjerk Meesters — 2012-09-01T03:59:53+00:00 URL: https://externals.io/message/62606#62617 · In reply to: #62613 > Yes, typo! sorry. > > $array = array(1001, 1002, 1003, 1004); > $number = $array[-1]; // => 1004 > $array[-1] = 1005; > $number = $array[-1]; // => ???? > > Looking at the resulting code, I would like to point out also that > it's extremely misleading... because $array[-1] references two > completely different elements depending on whether you're reading or > writing the value... unless $array[-1] = 1005 would actually overwrite > the last element in the array - in which case it gets even more > dangerous, as well as breaking backwards compatibility... That might actually be something I could use :) But the fun for me begins here: $numbers = array(); $numbers[-1] = 5; $numbers[] = 6; What would have happened to the keys? Normally [] is equivalent to [count($numbers)]. Perhaps this is why JS treats negative indices as properties instead :) > > > > > […] --- ## #62618 — Sherif Ramadan — 2012-09-01T05:24:10+00:00 URL: https://externals.io/message/62606#62618 · In reply to: #62617 > > That might actually be something I could use :) But the fun for me begins here: > > $numbers = array(); > $numbers[-1] = 5; > $numbers[] = 6; > > What would have happened to the keys? Normally [] is equivalent to [count($numbers)]. > This is incorrect, $numbers[] = 6; is not equivalent to $numbers[count($numbers)] = 6; at all, because PHP uses an internal value for this that is represented an unsigned long in the Hashtable ht struct. As it stands today, that code would result in the following: $numbers = array(); $numbers[-1] = 5; $numbers[] = 6; var_dump($numbers); /* array(2) { [-1]=> int(5) [0]=> int(6) } */ Which looks awkward, I know, but the reason is that the internal value that represents the next key to be used when no key is supplied during an array push is an unsigned figure while integer values in PHP are all signed. Since one's complement is used this means the unsigned value will always wrap-around causing -1 to take us back to zero (you can't overflow an unsigned long). Here's how we can see this quirk rearing it's ugly head: $numbers = array(); $numbers[PHP_INT_MAX] = 'foo'; $numbers[] = 'bar'; var_dump($numbers); /* Warning: Cannot add element to the array as the next element is already occupied in ... on line 1 array(1) { [9223372036854775807]=> string(3) "foo" } */ As you can see PHP has a bit of a hickup here. The implementation is such that the index is always initialized to 0 by default and clamped by the API (so when a negative value is supplied we still start back at 0). Here's the actual implementation of array_psuh: for (i = 0; i < argc; i++) { new_var = *args[i]; Z_ADDREF_P(new_var); if (zend_hash_next_index_insert(Z_ARRVAL_P(stack), &new_var, sizeof(zval *), NULL) == FAILURE) { Z_DELREF_P(new_var); php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot add element to the array as the next element is already occupied"); efree(args); RETURN_FALSE; } } $array = array(); $array[PHP_INT_MAX+1] = 'foo'; $array[] = 'bar'; var_dump($array); /* array(2) { [-9223372036854775808]=> string(3) "foo" [0]=> string(3) "bar" } */ Another gotchya, of the API is that the representation of the key used in the array and the next available key can be somewhat two misleading. --- On a final note the confusion behind proposing the use of negative indexes for array lookup in PHP is that PHP doesn't really have an array. Something I believe has been discussed at length in the past. Which is why it makes sense to separate the "key" used in the array from the "offset" of the element in the ordered map. $array = array(-1 => "foo",0 => "bar",3 => "baz") array_slice($array, -1); // makes perfect sense to me that this would return "baz" $array = array(-1 => "foo",0 => "bar",3 => "baz") echo $array[-1]; // makes perfect sense to me that this would return "foo" --- ## #62620 — Kris Craig — 2012-09-01T08:17:29+00:00 URL: https://externals.io/message/62606#62620 · In reply to: #62618 On Fri, Aug 31, 2012 at 10:24 PM, Sherif Ramadan wrote: > > […] > begins here: > > […] > [count($numbers)]. > > […] > > This is incorrect, $numbers[] = 6; is not equivalent to > $numbers[count($numbers)] = 6; at all, because PHP uses an internal > value for this that is represented an unsigned long in the Hashtable > ht struct. > > As it stands today, that code would result in the following: > > $numbers = array(); > $numbers[-1] = 5; > $numbers[] = 6; > var_dump($numbers); > /* > array(2) { > [-1]=> > int(5) > [0]=> > int(6) > } > */ > > Which looks awkward, I know, but the reason is that the internal value > that represents the next key to be used when no key is supplied during > an array push is an unsigned figure while integer values in PHP are > all signed. Since one's complement is used this means the unsigned > value will always wrap-around causing -1 to take us back to zero (you > can't overflow an unsigned long). > > Here's how we can see this quirk rearing it's ugly head: > > $numbers = array(); > $numbers[PHP_INT_MAX] = 'foo'; > $numbers[] = 'bar'; > var_dump($numbers); > /* > Warning: Cannot add element to the array as the next element is > already occupied in ... on line 1 > array(1) { > [9223372036854775807]=> > string(3) "foo" > } > */ > > As you can see PHP has a bit of a hickup here. The implementation is > such that the index is always initialized to 0 by default and clamped > by the API (so when a negative value is supplied we still start back > at 0). > > Here's the actual implementation of array_psuh: > > for (i = 0; i < argc; i++) { > new_var = *args[i]; > Z_ADDREF_P(new_var); > > if (zend_hash_next_index_insert(Z_ARRVAL_P(stack), &new_var, > sizeof(zval *), NULL) == FAILURE) { > Z_DELREF_P(new_var); > php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot add > element to the array as the next element is already occupied"); > efree(args); > RETURN_FALSE; > } > } > > > > $array = array(); > $array[PHP_INT_MAX+1] = 'foo'; > $array[] = 'bar'; > var_dump($array); > /* > array(2) { > [-9223372036854775808]=> > string(3) "foo" > [0]=> > string(3) "bar" > } > */ > > Another gotchya, of the API is that the representation of the key used > in the array and the next available key can be somewhat two > misleading. > > --- > > On a final note the confusion behind proposing the use of negative > indexes for array lookup in PHP is that PHP doesn't really have an > array. Something I believe has been discussed at length in the past. > Which is why it makes sense to separate the "key" used in the array > from the "offset" of the element in the ordered map. > > $array = array(-1 => "foo",0 => "bar",3 => "baz") > array_slice($array, -1); // makes perfect sense to me that this would > return "baz" > > $array = array(-1 => "foo",0 => "bar",3 => "baz") > echo $array[-1]; // makes perfect sense to me that this would return "foo" > > -- > > > > This discussion kinda reminds me of some of the debates over AUTO_INCREMENT behavior in the MySQL community. Specifically, they end up having to tackle the same funcamental, conceptual dilemma: If I assign/insert/whatever an arbitrary value to a container that can be incremented, and then I direct said container to generate the next increment, what value should that be? What's the most sensible (or perhaps the least unsensible) way to determine that? Of course, I don't really know what the answer is. From a strictly logical standpoint (ignoring the fact that it just doesn't work this way in PHP), my thinking would be that the count-based index approach makes the most sense. I think many (if not most) PHP developers incorrectly assume this to already be the case now. What's my point? I'm not sure that I have one-- but if I did, the point would be that any approach to handling this will likely be problematic in some way or another. That said, I think we should find a way to support negative indexes. There were times when being able to do that would have been very convenient for me. =) --Kris --- ## #62621 — Lester Caine — 2012-09-01T08:52:55+00:00 URL: https://externals.io/message/62606#62621 · In reply to: #62618 Sherif Ramadan wrote: > As it stands today, that code would result in the following: > > $numbers = array(); > $numbers[-1] = 5; > $numbers[] = 6; > var_dump($numbers); > /* > array(2) { > [-1]=> > int(5) > [0]=> > int(6) > } > */ I think that it is just clarifying this further by adding $numbers = array(0,1,2,3,4); Which gives array(7) { [0]=> int(0) [1]=> int(1) [2]=> int(2) [3]=> int(3) [4]=> int(4) [-1]=> int(5) [5]=> int(6) } and foreach( $numbers as $value ) { print($value); } gives 0123456 as I would have expected! AND print( $numbers[-1] ); still returns '5' I can NOW use `ksort()` to really move the '5' to the top of the list. array(7) { [-1]=> int(5) [0]=> int(0) [1]=> int(1) [2]=> int(2) [3]=> int(3) [4]=> int(4) [5]=> int(6) } Bottom line ... perhaps we need a DIFFERENT means of identifying position in the array since using -ve keys IS a valid situation in real life. Personally I use -ve positions in the database tables to save having to process all records when adding items at the top, so the associative array quite correctly has -ve key values. I would not want PHP screwing around with that when reading the array in :) I think I am right in saying that 'real array' always start at 0 or 1, and positioning is explicit. So for simple stings then ne6ative indexes make perfect sense. But for PHP 'container arrays' we can legally have -ve keys, and removing that is simply not acceptable. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk --- ## #62622 — Sherif Ramadan — 2012-09-01T09:07:30+00:00 URL: https://externals.io/message/62606#62622 · In reply to: #62620 > > This discussion kinda reminds me of some of the debates over AUTO_INCREMENT > behavior in the MySQL community. Specifically, they end up having to tackle > the same funcamental, conceptual dilemma: If I assign/insert/whatever an > arbitrary value to a container that can be incremented, and then I direct > said container to generate the next increment, what value should that be? > What's the most sensible (or perhaps the least unsensible) way to determine > that? > > Of course, I don't really know what the answer is. From a strictly logical > standpoint (ignoring the fact that it just doesn't work this way in PHP), my > thinking would be that the count-based index approach makes the most sense. > I think many (if not most) PHP developers incorrectly assume this to already > be the case now. > > What's my point? I'm not sure that I have one-- but if I did, the point > would be that any approach to handling this will likely be problematic in > some way or another. That said, I think we should find a way to support > negative indexes. There were times when being able to do that would have > been very convenient for me. =) > > --Kris > I really think people are misunderstanding both the problem and the solution here so I will try to make this very simple. * Problem: Finding an element in a PHP array by its offset * Solution: array_slice($array, -1) * Confusion: Array offsets are not indexes or array keys * Deep confusion: $array[-1] can be used to access an element by its offset * Twilight Zone: PHP Arrays are ordered by their keys The problem here is not that PHP doesn't support or allow you to access the element in an array by its offset or that a negative offset can't reach into the bottom/end of the array. The problem is that people are confusing an offset with an index. PHP arrays have keys. The $array[] syntax allows us to access an array element by its key, not by its offset. The reason is simple: PHP Arrays are ordered hashmaps (they are made up of ordered key/value pairs). To confuse people now by modifying this syntax to allow you to do what `array_slice()` already does perfectly well, would make using the same syntax for accessing array values by their keys impossible. --- ## #62623 — Sherif Ramadan — 2012-09-01T09:17:40+00:00 URL: https://externals.io/message/62606#62623 · In reply to: #62621 > Sherif Ramadan wrote: > > […] > > > I think that it is just clarifying this further by adding > $numbers = array(0,1,2,3,4); > Which gives > array(7) { [0]=> int(0) [1]=> int(1) [2]=> int(2) [3]=> int(3) [4]=> int(4) > [-1]=> int(5) [5]=> int(6) } I have no idea how it would. I think you supplied the wrong code here. > and > foreach( $numbers as $value ) { print($value); } gives 0123456 as I would > have expected! > AND print( $numbers[-1] ); still returns '5' > > I can NOW use `ksort()` to really move the '5' to the top of the list. > array(7) { [-1]=> int(5) [0]=> int(0) [1]=> int(1) [2]=> int(2) [3]=> int(3) > [4]=> int(4) [5]=> int(6) } > > Bottom line ... perhaps we need a DIFFERENT means of identifying position in > the array since using -ve keys IS a valid situation in real life. Personally > I use -ve positions in the database tables to save having to process all > records when adding items at the top, so the associative array quite > correctly has -ve key values. I would not want PHP screwing around with that > when reading the array in :) I can't understand what you mean by "Different means of identifying position in the array"? If you mean a way to access an array's element by its position in the array then yes, we already have that. It's called `array_slice()` see http://php.net/array-slice which allows you to access elements in the array by their offset and that includes using a negative offset. The key and the offset are two completely different things. > > I think I am right in saying that 'real array' always start at 0 or 1, and > positioning is explicit. So for simple stings then ne6ative indexes make > perfect sense. But for PHP 'container arrays' we can legally have -ve keys, > and removing that is simply not acceptable. > Yes, the position is always explicit, the key isn't ever responsible for determining the position of an element in the array. The value of the PHP key has never been responsible for determining the position of the element in the array. $array[99999] = 'first element'; $array[0] = 'second element'; var_dump($array); /* array(2) { [99999]=> string(13) "first element" [0]=> string(14) "second element" } */ $array[0] = 'first element'; $array[99999] = 'second element'; var_dump($array); /* array(2) { [99999]=> string(14) "second element" [0]=> string(13) "first element" } */ The order has always been determined internally so the values of these keys mean nothing in terms of order as you can see. > -- > Lester Caine - G8HFL > ----------------------------- > Contact - http://lsces.co.uk/wiki/?page=contact > L.S.Caine Electronic Services - http://lsces.co.uk > EnquirySolve - http://enquirysolve.com/ > Model Engineers Digital Workshop - http://medw.co.uk > Rainbow Digital Media - http://rainbowdigitalmedia.co.uk --- ## #62624 — Sherif Ramadan — 2012-09-01T09:23:00+00:00 URL: https://externals.io/message/62606#62624 · In reply to: #62623 > > $array[0] = 'first element'; > $array[99999] = 'second element'; > var_dump($array); > /* > array(2) { > [99999]=> > string(14) "second element" > [0]=> > string(13) "first element" > } > */ Just correcting this as it was a copy/paste fail... The above code would produce: array(2) { [0]=> string(13) "first element" [99999]=> string(14) "second element" } --- ## #62626 — Lester Caine — 2012-09-01T09:35:43+00:00 URL: https://externals.io/message/62606#62626 · In reply to: #62623 Sherif Ramadan wrote: > I can't understand what you mean by "Different means of identifying > position in the array"? If you mean a way to access an array's element > by its position in the array then yes, we already have that. It's > called `array_slice()` seehttp://php.net/array-slice which allows you > to access elements in the array by their offset and that includes > using a negative offset. The key and the offset are two completely > different things. Never had to use it ;) But then there is a heck of a lot of stuff I've never even looked at. But it does blow this thread - when referring to collection arrays - out of the water? The problem I see is people trying to apply 'string' rules in the wrong way ... and I find that string handling works fine for me as it currently works but we probably just need to STOP people trying to apply the same rules to something that is totally different? And I though that 'string' -ve indexes had been sorted? -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk --- ## #62632 — Lester Caine — 2012-09-01T12:07:21+00:00 URL: https://externals.io/message/62606#62632 · In reply to: #62626 Lester Caine wrote: > Sherif Ramadan wrote: > > […] > > Never had to use it ;) But then there is a heck of a lot of stuff I've never > even looked at. > But it does blow this thread - when referring to collection arrays - out of the > water? The problem I see is people trying to apply 'string' rules in the wrong > way ... and I find that string handling works fine for me as it currently works > but we probably just need to STOP people trying to apply the same rules to > something that is totally different? And I though that 'string' -ve indexes had > been sorted? OK ... got back form the weekly chore of shopping, and was thinking about what I had said :) "Different means of identifying position in the array"? I was probably thinking more along the lines of $numbers[-1] is a perfectly valid key on that array, so there is no way that it can be redefined otherwise. It would have to be something completely different such as $numbers[(-1)] to differentiate from the normal use of a key in the array ... and I would have to object strongly to adding that since it is different to other methods. The problem here was accepting that [-1] works differently on a string, and now we are seeing the consequences? It is only valid on a string and so this discussion needs a big brick wall between strings and 'collection arrays' ? People who think they need a position based indexing obviously need educating in the use of the array_slice and the other functions that seem to go with that, but I can see that this method of handling array indexes ( rather than keys ) COULD benefit from a tidy up. As has been discussed in general on the whole array API? -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk