Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:128958
Precedence: list
MIME-Version: 1.0
References: <CAE1541vPXkgZsxfAiWJyosnhDksGT=2tDJyoYOcJtG16jZh9nA@mail.gmail.com>
 <4b605609-47c2-4df5-be71-8b962ad9ff1e@varteg.nz> <CAE1541vL=D2ni1L+Hg-WzuWdNcKHMwRoq64PeEgzSRmJ3_kAVQ@mail.gmail.com>
In-Reply-To: <CAE1541vL=D2ni1L+Hg-WzuWdNcKHMwRoq64PeEgzSRmJ3_kAVQ@mail.gmail.com>
Date: Fri, 24 Oct 2025 17:52:18 -0700
Message-ID: <CAE1541uu0ZFUw0Qj-m87h1=r+C07=6z67tcKkxnFrL1GVb+QWQ@mail.gmail.com>
Subject: Re: [PHP-DEV] RFC proposal for adding SORT_STRICT flag to array_unique()
To: Weedpacket@varteg.nz
Cc: internals@lists.php.net
Content-Type: multipart/alternative; boundary="000000000000a2a7d60641f114fa"
From: jmarble@intuitivetechnology.com (Jason Marble)

--000000000000a2a7d60641f114fa
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Quick POC:
https://github.com/jmarble/php-src/tree/feature/array-unique-sort-strict

~1.4x faster than this simple userland implementation on my local machine.
I purposefully avoided implementing a hash-bucket because I had already
tried that and encountered too many edge cases LOL:
https://gist.github.com/jmarble/1e08eb15274cd434e867baf96ffa301d

On Fri, Oct 24, 2025 at 4:51=E2=80=AFPM Jason Marble <
jmarble@intuitivetechnology.com> wrote:

> Correct! Basically:
> - SORT_STRINGS: reliable and predictable when you understand the value
> will be converted to a string
> - SORT_NUMERIC: same but _risky_, you should be certain you're working
> with numbers
> - SORT_REGULAR: the sort is unstable and will inevitably cause a bug that
> no one will understand LOL
>
> With the proposed SORT_STRICT, we will get super fast, reliable and
> predictable deduplication.
>
>
> On Fri, Oct 24, 2025 at 3:16=E2=80=AFPM Morgan <Weedpacket@varteg.nz> wro=
te:
>
>> On 2025-10-25 08:34, Jason Marble wrote:
>> > Hello everybody!
>> >
>> >
>> > The potential for a `SORT_NATURAL` flag also came to mind as another
>> > useful addition, but I believe `SORT_STRICT` is the more critical
>> > feature to discuss first.
>> >
>>
>> I know I find array_unique generally useless due to its insistence on
>> stringifying everything for comparison.
>>
>> ```
>> $uniques =3D [];
>> foreach($source_array as $a) {
>>      if(!in_array($a, $uniques, true)) {
>>          $uniques[] =3D $a;
>>      }
>> }
>> ```
>>
>> I seem to recall part of the issue is that array_unique works by sorting
>> its elements so that "equal" values are adjacent. I know this would be
>> done on O(n log(n)) vs. O(n^2) grounds, but that could be addressed at
>> least in part by a smarter sort criterion that sorts by type/class (in
>> some arbitrary order) before sorting by value. For uncomparable types
>> (i.e., instances of most classes) this would be by object ID, because we
>> don't _actually_ care about ordering.
>>
>

--000000000000a2a7d60641f114fa
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Quick POC:<br><a href=3D"https://github.com/jmarble/php-sr=
c/tree/feature/array-unique-sort-strict">https://github.com/jmarble/php-src=
/tree/feature/array-unique-sort-strict</a><br><br>~1.4x faster than=C2=A0th=
is simple userland implementation on my local machine. I purposefully avoid=
ed implementing a hash-bucket because I had already tried that and encounte=
red too many edge cases LOL:<br><a href=3D"https://gist.github.com/jmarble/=
1e08eb15274cd434e867baf96ffa301d">https://gist.github.com/jmarble/1e08eb152=
74cd434e867baf96ffa301d</a></div><br><div class=3D"gmail_quote gmail_quote_=
container"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, Oct 24, 2025 at 4:=
51=E2=80=AFPM Jason Marble &lt;<a href=3D"mailto:jmarble@intuitivetechnolog=
y.com">jmarble@intuitivetechnology.com</a>&gt; wrote:<br></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div>Correct! Basica=
lly:<br>- SORT_STRINGS: reliable and predictable when you understand the va=
lue will be converted to a string<br>- SORT_NUMERIC: same but _risky_, you =
should be certain you&#39;re working with numbers<br>- SORT_REGULAR: the so=
rt is unstable and will inevitably cause a bug that no one will understand =
LOL</div><div><br>With the proposed SORT_STRICT, we will get super fast, re=
liable and predictable deduplication.<br><br></div><br><div class=3D"gmail_=
quote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, Oct 24, 2025 at 3:16=
=E2=80=AFPM Morgan &lt;<a href=3D"mailto:Weedpacket@varteg.nz" target=3D"_b=
lank">Weedpacket@varteg.nz</a>&gt; wrote:<br></div><blockquote class=3D"gma=
il_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,2=
04,204);padding-left:1ex">On 2025-10-25 08:34, Jason Marble wrote:<br>
&gt; Hello everybody!<br>
&gt; <br>
&gt; <br>
&gt; The potential for a `SORT_NATURAL` flag also came to mind as another <=
br>
&gt; useful addition, but I believe `SORT_STRICT` is the more critical <br>
&gt; feature to discuss first.<br>
&gt; <br>
<br>
I know I find array_unique generally useless due to its insistence on <br>
stringifying everything for comparison.<br>
<br>
```<br>
$uniques =3D [];<br>
foreach($source_array as $a) {<br>
=C2=A0 =C2=A0 =C2=A0if(!in_array($a, $uniques, true)) {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0$uniques[] =3D $a;<br>
=C2=A0 =C2=A0 =C2=A0}<br>
}<br>
```<br>
<br>
I seem to recall part of the issue is that array_unique works by sorting <b=
r>
its elements so that &quot;equal&quot; values are adjacent. I know this wou=
ld be <br>
done on O(n log(n)) vs. O(n^2) grounds, but that could be addressed at <br>
least in part by a smarter sort criterion that sorts by type/class (in <br>
some arbitrary order) before sorting by value. For uncomparable types <br>
(i.e., instances of most classes) this would be by object ID, because we <b=
r>
don&#39;t _actually_ care about ordering.<br>
</blockquote></div></div>
</blockquote></div>

--000000000000a2a7d60641f114fa--