Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126180
Content-Type: multipart/alternative; boundary=Apple-Mail-ECA41D4D-3234-45FF-BE08-D6527C2992BE
Content-Transfer-Encoding: 7bit
Precedence: bulk
Mime-Version: 1.0 (1.0)
Subject: Re: [PHP-DEV] C Unit testing and mocking
Date: Thu, 26 Dec 2024 12:04:19 -0600
Message-ID: <32BBE792-FB6F-46CB-9621-05B311BB5C8B@benramsey.com>
References: <CAEKnhAEuZyiSLFOVg+A=iDRmMbib5uKxEjCwTqaw0wp1zDx9EA@mail.gmail.com>
Cc: PHP internals list <internals@lists.php.net>
In-Reply-To: <CAEKnhAEuZyiSLFOVg+A=iDRmMbib5uKxEjCwTqaw0wp1zDx9EA@mail.gmail.com>
To: Jakub Zelenka <bukka@php.net>
From: ben@benramsey.com (Ben Ramsey)


--Apple-Mail-ECA41D4D-3234-45FF-BE08-D6527C2992BE
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div dir=3D"ltr"><blockquote type=3D"cite">=
On Dec 26, 2024, at 06:19, Jakub Zelenka &lt;bukka@php.net&gt; wrote:<br><br=
></blockquote></div><blockquote type=3D"cite"><div dir=3D"ltr">=EF=BB=BF<div=
 dir=3D"ltr"><div dir=3D"ltr">On Tue, Dec 24, 2024 at 8:22=E2=80=AFPM Ben Ra=
msey &lt;<a href=3D"mailto:ben@benramsey.com">ben@benramsey.com</a>&gt; wrot=
e:</div><div class=3D"gmail_quote gmail_quote_container"><blockquote class=3D=
"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"ltr"><blockquote t=
ype=3D"cite">On Dec 16, 2024, at 07:21, Jakub Zelenka &lt;<a href=3D"mailto:=
bukka@php.net" target=3D"_blank">bukka@php.net</a>&gt; wrote:<br><br></block=
quote></div><blockquote type=3D"cite"><div dir=3D"ltr">=EF=BB=BF<div dir=3D"=
ltr">Hi,<div><br></div><div>I have been looking into how to test some cases w=
here integration tests are very difficult or even impossible to create for. T=
hose are often found in networking related and system specific code code (ne=
twork.c, streams, FPM and more). I was recently fixing one such bug and deci=
ded to give a try which resulted in this PR:&nbsp;<a href=3D"https://github.=
com/php/php-src/pull/16987" target=3D"_blank">https://github.com/php/php-src=
/pull/16987</a> .</div><div><br></div><div>There was a suggestion of RFC but=
 that might be a bit too much as it's just an internal change / addition. Bu=
t certainly some overview on internals should be done so writing this instea=
d.</div><div><br></div><div>I decided to use cmocka in that PR because I had=
 some experience with that. It's quite small and still very powerful and all=
ow vast mocking options. It's a bit manual but it gives a bigger control ove=
r the mock. It relies on --wrap linking option that replaces original functi=
ons with wraps. This is however available only on Linux (or maybe some other=
 Unix variants) but doesn't work on MacOS or Windows. The developers that wa=
nt to use it on those platforms would need to use some Linux Virtualisation o=
ption (e.g. Docker). It also requires static library which is supported by e=
mbed SAPI that can be compiled statically. That limits number of extensions t=
o use but the main use cases don't really have deps so it should be fine.</d=
iv><div><br></div><div>I did also some research into the other mocking libra=
ries in C. There is a Unity with CMock, FFF and some C++ libs like GUnit, Cr=
iterion and Trompeloeil that I looked into. I quickly discarded&nbsp;GUnit a=
nd&nbsp;Trompeloeil as they relay on C++ virtual methods and require wrappin=
g C code to C++ which is very inconvenient. FFF seems too simple and maybe q=
uite inflexible for our needs as well. Criterion also optionally uses wrap s=
o I didn't see much advantages compare to cmocka. So it left Unity with CMoc=
k that allows generating custom mocks using a Ruby script. That seemed initi=
ally quite nice but after spending around two hours with trying to make it w=
orks for PHP codebase, I just gave up. It gets quite messy for complex scena=
rios and I just didn't figure out how to nicely mock libc functions without a=
ny modification to php-src.</div><div><br></div><div>In terms of CI. It has g=
ot its own build which is very simple and it tests just specific parts so we=
 could just limit it to run only for changed files which might be quite conv=
enient.</div><div><br></div><div>So the proposed PR is probably the only rea=
sonable unit testing that I can come up with. I think it should be completel=
y optional initially for people to use - more like an experiment. If it beco=
mes used, then good of course. And if it becomes pain, we can just get rid o=
f it. Has anyone got any objections to get this merged? If not I plan to mer=
ge it early in January.</div><div><br></div><div>Cheers</div><div><br></div>=
<div>Jakub</div></div>
</div></blockquote><br><div><br></div><div>FWIW, as someone still very new t=
o C, I found Criterion quite easy to use, and I was able to quickly grasp it=
s concepts and start using it right away. I can=E2=80=99t say the same for o=
ther C testing libraries I tried.</div></div></blockquote><div><br></div><di=
v>I just checked Criterion a bit more and it actually does not have built-in=
 mocking. It could be used with --wrap but it doesn't have any expectations l=
ike cmocka. There's Mimick [1] from the same author which is indendent but i=
ts docs still say that it's experimental so probably not a good idea to use a=
n experimental tool as its API can change. So all in all it's not probably a=
n option for us.</div><div><br></div><div>[1]&nbsp;<a href=3D"https://github=
.com/Snaipe/Mimick">https://github.com/Snaipe/Mimick</a></div></div></div>
</div></blockquote><div><br></div><div>I=E2=80=99ve not done any mocking in C=
 yet, so I can=E2=80=99t offer much there, but I see Mimick hasn=E2=80=99t h=
ad a tagged release since 2017, even though development still appears active=
, but I agree it might not be mature enough to standardize on.</div><div><br=
></div><div>It=E2=80=99s interesting to note David Carlier (a.k.a. devnexen)=
 has made some contributions to Mimick. :-)</div><div><br></div><div>Cheers,=
</div><div>Ben</div><div><br></div></body></html>=

--Apple-Mail-ECA41D4D-3234-45FF-BE08-D6527C2992BE--