Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126128
Content-Type: text/plain;
	charset=utf-8
Precedence: bulk
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\))
Subject: Re: [PHP-DEV] C Unit testing and mocking
In-Reply-To: <CAEKnhAEFJKPWWL0EKu_JBPgwBEe+Pk3GDzR7HQm3J-g7=Wk0BA@mail.gmail.com>
Date: Mon, 16 Dec 2024 11:04:43 -0400
Cc: PHP internals list <internals@lists.php.net>
Content-Transfer-Encoding: quoted-printable
Message-ID: <A246ACD2-A649-47B2-8175-259248FAC85C@cmpct.info>
References: <CAEKnhAEFJKPWWL0EKu_JBPgwBEe+Pk3GDzR7HQm3J-g7=Wk0BA@mail.gmail.com>
To: Jakub Zelenka <bukka@php.net>
From: calvin@cmpct.info (Calvin Buckley)

On Dec 16, 2024, at 9:18=E2=80=AFAM, Jakub Zelenka <bukka@php.net> =
wrote:
>=20
> Hi,
>=20
> I have been looking into how to test some cases where integration =
tests are very difficult or even impossible to create for. Those are =
often found in networking related and system specific code code =
(network.c, streams, FPM and more). I was recently fixing one such bug =
and decided to give a try which resulted in this PR: =
https://github.com/php/php-src/pull/16987 .
>=20
> There was a suggestion of RFC but that might be a bit too much as it's =
just an internal change / addition. But certainly some overview on =
internals should be done so writing this instead.
>=20
> I decided to use cmocka in that PR because I had some experience with =
that. It's quite small and still very powerful and allow vast mocking =
options. It's a bit manual but it gives a bigger control over the mock. =
It relies on --wrap linking option that replaces original functions with =
wraps. This is however available only on Linux (or maybe some other Unix =
variants) but doesn't work on MacOS or Windows. The developers that want =
to use it on those platforms would need to use some Linux Virtualisation =
option (e.g. Docker). It also requires static library which is supported =
by embed SAPI that can be compiled statically. That limits number of =
extensions to use but the main use cases don't really have deps so it =
should be fine.
>=20
> I did also some research into the other mocking libraries in C. There =
is a Unity with CMock, FFF and some C++ libs like GUnit, Criterion and =
Trompeloeil that I looked into. I quickly discarded GUnit and =
Trompeloeil as they relay on C++ virtual methods and require wrapping C =
code to C++ which is very inconvenient. FFF seems too simple and maybe =
quite inflexible for our needs as well. Criterion also optionally uses =
wrap so I didn't see much advantages compare to cmocka. So it left Unity =
with CMock that allows generating custom mocks using a Ruby script. That =
seemed initially quite nice but after spending around two hours with =
trying to make it works for PHP codebase, I just gave up. It gets quite =
messy for complex scenarios and I just didn't figure out how to nicely =
mock libc functions without any modification to php-src.
>=20
> In terms of CI. It has got its own build which is very simple and it =
tests just specific parts so we could just limit it to run only for =
changed files which might be quite convenient.
>=20
> So the proposed PR is probably the only reasonable unit testing that I =
can come up with. I think it should be completely optional initially for =
people to use - more like an experiment. If it becomes used, then good =
of course. And if it becomes pain, we can just get rid of it. Has anyone =
got any objections to get this merged? If not I plan to merge it early =
in January.
>=20
> Cheers
>=20
> Jakub
>=20

I'm assuming that uses ELF symbol interposition or something like that,
which is why it seems Linux/BSD specific. That seems fragile to me.

I think currently for wanting to test C functions, we're adding custom
functions into ext/zend_test and writing PHPT. Would this work? We
already have that, after all. If not, it'd be helpful to list the
challenges that approach faces.=