# Feedback requested on using #defines to improve the performance of the TSRMG macro

- Thread: https://externals.io/message/45953
- Original archive: https://news-web.php.net/php.internals/45953
- Started by Arvind Srinivasan on 2009-11-05T11:52:34+00:00
- Messages: 8
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #45953 — Arvind Srinivasan — 2009-11-05T11:52:34+00:00

URL: https://externals.io/message/45953#45953

When compiled for multi-threaded (#ifdef ZTS ) operation, the various
subsystems in PHP use dynamically allocated (ts_allocate_id)
identifiers to index into the thread-local storage for each subsystem.
These dynamically allocated ids are used by macros such as CG, EG, PG,
AG.

The TSRMG macro is defined as:
#define TSRMG(id, type, element)	(((type) (*((void ***)
tsrm_ls))[TSRM_UNSHUFFLE_RSRC_ID(id)])->element)

The PG macro is defined as:
# define PG(v) TSRMG(core_globals_id, php_core_globals *, v)

where core_globals_id is
extern PHPAPI int core_globals_id;

cc -E of
	PG(connection_status) = PHP_CONNECTION_ABORTED;
translates to:
 (((php_core_globals *) (*((void ***)
tsrm_ls))[((core_globals_id)-1)])->connection_status) = 1;

and cc -S of the same code results in:
	.loc 1 108 0
	movl	%ebx, %eax
	subl	core_globals_id, %eax
	movl	(%esi), %edx
	sall	$2, %eax
	negl	%eax
	movl	(%edx,%eax), %eax
	movw	$1, 144(%eax)

I used fixed IDs instead of dynamically allocated ones and noticed a
decent improvement in performance (few percentage points) when
running a web-based ecommerce-site workload on SPARC CMT hardware.

With my changes the PG macro is defined as:
# define PG(v) TSRMG(CORE_GLOBALS_ID, php_core_globals *, v)

#define CORE_GLOBALS_ID					10

and core_globals_id is
extern PHPAPI const ts_rsrc_id core_globals_id;

cc -E of the same line of code translates to:
 (((php_core_globals *) (*((void ***)
tsrm_ls))[((10)-1)])->connection_status) = 1;

cc -S (my version):
	.loc 1 108 0
	movl	(%eax), %eax
	movl	36(%eax), %eax
	movw	$1, 144(%eax)

The patch is fairly long, so rather than attach it to this email, i'll
point to it.

It'd be great if someone could give me feedback on my changes -
http://bitbucket.org/arvi/arviq/src/tip/arvi-16-ts_allocate_reserved_id
- for using reserved/fixed IDs for accessing thread-local storage of
frequently used/known/core subsystems. I think the changes only affect
the #ifdef ZTS codepaths.

Thanks,
Arvi

---

## #45955 — Christopher Jones — 2009-11-05T18:58:53+00:00

URL: https://externals.io/message/45953#45955 · In reply to: #45953

Hi Arvind,

Does the GLOBALS_ID_BASE idea work?  In
"ts_allocate_reserved_id(GLOBALS_ID_BASE+1...)" each extension would
anyway need to reserve an increment to avoid clashes.  Also, why is
GLOBALS_ID_BASE 30 when the largest reserved value is 18?  Maybe I'm
missing something.

Would there be significant memory space or locality issues if one ID
per extension in the PHP source bundle was reserved upfront, even if
those extensions were never enabled?

Chris


Arvind Srinivasan wrote:
 > When compiled for multi-threaded (#ifdef ZTS ) operation, the various
 > subsystems in PHP use dynamically allocated (ts_allocate_id)
 > identifiers to index into the thread-local storage for each subsystem.
 > These dynamically allocated ids are used by macros such as CG, EG, PG,
 > AG.
 >
 > The TSRMG macro is defined as:
 > #define TSRMG(id, type, element)	(((type) (*((void ***)
 > tsrm_ls))[TSRM_UNSHUFFLE_RSRC_ID(id)])->element)
 >
 > The PG macro is defined as:
 > # define PG(v) TSRMG(core_globals_id, php_core_globals *, v)
 >
 > where core_globals_id is
 > extern PHPAPI int core_globals_id;
 >
 > cc -E of
 > 	PG(connection_status) = PHP_CONNECTION_ABORTED;
 > translates to:
 >  (((php_core_globals *) (*((void ***)
 > tsrm_ls))[((core_globals_id)-1)])->connection_status) = 1;
 >
 > and cc -S of the same code results in:
 > 	.loc 1 108 0
 > 	movl	%ebx, %eax
 > 	subl	core_globals_id, %eax
 > 	movl	(%esi), %edx
 > 	sall	$2, %eax
 > 	negl	%eax
 > 	movl	(%edx,%eax), %eax
 > 	movw	$1, 144(%eax)
 >
 > I used fixed IDs instead of dynamically allocated ones and noticed a
 > decent improvement in performance (few percentage points) when
 > running a web-based ecommerce-site workload on SPARC CMT hardware.
 >
 > With my changes the PG macro is defined as:
 > # define PG(v) TSRMG(CORE_GLOBALS_ID, php_core_globals *, v)
 >
 > #define CORE_GLOBALS_ID					10
 >
 > and core_globals_id is
 > extern PHPAPI const ts_rsrc_id core_globals_id;
 >
 > cc -E of the same line of code translates to:
 >  (((php_core_globals *) (*((void ***)
 > tsrm_ls))[((10)-1)])->connection_status) = 1;
 >
 > cc -S (my version):
 > 	.loc 1 108 0
 > 	movl	(%eax), %eax
 > 	movl	36(%eax), %eax
 > 	movw	$1, 144(%eax)
 >
 > The patch is fairly long, so rather than attach it to this email, i'll
 > point to it.
 >
 > It'd be great if someone could give me feedback on my changes -
 > http://bitbucket.org/arvi/arviq/src/tip/arvi-16-ts_allocate_reserved_id
 > - for using reserved/fixed IDs for accessing thread-local storage of
 > frequently used/known/core subsystems. I think the changes only affect
 > the #ifdef ZTS codepaths.
 >
 > Thanks,
 > Arvi
 >

-- 
Blog: http://blogs.oracle.com/opal
Twitter:  http://twitter.com/ghrd

---

## #45956 — Arvind Srinivasan — 2009-11-05T19:20:09+00:00

URL: https://externals.io/message/45953#45956 · In reply to: #45955

> Does the GLOBALS_ID_BASE idea work?  In
> "ts_allocate_reserved_id(GLOBALS_ID_BASE+1...)" each extension would
> anyway need to reserve an increment to avoid clashes.  Also, why is

I didn't really try using this. When I added it, I thought it might be
useful for modules that live outside the PHP source tree. They could
then define their constants using
#define FOO_ID (GLOBALS_ID_BASE + 3)
rather than hardcoding 33. As you point out, they would still need to
reserve an increment.

> GLOBALS_ID_BASE 30 when the largest reserved value is 18?  Maybe I'm
> missing something.
>
I reserved IDs for the subsystems I thought were core. I was sure
there'd be others that I'd missed and so I left space for more.


> Would there be significant memory space or locality issues if one ID
> per extension in the PHP source bundle was reserved upfront, even if
> those extensions were never enabled?
>

I'll run some tests and see what impact this has.

Arvi

---

## #45957 — mm w — 2009-11-05T19:29:50+00:00

URL: https://externals.io/message/45953#45957 · In reply to: #45956

so should be a UUID rather than a user defined int you cannot avoid
collision with your system, it's a dangerous way to go.

Best,


> > […]
>
> I didn't really try using this. When I added it, I thought it might be
> useful for modules that live outside the PHP source tree. They could
> then define their constants using
> #define FOO_ID (GLOBALS_ID_BASE + 3)
> rather than hardcoding 33. As you point out, they would still need to
> reserve an increment.
>
> > […]
> I reserved IDs for the subsystems I thought were core. I was sure
> there'd be others that I'd missed and so I left space for more.
>
>
> > […]
>
> I'll run some tests and see what impact this has.
>
> Arvi

---

## #45958 — Christopher Jones — 2009-11-05T20:12:03+00:00

URL: https://externals.io/message/45953#45958 · In reply to: #45956

Arvind Srinivasan wrote:
> > […]
 >
 > I didn't really try using this. When I added it, I thought it might be
 > useful for modules that live outside the PHP source tree. They could
 > then define their constants using
 > #define FOO_ID (GLOBALS_ID_BASE + 3)
 > rather than hardcoding 33. As you point out, they would still need to
 > reserve an increment.
 >
> > […]
 > I reserved IDs for the subsystems I thought were core. I was sure
 > there'd be others that I'd missed and so I left space for more.
 >
 >
> > […]
 >
 > I'll run some tests and see what impact this has.

I'd vote that the reserved slots be assigned from day 1 for one of (i)
always-enabled core extensions or (ii) for core + common extensions or
{iii} the whole set of PHP-bundled extensions + common extensions from
PECL (i.e. APC?).  The choice of (i), (ii) or (iii) would depend on
your tests.

All other extensions should use a run-time allocated non-reserved
slot.  Extensions that become popular can add an xxx_GLOBALS_ID entry
to zend_globals_macros.h.

Chris

-- 
Blog: http://blogs.oracle.com/opal
Twitter:  http://twitter.com/ghrd

---

## #45962 — Stanislav Malyshev — 2009-11-05T23:33:32+00:00

URL: https://externals.io/message/45953#45962 · In reply to: #45953

Hi!

> When compiled for multi-threaded (#ifdef ZTS ) operation, the various
> subsystems in PHP use dynamically allocated (ts_allocate_id)
> identifiers to index into the thread-local storage for each subsystem.
> These dynamically allocated ids are used by macros such as CG, EG, PG,
> AG.

I think it makes sense. One note on the patch - you don't have to 
replace XXX_core_globals_id with XXX_GLOBALS_ID everywhere, especially 
in init code, etc. which is not performance-sensitive, as you already 
have these values assigned.
Also, ts_allocate_reserved_id doesn't really allocate anything anymore, 
as far as I can see.
50 reserved IDs can be an overkill though... I'd say only core IDs get 
revervations, all extensions should be dynamic.
Also, ts_allocate_reserved_id seems to allow zero size as argument, but 
then allows to override such ID with another call of 
ts_allocate_reserved_id, while it does not allow this for non-zero sizes.
-- 
Stanislav Malyshev, Zend Software Architect
stas@zend.com   http://www.zend.com/
(408)253-8829   MSN: stas@zend.com

---

## #45966 — Basant Kukreja — 2009-11-06T06:18:51+00:00

URL: https://externals.io/message/45953#45966 · In reply to: #45953

+1

Regards,
Basant.


> When compiled for multi-threaded (#ifdef ZTS ) operation, the various
> subsystems in PHP use dynamically allocated (ts_allocate_id)
> identifiers to index into the thread-local storage for each subsystem.
> These dynamically allocated ids are used by macros such as CG, EG, PG,
> AG.
> 
> The TSRMG macro is defined as:
> #define TSRMG(id, type, element)	(((type) (*((void ***)
> tsrm_ls))[TSRM_UNSHUFFLE_RSRC_ID(id)])->element)
> 
> The PG macro is defined as:
> # define PG(v) TSRMG(core_globals_id, php_core_globals *, v)
> 
> where core_globals_id is
> extern PHPAPI int core_globals_id;
> 
> cc -E of
> 	PG(connection_status) = PHP_CONNECTION_ABORTED;
> translates to:
>  (((php_core_globals *) (*((void ***)
> tsrm_ls))[((core_globals_id)-1)])->connection_status) = 1;
> 
> and cc -S of the same code results in:
> 	.loc 1 108 0
> 	movl	%ebx, %eax
> 	subl	core_globals_id, %eax
> 	movl	(%esi), %edx
> 	sall	$2, %eax
> 	negl	%eax
> 	movl	(%edx,%eax), %eax
> 	movw	$1, 144(%eax)
> 
> I used fixed IDs instead of dynamically allocated ones and noticed a
> decent improvement in performance (few percentage points) when
> running a web-based ecommerce-site workload on SPARC CMT hardware.
> 
> With my changes the PG macro is defined as:
> # define PG(v) TSRMG(CORE_GLOBALS_ID, php_core_globals *, v)
> 
> #define CORE_GLOBALS_ID					10
> 
> and core_globals_id is
> extern PHPAPI const ts_rsrc_id core_globals_id;
> 
> cc -E of the same line of code translates to:
>  (((php_core_globals *) (*((void ***)
> tsrm_ls))[((10)-1)])->connection_status) = 1;
> 
> cc -S (my version):
> 	.loc 1 108 0
> 	movl	(%eax), %eax
> 	movl	36(%eax), %eax
> 	movw	$1, 144(%eax)
> 
> The patch is fairly long, so rather than attach it to this email, i'll
> point to it.
> 
> It'd be great if someone could give me feedback on my changes -
> http://bitbucket.org/arvi/arviq/src/tip/arvi-16-ts_allocate_reserved_id
> - for using reserved/fixed IDs for accessing thread-local storage of
> frequently used/known/core subsystems. I think the changes only affect
> the #ifdef ZTS codepaths.
> 
> Thanks,
> Arvi

---

## #45968 — Arvind Srinivasan — 2009-11-06T10:35:12+00:00

URL: https://externals.io/message/45953#45968 · In reply to: #45962

Hi,

I've updated my patch based on your feedback.

http://bitbucket.org/arvi/arviq/src/tip/arvi-16-ts_allocate_reserved_id
incorporates the following changes:

* xxx_globals_id replaced with XXX_GLOBALS_ID only when used in macros
and not in init code etc
* ts_allocate_reserved_id renamed to ts_reserve_id as Stas pointed out
that it doesn't actually allocate anything
* MAX_RESERVED_IDS set to 20. I have assigned upto 18 so far, but I
didn't do the win32 ones as I don't have a dev environment on Windows
* ts_reserve_id doesn't allow 0 as a value for size or for rsrc_id

The memory impact of reserving ids
= sizeof(tsrm_resource_type) * MAX_RESERVED_IDS * NUM_THREADS
= 16 * MAX_RESERVED_IDS * NUM_THREADS

Please let me know if the diffs above look okay and if so I'll create
a bug to track the patch. I ran 'make test' to verify that the patch
didn't cause regressions.

I haven't changed cwd_globals_id to use ts_reserve_id/CWD_GLOBALS_ID,
since tsrm_virtual_cwd is in the TSRM directory and all the
XXX_GLOBALS_ID are defined in the Zend directory. I will shortly be
proposing to move tsrm_virtual_cwd to the Zend directory and so I left
CWD_GLOBALS_ID as is in the list of reserved ids.

Thanks,
Arvi