# Problems with iconv on AIX 5.3 and 6.1 - maybe a bug - Thread: https://externals.io/message/49892 - Original archive: https://news-web.php.net/php.internals/49892 - Started by Michael Felt on 2010-10-09T20:13:05+00:00 - Messages: 7 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #49892 — Michael Felt — 2010-10-09T20:13:05+00:00 URL: https://externals.io/message/49892#49892 I think it is a bug, but it might also be a misunderstanding. Background: I am trying to convert Joomla 1.0.X to 1.5.X and ran into problems with the conversion of the database that is done during the migration. I started out by doing several of the tests/samples in the php manuals, but always got blank results. I tried changing the php.ini settings: [iconv] iconv.input_encoding = ISO8859-1 iconv.internal_encoding = ISO8859-1 iconv.output_encoding = ISO8859-1 and I dont think it is really necessary - as the iconv command interface also accepts ISO-8859-1 as an argument. Initially I was confused by the information provided by `phpinfo()` from Joomla 1.0.X SysInfo tab: iconv iconv support enabled iconv implementation unknown iconv library version unknown Only when I looked in iconv.c did I find where the "unknown" string was coming from: #ifdef PHP_ICONV_IMPL REGISTER_STRING_CONSTANT("ICONV_IMPL", PHP_ICONV_IMPL, CONST_CS | CONST_PERSISTENT); #elif HAVE_LIBICONV REGISTER_STRING_CONSTANT("ICONV_IMPL", "libiconv", CONST_CS | CONST_PERSISTENT); #elif defined(NETWARE) REGISTER_STRING_CONSTANT("ICONV_IMPL", "Novell", CONST_CS | CONST_PERSISTENT); #else REGISTER_STRING_CONSTANT("ICONV_IMPL", "unknown", CONST_CS | CONST_PERSISTENT); #endif REGISTER_STRING_CONSTANT("ICONV_VERSION", version, CONST_CS | CONST_PERSISTENT); It seems my system is being configured, but under #else - "unknown". I did some digging in configure. Once configures finds iconv (PHP version 5.2.13 line numbers!) Please note the bold lines: *configure:45884: checking for iconv support configure:45947: checking for iconv* configure:45975: xlc -o conftest -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysql/lib -blibpat h:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/lib -L/usr/local/lib conftest.c -lf reetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 -liconv -lm 1>&5 ld: 0706-027 The -R /usr/local/lib flag is ignored. *configure:46563: checking if iconv is glibc's* configure:46572: xlc -o conftest -I/usr/include -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysq l/lib -L/usr/lib -blibpath:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/lib -L/us r/local/lib conftest.c -lfreetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 -liconv - lm 1>&5 "configure", line 46566.10: 1506-296 (S) #include file not found. configure: failed program was: #line 46565 "configure" #include "confdefs.h" #include int main() { gnu_get_libc_version(); ; return 0; } *configure:46590: checking if using GNU libiconv* configure:46610: xlc -o conftest -I/usr/include -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysq l/lib -liconv -L/usr/lib -blibpath:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/l ib -L/usr/local/lib conftest.c -lfreetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 - liconv -lm 1>&5 "configure", line 46604.22: 1506-045 (S) Undeclared identifier _libiconv_version. configure: failed program was: #line 46600 "configure" #include "confdefs.h" #include int main() { printf("%d", _libiconv_version); return 0; } *configure:46632: checking if iconv is Konstantin Chuguev's* configure:46641: xlc -o conftest -I/usr/include -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysq l/lib -L/usr/lib -blibpath:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/lib -L/us r/local/lib conftest.c -lfreetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 -liconv - lm 1>&5 ld: 0706-027 The -R /usr/local/lib flag is ignored. ld: 0711-317 ERROR: Undefined symbol: .iconv_ccs_init ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. configure: failed program was: #line 46634 "configure" #include "confdefs.h" #include int main() { iconv_ccs_init(NULL, NULL); ; return 0; } *configure:46809: checking if iconv supports errno* configure:46844: xlc -o conftest -I/usr/include -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysq l/lib -L/usr/lib -blibpath:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/lib -L/us r/local/lib conftest.c -lfreetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 -liconv - lm 1>&5 ld: 0706-027 The -R /usr/local/lib flag is ignored. configure:46876: checking if your cpp allows macro usage in include lines ================ At this point configure has found iconv, and that it supports errno, but not the "version". I felt that it "should" work with *libiconv *as it's variable. In configure, as a hack to test, I added the lines: if { (eval echo configure:45975: \"$ac_link\") 1>&5; (eval $ac_link) 2>&5; } && test -s conftest${ac_exeext}; then rm -rf conftest* eval "ac_cv_func_iconv=yes" * iconv_impl_name="aix_libiconv" echo "MAMF TESTING:host:" $host_alias "iconv_impl_name is:" $iconv_impl_name >&5* else echo "configure: failed program was:" >&5 cat conftest.$ac_ext >&5 rm -rf conftest* eval "ac_cv_func_iconv=no" fi and farther below: *echo "MAMF TESTING:host:" $host_alias "iconv_impl_name is:" $iconv_impl_name >&5* case "$iconv_impl_name" in * aix_libiconv | gnu_libiconv )* echo "#define PHP_ICONV_IMPL \"libiconv\"" > ext/iconv/php_php_iconv_impl.h cat >> confdefs.h <<\EOF #define PHP_ICONV_IMPL "libiconv" EOF echo "#define HAVE_LIBICONV 1" > ext/iconv/php_have_libiconv.h Now I get in configure.log configure:45884: checking for iconv support configure:45947: checking for iconv configure:45975: xlc -o conftest -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysql/lib -blibpat h:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/lib -L/usr/local/lib conftest.c -lf reetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 -liconv -lm 1>&5 ld: 0706-027 The -R /usr/local/lib flag is ignored. *MAMF TESTING:host: powerpc-ibm-aix5.3.0.0 iconv_impl_name is: aix_libiconv* *configure:46563: checking if iconv is glibc's* configure:46572: xlc -o conftest -I/usr/include -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysq l/lib -L/usr/lib -blibpath:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/lib -L/us r/local/lib conftest.c -lfreetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 -liconv - lm 1>&5 "configure", line 46566.10: 1506-296 (S) #include file not found. configure: failed program was: #line 46565 "configure" #include "confdefs.h" *#include * int main() { gnu_get_libc_version(); ; return 0; } *MAMF TESTING:host: powerpc-ibm-aix5.3.0.0 iconv_impl_name is: aix_libiconv configure:46809: checking if iconv supports errno* configure:46844: xlc -o conftest -I/usr/include -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysq l/lib -L/usr/lib -blibpath:/usr/local/lib:/usr/lib:/usr/local/ssl/lib:/usr/local/mysql/lib -R/usr/local/lib -L/us r/local/lib conftest.c -liconv -lfreetype -lpng -lz -ljpeg -lcurl -lz -lrt -lm -lxml2 -liconv -lm -lcurl -lxml2 - liconv -lm 1>&5 ... snip configure finishes. make runs fine until iconv.c: root@x105:[/data/prj/php-5.2.13.a]make /bin/sh /data/prj/php-5.2.13.a/libtool --silent --preserve-dup-deps --mode=compile xlc -I"/usr/include" -Iext/iconv/ -I/data/prj/php-5.2.13.a/ext/iconv/ -DPHP_ATOM_INC -I/data/prj/php-5.2.13.a/include -I/data/prj/php-5.2.13.a/main -I/data/prj/php-5.2.13.a -I/data/prj/php-5.2.13.a/ext/date/lib -I/usr/local/include/libxml2 -I/usr/local/include -I/usr/local/include/freetype2 -I/usr/local/mysql/include -I/usr/local/mysql/include/mysql -I/data/prj/php-5.2.13.a/TSRM -I/data/prj/php-5.2.13.a/Zend -I/usr/include -L/usr/local/ssl/lib -L/usr/local/lib -L/usr/lib -L/usr/local/mysql/lib -c /data/prj/php-5.2.13.a/ext/iconv/iconv.c -o ext/iconv/iconv.lo "/data/prj/php-5.2.13.a/ext/iconv/iconv.c", line 265.23: 1506-045 (S) Undeclared identifier _libiconv_version. "/data/prj/php-5.2.13.a/ext/iconv/iconv.c", line 686.33: 1506-280 (W) Function argument assignment between types "unsigned int*" and "int*" is not allowed. make: 1254-004 The error code from the last command is 1. So I modified iconv.c to read: /* {{{ PHP_MINIT_FUNCTION */ PHP_MINIT_FUNCTION(miconv) { char *version = "unknown"; REGISTER_INI_ENTRIES(); *#ifndef _AIX* #if HAVE_LIBICONV { static char buf[16]; snprintf(buf, sizeof(buf), "%d.%d", ((_libiconv_version >> 8) & 0x0f), (_libiconv_version & 0x0f)); version = buf; } #elif HAVE_GLIBC_ICONV version = (char *)gnu_get_libc_version(); #elif defined(NETWARE) version = "OS built-in"; #endif *#else version = "OS built-in"; #endif* #ifdef PHP_ICONV_IMPL REGISTER_STRING_CONSTANT("ICONV_IMPL", PHP_ICONV_IMPL, CONST_CS | CONST_PERSISTENT); ======== To be continued. The sapi/cli/php5 is not being created now. Will be updated with changes in the failed make test output once I figure what has gone wrong. :( --- ## #49893 — Michael Felt — 2010-10-09T20:22:29+00:00 URL: https://externals.io/message/49892#49893 · In reply to: #49892 The bold feature of the mailer got lost - look for *.....* for highlighted text. My apologies. --- ## #49894 — Michael Felt — 2010-10-09T20:33:34+00:00 URL: https://externals.io/message/49892#49894 · In reply to: #49892 > > ======== > To be continued. The sapi/cli/php5 is not being created now. Will be > updated with changes in the > failed make test output once I figure what has gone wrong. :( > Actually, it is there, but for some reason it is not starting. Please excuse my ignorance.... root@x105:[/data/prj/php-5.2.13]make test echo '\ \ Build complete. Don't forget to run 'make test'. ERROR: Cannot run tests without CLI sapi. ======== However, the SAPI client is there, but being ignored. ========= root@x105:[/data/prj/php-5.2.13]./sapi/cli/php5 -? Usage: php5 [options] [-f] [--] [args...] php5 [options] -r [--] [args...] php5 [options] [-B ] -R [-E ] [--] [args...] php5 [options] [-B ] -F [-E ] [--] [args...] php5 [options] -- [args...] php5 [options] -a -a Run interactively -c | Look for php.ini file in this directory -n No php.ini file will be used -d foo[=bar] Define INI entry foo with value 'bar' -e Generate extended information for debugger/profiler -f Parse and execute . -h This help -i PHP information -l Syntax check only (lint) -m Show compiled in modules -r Run PHP without using script tags -B Run PHP before processing input lines -R Run PHP for every input line -F Parse and execute for every input line -E Run PHP after processing all input lines -H Hide any passed arguments from external tools. -s Output HTML syntax highlighted source. -v Version number -w Output source with stripped comments and whitespace. -z Load Zend extension . args... Arguments passed to script. Use -- args when first argument starts with - or script is read from stdin --ini Show configuration file names --rf Show information about function . --rc Show information about class . --re Show information about extension . --ri Show configuration for extension . root@x105:[/data/prj/php-5.2.13] --- ## #49895 — Antony Dovgal — 2010-10-11T10:02:46+00:00 URL: https://externals.io/message/49892#49895 · In reply to: #49892 > I think it is a bug, but it might also be a misunderstanding. > > Background: I am trying to convert Joomla 1.0.X to 1.5.X and ran into > problems with the conversion of the database that is done during the > migration. I started out by doing several of the tests/samples in the > php manuals, but always got blank results. I tried changing the php.ini > settings: > [iconv] > iconv.input_encoding = ISO8859-1 > iconv.internal_encoding = ISO8859-1 > iconv.output_encoding = ISO8859-1 Well, yes, it's a known problem that AIX contains it's very own version of iconv. At least it differs from everything else I've seen so far (in terms of encoding names and functionality). > #else > REGISTER_STRING_CONSTANT("ICONV_IMPL", "unknown", CONST_CS | > CONST_PERSISTENT); > #endif > REGISTER_STRING_CONSTANT("ICONV_VERSION", version, CONST_CS | > CONST_PERSISTENT); > > It seems my system is being configured, but under #else - "unknown". Probably "ibm" might be better, but it doesn't change much.. > I did some digging in configure. Once configures finds iconv (PHP > version 5.2.13 line numbers!) You should look into ext/iconv/config.m4 instead. ./configure is generated and it's not supposed to be read. Also I'd suggest you to try 5.3 branch, I can see that AIX iconv support exists only there. -- Wbr, Antony Dovgal --- http://pinba.org - realtime statistics for PHP --- ## #49897 — Michael Felt — 2010-10-11T18:35:38+00:00 URL: https://externals.io/message/49892#49897 · In reply to: #49895 Thanks Antony, I sort of found that out. I downloaded the gnu iconv and gettext, and ran them as libiconv, configure, install; tettext, configure, install; libiconv, distclean, configure, install; php5.2.13 config, make, test (better results), install - and the joomla migrate worked. Now - as one of my goals is to make installable packages - I need to figure out why "make dist" for libiconv (and maybe for gettext as well) does not work. As to 5.3 - personally, I have some oldish PHP code that does not like some of the new defaults in 5.3 (as least when I tried when it first came out). However, I shall soon do 5.2.14. There is another AIX idiosycrancy I found. It does not like the BUILD_CLI string in the Makefile. If I edit it into seperate lines and put those where $(BUILD_CLI) stands, the sapi/cli/php program configures fine. The other issue I run into is the, also oldish, libmysqlclient library I link against also has all the libz routines. I am assuming the code is using the libz routines - because libz is first in the shared libraries list. Fun in porting :) Off topic I know - but how much interest is there is a more explicit "debug" info of the BUILD_CLI issue. Personally I find it very strange as it used to work fine (the string does not look to have changed in a long time. I would guess it is some security 'improvement' in make or ksh. > > > […] > > Well, yes, it's a known problem that AIX contains it's very own version of > iconv. > At least it differs from everything else I've seen so far (in terms of > encoding names and functionality). > > > […] > > Probably "ibm" might be better, but it doesn't change much.. > > > […] > > You should look into ext/iconv/config.m4 instead. > ./configure is generated and it's not supposed to be read. > > Also I'd suggest you to try 5.3 branch, I can see that AIX iconv support > exists only there. > > -- > Wbr, > Antony Dovgal > --- > http://pinba.org - realtime statistics for PHP --- ## #49900 — Antony Dovgal — 2010-10-12T12:00:38+00:00 URL: https://externals.io/message/49892#49900 · In reply to: #49897 > As to 5.3 - personally, I have some oldish PHP code that does not like > some of the new defaults in 5.3 (as least when I tried when it first > came out). However, I shall soon do 5.2.14. I'd still recommend at least _to try_ 5.3 just to make sure it really does compile & work fine. If that is the case, you can start backporting AIX-related changes from 5.3 to 5.2. > There is another AIX idiosycrancy I found. It does not like the > BUILD_CLI string in the Makefile. If I edit it into seperate lines and > put those where $(BUILD_CLI) stands, the sapi/cli/php program configures > fine. Make sure you use GNU make and as much other GNU utils as possible - the native ones are broken. > Fun in porting :) Well, I wouldn't call that fun.. it's more of a frustration to me. > Off topic I know - but how much interest is there is a more explicit > "debug" info of the BUILD_CLI issue. Personally I find it very strange > as it used to work fine (the string does not look to have changed in a > long time. I would guess it is some security 'improvement' in make or ksh. GNU bash and GNU make should certainly fix it. -- Wbr, Antony Dovgal --- http://pinba.org - realtime statistics for PHP --- ## #49903 — Michael Felt — 2010-10-12T21:15:45+00:00 URL: https://externals.io/message/49892#49903 · In reply to: #49900 > > > […] > > I'd still recommend at least _to try_ 5.3 just to make sure it really does > compile & work fine. > If that is the case, you can start backporting AIX-related changes from 5.3 > to 5.2. > > I'll try it, of course. Just as I did when it first came out. I have even tried the Alpha version, but not had much luck (then - I'll look again). > > […] > > Make sure you use GNU make and as much other GNU utils as possible - the > native ones are broken. > Native make might be broken - but I understand legacy make (or is it POSIX) better than gnu make. Sometimes I have to use gnumake (One of the packages I port requires it). > > > […] > This is a poke at myself - and trying to acknowledge that I know it is a very difficult challenge to write portable code. That is why I hesitate to call anything a bug. Just because it isn't working on my system effortlessly does not make it a bug. And you are quite correct. It does get very frustrating. > Well, I wouldn't call that fun.. it's more of a frustration to me. > > > […] > ksh. > > GNU bash and GNU make should certainly fix it. > > And, maybe I'll have to adapt my development system to use those. But after over 30 years of using sh (the original shell, short time at csh, and the last 15+ years using ksh - I dont really want to learn a new shell. AIX is also my work, and bash is an option. ksh is there all the time. I cannot demand that the people I support use bash, so I dont want to be overly dependent on it professionally either. I shall try gnu make and see if the problem with BUILDCLI goes away - as gnumake already built. And if that does not work I'll considering loading bash for a test run. wbr, Michael (p.s. I had to google wbr!) > -- > Wbr, > Antony Dovgal > --- > http://pinba.org - realtime statistics for PHP