Solved my build problems

Solved my build problems

Postby Worldblender » 10 Jul 2017, 01:00

For the past 30 days, I've struggled with trying to get SuperTuxKart to build successfully on my computer running Ubuntu 16.04, the amd64 version. All of these problems cropped up after June 5, when the libpng library shipping with this source code was upgraded from 1.5.9 to 1.5.28. Here is what happened, chronologically (all filepaths are relative to the structure of the stk-code directory):
  1. I received errors about ./lib/libpng/pnglibconf.h not being found while stkirrlicht was being compiled. Before the upgrade, pnglibconf.h was already included. Now, it's no longer present by default, but today, I was able to workaround this problem by building libpng separately, in which that process will generate the correct pnglibconf.h so that it will be picked up during compilation.
  2. Using .lib/libpng/scripts/pnglibconf.h.prebuilt, I was able to complete more compilation, up to the point where the main binary, 'supertuxkart', began being linked.
  3. But now, I receive errors about something that couldn't be linked:
    {l Code}: {l Select All Code}
    lib/irrlicht/libstkirrlicht.a(CImageLoaderPNG.cpp.o): In function `irr::video::CImageLoaderPng::loadImage(irr::io::IReadFile*, bool) const':
    /home/richard/stk-code/lib/irrlicht/source/Irrlicht/CImageLoaderPNG.cpp:(.text+0x225): undefined reference to `png_set_longjmp_fn'
    /home/richard/stk-code/lib/irrlicht/source/Irrlicht/CImageLoaderPNG.cpp:(.text+0x45a): undefined reference to `png_set_longjmp_fn'
    lib/irrlicht/libstkirrlicht.a(CImageLoaderPNG.cpp.o): In function `irr::video::png_cpexcept_error(png_struct_def*, char const*)':
    /home/richard/stk-code/lib/irrlicht/source/Irrlicht/CImageLoaderPNG.cpp:(.text+0x50e): undefined reference to `png_set_longjmp_fn'
    lib/irrlicht/libstkirrlicht.a(CImageWriterPNG.cpp.o): In function `irr::video::CImageWriterPNG::writeImage(irr::io::IWriteFile*, irr::video::IImage*, unsigned int) const':
    /home/richard/stk-code/lib/irrlicht/source/Irrlicht/CImageWriterPNG.cpp:(.text+0x20b): undefined reference to `png_set_longjmp_fn'
    /home/richard/stk-code/lib/irrlicht/source/Irrlicht/CImageWriterPNG.cpp:(.text+0x4ca): undefined reference to `png_set_longjmp_fn'
    lib/irrlicht/libstkirrlicht.a(CImageWriterPNG.cpp.o):/home/richard/stk-code/lib/irrlicht/source/Irrlicht/CImageWriterPNG.cpp:(.text+0x5b1): more undefined references to `png_set_longjmp_fn' follow
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
  4. After weeks passed seemingly with no solution in sight, I decided to investigate this more. I had enough time to do so that giving up wasn't going to be on my mind.
  5. First, I found out that libpng.so points to libpng12.so, which is PNG version 1.2.54. I knew that this version was too old to support all of the features from 1.5.28. However, my distro had packaged libpng16, at version 1.6.20, but not libpng15, and it's a reflection of the Debian packages. I found out that, through the results on https://pkgs.org/, that only RPM-based distros packaged libpng15, while DEB-based distros did not, although both packaged libpng16.
  6. I could've just installed libpng16-dev onto my system, but since libgdk-pixbuf2.0-dev explicitly depends on libpng12-dev, and both libpng16-dev and libpng12-dev cannot coexist, it would've resulted in the removal of many more header files I've installed, including those for GTK-2.0 and GTK-3.0. However, I would've not had this problem if I installed only libpng16-16, which I eventually did. I almost thought that Ubuntu 16.04 was no longer supported as a build host for SuperTuxKart, but I did not give up.
  7. I tried to build with the internal libpng with version 1.5.28, and linked with libpng16, and the build succeeded. However, while I could launch the game successfully with the --no-graphics option, the game would end with a fatal error upon reading in its data files displace.png, and I learned that it was a bad idea to link code that was compiled with its own version of a library with the system library, especially when the own version has too many changes or is too old.
  8. I decided to see what's inside the ./lib/ directory. I have already noted the changes in this message from commit 9ebdb5fd00ea6df43c04517b30c77ab575bcfecf, and following 2 commits after this one (not shown here):
    deve {l Wrote}:
    {l Code}: {l Select All Code}
    Update libpng to newer version in order to fix some vulnerabilities.

    I decided to just make a bug-fix update and keep the 1.5.x series because newer versions may also need the zlib update.

    After reading this again today, I decided to be brave and upgrade both libpng and zlib from versions 1.5.28 and 1.2.7, to versions 1.6.30 and 1.2.11, respectively. I knew that project-specific fixes were usually important, so I also applied them to the new versions where necessary. In addtion, I also altered the following lines in my CMakeCache.txt, so that they point to the correct libpng, libpng16 (this step is not needed for Ubuntu 16.10 and later, since libpng16 is made the only libpng available from the repositories):
    {l Code}: {l Select All Code}
    //Path to a library.
    PNG_LIBRARY_DEBUG:FILEPATH=/usr/lib/x86_64-linux-gnu/libpng16.so.16

    //Path to a library.
    PNG_LIBRARY_RELEASE:FILEPATH=/usr/lib/x86_64-linux-gnu/libpng16.so.16
  9. After upgrading libpng and zlib, I reran my build process, and at the end, to my surprise, the build succeeded, and the game launched without any fatal errors, with graphics.
I would've thought about about upgrading the Irrlicht Engine from version 1.8.0 to 1.8.5, but that would too much work for me to do for now. The ./lib/irrlicht/update doesn't seem to work for some reason, but here, it's starting to feel outdated.
Extra notes:
  • ldd output shows my binary to linked to both libpng12 and libpng16.
    {l Code}: {l Select All Code}
    richard@M4A785TD-V:~/stk-code$ ldd ./bin/supertuxkart | grep png
            libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f1c92d23000)
            libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007f1c90286000)
  • The resulting output can be difficult to uninstall, but I found that CPack, which ships with CMake since version 2.4.2, can help simplify the creation of packages which can then be easily installed and later on uninstalled. See https://cmake.org/Wiki/CMake:Packaging_With_CPack for information on this tool. To use it with CMake, simply add this line into CMakeLists.txt:
    {l Code}: {l Select All Code}
    include(CPack)
  • I tried this tool, and here is the generated CPackConfig.cmake (the related CPackSourceConfig.cmake is similar in format), excluding the topmost comment:
    {l Code}: {l Select All Code}
    SET(CPACK_BINARY_7Z "")
    SET(CPACK_BINARY_BUNDLE "")
    SET(CPACK_BINARY_CYGWIN "")
    SET(CPACK_BINARY_DEB "OFF")
    SET(CPACK_BINARY_DRAGNDROP "")
    SET(CPACK_BINARY_IFW "OFF")
    SET(CPACK_BINARY_NSIS "OFF")
    SET(CPACK_BINARY_OSXX11 "")
    SET(CPACK_BINARY_PACKAGEMAKER "")
    SET(CPACK_BINARY_RPM "OFF")
    SET(CPACK_BINARY_STGZ "ON")
    SET(CPACK_BINARY_TBZ2 "OFF")
    SET(CPACK_BINARY_TGZ "ON")
    SET(CPACK_BINARY_TXZ "OFF")
    SET(CPACK_BINARY_TZ "ON")
    SET(CPACK_BINARY_WIX "")
    SET(CPACK_BINARY_ZIP "")
    SET(CPACK_CMAKE_GENERATOR "Unix Makefiles")
    SET(CPACK_COMPONENTS_ALL "")
    SET(CPACK_COMPONENT_UNSPECIFIED_HIDDEN "TRUE")
    SET(CPACK_COMPONENT_UNSPECIFIED_REQUIRED "TRUE")
    SET(CPACK_GENERATOR "STGZ;TGZ;TZ")
    SET(CPACK_INSTALL_CMAKE_PROJECTS "/home/richard/stk-code;SuperTuxKart;ALL;/")
    SET(CPACK_INSTALL_PREFIX "/usr/local")
    SET(CPACK_MODULE_PATH "/home/richard/stk-code/cmake")
    SET(CPACK_NSIS_DISPLAY_NAME "SuperTuxKart 0.1.1")
    SET(CPACK_NSIS_INSTALLER_ICON_CODE "")
    SET(CPACK_NSIS_INSTALLER_MUI_ICON_CODE "")
    SET(CPACK_NSIS_INSTALL_ROOT "$PROGRAMFILES")
    SET(CPACK_NSIS_PACKAGE_NAME "SuperTuxKart 0.1.1")
    SET(CPACK_OUTPUT_CONFIG_FILE "/home/richard/stk-code/CPackConfig.cmake")
    SET(CPACK_PACKAGE_DEFAULT_LOCATION "/")
    SET(CPACK_PACKAGE_DESCRIPTION_FILE "/usr/share/cmake-3.5/Templates/CPack.GenericDescription.txt")
    SET(CPACK_PACKAGE_DESCRIPTION_SUMMARY "SuperTuxKart built using CMake")
    SET(CPACK_PACKAGE_FILE_NAME "SuperTuxKart-0.1.1-Linux")
    SET(CPACK_PACKAGE_INSTALL_DIRECTORY "SuperTuxKart 0.1.1")
    SET(CPACK_PACKAGE_INSTALL_REGISTRY_KEY "SuperTuxKart 0.1.1")
    SET(CPACK_PACKAGE_NAME "SuperTuxKart")
    SET(CPACK_PACKAGE_RELOCATABLE "true")
    SET(CPACK_PACKAGE_VENDOR "Humanity")
    SET(CPACK_PACKAGE_VERSION "0.1.1")
    SET(CPACK_PACKAGE_VERSION_MAJOR "0")
    SET(CPACK_PACKAGE_VERSION_MINOR "1")
    SET(CPACK_PACKAGE_VERSION_PATCH "1")
    SET(CPACK_RESOURCE_FILE_LICENSE "/usr/share/cmake-3.5/Templates/CPack.GenericLicense.txt")
    SET(CPACK_RESOURCE_FILE_README "/usr/share/cmake-3.5/Templates/CPack.GenericDescription.txt")
    SET(CPACK_RESOURCE_FILE_WELCOME "/usr/share/cmake-3.5/Templates/CPack.GenericWelcome.txt")
    SET(CPACK_SET_DESTDIR "OFF")
    SET(CPACK_SOURCE_7Z "")
    SET(CPACK_SOURCE_CYGWIN "")
    SET(CPACK_SOURCE_GENERATOR "TBZ2;TGZ;TXZ;TZ")
    SET(CPACK_SOURCE_OUTPUT_CONFIG_FILE "/home/richard/stk-code/CPackSourceConfig.cmake")
    SET(CPACK_SOURCE_TBZ2 "ON")
    SET(CPACK_SOURCE_TGZ "ON")
    SET(CPACK_SOURCE_TXZ "ON")
    SET(CPACK_SOURCE_TZ "ON")
    SET(CPACK_SOURCE_ZIP "OFF")
    SET(CPACK_SYSTEM_NAME "Linux")
    SET(CPACK_TOPLEVEL_TAG "Linux")
    SET(CPACK_WIX_SIZEOF_VOID_P "8")

    if(NOT CPACK_PROPERTIES_FILE)
      set(CPACK_PROPERTIES_FILE "/home/richard/stk-code/CPackProperties.cmake")
    endif()

    if(EXISTS ${CPACK_PROPERTIES_FILE})
      include(${CPACK_PROPERTIES_FILE})
    endif()

    If CPack is to be considered as a way of creating packages for this project, the description and the package names should be changed to reflect the current state of this project (they actually should be set in CMakeLists.txt).
In summary, I got the latest git version of SuperTuxKart to build successfully after lots of changes. In the near future, consider one or more of the following changes I suggest (actually, I could do the first 2 points easily myself):
  • Including the CPack module in CMakeLists.txt, to make it easy to package the compiled output.
  • Upgrading both libpng and zlib from versions 1.5.28 and 1.2.7, to versions 1.6.30 and 1.2.11, respectively.
  • Upgrading the Irrlicht Engine from version 1.8.0 to 1.8.5, in order to take advantage of bugfixes and some new features provided upstream.
User avatar
Worldblender
 
Posts: 34
Joined: 15 Aug 2014, 18:46

Re: Solved my build problems

Postby Auria » 10 Jul 2017, 01:23

If someone wants to submit a pull request enabling to build STK with the newer versions of libpng/zlib, that would be appreciated! (for the moment, that should probably be optional as many distributions probably still ship the older versions).

Upgrading irrlicht will not happen because we're no longer using irrlicht (some irrlicht files are still left in the code but we're mostly using our own engine now)
Image
User avatar
Auria
STK Moderator
 
Posts: 2812
Joined: 07 Dec 2009, 03:52

Re: Solved my build problems

Postby deve » 10 Jul 2017, 07:23

So it looks that it's a regression since our libpng upgrade. I wonder what happens, because the only line that I added is "${CMAKE_CURRENT_BINARY_DIR}/../libpng/" in include_directories in irrlicht cmake file. And as long as you don't build static libpng and you don't have "build/lib/libpng" directory, it shouldn't matter... Other modifications are inside "lib/libpng" directory, so it shouldn't matter for linux build at all.

Maybe cmake finds our libpng copy in ./lib/libpng? You can see what contains the PNG_INCLUDE_DIR in CMakeCache.txt.

The message "undefined reference to `png_set_longjmp_fn'" suggests that it's linking with libpng 1.2.

If ldd shows that binary is linked with both libpng versions, it may be that other libraries depend on libpng and are linked with different version of the library (i.e libfreetype). You can precisely see linked libraries with "objdump -x filename | grep "\.so" ".

Most probable for me is that PNG_INCLUDE_DIR is "./lib/libpng" and the directory now doesn't contain pnglibconf.h, so it shows an error. I will see if I can reporoduce it...

Btw. what is your cmake version?

And using newest libpng/zlib should be fine too, but it needs some testing to make sure that everyting is ok, so I didn't care...
deve
 
Posts: 182
Joined: 23 Jan 2014, 13:45

Re: Solved my build problems

Postby Worldblender » 10 Jul 2017, 16:17

The current CMake version I have is 3.5.1. I also found my PNG_INCLUDE_DIR to be /usr/include/, looking at my main CMakeCache.txt.
Edit: I'm also having some problems with building the Android version, as I'm getting errors about undefined references to 'stdout', 'stdin', 'stderr', and 'atof', but only for the dependencies in dependencies-android.tar.xz. I used the Android NDK r15 to attempt this build.
Sample error output:
{l Code}: {l Select All Code}
/home/richard/stk-code/android/obj/openal/Alc/ALc.c:1220: error: undefined reference to 'stderr'
/home/richard/stk-code/android/obj/openal/Alc/ALc.c:874: error: undefined reference to 'stderr'
/home/richard/stk-code/android/obj/openal/Alc/helpers.c:285: error: undefined reference to 'fegetenv'
/home/richard/stk-code/android/obj/openal/Alc/helpers.c:294: error: undefined reference to 'fesetround'
/home/richard/stk-code/android/obj/openal/Alc/helpers.c:331: error: undefined reference to 'fesetenv'
/home/richard/stk-code/android/obj/openal/Alc/helpers.c:932: error: undefined reference to 'mmap64'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-mprintf.o):mprintf.c:function curl_mprintf: error: undefined reference to 'stdout'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-mprintf.o):mprintf.c:function curl_mvprintf: error: undefined reference to 'stdout'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-cookie.o):cookie.c:function Curl_cookie_init: error: undefined reference to 'stdin'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-cookie.o):cookie.c:function Curl_flush_cookies: error: undefined reference to 'stdout'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-url.o):url.c:function Curl_init_userdefined: error: undefined reference to 'stdout'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-url.o):url.c:function Curl_init_userdefined: error: undefined reference to 'stdin'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-url.o):url.c:function Curl_init_userdefined: error: undefined reference to 'stderr'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-url.o):url.c:function Curl_setopt: error: undefined reference to 'stderr'
/home/richard/stk-code/android/obj/curl/lib/.libs/libcurl.a(libcurl_la-formdata.o):formdata.c:function Curl_getformdata: error: undefined reference to 'stdin'
/home/richard/stk-code/android/obj/libpng/libpng.a(pngget.c.o):pngget.c:function png_get_sCAL_fixed: error: undefined reference to 'atof'
/home/richard/stk-code/android/obj/libpng/libpng.a(pngget.c.o):pngget.c:function png_get_sCAL_fixed: error: undefined reference to 'atof'
/home/richard/stk-code/android/obj/libpng/libpng.a(pngget.c.o):pngget.c:function png_get_sCAL: error: undefined reference to 'atof'
/home/richard/stk-code/android/obj/libpng/libpng.a(pngget.c.o):pngget.c:function png_get_sCAL: error: undefined reference to 'atof'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

I believe this to be one or more of these things:
  • The namespace 'std' hasn't been explicitly declared, as in 'using namespace std;'.
  • The dependencies-android.tar.xz is outdated, and needs an update.
User avatar
Worldblender
 
Posts: 34
Joined: 15 Aug 2014, 18:46

Re: Solved my build problems

Postby deve » 10 Jul 2017, 20:27

You can see if this helped:
https://github.com/supertuxkart/stk-cod ... 01ab867bee

NDK r15 is a bigger problem :/ I already looked at this and I wasn't able to compile STK dependiencies using it.

A bit more info:
https://android.googlesource.com/platfo ... Headers.md
https://github.com/android-ndk/ndk/issues/445
https://github.com/openssl/openssl/issues/3826

It should be possible to compile it, but it needs at least patched openssl, so it seems to not have a sense. I also trust gcc much more than clang (clang aarch64 build was causing a crash on startup that was unrelated to STK on some devices). So I suggest to stay with NDK r12b for compilation. It actually has everything that we need.
deve
 
Posts: 182
Joined: 23 Jan 2014, 13:45

Who is online

Users browsing this forum: No registered users and 1 guest