Swig typemaps cheatsheet

The documentation of SWIG is pretty extensive, but there isn’t a single point that document all available macros / substitution mechanisms available while writing a typemap. It’s a nightmare to find what you’re looking for when you’re not already really familiar with it, so to save myself and maybe a few others from early hair loss, here is a quick list of some useful substitutions I found.

Assuming I’m writing an out typemap for a template<class T> class Handle;

$1The local copy of the C/C++ object
$resultThe output PyObject to return to Python
$1_typeThe type of the C/C++ object (Handle<T>)
$1_descriptor The SWIG type descriptor corresponding to Handle<T>
$&1_descriptor The SWIG type descriptor corresponding to Handle<T> *

The case I was trying to cover when I almost became bald is the following: I have a templated Handle class (as you’ve probably already guessed) that’s used to encapsulate a heap allocated data. Some API functions / methods bound in Python return such handles in place of pointers, but I wanted the those to return None in Python instead of returning an invalid handle.

So I basically needed a typemap that checked the validity of my handle, and if invalid returned None, otherwise returned the usual SWIG object. Here is the simplified typemap:

%typemap(out) Handle {
    if ($1.valid() == false) {
        $result = Py_None;
    } else {
        // this fucking $&1_descriptor is at the origin of this post...
        $result = SWIG_NewPointerObj(new $type(std::move($1)), $&1_descriptor, SWIG_POINTER_OWN);
    }
}

Linking against a static Qt5 build using CMake

The previous post was the script I use to statically build Qt5 on Windows, but the real fun part is then linking an application using that version ! Here is the part of the CMake script I’m using (as usual, for reference)

For the moment I only build a basic Widgets application. As I use more modules / functionalities, I’ll update the script. Also please note that the very beginning (finding Qt) is tailored to my development setups, so you might need to update the paths (or provide CMake with a valid QT_DIR option)

Edit 2019-10-17: Since I removed accessibility from my build script, WindowsUIAutomationSupport is no longer needed as a dependency of the Windows integration plugin.

Edit 2019-10-18: I created a repository to start collecting my various CMake helper scripts, and the following script is now hosted there (it’s also a lot cleaner, modular and easy to use) You can see it here: https://github.com/dcourtois/CMakeUtils/blob/master/Qt.cmake

#
# Set to On to use a statically built version of Qt
#
set (QT_STATIC On)
if (QT_STATIC)
    set (QT_STATIC_SUFFIX "static/lib/cmake")
endif ()

#
# Find Qt.
#
find_path (QT_DIR Qt5
    HINTS
        # User set directory
        "${QT_DIR}"

        # My dev directories, depending on which machine I'm on
        "C:/Development/Libs/Qt/5.13.1"
        "D:/Development/Libs/Qt/5.13.1"
        "~/Development/Libs/Qt/5.13.1"

    PATH_SUFFIXES
        # As for the dev dirs, these are the suffixes to the CMake folder on the various machines I own.
        # Note that those are the default subfolder when installing Qt from its official installer.
        "${QT_STATIC_SUFFIX}"
        "msvc2017_64/lib/cmake"
        "gcc_64/lib/cmake"
        "clang_64/lib/cmake"
)

#
# Log / Error
#
if (NOT QT_DIR)
    message (FATAL_ERROR "Couldn't find Qt. Use QT_DIR variable to point to a valid Qt install.")
else ()
    message (STATUS "Found Qt in '${QT_DIR}'")
endif ()

#
# Find our Qt components
#
set (CMAKE_PREFIX_PATH ${QT_DIR})
find_package (Qt5 5
    COMPONENTS
        Widgets
    REQUIRED
)

#
# When using static build, exported Qt targets miss a awefull lot of dependencies (on Windows
# at least, didn't check the other platforms) so to avoid bothering, patch Qt5::Widgets
#
if (QT_STATIC)

    #
    # Set a few paths
    #
    set (QT_LIB_DIR "${QT_DIR}/..")
    set (QT_PLUGIN_DIR "${QT_DIR}/../../plugins")

    #
    # Qt5::QWindowsIntegrationPlugin
    # note that this target somehow is already there even if we didn't search for it in the
    # find_package command. And since it's mandatory to run the most basic Widgets application
    # on Windows ...
    #
    if (TARGET Qt5::QWindowsIntegrationPlugin)

        # find additional components needed by the windows platform plugin
        find_package (Qt5 5
            COMPONENTS
                EventDispatcherSupport
                FontDatabaseSupport
                ThemeSupport
            REQUIRED
        )

        # configure direct dependencies of the plugin
        target_link_libraries(Qt5::QWindowsIntegrationPlugin
            INTERFACE
                # Qt targets
                Qt5::EventDispatcherSupport
                Qt5::FontDatabaseSupport
                Qt5::ThemeSupport

                # Windows libs
                Dwmapi.lib
                Imm32.lib
                Wtsapi32.lib

                # The following is needed if you want to use the Windows vista style plugin.
                # If you provide your own style or CSS, you can comment out the following libs.
                ${QT_PLUGIN_DIR}/styles/qwindowsvistastyle.lib
                UxTheme.lib
        )

    endif ()

    #
    # Qt5::FontDatabaseSupport
    #
    if (TARGET Qt5::FontDatabaseSupport)

        target_link_libraries(Qt5::FontDatabaseSupport
            INTERFACE
                # Qt libs
                ${QT_LIB_DIR}/qtfreetype.lib
        )

    endif ()

    #
    # Qt5::Gui
    #
    if (TARGET Qt5::Gui)

        target_link_libraries(Qt5::Gui
            INTERFACE
                # Qt targets
                Qt5::QWindowsIntegrationPlugin

                # Qt libs
                ${QT_LIB_DIR}/qtlibpng.lib
        )

    endif ()

    #
    # Qt5::Core
    #
    if (TARGET Qt5::Core)

        target_link_libraries(Qt5::Core
            INTERFACE
                # Qt libs
                ${QT_LIB_DIR}/qtpcre2.lib

                # Windows libs
                Netapi32.lib
                Ws2_32.lib
                UserEnv.lib
                Version.lib
                Winmm.lib
        )

        target_compile_definitions (Qt5::Core
            INTERFACE
                # Remove debug stuff from Qt
                $<$<CONFIG:Release>:QT_NO_DEBUG>
                $<$<CONFIG:Release>:QT_NO_DEBUG_OUTPUT>
                $<$<CONFIG:Release>:QT_NO_INFO_OUTPUT>
                $<$<CONFIG:Release>:QT_NO_WARNING_OUTPUT>

                # Since Qt was built in release, we need to match it on Windows
                _ITERATOR_DEBUG_LEVEL=0
        )

    endif ()

endif ()

So this needs to be used after your project has been defined, and then you’ll be able the just add Qt5::Widgets to your target’s link libraries, and it should correctly link everything needed to run your Widgets application.

You’ll still need to manually import plugins though, and since the platform integration one is mandatory, here is how to do it (put that in your main compilation unit or wherever you want)

#if defined(QT_STATIC)

// needed to import statically linked plugins
#include <QtPlugin>

// The minimum plugin needed is the platform integration. Without it, the application just crashes on startup.
Q_IMPORT_PLUGIN(QWindowsIntegrationPlugin);

// This one is for the Windows 'Vista' theme. As I wrote in the CMake part, this is
// optional if you plan on using your own CSS or style.
Q_IMPORT_PLUGIN(QWindowsVistaStylePlugin);

#endif

Qt5 static Windows build script

To avoid loosing the script I use to build Qt 5 on Windows, here is the latest version I’m using. Note that I’m compiling using Visual Studio 2019 and I disable things I don’t need to try to minimize build time.

Edit 2019-10-17: Removed accessibility and harfbuzz

@echo off

::
:: Configure Visual Studio 2019 Community for x64 build.
::
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64

::
:: Set Qt directory and set it as the current one.
::
set QT_DIR=C:\Development\Libs\Qt\5.13.1
pushd "%QT_DIR%\Src"

::
:: Configure Qt
::
call configure -release             ^
    -static                         ^
    -opensource -confirm-license    ^
    -prefix "%QT_DIR%/static"       ^
    -platform win32-msvc            ^
    -mp                             ^
    -c++std c++17                   ^
    -no-accessibility               ^
    -no-icu                         ^
    -no-harfbuzz                    ^
    -nomake examples                ^
    -nomake tests                   ^
    -skip qtwebengine               ^
    -skip qt3d

::
:: Build and install
::
nmake
nmake install

::
:: Restore the previous working directory
::
popd

Overwrite all git commit’s author

As a little post-it, here’s a script to amend author’s name and email of all commits of a repository. Use with care, it will override all commits, no matter what the original author is.

#!/bin/sh

if [ "$#" -ne 2 ]; then
	echo "usage: amend 'name' 'email'"
	exit 1
fi

git filter-branch -f --env-filter "
	GIT_AUTHOR_NAME='$1'
	GIT_AUTHOR_EMAIL='$2'
	GIT_COMMITTER_NAME='$1'
	GIT_COMMITTER_EMAIL='$2'
" HEAD

Credit: https://stackoverflow.com/a/750191/913135

Visual Studio 2017, CMake and launch settings

Long story short, documentation for CMake related features in Visual Studio 2017 sucks. It’s scattered among a ton of blog post where most of the code samples are outdated or contain errors. Here is an example launch.vs.json that I’m using for reference:

{
	"version": "0.2.1",
	"defaults": {},
	"configurations": [
		{
			"type": "Debug",
			"project": "CMakeLists.txt",
			"projectTarget": "foo.exe",
			"name": "foo.exe",

			// currentDir and not cwd like we can read in most examples ...
			"currentDir": "${workspaceRoot}",

			// this one is pretty straightforward
			"args": [ "arg0", "arg1" ],

			// env overwrites all the environment variables ...
			// separate vars by a null (\u0000) character. You can use existing env vars, see PATH
			"env": "TEST_ENV=foo\u0000HELLO=World\u0000PATH=${env.PATH};${workspaceRoot}"
		}
	]
}

Building & Debugging Blender with Visual Studio Code

Visual Studio Code is an open source and multiplatform IDE for code editing (and compiling, debugging, etc.) available here : https://code.visualstudio.com/

I wanted to see if I could build, run and debug Blender using this IDE, and the anwser is : yes. Here is how to do it.

Edit: Updated the scripts and configuration of Visual Studio Code (no longer requires an CMake install, will load scripts and datafiles directly from the sources)

Setup

Based on : https://wiki.blender.org/index.php/Dev:Doc/Building_Blender/Linux/Generic_Distro/CMake.

I’m using the following folder hierarchy to build:

Blender         # with a B.
   |--- blender # with a b, this is a clone of the Blender source repository
   |--- libs    # will contains Blender's dependencies (external libraries)

So, first step is to create this structure, retrieve the Blender sources, and build the dependencies. To do that, here is the bootstrap script I’m using (put that in a bootstrap.sh file and execute it)

# create our root folder
mkdir Blender
cd Blender

# this will download Blender's sources
git clone --recursive https://git.blender.org/blender.git

# now create the libs folders
mkdir -p libs/sources
mkdir -p libs/install

# and build them. Note that the source and install path need
# to be absolute
./blender/build_files/build_environment/install_deps.sh \
    --with-all                                          \
    --source $PWD/deps/src                              \
    --install $PWD/deps/install

This can take quite a bit of time, depending on your computer. But you only need to do it once (well, sometime the libs are updated, but it doesn’t happen very often)

Also, after this is done, you’ll notice a file BUILD_NOTES.txt in the root Blender folder. This is a generated file which contains useful information about the dependencies (we’ll use this file to configure Visual Studio Code to build Blender)

Install Visual Studio Code

Once you’ve installed Visual Studio Code, install the following addons :

  • C/C++ (Microsoft) – C/C++ support. Required
  • CMake (twxs) – CMake syntax coloring, code completion, etc. Optional but useful if you have to modify a CMake script file
  • CMake Tools (Microsoft) – Add Configure/Build features based on CMake. Required
  • Clang-Format (xaver) – Used to automatically format code to follow Blender’s coding style. Optional but highly recommended if you want to contribute.

Note that CMake Tools plugin requires CMake version 3.7.1 or greater, with server mode enabled (if you installed a recent prebuilt version from the CMake website, it should be enabled by default. But if you build your own, make sure it’s enabled)

Build

Now, open Visual Studio Code, and open the Blender/blender folder. If CMake Tools worked correctly, the status bar should look like this:

Before going further, we will setup the project to configure CMake to use the dependencies. In the sources (Blender/blender) create a .vscode folder. In this folder, create a settings.json file:

{
    // Where Blender is built. DO NOT BUILD IN SOURCES !
    "cmake.buildDirectory": "${workspaceRoot}/../build",

    // If you want to generate an install of Blender, setup this path:
    "cmake.installPrefix":  "${workspaceRoot}/../install"

    // The options used by CMake to configure how Blender is built
    "cmake.configureSettings": {
        // this part comes from the BUILD_NOTES.txt file generated when we build the libs
        "WITH_CODEC_SNDFILE": "ON",
        "PYTHON_VERSION": "3.7",
        "WITH_OPENCOLORIO": "ON",
        // ... the rest of the file
    },

    // Clang-format options (only if your installed the extension)
    "editor.formatOnSave": true,
    "editor.defaultFormatter": "xaver.clang-format"
}

Notice that the options in BUILD_NOTES.txt are not formatted the same way as CMake Tools want, so you need to format them like in the example above.

Now after this step, you’ll be able to build Blender. Just setup the kit you want to use, select the build type (both actions are done using the CMake Tools helpers that appear on the status bar, on the bottom-left of Visual Studio Code) and hit F7

Debug

Time for fun! Blender is built, we’d like to dive in the code. There’s one last thing to do: add a launch configuration. Indeed, unless you make an install, the scripts and datafiles are not accessible to the Blender’s executable. So to debug, we need to create a launch.json next to our settings.json file.

This is the content on my file on Linux, with comments on the relevent sections:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) Launch",
            "type": "cppdbg",
            "request": "launch",
            // path to the Blender's executable
            "program": "${command:cmake.buildDirectory}/bin/blender",
            // in here you can add arguments that will be passed to blender when the debugging session starts
            "args": [],
            "stopAtEntry": false,
            // by default it runs in the source folder, but you can use any other you like
            "cwd": "${workspaceFolder}",
            // this one is the most important! Without those environment variable, Blender will fail to start because it won't find its Python scripts and data files.
            "environment": [
                { "name": "BLENDER_SYSTEM_SCRIPTS", "value": "${workspaceFolder}/release/scripts" },
                { "name": "BLENDER_SYSTEM_DATAFILES", "value": "${workspaceFolder}/release/datafiles" }
            ],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [ { "description": "", "text": "-enable-pretty-printing", "ignoreFailures": true } ]
        }
    ],
    "compounds": []
}

And there, you’re done. Now you can run Blender from inside Visual Studio Code by hitting F5. If you selected a Debug build type, breakpoints will work as excepted, etc.

Conclusion

It was my first time using Visual Studio Code, and I must say I’m very impressed by the ease of use and feature set ! Out of the box, it has Git support, and through the embedded extension manager you can enable C/C++ building/debugging/etc., CMake support, etc. with only a few clicks !

And then the setup part is completely straightforward ! You have a .vscode folder in your project containing the various .json setting files, and in each of those files, autocompletion works like a charm so you can easily find what you want, without even having to bother looking online for documentation / help !! (in my first tests, I was on an old computer where I had to specify the path to my custom CMake, and the path to my custom GDB, and I found them in a few seconds without having to leave the IDE)

Anyway, I highly recommend you to give a try to this software, CMake support is almost perfect (I still have to delete the build folder manually whenever I want to change the options CMake was configured with, but other than that, perfect) and building / debugging is fast, intuitive, user friendly and powerful.

Fontainebleau

Deux petites vidéos pour deux WE de grimpe, avec :

  • Renversement Dialectique (7a)
  • La Memel (7a+)
  • Free Hug (7b)
  • Noir Désir gauche (7b)
  • Dark Room (7a+)

Change default location of Visual Studio’s intellisense databases

A small post-it on how to change the default location of intellisense databases for Visual Studio. By default, those huge .sdf files (which get updated again and again) are stored next to your project location.

If you’re working on an SSD, or on a slow USB drive, you might want to avoid storing those databases at their usual location (or just to avoid having huge files around, or whatever other reason)

To do that, just go to Tools > Options then on the Text Editor > C/C++ > Advanced tab, locate the section Fallback Location. Here, Always use Fallback Location will need to be set to true to tell Visual Studio to always use the Fallback Location (which you can then set to any folder you want, or leave empty if you want to use the windows temp folder)

You can now close Visual Studio, remove your sdf files, launch it again. It will prompt you about your fallback location, but you just have to click OK and don’t forget to check the Don’t Prompt Me Again checkbox to avoid this at every launch. Tadaaa, no more sdf files next to your projects !

[Source]

Apremont Ouest

Beau ciel bleu et grosse collante pour un après-midi à Apremont. Et quelques belles croix :

  • Tailler en Pièce (7a)
  • Onde de Choc (7b)
  • Fleurs de Rhum (7a+)
  • Crazy Horse (7b)

Starring Carlos la Machina, Chukky le sac à pouf cool, et moiself.

Compiling ICU with Visual Studio 2013

In my previous post on how to build Qt on Windows, I explained how to build Qt for Windows, using Visual Studio 2010 and prebuilt ICU libraries. If we want to build Qt with Visual Studio 2013, we’ll need to build ICU ourself, and here’s how :

  • Download ICU sources from here : http://site.icu-project.org/download
  • Unzip in C:\ (you will have a C:\icu folder with C:\icu\readme.html (among others))
  • Go into C:\icu\source\common\unicode, edit platform.h and add the following somewhere at the beginning. It’s needed if you want to build Qt :
#define U_CHARSET_IS_UTF8 1
  • Go into C:\icu and create a build.bat file, with the following content and run it :
@echo off

::
:: setup Visual Studio 2013 environment for x64 builds
::
call "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" x86_amd64

::
:: build
::
msbuild source\allinone\allinone.sln /m /target:Build /property:Configuration=Release;Platform=x64
msbuild source\allinone\allinone.sln /m /target:Build /property:Configuration=Debug;Platform=x64

This will build ICU in x64. You can run the same script again and remove the “x86_amd64” argument to the vcvarsall.bat script, and change the Platform property to Win32 to build ICU in x86.