TL;DR: Do not use the Docker-based Sailfish SDK on an out-of-the-box installation of Fedora. Either use the VirtualBox based setup or use Docker after enabling cgroups v1 compatibility.
If you really want to try to use Podman, you can try the “wrapper script” from this repository.
This blog post covers my many attempts to use the Sailfish SDK on Fedora 33 to build a Rust-based application.
Spoiler alert: it didn’t went well .
The Fedora experience
Long time ago, after distro-hopping from Mandrivas, Ubuntus and OpenSUSE, I have settled on using Fedora for my daily driver. It is a nice distro, being both reliable and innovative. I’m trying to not tweak it’s out-of-the-box configuration, to “benefit” from all the features that are shipped with it.
While most of these features were welcoming or at least invisible, some of them are more controversial for end-users. For example, I still don’t see the use of SELinux, that often get into my way.
But today, we are going to talk about another of these kind of features, introduced by Fedora 31: cgroups v2. While I
don’t know what it brings (and didn’t read about it), this change made Docker unavailable for Fedora 1.
Fortunately, podman is proposed as a drop-in replacement. The project website even recommend
After using Podman for some time, I was ready to state that it was indeed a good replacement for Docker, until that day where I wanted to (re)use the Sailfish SDK to build some Rust code.
The not-so drop-in replacement
The Sailfish SDK packages a cross-compiler, Scratchbox 2, inside an OS image. The SDK boots this image to start the “build engine”. It can either use Virtual Box or Docker. When the build engine is started, the SDK communicates with it with SSH.
Armed with my trust in Podman, I created a symbolic link from
docker and installed the SDK by choosing
the Docker option. The installation went well but unfortunately, inside the SDK, the build engine could not be started.
After looking at the logs 2 and into the code 3, I found out that the SDK is trying to find an image named
sailfish-os-build-engine and, when found, starting a container with the same name. This works fine with Docker, but
Podman prefixes all local images with
localhost. On my machine, the image is then named
I tried to cheat a bit and configure a build engine named
localhost/sailfish-os-build-engine, but this didn’t worked
either. The SDK will always use the image name for the container name and slashes are forbidden in container
After countless hours searching for various solution on the Internet, in the hope of finding the magic switch that
localhost prefix, I ended by concluding that this is impossible 4. Podman isn’t a drop-in replacement
for Docker after all.
I still had a way to make the SDK work. By introducing a little change in the SDK, I could remove the prefix when listing images. This should make the SDK compatible with Podman. After a rebuild of Qt Creator and a tentative pull-request, my patched SDK was ready.
A bus problem
The rebuilt SDK allowed me to start the engine, but didn’t seems to acknowledge that it started. Trying to SSH also lead to an error.
$ ssh -p 2222 -i ~/Dev/SailfishOS/SDK/vmshare/ssh/private_keys/engine/mersdk mersdk@localhost kex_exchange_identification: read: Connection reset by peer Connection reset by ::1 port 2222
The engine seems to reject SSH connections, so as a last resort, I executed a shell directly in the container …
$ podman exec -it sailfish-os-build-engine /bin/bash Failed to connect to bus: No such file or directory
… and discovered another class of issues.
Remember that the Sailfish OS SDK build engine run either as a VM or a container. When running in a VM, a fully fledged Linux distribution is booted, including a SSH server and a web-service.
The Docker based build engine should behave similarly as the VM. In fact, the entrypoint of the container is
systemd, so when the container is started,
systemd will be in charge of initializing various daemons
in the container, including the SSH server.
The bus error I’m receiving seems to indicate that the container could not interact with systemd running on host. I’m suspecting that the change to cgroups v2 is still causing trouble here. Enabling cgroups v1 compatibility seems to fix this issue as reported by this thread.
Introducing second order hacks
In the meantime, SDK developers suggested to use an alternative docker executable. In fact, the docker command to be used can be configured via an ini file.
# <sdk-root>/share/qtcreator/SailfishSDK/libsfdk.ini # Set the path to your docker executable DockerPath=/home/sk/Dev/SailfishOS/sdk-podocker/target/debug/sdk-podocker
So instead of changing how the SDK lists images or starts container, I could simply write a script that wraps Docker or Podman. This script would be in charge of
- Patching the output of
- Start the SSH server inside the build engine container after
I have written a very small Rust program, located in this repository. You can try it but, as you will see later, I do not recommend using it as it didn’t resolve all my issues.
Will it build ?
When using the wrapper script, the SDK finally recognizes the build engine and could start to build. However, I faced a lot of permissions errors when writing to the host’s file system from inside the container.
Podman in a rootless mode performs id mapping: files on the host file system and mapped using volumes will appear as
root inside the container. And when root writes files on the mapped volumes, they will be owned by the host’s
(non-root) user. The build engine do not use root for most of it’s operations. Instead it is using the
that do not have permission to write in
While this permissions issue is still something manageable 5, I feel that the giant pile of hacks I’m starting to
build didn’t feel sustainable. But the final nail in the
coffin container came with Rust: when following this
forum.sailfishos.org thread to install Rust,
I managed to break my build engine 6.
At start, I simply wanted to build some Rust code. But I got carried away quite far. In the meantime, we have seen the differences between Docker and Podman, Podman specificities and how the Sailfish SDK works.
For Fedora users, I recommend to stick with Virtual Box based SDK. You can also try to using Docker and reenable
cgroups v1 by passing
systemd.unified_cgroup_hierarchy=false to the kernel command line. I do not recommend using
Podman for the SDK 7.
I have learned today that Docker is now compatible with cgroups v2. I might revisit one day my decision to live without Docker. ↩
I take this opportunity to give Kudos to Jolla developers for caring and improving their SDK. I was amazed by the amount of changes since the last time I used it ! Some highlights: Docker support and the CLI. Running the CLI with debug logs helped me a lot. ↩
chmod 777is still an option. ↩
More on this on a follow-up article. ↩
I do recommend using Podman for containers though. The daemon-less, root-less design is really nice, but simply doesn’t work with the Sailfish SDK. ↩