1. 6

I need a C build environment (libc / compiler / make / shell) for testing Oil [1].

My desktop is an Ubuntu 16.04 box, so ideally it should work there. Ideally it would let me test some combo of:

  • Linux distro: Ubuntu vs. Arch
  • OS/kernel: Linux vs OpenBSD
  • Architecture: x64 vs x86 vs ARM variants etc.

Things I thought of:

  • VirtualBox: there is osboxes.org, not sure I trust it

  • QEMU: I have to build all my own images?

  • Vagrant: has some official images, but doesn’t seem to support ARM ?

  • Docker: has images, doesn’t seem to support non-Linux or ARM

  • Various Cloud solutions: these are mostly x86_64?

  • Real Hardware: well I bought a Pi Zero, but it’s just one arm variant and it’s slow…

It seems kinda weird that this doesn’t exist already? I guess the problem is that servers are almost all x86, and anybody using ARM typically has non-trivial hardware, so they don’t use virtualization to test.

I did watch a tech talk about the Golang continuous build and it seems like they had to build a very extensive infrastructure for testing across architectures.

The Aboriginal Linux project is sort of like what I want, with pre-built QEMU images, but it’s more of an educational thing and it’s been discontinued:


(It also uses really old versions of some things.)

[1] https://github.com/oilshell/oil/issues/16


  2. 4

    I’ve written a tool to do this using Docker “build containers”. For the non-amd64 linux stuff you’d have to setup a cross compiler, but it is doable. You can find the tool at https://bitbucket.org/rw_grim/convey.

    1. 1

      Thanks! This looks like it’s solving a very similar problem as the one I have. So this is a tool to specify the build environment for Pidgin across multiple platforms? And I assume you have some kind of continuous build/testing, like either pre-release or pre/post-commit?

      Is there a dashboard for the continuous build? I’d be interested to see what it looks like.

      It seems like newer open source projects use cloud services for this, but older ones roll have their own servers… I might want to use my own servers too.

      1. 1

        Right now it’s just being triggered by bamboo. You can see it at https://bamboo.pidgin.im but it’s not setup to use all of this yet. I need to finish building some Docker images for some dependencies and stuff.

        Aside from that I am working on (albeit slowly) a webapp that’s specially for this so you don’t need to use Jenkins/Bamboo/etc.

        1. 1

          Oh what’s the issue with Bamboo? (I’ve never used it.) Is it an issue of not being open source, or not having features you need? What about Jenkins?

          1. 2

            The problem with all of them is having to maintain configurations for a ton of platforms outside of the repo which drives their changes.

      1. 3

        If I had to pick a single second architecture for testing, I would go for SPARC - it’s big endian and disallows unaligned accesses.

        Making your own images in QEMU is no different to doing a regular install. make a file big enough to act as a disk (dd if=/dev/zero of=file.img bs=1g count=4) , and ‘qemu -cdrom installer.iso -hda file.img’. Now you can use file.img as an image for as long as you like.

        1. 2

          There are few options to test your code on a various hardware platforms.

          First solution is using real hardware with required hardware architecture, like OpenBSD/NetBSD projects do. The disadvantage with this is you need to host hardware somewhere and maintain it. This activity can be a challenge, I remember Miod Vallat wrote about his machineroom and told he spend a lot of time to maintain his hardware zoo. If you don’t want to waste a time and host your own hardware you can rent hardware: Oracle sell access to a MIPS servers in Oracle Cloud, SoftLayer sell access to Power servers, Scaleway - ARM baremetal servers. But it will costs some amount of money. Last option for using real hardware with rare CPU architecture is an access to a hardware farm. For example GCC project has one and PostgreSQL has it’s own buildfarm.

          Second option is using hardware virtualization. QEMU supports various different hardware architectures and it is possible to test your software with binary translation using qemu-user-static. For example CRIU project use it in Travis CI containers, take a look on their scripts.

          1. 1

            I remember Miod Vallat wrote about his machineroom and told he spend a lot of time to maintain his hardware zoo.


            1. 1

              Thanks this is exactly what I was looking for! I’m curious how other open source projects do it. Interesting that GNU has one you can apply for!