Support for AARCH64

Tue Mar 24 15:18:58 UTC 2020

Compare your configure line with what's currently in use (or the apkbuild
file), there are a few options (with-unwind, without-jemalloc, etc.) That
need to be set

On Tue, Mar 24, 2020, 08:05 Martin Grigorov <martin.grigorov at gmail.com>
wrote:

> Hi,
>
> On Tue, Mar 24, 2020 at 11:00 AM Martin Grigorov <
> martin.grigorov at gmail.com> wrote:
>
>> Hi Guillaume,
>>
>> On Mon, Mar 23, 2020 at 8:01 PM Guillaume Quintard <
>> guillaume at varnish-software.com> wrote:
>>
>>> Hi Martin,
>>>
>>> Thank you for that.
>>> A few remarks and questions:
>>> - how much time does the "docker build" step takes? We can possibly
>>> speed things up by push images to the dockerhub, as they don't need to
>>> change very often.
>>>
>>
>> Definitely such optimization would be a good thing to do!
>> At the moment, with 'machine' executor it fetches the base image and then
>> builds all the Docker layers again and again.
>> Here are the timings:
>> 1) Spinning up a VM - around 10secs
>> 2) prepare env variables - 0secs
>> 3) checkout code (varnish-cache) - 5secs
>> 4) activate QEMU - 2secs
>> 5) build packages
>> 5.1) x86 deb - 3m 30secs
>> 5.2) x86 rpm - 2m 50secs
>> 5.3) aarch64 rpm - 35mins
>> 5.4) aarch64 deb - 45mins
>>
>>
>>> - any reason why you clone pkg-varnish-cache in each job? The idea was
>>> to have it cloned once in tar-pkg-tools for consistency and
>>> reproducibility, which we lose here.
>>>
>>
>> I will extract the common steps once I see it working. This is my first
>> CircleCI project and I still find my ways in it!
>>
>>
>>> - do we want to change things for the amd64 platforms for the sake of
>>> consistency?
>>>
>>
>> So far there is nothing specific for amd4 or aarch64, except the base
>> Docker images.
>> For example make-deb-packages.sh is reused for both amd64 and aarch64
>> builds. Same for -rpm- and now for -apk- (alpine).
>>
>> Once I feel the change is almost finished I will open a Pull Request for
>> more comments!
>>
>> Martin
>>
>>
>>>
>>> --
>>> Guillaume Quintard
>>>
>>>
>>> On Mon, Mar 23, 2020 at 6:25 AM Martin Grigorov <
>>> martin.grigorov at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>>
>>>> On Wed, Mar 18, 2020 at 5:31 PM Martin Grigorov <
>>>> martin.grigorov at gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On Thu, Mar 12, 2020 at 4:35 PM Martin Grigorov <
>>>>> martin.grigorov at gmail.com> wrote:
>>>>>
>>>>>> Hi Guillaume,
>>>>>>
>>>>>> On Thu, Mar 12, 2020 at 3:23 PM Guillaume Quintard <
>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Offering arm64 packages requires a few things:
>>>>>>> - arm64-compatible code (all good in
>>>>>>> https://github.com/varnishcache/varnish-cache)
>>>>>>> - arm64-compatible package framework (all good in
>>>>>>> https://github.com/varnishcache/pkg-varnish-cache)
>>>>>>> - infrastructure to build the packages (uhoh, see below)
>>>>>>> - infrastructure to store and deliver (
>>>>>>> https://packagecloud.io/varnishcache)
>>>>>>>
>>>>>>> So, everything is in place, expect for the third point. At the
>>>>>>> moment, there are two concurrent CI implementations:
>>>>>>> - travis:
>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.travis.yml It's
>>>>>>> the historical one, and currently only runs compilation+test for OSX
>>>>>>>
>>>>>>
>>>>>> Actually it tests Linux AMD64 and ARM64 too.
>>>>>>
>>>>>>
>>>>>>> - circleci:
>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.circleci/config.yml the
>>>>>>> new kid on the block, that builds all the packages and distchecks for all
>>>>>>> the packaged platforms
>>>>>>>
>>>>>>> The issue is that cirecleci doesn't support arm64 containers (for
>>>>>>> now?), so we would need to re-implement the packaging logic in Travis. It's
>>>>>>> not a big problem, but it's currently not a priority on my side.
>>>>>>>
>>>>>>> However, I am totally ready to provide help if someone wants to take
>>>>>>> that up. The added benefit it that Travis would be able to handle
>>>>>>> everything and we can retire the circleci experiment
>>>>>>>
>>>>>>
>>>>>> I will take a look in the coming days and ask you if I need help!
>>>>>>
>>>>>
>>>>> I've took a look at the current setup and here is what I've found as
>>>>> problems and possible solutions:
>>>>>
>>>>> 1) Circle CI
>>>>> 1.1) problem - the 'machine' and 'Docker' executors run on x86_64, so
>>>>> there is no way to build the packages in a "native" environment
>>>>> 1.2) possible solutions
>>>>> 1.2.1) use multiarch cross build
>>>>> 1.2.2) use 'machine' executor that registers QEMU via
>>>>> https://hub.docker.com/r/multiarch/qemu-user-static/ and then builds
>>>>> and runs a custom Docker image that executes a shell script with the build
>>>>> steps
>>>>> It will look something like
>>>>> https://github.com/yukimochi-containers/alpine-vpnserver/blob/69bb0a612c9df3e4ba78064d114751b760f0df9d/.circleci/config.yml#L19-L38 but
>>>>> instead of uploading the Docker image as a last step it will run it.
>>>>> The RPM and DEB build related code from current config.yml will be
>>>>> extracted into shell scripts which will be copied in the custom Docker
>>>>> images
>>>>>
>>>>> From these two possible ways I have better picture in my head how to
>>>>> do 1.2.2, but I don't mind going deep in 1.2.1 if this is what you'd prefer.
>>>>>
>>>>
>>>> I've decided to stay with Circle CI and use 'machine' executor with
>>>> QEMU.
>>>>
>>>> The changed config.yml could be seen at
>>>> https://github.com/martin-g/varnish-cache/tree/feature/aarch64-packages/.circleci and
>>>> the build at
>>>> https://app.circleci.com/pipelines/github/martin-g/varnish-cache/71/workflows/3a275d79-62a9-48b4-9aef-1585de1c87c8
>>>> The builds on x86 arch take 3-4 mins, but for aarch64 (emulation!)
>>>> ~40mins
>>>> For now the jobs just build the .deb & .rpm packages for CentOS 7 and
>>>> Ubuntu 18.04, both amd64 and aarch64.
>>>> TODOs:
>>>> - migrate Alpine
>>>>
>>>
> Build on Alpine aarch64 fails with:
> ...
> automake: this behaviour will change in future Automake versions: they will
> automake: unconditionally cause object files to be placed in the same
> subdirectory
> automake: of the corresponding sources.
> automake: project, to avoid future incompatibilities.
> parallel-tests: installing 'build-aux/test-driver'
> lib/libvmod_debug/Makefile.am:12: warning: libvmod_debug_la_LDFLAGS
> multiply defined in condition TRUE ...
> lib/libvmod_debug/automake_boilerplate.am:19: ...
> 'libvmod_debug_la_LDFLAGS' previously defined here
> lib/libvmod_debug/Makefile.am:9:   'lib/libvmod_debug/
> automake_boilerplate.am' included from here
> + autoconf
> + CONFIG_SHELL=/bin/sh
> + export CONFIG_SHELL
> + ./configure '--prefix=/opt/varnish' '--mandir=/opt/varnish/man'
> --enable-maintainer-mode --enable-developer-warnings
> --enable-debugging-symbols --enable-dependency-tracking
> --with-persistent-storage --quiet
> configure: WARNING: dot not found - build will fail if svg files are out
> of date.
> configure: WARNING: No system jemalloc found, using system malloc
> configure: error: Could not find backtrace() support
>
> Does anyone know a workaround ?
> I use multiarch/alpine:aarch64-edge as a base Docker image
>
> Martin
>
>
>
>> - store the packages as CircleCI artifacts
>>>> - anything else that is still missing
>>>>
>>>> Adding more architectures would be as easy as adding a new Dockerfile
>>>> with a base image from the respective type.
>>>>
>>>> Martin
>>>>
>>>>
>>>>> 2) Travis CI
>>>>> 2.1) problems
>>>>> 2.1.1) generally Travis is slower than Circle!
>>>>> Althought if we use CircleCI 'machine' executor it will be slower than
>>>>> the current 'Docker' executor!
>>>>> 2.1.2) Travis supports only Ubuntu
>>>>> Current setup at CircleCI uses CentOS 7.
>>>>> I guess the build steps won't have problems on Ubuntu.
>>>>>
>>>>> 3) GitHub Actions
>>>>> GH Actions does not support ARM64 but it supports self hosted ARM64
>>>>> runners
>>>>> 3.1) The problem is that there is no way to make a self hosted runner
>>>>> really private. I.e. if someone forks Varnish Cache any commit in the fork
>>>>> will trigger builds on the arm64 node. There is no way to reserve the
>>>>> runner only for commits against
>>>>> https://github.com/varnishcache/varnish-cache
>>>>>
>>>>> Do you see other problems or maybe different ways ?
>>>>> Do you have preferences which way to go ?
>>>>>
>>>>> Regards,
>>>>> Martin
>>>>>
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Guillaume Quintard
>>>>>>> _______________________________________________
>>>>>>> varnish-dev mailing list
>>>>>>> varnish-dev at varnish-cache.org
>>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>>>>>>>
>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20200324/1d2e18e4/attachment.html>