I'm still working on perl-cross-debian (just uploaded 0.0.2) and there's more to do on that with upstream but part of the reason to work on perl cross-building is to do what I can to help with the ARM64 port.

So, I went back to Wookey's talk at DebConf12 which leads to the current list of cross-build results for arm64 and started through the list.

coreutils is listed as failing but that was an archive error (MD5sum hash mismatch), so that just needs a retry. I don't have access to that buildd, yet, so nothing I can do there.

Next on the list (excluding those just waiting for build-deps) was klibc.

  aarch64-linux-gnu-gcc -Wp,-MD,usr/klibc/.__static_init.o.d  -nostdinc -iwithprefix 

include -I/ PKGBUILDDIR /usr/include/arch/x86_64 -Iusr/include/arch/x86_64 

-I/ PKGBUILDDIR /usr/include/bits64 

-Iusr/include/bits64 -I/ PKGBUILDDIR /usr/klibc/../include 

-Iusr/klibc/../include -I/ PKGBUILDDIR /usr/include -Iusr/include 

-I/ PKGBUILDDIR /linux/include -Ilinux/include 

-I/ PKGBUILDDIR /linux/arch/x86/include -Ilinux/arch/x86/include 

-D__KLIBC__=2 -D__KLIBC_MINOR__=0 -D_BITSIZE=64 -fno-stack-protector -fwrapv -m64 -Os 

-fno-asynchronous-unwind-tables -fomit-frame-pointer -falign-functions=1 -falign-jumps=1 -falign-loops=1 

-W -Wall -Wno-sign-compare -Wno-unused-parameter -c -o usr/klibc/__static_init.o usr/klibc/__static_init.c

aarch64-linux-gnu-gcc: error: unrecognized command line option '-m64'

make[4]: *** [usr/klibc/__static_init.o] Error 1

Turns out that this is a build failure I understood, at least initially. A little digging and a trivial patch was begun:

--- a/usr/klibc/arch/arm/MCONFIG

+++ b/usr/klibc/arch/arm/MCONFIG

@@ -30,6 +30,13 @@

 ifeq ($(CONFIG_AEABI),y)

 KLIBCREQFLAGS += -mabi=aapcs-linux -mno-thumb-interwork

 else

+# aarch64

+ifeq ($(CONFIG_AARCH64),y)

+KLIBCREQFLAGS +=

+KLIBCOPTFLAGS += -mgeneral-regs-only

+else

 KLIBCREQFLAGS += -mabi=apcs-gnu -mno-thumb-interwork

 endif

 endif

+endif

+

Alongside a trivial change to debian/rules

ifeq ($(DEB_HOST_ARCH),arm64)

DEB_MAKE_ENVVARS := ARCH=arm CONFIG_AARCH64=y CPU_ARCH=armv8-a CPU_TUNE=generic

endif

OK, then things get a bit more awkward,

....

code/debian/cross/klibc/klibc-2.0.1/linux/arch/arm/include -Ilinux/arch/arm/include -D__KLIBC__=2 

-D__KLIBC_MINOR__=0 -D_BITSIZE=32 -fno-stack-protector -fwrapv -fno-exceptions -Os 

-march=armv8-a -mtune=generic -mgeneral-regs-only -W -Wall -Wno-sign-compare -Wno-unused-parameter 

-c -o ../foo.o usr/klibc/arch/arm/crt0.S

usr/klibc/arch/arm/crt0.S: Assembler messages:

usr/klibc/arch/arm/crt0.S:19: Error: operand 1 should be an integer register --  mov r0,sp'

usr/klibc/arch/arm/crt0.S:20: Error: operand 1 should be an integer register --  mov r1,#0'

Hmm. Assembly, well, yes, I've done assembly before, I know what mov should normally do, sp is likely to be the stack pointer .... where's my AARCH64 assembly PDF again... PRD03-GENC-010197 ...

OK, so maybe the r0 and r1 should be x0 and x1, hmm, that at least doesn't raise assembly errors. So a tentative change:

--- a/usr/klibc/arch/arm/crt0.S

+++ b/usr/klibc/arch/arm/crt0.S

@@ -15,9 +15,13 @@

 #ifdef __thumb__

 	.thumb_func

 #endif

-

+#ifdef __aarch64__

+_start: mov x0, sp

+	mov x1, #0

+	bl __libc_init

+#else

 _start:	mov	r0, sp

 	mov	r1, #0

 	bl	__libc_init

-

+#endif

 	.size _start,.-_start

Next stage, however, leaves me quite a bit more lost:

....

klibc-2.0.1/linux/arch/arm/include -Ilinux/arch/arm/include 

-D__KLIBC__=2 -D__KLIBC_MINOR__=0 -D_BITSIZE=32 -fno-stack-protector -fwrapv -fno-exceptions -Os 

-march=armv8-a -mtune=generic -mgeneral-regs-only -W -Wall -Wno-sign-compare -Wno-unused-parameter 

-c -o usr/klibc/arch/arm/setjmp.o usr/klibc/arch/arm/setjmp.S

usr/klibc/arch/arm/setjmp.S: Assembler messages:

usr/klibc/arch/arm/setjmp.S:32: Error: unknown mnemonic  stmia' --  stmia r0, r4,r5,r6,r7,r8,r9,r10,fp,sp,lr '

usr/klibc/arch/arm/setjmp.S:33: Error: operand 1 should be an integer register --  mov r0,#0'

usr/klibc/arch/arm/setjmp.S:34: Error: operand 1 should be an integer register --  mov pc,lr'

usr/klibc/arch/arm/setjmp.S:42: Error: unknown mnemonic  ldmia' --  ldmia r0, r4,r5,r6,r7,r8,r9,r10,fp,sp,lr '

usr/klibc/arch/arm/setjmp.S:43: Error: unknown mnemonic  movs' --  movs r0,r1'

usr/klibc/arch/arm/setjmp.S:44: Error: unknown mnemonic  moveq' --  moveq r0,#1'

usr/klibc/arch/arm/setjmp.S:45: Error: operand 1 should be an integer register --  mov pc,lr'

make[5]: *** [usr/klibc/arch/arm/setjmp.o] Error 1

So now I'm out of my depth of AARCH64 assembly (apart from the recurrence of mov r0 vs mov x0 etc.). If the above is useful then maybe someone can work out what is wrong with setjmp.S or whether AARCH64 just means that klibc needs to gain a arch/arm64/ directory and not try to duplicate each entire assembly block within #ifdef clauses.

I don't really know where else to put an incomplete investigation like this, so it's here for anyone to find.

(Oh, and if you're reading those arm64 cross-build logs, then a few hundred occurrences of ldconfig: /usr/lib/aarch64-linux-gnu/*.so is for unknown machine 183. in every build log (success or fail) is apparently entirely normal until more packages get fixed. Really slows down scanning the build log.

I may try busybox or libusb next. libusb looks like a classic "you might have told me to cross-compile but I'm going to use g++ anyway because I know best" cross-building problem, indicative of yet another BDBS. sigh.

Resources:
Getting started with 64-bit ARM development
ARMv8 images for developers
AArch64 for everyone, Marcin Juszkiewicz
Howto/HelloAarch64 - Linaro wiki
AArch64 gcc options

When packaging software for Debian, there exist two important assumptions:

1. Compilation is done natively
2. Potentially all of Debian is available at compile time

Both assumptions make the life of a package maintainer much easier and they do not create any problem unless you are one of the unlucky few who want to run Debian on an architecture that it does not yet exist for. You will then have to use other distributions like OpenEmbedded or Gentoo which you compiled (or retrieved otherwise) for that new architecture to hack a core of Debian source packages until they build a minimal Debian system that you can chroot into and continue natively building the rest of it. But even if you manage to get that far you will continue to be plagued by cyclic build and runtime dependencies. So you start to hack source packages so that they drop some dependencies and you can break enough cycles to advance step by step. The Debian ports page lists 24 ports of Debian, so despite its unpleasant nature, porting it is something that is not done seldom. The process as laid out above has a number of drawbacks:

• The process is mostly manual and reinvented every time it is done.
• You need another distribution for the bootstrapping process. Debian itself should be sufficient.
• Its complexity and manual nature prevents architectures with little workforce behind them from catching up to the main archive.
• It also avoids that Debian exists in CPU optimized sub-arch builds.

If Debian would provide a set of core packages that are cross-compilable and which suffice for a minimal foreign build system, and if it would also have enough source packages that provide a reduced build dependency set so that all dependency cycles can be broken, building Debian for a yet unknown architecture could be mostly automated. The benefits would be:

• Putting Debian on a foreign architecture would (in the best case) boil down to making the code cross-compile for and native-compile on that architecture.
• Debian would not need any other distribution to be ported to a different architecture. This would make Debian even more "universal".
• Lagging architectures can be more easily updated or rebooted than when they were initially created.
• Debian optimized for specific CPUs (Raspberry Pie, OpenMoko...) would be more attractive.

With three of this year's GSoC projects, this dream seems to come into reach. There is the "Multiarch Cross-Toolchains" project by Thibaut Girka and mentored by Hector Oron and Marcin Juszkiewicz. Cross-compiling toolchains need packages from the foreign architecture to be installed alongside the native libraries. Cross-compiler packages have been available through the emdebian repositories but always were more of a hack. With multiarch, it is now possible to install packages from multiple architectures at once, so that cross-compilation toolchains can be realized in a proper manner and therefor can also enter the main archives. Besides creating multiarch enabled toolchains, he will also be responsible for making them build on the Debian builld system as cross-architecture dependencies are not yet supported. There is also the "Bootstrappable Debian" project by Patrick "P. J." McDermott and mentored by Wookey and Jonathan Austin. He will make a small set of source packages multiarch cross-compilable (using cross-compilers provided by Thibaut Girka) and add a Build-Depends-StageN header to critical packages so that they can be built with reduced build dependencies for breaking dependency cycles. He will also patch tools as necessary to recognize the new control header. And then there is my project: "Port bootstrap build-ordering tool" (Application). It is mentored by Wookey and Pietro Abate. In contrast to the other two, my output will be more on the meta-level as I will not modify any actual Debian package or patch Debian tools with more functionality. Instead the goal of this project is threefold:

1. find the minimal set of source packages that have to be cross compiled
2. help the user to find packages that are good candidates for breaking build dependency cycles through added staged build dependencies or by making them cross-compilable
3. develop a tool that takes the information about packages that can be cross compiled or have staged build dependencies to output an ordering with which packages must be built to go from nothing to a full archive

More on that project in my follow-up post.