前言

最近甲骨文首尔 ARM 免费服务器大量放货,我也开了一台 4C 24GB 的机器。这么大内存的机器,我想要在上面托管一些项目,由于一些项目的 Docker 镜像未提供 ARM 架构的支持,所以我了解了一下使用 Docker Buildx 来构建支持多架构的 Docker 镜像。

Docker Buildx 是 Docker CLI 的一个插件,它是默认 Docker Build 命令的拓展,完全支持 Moby BuildKit 构建器的特性并提供与 Docker Build 一致的体验,它带来的新特性之一就是支持构建多架构镜像,更多信息可以查看 Docker 文档 - Docker Buildx

在构建机器上构建与本机 CPU 架构不同的 Docker 镜像会用到 QEMU 的用户空间 (userspace) 仿真和虚拟化,由于虚拟化和模拟有一定性能损失,所以整个构建过程会花费更多时间。

本文不讨论 Dockerfile 的写法,我们以 book-searcher 项目为例,在甲骨文 ARM 服务器 1 上构建支持 AMD64, ARM/V7, ARM64 的 Docker 镜像并推送到 Docker Hub

$ uname -a
Linux instance-oracle-arm 5.10.0-20-arm64 #1 SMP Debian 5.10.158-2 (2022-12-13) aarch64 GNU/Linux

准备

Docker Buildx 在 Docker 19.03 以上版本默认支持,在开始前,建议将构建机器上的 Docker 升级到最新版本

# 查看 buildx 版本信息
$ sudo docker buildx version
github.com/docker/buildx v0.10.0-docker 876462897612d36679153c3414f7689626251501

# 命令帮助
$ sudo docker buildx

Usage:  docker buildx [OPTIONS] COMMAND

Extended build capabilities with BuildKit

Options:
      --builder string   Override the configured builder instance

Management Commands:
  imagetools  Commands to work on images in registry

Commands:
  bake        Build from a file
  build       Start a build
  create      Create a new builder instance
  du          Disk usage
  inspect     Inspect current builder instance
  ls          List builder instances
  prune       Remove build cache
  rm          Remove a builder instance
  stop        Stop builder instance
  use         Set the current builder instance
  version     Show buildx version information

Run 'docker buildx COMMAND --help' for more information on a command.

tonistiigi/binfmt 是一使用 Docker 镜像分发的跨平台仿真模拟器集合,在 Windows/macOS 上的 Docker Desktop 一般默认已经安装了

# 安装所有 CPU 架构模拟器
$ sudo docker run --privileged --rm tonistiigi/binfmt --install all

# 安装指定 CPU 架构模拟器
$ sudo docker run --privileged --rm tonistiigi/binfmt --install arm64,riscv64,arm

在 Linux 机器上安装

$ sudo docker run --privileged --rm tonistiigi/binfmt --install all
Unable to find image 'tonistiigi/binfmt:latest' locally
latest: Pulling from tonistiigi/binfmt
6dda554f4baf: Pull complete
2b0720d7a501: Pull complete
Digest: sha256:66e11bea77a5ea9d6f0fe79b57cd2b189b5d15b93a2bdb925be22949232e4e55
Status: Downloaded newer image for tonistiigi/binfmt:latest
{
  "supported": [
    "linux/arm64",
    "linux/amd64",
    "linux/riscv64",
    "linux/ppc64le",
    "linux/s390x",
    "linux/386",
    "linux/mips64le",
    "linux/mips64",
    "linux/arm/v7",
    "linux/arm/v6"
  ],
  "emulators": [
    "qemu-i386",
    "qemu-mips64",
    "qemu-mips64el",
    "qemu-ppc64le",
    "qemu-riscv64",
    "qemu-s390x",
    "qemu-x86_64"
  ]
}

前面也提到这个构建过程一般会花费更多时间,我们可以使用 screen 来保证会话不会中断 2

# Debian/Ubuntu 安装
$ sudo apt install -y screen

# 新建一个名为 build 的会话
$ screen -S build

# 列出开启的 screen 会话
$ screen -ls

# 恢复某个 screen 会话(比如,上面的 build)
$ screen -r build

在 Docker Hub 创建一个仓库,创建一个登录令牌用于登录

$ sudo docker login -u [username]
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

创建一个名称为 multiarch 的 buildx 构建实例并立即使用

$ sudo docker buildx create --name multiarch --use

列出所有的 buildx 构建实例状态,可以看到这个新的 buildx 构建实例还未启动

$ sudo docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS   BUILDKIT PLATFORMS
multiarch *  docker-container
  multiarch0 unix:///var/run/docker.sock inactive
default      docker
  default    default                     running  20.10.22 linux/arm64, linux/arm/v7, linux/arm/v6, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386

我们可以立即启动这个构建实例(不是必须的)

$ sudo docker buildx inspect --bootstrap
Name:   multiarch
Driver: docker-container

Nodes:
Name:      multiarch0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Buildkit:  v0.11.0
Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

再次查看所有 buildx 构建实例状态,可以看到已经在运行了

$ sudo docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS  BUILDKIT PLATFORMS
multiarch *  docker-container
  multiarch0 unix:///var/run/docker.sock running v0.11.0  linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default      docker
  default    default                     running 20.10.22 linux/arm64, linux/arm/v7, linux/arm/v6, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386

开始

现在可以开始构建多架构的 Docker 镜像了,以 book-searcher 为例

# 拉取 book-searcher 源码
$ git clone https://github.com/book-searcher-org/book-searcher.git
$ cd book-searcher

# 开始构建并推送到 Docker Hub 仓库(注意替换 dejavumoe/book-searcher:0.8.3)
$ sudo docker buildx build -t dejavumoe/book-searcher:0.8.3 --platform=linux/arm/v7,linux/arm64,linux/amd64 . --push

开始构建过程

Buildx

构建成功后,镜像会推送到 Docker Hub,比如文中示例的 book-searcher

清理

镜像构建并推送完成后,如果没有特殊需求,可以删除构建缓存

# 查看 buildx 构建缓存
$ sudo docker buildx du
ID						RECLAIMABLE	SIZE		LAST ACCESSED
no7ihggdz0dg9w916xfqdgpo4*              	true 		1.649GB   	19 hours ago
zfsooeun4nk9wuuhzkbs13bvg*              	true 		1.631GB   	19 hours ago
17luobonj5ah02zvbvvjemxhr               	true 		981.7MB   	19 hours ago
g4s8qwcxz9fygcwshw4twe31q               	true 		755.2MB   	20 hours ago
nxvw4vi5tb4hqzgc4odldltow               	true 		752.3MB   	19 hours ago
w0acj0yo862qpgkz1uc1wv7uv               	true 		735.2MB   	20 hours ago
7c3a6xlxsptavooi79v1ous0s               	true 		707.1MB   	20 hours ago
4oo7lyzdk6qkbbkl7pp52n5lr*              	true 		706MB     	19 hours ago
qut0jkq66o0c2ym8c7ur01hi6*              	true 		699.6MB   	19 hours ago
oq1s76dyzvwzducjkyymfu2yf               	true 		682.4MB   	20 hours ago
ixdur7y808t04lg654wwnm9f4               	true 		667.3MB   	19 hours ago
25qhl922ornrrslztt4jbw74d               	true 		586.3MB   	20 hours ago
uvybomtbmtsoun7cog8qdb8t0               	true 		583.4MB   	20 hours ago
1wvd1baf9mmv78q43gzreivf2               	true 		222.2MB   	20 hours ago
tr82ohutgbuzt2nmmhtwpp33f               	true 		220.6MB   	20 hours ago
edsjka5ipnayc0u0lbc354kd8               	true 		216MB     	20 hours ago
hx6x9frep5kojti0babn0edfp               	true 		215.4MB   	20 hours ago
yl7xogsqbfp3vqumlex9z70tb               	true 		212.3MB   	20 hours ago
mn480l48qvgixqkhhng3vs7kx               	true 		211.1MB   	20 hours ago
6hdhnlfc1d6vv0uei4h8x0mbl               	true 		200.9MB   	20 hours ago
voaflaun5zqg7rowaeleppipd               	true 		196.2MB   	20 hours ago
kmjaktli4z18wwn91lrxwh3ys               	true 		194.3MB   	20 hours ago
tson88ja1xdzyvkik9e2rb96r               	true 		188.7MB   	20 hours ago
t4t91979x80yuxgs0ym1bc6s6               	true 		183.6MB   	20 hours ago
uzp98pdsf5x3xmr1d1h88jfnt               	true 		180.6MB   	20 hours ago
4n9pmxco52u80kgsztr7ja17r               	true 		173.5MB   	20 hours ago
ubx1qiodeul94edl5piiwl9ii               	true 		168MB     	20 hours ago
wzej45wgvms0cjtd9krs83ac2               	true 		155.3MB   	20 hours ago
c3a2t1xt0wkvjdjdbv6itvv3w               	true 		117.9MB   	19 hours ago
iy9bmhb7sr3mkmwnp44m14xkb               	true 		107.3MB   	19 hours ago
qjqbzl6oeaccgbaxsaknbszuy               	true 		92.6MB    	19 hours ago
uelyg10c9yqlqgoowt89f29lk               	true 		31.22MB   	20 hours ago
ly1i0mkejsrjabkukyn8nqr6r               	true 		31.08MB   	20 hours ago
ijeolkpn70jlddhbms3vmozwz               	true 		28.79MB   	20 hours ago
mzzeina69yltq7jywr20wpl5j               	true 		28.61MB   	20 hours ago
up38aojz3dmcyouykvlwe3nh6               	true 		27.66MB   	20 hours ago
9uni4lgitbxyp26d5r24fcyl5               	true 		26.77MB   	20 hours ago
uowpw2a4879redczhhi9kjone               	true 		26.01MB   	20 hours ago
fz43v8gbbnqijvrq88kqyy2xo               	true 		25.36MB   	20 hours ago
y0xcjgz4ev6fhabfe6x9xr9ye               	true 		22.62MB   	20 hours ago
r4qptfigvkvv10vbm3ixl0324               	true 		21.36MB   	19 hours ago
cfdcdeahdo2chctwwascpaf21               	true 		19.91MB   	19 hours ago
v5bagyzlk48ve18j3p3u4bp5g               	true 		19.66MB   	19 hours ago
jvd0ew1rxqceq1pf6cuyajpaj               	true 		18.01MB   	20 hours ago
ta5440lqabiw13a522z3y17u6               	true 		17.83MB   	20 hours ago
n5zooon2p4vrwf4egvg5g392g               	true 		16.52MB   	20 hours ago
mqrnwaojp1f4jbmmcko3qvngj               	true 		9.983MB   	20 hours ago
ntec9jovchjk8agmarrbzkpjj               	true 		9.983MB   	20 hours ago
pj4x8t87yuzr8m7il57kk2yxt               	true 		9.761MB   	20 hours ago
yvj7uskroed3zo9i90ha8jgq1               	true 		1.139MB   	19 hours ago
ibwn6vad7njt1bjmovbi5yd2t               	true 		1.139MB   	19 hours ago
yq1ovmhkilofp8ld66d005uzv               	true 		970.8kB   	19 hours ago
uhoebp03cgtpt1r85trqke1me               	true 		970.8kB   	19 hours ago
phylad5rcv1q9gffeviucbmy3               	true 		970.8kB   	19 hours ago
hdwzqzw6l7558u7tb3x91ax9o               	true 		970.8kB   	19 hours ago
hc6667esnksz7svywlorbr29g               	true 		970.8kB   	19 hours ago
b1z4vovtou07n78o9yg1kjnw9               	true 		970.8kB   	19 hours ago
toqgn78uygmlj3vjdd8af6207*              	true 		966.7kB   	19 hours ago
1kjddgcyuhgcv1pxmtr2v2auc               	true 		413.8kB   	20 hours ago
b4rablycd81t6dzpbhn2vilbm               	true 		409.7kB   	20 hours ago
uf8p6t2e1z2ydukbg8vwr1mud               	true 		401.5kB   	20 hours ago
ot84yzon5gcbhqoh2erzaihtk               	true 		20.93kB   	19 hours ago
rb7m8s3o1w2fsfb0keurssyl9               	true 		20.93kB   	19 hours ago
pioklpgrcz3ftdrexyx2whxmy               	true 		20.93kB   	19 hours ago
yqq29ynpp5bd9vfio306krccr*              	true 		8.192kB   	19 hours ago
9kdh9ldirubp6ulj6h5anjsrn*              	true 		8.192kB   	19 hours ago
Reclaimable:	14.79GB
Total:		14.79GB

# 删除 buildx 构建缓存
$ sudo docker buildx prune

也可以删除 buildx 实例

$ sudo docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS  BUILDKIT PLATFORMS
multiarch *  docker-container
  multiarch0 unix:///var/run/docker.sock running v0.11.0  linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default      docker
  default    default                     running 20.10.22 linux/arm64, linux/arm/v7, linux/arm/v6, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386
  
# 停止 buildx 实例
$ sudo docker buildx stop multiarch

# 删除 buildx 实例
$ sudo docker buildx rm multiarch

如果构建 Dockerfile 中的基础镜像不支持某些特定的 CPU 架构、某些项目缺少特定的的链接库……等原因会导致镜像构建失败,这就不在本文讨论范畴之内了 😂。

参考信息


  1. 在 AMD64 架构计算机上构建支持 ARM 或其他架构的 Docker 镜像也一样 ↩︎

  2. 进入某个 screen 会话后,输入 exit 会直接退出并关闭这个会话 ↩︎