packer创建自定义镜像
作者建议
我将文档发布到了多个平台,不同的平台阅读体验不同,排版也可能有区别,本人强烈建议直接点击下面的链接查看原始文档,因为下面的链接是原始文档,阅读体验极佳,排版美观,有目录结构,您可以很容易找到您想要阅读的章节。另外,文档一直持续更新,不断完善,内容更加准确且与时俱进。原始文档始终是最新版本的,其它平台中的文档可能已经过时了。
单击查看原始文档(持续更新中):《packer创建自定义镜像》
如果您是在微信上查看此文章的,也可以直接点击文末的“阅读原文”查看原始文档
原创不易,如果对您有帮助,还请您一键三连[抱拳]
有任何问题都可以联系作者,文末有作者联系方式,欢迎交流。
简介:
packer是一个非常好用的开源工具,可以自定义各种操作系统,用于各种平台,如kvm/qemu,各种公有云和私有云、VMware、hyper-v、MAAS(裸金属即服务)等等。
canonical官方有个开源项目,可以让我们更好地创建自定义镜像,虽然这个项目注意是针对maas的,但是稍加调整也能给kvm/qemu、OpenStack使用。
安装必须的软件
#sudo apt install packer
#如果提示没有可用的包,则可以去官方找安装教程
#https://developer.hashicorp.com/packer/tutorials/docker-get-started/get-started-install-cli
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install packer -y
sudo apt install qemu-utils -y
sudo apt install qemu-system -y
sudo apt install ovmf -y
sudo apt install cloud-image-utils -y
sudo apt install make libnbd-bin nbdkit fuse2fs -y
cd ~
git clone https://github.com/canonical/packer-maas.git
cd packer-maas/ubuntu
mkdir iso
克隆代码后,看到有这样一些目录:
从上图可知,可以自定义的系统种类很多。
自定义ubuntu server 24.04(为kvm/qemu)
参考资料:
QEMU Builder | Integrations | Packer | HashiCorp Developer
Cloud config examples - cloud-init 24.3.1 documentation (cloudinit.readthedocs.io)
分区采用lvm
原始的HCL模板文件,是通过http从官网下载ubuntu iso镜像文件的,我发现对于网速不快的情况下,容易出问题,因为qemu插件规定了下载iso文件不能超过半小时,否则就会终止。为了方便我改为了直接用本地的iso文件,下载网址为http://releases.ubuntu.com/noble/ubuntu-24.04.1-live-server-amd64.iso
ubuntu-lvm.pkr.hcl文件内容:
注意iso_checksum必须要与镜像文件匹配。
source "qemu" "lvm" {
boot_command = ["<wait>e<wait5>", "<down><wait><down><wait><down><wait2><end><wait5>", "<bs><bs><bs><bs><wait>autoinstall ---<wait><f10>"]
boot_wait = "2s"
cpus = 8
disk_size = "200G"
# net_device = "virtio-net"
# disk_interface = "virtio"
format = "raw"
headless = var.headless
http_directory = var.http_directory
iso_checksum = "file:http://releases.ubuntu.com/${var.ubuntu_series}/SHA256SUMS"
# iso_target_path = "packer_cache/${var.ubuntu_lvm_iso}"
iso_url = "file:///home/mywind/packer-maas/ubuntu/iso/${var.ubuntu_lvm_iso}"
memory = 8192
qemuargs = [
["-vga", "qxl"],
["-device", "virtio-blk-pci,drive=drive0,bootindex=0"],
["-device", "virtio-blk-pci,drive=cdrom0,bootindex=1"],
["-device", "virtio-blk-pci,drive=drive1,bootindex=2"],
[
"-netdev",
"user,id=mynet0,",
# "hostfwd=hostip:hostport-guestip:guestport",
""
],
["-device", "virtio-net,netdev=mynet0" ],
["-drive", "if=pflash,format=raw,id=ovmf_code,readonly=on,file=/usr/share/${lookup(local.uefi_imp, var.architecture, "")}/${lookup(local.uefi_imp, var.architecture, "")}_CODE${var.ovmf_suffix}.fd"],
["-drive", "if=pflash,format=raw,id=ovmf_vars,file=${lookup(local.uefi_imp, var.architecture, "")}_VARS.fd"],
["-drive", "file=output-lvm/packer-lvm,if=none,id=drive0,cache=writeback,discard=ignore,format=raw"],
["-drive", "file=seeds-lvm.iso,format=raw,cache=none,if=none,id=drive1,readonly=on"],
["-drive", "file=iso/${var.ubuntu_lvm_iso},if=none,id=cdrom0,media=cdrom"]
]
shutdown_command = "sudo -S shutdown -P now"
ssh_handshake_attempts = 500
ssh_password = var.ssh_ubuntu_password
ssh_timeout = var.timeout
ssh_username = "ubuntu"
ssh_wait_timeout = var.timeout
}
build {
sources = ["source.qemu.lvm"]
provisioner "file" {
destination = "/tmp/curtin-hooks"
source = "${path.root}/scripts/curtin-hooks"
}
provisioner "file" {
source = "${path.root}/scripts/.bashrc"
destination = "/home/ubuntu/.bashrc"
}
provisioner "shell" {
environment_vars = ["HOME_DIR=/home/ubuntu", "http_proxy=${var.http_proxy}", "https_proxy=${var.https_proxy}", "no_proxy=${var.no_proxy}"]
execute_command = "echo 'ubuntu' | {{ .Vars }} sudo -S -E sh -eux '{{ .Path }}'"
expect_disconnect = true
scripts = ["${path.root}/scripts/curtin.sh", "${path.root}/scripts/networking.sh", "${path.root}/scripts/cleanup.sh"]
}
post-processor "compress" {
output = "custom-ubuntu2404-server-lvm.dd.gz"
}
}
ubuntu-lvm.variables.pkr.hcl
variable "ubuntu_lvm_iso" {
type = string
default = "ubuntu-24.04.1-live-server-amd64.iso"
description = "The ISO name to build the image from"
}
variables.pkr.hcl
注意:headless默认值为true,这个选项的作用是packer自定义镜像时,是否要弹出vnc窗口,为true表示不弹出,为false表示自动弹出,新手建议选择false,可以更直观了解packer自定义镜像的过程
packer {
required_version = ">= 1.7.0"
required_plugins {
qemu = {
version = "~> 1.0"
source = "github.com/hashicorp/qemu"
}
}
}
variable "headless" {
type = bool
default = false
description = "Whether VNC viewer should not be launched."
}
variable "http_directory" {
type = string
default = "http"
}
variable "http_proxy" {
type = string
default = "${env("http_proxy")}"
}
variable "https_proxy" {
type = string
default = "${env("https_proxy")}"
}
variable "no_proxy" {
type = string
default = "${env("no_proxy")}"
}
variable "ssh_password" {
type = string
default = "ubuntu"
}
variable "ssh_username" {
type = string
default = "root"
}
variable "ssh_ubuntu_password" {
type = string
default = "ubuntu"
}
variable "timeout" {
type = string
default = "1h"
description = "Timeout for building the image"
}
user-data-lvm
注意:packers必须要位于package_update和package_upgrade的下面,否则安装系统时会报错。
#cloud-config
autoinstall:
version: 1
identity:
hostname: custom-ubuntu2404-desktop
username: ubuntu
password: "$6$canonical.$0zWaW71A9ke9ASsaOcFTdQ2tx1gSmLxMPrsH0rF0Yb.2AEKNPV1lrF94n6YuPJmnUy2K2/JSDtxuiBDey6Lpa/"
keyboard:
layout: us
variant: ''
# network:
# ethernets:
# enp1s0:
# dhcp4: true
# version: 2
ssh:
install-server: true
# manage_resolv_conf: true
# resolv_conf:
# domain: dltornado2.com
# nameservers: [192.168.124.7, 172.25.0.250]
# searchdomains: [dltornado2.com, dltornado.com]
drivers:
nvidia: {license-accepted: true}
storage:
grub:
update_nvram: true
swap:
size: 0
layout:
name: lvm
#下面这行命令的作用是,分区时,将所有空余的空间都分配给根逻辑卷(root LV),这样就不会导致空间浪费
sizing-policy: all
timezone: Asia/Shanghai
apt:
primary:
- arches: [default]
uri: https://mirrors.aliyun.com/ubuntu/
late-commands:
- echo 'ubuntu ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/ubuntu
package_update: true
package_upgrade: true
packages:
- pwgen
- nmap
- arp-scan
- smbclient
- tree
Makefile
#!/usr/bin/make -f
include ../scripts/check.mk
PACKER ?= packer
PACKER_LOG ?= 1
export PACKER_LOG
SERIES ?= noble
ARCH ?= amd64
URL ?= http://releases.ubuntu.com
SUMS ?= SHA256SUMS
TIMEOUT ?= 1h
ifeq ($(wildcard /usr/share/OVMF/OVMF_CODE.fd),)
OVMF_SFX ?= _4M
else
OVMF_SFX ?=
endif
ISO=$(shell wget -O- -q ${URL}/${SERIES}/${SUMS} | grep desktop | cut -d'*' -f2)
.PHONY: all clean
all: custom-cloudimg.tar.gz
$(eval $(call check_packages_deps,cloud-image-utils ovmf parted,cloud-image-utils ovmf parted))
lint:
packer validate .
packer fmt -check -diff .
format:
packer fmt .
seeds-lvm.iso: user-data-lvm meta-data
cloud-localds $@ $^
seeds-flat.iso: user-data-flat meta-data
cloud-localds $@ $^
OVMF_VARS.fd: /usr/share/OVMF/OVMF_VARS*.fd
cp -v $< $@
custom-cloudimg.tar.gz: check-deps clean
${PACKER} init . && ${PACKER} build \
-only='cloudimg.*' \
-var ubuntu_series=${SERIES} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
custom-ubuntu2404-server-flat.tar.gz: check-deps clean seeds-flat.iso OVMF_VARS.fd \
packages/custom-packages.tar.gz
${PACKER} init . && ${PACKER} build -only=qemu.flat \
-var ubuntu_series=${SERIES} \
-var ubuntu_iso=${ISO} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
custom-ubuntu2404-server-lvm.dd.gz: check-deps clean seeds-lvm.iso OVMF_VARS.fd
${PACKER} init . && ${PACKER} build -only=qemu.lvm \
-var ubuntu_series=${SERIES} \
-var ubuntu_lvm_iso=${ISO} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
custom-ubuntu2404-desktop-lvm.dd.gz: check-deps clean seeds-lvm.iso OVMF_VARS.fd
${PACKER} init . && ${PACKER} build -only=qemu.lvm \
-var ubuntu_series=${SERIES} \
-var ubuntu_lvm_iso=${ISO} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
clean:
${RM} -rf output-* custom-*.gz \
seeds-flat.iso seeds-lvm.iso seeds-cloudimg.iso \
OVMF_VARS.fd
CUSTOM_PKGS:=${wildcard packages/*.deb}
packages/custom-packages.tar.gz: ${CUSTOM_PKGS}
ifeq ($(strip $(CUSTOM_PKGS)),)
tar czf $@ -C packages -T /dev/null
else
tar czf $@ -C packages ${notdir $^}
endif
.INTERMEDIATE: OVMF_VARS.fd packages/custom-packages.tar.gz \
seeds-flat.iso seeds-lvm.iso seeds-cloudimg.iso
最后开始自定义
make custom-ubuntu2404-server-lvm.dd.gz
时间会比较长,最后会自动在当前目录下创建一个custom-ubuntu2404-server-lvm.dd.gz文件
解压
gunzip custom-ubuntu2404-server-lvm.dd.gz
解压后得到的文件为custom-ubuntu2404-server-lvm.dd,此文件大小与qemu插件中设置的磁盘大小一样,本文中设置的是200GB,所以此文件也是200GB,kvm无法直接用这个格式的虚拟磁盘,还需要将其转换为qcow2格式
qemu-img convert -f raw -O qcow2 custom-ubuntu2404-server-lvm.dd custom-ubuntu2404-server-lvm.qcow2
转换后的qcow2文件,大小大大压缩了,只有4GB多
virt-manager创建虚拟机
先要将得到的custom-ubuntu2404-server-lvm.qcow2文件拷贝到/var/lib/libvirt/images/目录中,合适命名
sudo cp ~/packer-maas/ubuntu/custom-ubuntu2404-server-lvm.qcow2 /var/lib/libvirt/images//custom-ubuntu2404-server-lvm-test.qcow2
创建和启动虚拟机的过程非常快
可以看到网络并没有激活,这是因为用packer创建自定义镜像时,ubuntu系统将网卡识别为ens6而不是enp1s0,解决办法也很简单,编辑网络yaml文件将ens6改为enp1s即可
自定义ubuntu desktop 24.04(为kvm/qemu)
分区采用lvm
cd ~/packer-maas/ubuntu-desktop
mkdir iso
从canonical官网release网站下载相应版本的iso镜像文件,然后放入iso文件夹中http://releases.ubuntu.com/noble/ubuntu-24.04.1-desktop-amd64.iso
ubuntu-lvm.pkr.hcl
source "qemu" "lvm" {
boot_command = ["<wait>e<wait5>", "<down><wait><down><wait><down><wait2><end><wait5>", "<bs><bs><bs><bs><wait>autoinstall ---<wait><f10>"]
boot_wait = "2s"
cpus = 8
disk_size = "200G"
format = "raw"
headless = var.headless
http_directory = var.http_directory
iso_checksum = "c2e6f4dc37ac944e2ed507f87c6188dd4d3179bf4a3f9e110d3c88d1f3294bdc"
# iso_target_path = "packer_cache/${var.ubuntu_lvm_iso}"
iso_url = "file:///home/mywind/packer-maas/ubuntu-desktop/iso/${var.ubuntu_lvm_iso}"
memory = 8192
qemuargs = [
["-vga", "qxl"],
["-device", "virtio-blk-pci,drive=drive0,bootindex=0"],
["-device", "virtio-blk-pci,drive=cdrom0,bootindex=1"],
["-device", "virtio-blk-pci,drive=drive1,bootindex=2"],
["-drive", "if=pflash,format=raw,id=ovmf_code,readonly=on,file=/usr/share/${lookup(local.uefi_imp, var.architecture, "")}/${lookup(local.uefi_imp, var.architecture, "")}_CODE${var.ovmf_suffix}.fd"],
["-drive", "if=pflash,format=raw,id=ovmf_vars,file=${lookup(local.uefi_imp, var.architecture, "")}_VARS.fd"],
["-drive", "file=output-lvm/packer-lvm,if=none,id=drive0,cache=writeback,discard=ignore,format=raw"],
["-drive", "file=seeds-lvm.iso,format=raw,cache=none,if=none,id=drive1,readonly=on"],
["-drive", "file=iso/${var.ubuntu_lvm_iso},if=none,id=cdrom0,media=cdrom"]
]
shutdown_command = "sudo -S shutdown -P now"
ssh_handshake_attempts = 500
ssh_password = var.ssh_ubuntu_password
ssh_timeout = var.timeout
ssh_username = "ubuntu"
ssh_wait_timeout = var.timeout
}
build {
sources = ["source.qemu.lvm"]
provisioner "file" {
destination = "/tmp/curtin-hooks"
source = "${path.root}/scripts/curtin-hooks"
}
provisioner "file" {
source = "${path.root}/scripts/.bashrc"
destination = "/home/ubuntu/.bashrc"
}
provisioner "shell" {
environment_vars = ["HOME_DIR=/home/ubuntu", "http_proxy=${var.http_proxy}", "https_proxy=${var.https_proxy}", "no_proxy=${var.no_proxy}"]
execute_command = "echo 'ubuntu' | {{ .Vars }} sudo -S -E sh -eux '{{ .Path }}'"
expect_disconnect = true
scripts = ["${path.root}/scripts/curtin.sh", "${path.root}/scripts/networking.sh", "${path.root}/scripts/cleanup.sh"]
}
post-processor "compress" {
output = "custom-ubuntu2404-desktop-lvm.dd.gz"
}
}
ubuntu-lvm.variables.pkr.hcl
variable "ubuntu_lvm_iso" {
type = string
default = "ubuntu-24.04.1-desktop-amd64.iso"
description = "The ISO name to build the image from"
}
variables.pkr.hcl
注意:如果是创建自定义的ubuntu desktop镜像,则必须要将headless的值设置为false,这是因为安装系统过程中,需要手工单击一个按钮
packer {
required_version = ">= 1.7.0"
required_plugins {
qemu = {
version = "~> 1.0"
source = "github.com/hashicorp/qemu"
}
}
}
variable "headless" {
type = bool
default = false
description = "Whether VNC viewer should not be launched."
}
variable "http_directory" {
type = string
default = "http"
}
variable "http_proxy" {
type = string
default = "${env("http_proxy")}"
}
variable "https_proxy" {
type = string
default = "${env("https_proxy")}"
}
variable "no_proxy" {
type = string
default = "${env("no_proxy")}"
}
variable "ssh_password" {
type = string
default = "ubuntu"
}
variable "ssh_username" {
type = string
default = "root"
}
variable "ssh_ubuntu_password" {
type = string
default = "ubuntu"
}
variable "timeout" {
type = string
default = "1h"
description = "Timeout for building the image"
}
user-data-lvm
#cloud-config
autoinstall:
version: 1
identity:
hostname: custom-ubuntu2404-desktop
username: ubuntu
password: "$6$canonical.$0zWaW71A9ke9ASsaOcFTdQ2tx1gSmLxMPrsH0rF0Yb.2AEKNPV1lrF94n6YuPJmnUy2K2/JSDtxuiBDey6Lpa/"
keyboard:
layout: us
variant: ''
# network:
# ethernets:
# enp1s0:
# dhcp4: true
# version: 2
ssh:
install-server: true
manage_resolv_conf: true
resolv_conf:
domain: dltornado2.com
nameservers: [192.168.124.7, 172.25.0.250]
searchdomains: [dltornado2.com, dltornado.com]
drivers:
nvidia: {license-accepted: true}
storage:
grub:
update_nvram: true
swap:
size: 0
layout:
name: lvm
sizing-policy: all
timezone: Asia/Shanghai
apt:
primary:
- arches: [default]
uri: https://mirrors.aliyun.com/ubuntu/
late-commands:
- echo 'ubuntu ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/ubuntu
package_update: true
package_upgrade: true
packages:
- pwgen
- nmap
- arp-scan
- smbclient
- tree
Makefile
#!/usr/bin/make -f
include ../scripts/check.mk
PACKER ?= packer
PACKER_LOG ?= 1
export PACKER_LOG
SERIES ?= noble
ARCH ?= amd64
URL ?= http://releases.ubuntu.com
SUMS ?= SHA256SUMS
TIMEOUT ?= 1h
ifeq ($(wildcard /usr/share/OVMF/OVMF_CODE.fd),)
OVMF_SFX ?= _4M
else
OVMF_SFX ?=
endif
ISO=$(shell wget -O- -q ${URL}/${SERIES}/${SUMS} | grep desktop | cut -d'*' -f2)
.PHONY: all clean
all: custom-cloudimg.tar.gz
$(eval $(call check_packages_deps,cloud-image-utils ovmf parted,cloud-image-utils ovmf parted))
lint:
packer validate .
packer fmt -check -diff .
format:
packer fmt .
seeds-lvm.iso: user-data-lvm meta-data
cloud-localds $@ $^
seeds-flat.iso: user-data-flat meta-data
cloud-localds $@ $^
OVMF_VARS.fd: /usr/share/OVMF/OVMF_VARS*.fd
cp -v $< $@
custom-cloudimg.tar.gz: check-deps clean
${PACKER} init . && ${PACKER} build \
-only='cloudimg.*' \
-var ubuntu_series=${SERIES} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
custom-ubuntu2404-server-flat.tar.gz: check-deps clean seeds-flat.iso OVMF_VARS.fd \
packages/custom-packages.tar.gz
${PACKER} init . && ${PACKER} build -only=qemu.flat \
-var ubuntu_series=${SERIES} \
-var ubuntu_iso=${ISO} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
custom-ubuntu2404-server-lvm.dd.gz: check-deps clean seeds-lvm.iso OVMF_VARS.fd
${PACKER} init . && ${PACKER} build -only=qemu.lvm \
-var ubuntu_series=${SERIES} \
-var ubuntu_lvm_iso=${ISO} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
custom-ubuntu2404-desktop-lvm.dd.gz: check-deps clean seeds-lvm.iso OVMF_VARS.fd
${PACKER} init . && ${PACKER} build -only=qemu.lvm \
-var ubuntu_series=${SERIES} \
-var ubuntu_lvm_iso=${ISO} \
-var architecture=${ARCH} \
-var ovmf_suffix=${OVMF_SFX} \
-var timeout=${TIMEOUT} .
clean:
${RM} -rf output-* custom-*.gz \
seeds-flat.iso seeds-lvm.iso seeds-cloudimg.iso \
OVMF_VARS.fd
CUSTOM_PKGS:=${wildcard packages/*.deb}
packages/custom-packages.tar.gz: ${CUSTOM_PKGS}
ifeq ($(strip $(CUSTOM_PKGS)),)
tar czf $@ -C packages -T /dev/null
else
tar czf $@ -C packages ${notdir $^}
endif
.INTERMEDIATE: OVMF_VARS.fd packages/custom-packages.tar.gz \
seeds-flat.iso seeds-lvm.iso seeds-cloudimg.iso
开始自定义ubuntu镜像
custom-ubuntu2404-desktop-lvm.dd.gz
到了如下图时,一定要手工单击一下“install”
解压
gunzip custom-ubuntu2404-desktop-lvm.dd.gz
解压后得到的文件为custom-ubuntu2404-desktop-lvm.dd,此文件大小与qemu插件中设置的磁盘大小一样,本文中设置的是200GB,所以此文件也是200GB,kvm无法直接用这个格式的虚拟磁盘,还需要将其转换为qcow2格式
qemu-img convert -f raw -O qcow2 custom-ubuntu2404-desktop-lvm.dd custom-ubuntu2404-desktop-lvm.qcow2
转换后的qcow2文件,大小大大压缩了,只有6GB多
自定义ubuntu server 24.04(为maas)
这个很简单,关键是不用解压,也不用将文件转换为qcow2格式的,直接上传到maas服务器中,maas服务器自会处理,官方文档:MAAS | How to build an Ubuntu image
疑难解答
如果开启虚拟机后,并没有开始安装操作系统的过程,而是出现了一个shell窗口,这说明虚拟光驱没有正确加载iso镜像,检查qemu插件的基本配置