KubeVirt 虚拟机 CPU 和内存的热添加
|
Schrodingerscat
|
Kubernetes,KubeVirt,KVM
|
105
|
0

1030 字
|
20 分钟

虚拟机 Rollout 策略

虚拟机 Rollout 策略就是设置虚拟机配置修改后如何生效，目前虚拟机 Rollout 策略有两种，LiveUpdate 和 Stage。

  • LiveUpdata：会在虚拟机配置修改后尽快尝试对正在运行的虚拟机进行生效，即热添加。
  • Stage：会在虚拟机配置修改后会暂存配置，等待虚拟机重启生效。

LiveUpdate 的效果也需要其他功能的支持，比方说 虚拟机热迁移、CPU 热添加 或 内存热添加。

默认策略为 Stage，想开启 LiveUpdate 需要开启 VMLiveUpdateFeatures 特性门控。

测试两种策略

Stage

Stage 是默认策略，直接创建虚拟机测试。

[root@base-k8s-master-1 ~]# kubectl get vmi
NAME    AGE   PHASE     IP              NODENAME                        READY
my-vm   47h   Running   10.100.171.5    base-k8s-worker-2.example.com   True
[root@base-k8s-master-1 ~]# virtctl ssh fedora@my-vm
Last login: Tue Mar  4 15:12:55 2025 from 10.100.239.49
[fedora@vm-dv ~]$
[fedora@vm-dv ~]$ lscpu | grep CPU
CPU op-mode(s):                       32-bit, 64-bit
CPU(s):                               1
On-line CPU(s) list:                  0
CPU family:                           6
NUMA node0 CPU(s):                    0
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT Host state unknown
[fedora@vm-dv ~]$ free -h
               total        used        free      shared  buff/cache   available
Mem:           450Mi       134Mi       156Mi       692Ki       170Mi       315Mi
Swap:          449Mi          0B       449Mi

# 查看 VM 当前的配置
[root@base-k8s-master-1 ~]# kubectl get vm my-vm -o jsonpath="{.spec.template.spec.domain}" | jq
{
  "cpu": {
    "cores": 1
  },
  "devices": {},
  "machine": {
    "type": "q35"
  },
  "memory": {
    "guest": "512Mi"
  },
  "resources": {}
}

可以看到 CPU 为 1，内存为 450Mi。，尝试修改虚拟机配置。

[root@base-k8s-master-1 ~]# kubectl patch vm my-vm --type merge \
    -p '{"spec": {"template": {"spec": {"domain": {"cpu": {"cores": 2}}}}}}'
virtualmachine.kubevirt.io/my-vm patched
[root@base-k8s-master-1 ~]# kubectl patch vm my-vm --type merge \
    -p '{"spec": {"template": {"spec": {"domain": {"memory": {"guest": "1024Mi"}}}}}}'
virtualmachine.kubevirt.io/my-vm patched
[root@base-k8s-master-1 ~]# kubectl get vm my-vm \
    -o jsonpath="{.spec.template.spec.domain}" | jq
{
  "cpu": {
    "cores": 2
  },
  "devices": {},
  "machine": {
    "type": "q35"
  },
  "memory": {
    "guest": "1Gi"
  },
  "resources": {}
}

不重启虚拟机，进入虚拟机查看当前配置：

[root@base-k8s-master-1 ~]# virtctl ssh fedora@my-vm
Last login: Tue Mar  4 15:19:53 2025 from 10.100.239.49
[fedora@vm-dv ~]$ lscpu | grep CPU
CPU op-mode(s):                       32-bit, 64-bit
CPU(s):                               1
On-line CPU(s) list:                  0
CPU family:                           6
NUMA node0 CPU(s):                    0
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT Host state unknown
[fedora@vm-dv ~]$ free -h
               total        used        free      shared  buff/cache   available
Mem:           450Mi       135Mi       181Mi       696Ki       146Mi       315Mi
Swap:          449Mi          0B       449Mi

重启虚拟机，等虚拟机启动后查看配置：

[root@base-k8s-master-1 ~]# virtctl restart my-vm
VM my-vm was scheduled to restart
[root@base-k8s-master-1 ~]# kubectl get pod
NAME                        READY   STATUS        RESTARTS   AGE
virt-launcher-my-vm-pg2sv   2/2     Terminating   0          15m
[root@base-k8s-master-1 ~]# kubectl get pod
NAME                        READY   STATUS    RESTARTS   AGE
virt-launcher-my-vm-jttz7   2/2     Running   0          65s

# 查看配置
[root@base-k8s-master-1 ~]# virtctl ssh fedora@my-vm
Last login: Tue Mar  4 15:32:59 2025 from 10.100.239.50
[fedora@vm-dv ~]$ lscpu | grep CPU
CPU op-mode(s):                       32-bit, 64-bit
CPU(s):                               2
On-line CPU(s) list:                  0,1
CPU family:                           6
NUMA node0 CPU(s):                    0,1
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT Host state unknown
[fedora@vm-dv ~]$ free -h
               total        used        free      shared  buff/cache   available
Mem:           953Mi       275Mi       665Mi       704Ki       143Mi       677Mi
Swap:          952Mi          0B       952Mi

这里注意不能在虚拟机内启动，虚拟机内启动的话当前运行虚拟机的 POD 不会重建，不重建配置就不会生效，需要通过 virtctl restart 来重启，这样 POD 会重建，配置才能生效（可以看上边命令内容）。

LiveUpdate

通过 LiveUpdate 可以实现虚拟机 CPU、内存的热添加。

LiveUpdate 需要开启特性门控 VMLiveUpdateFeatures 并设置 Rollout 策略为 LiveUpdate。

目前能实现的是热添加，热减少不行。

为了实现热添加，还需要开启虚拟机热迁移特性门控 LiveMigrate。

CPU 热添加有以下限制：

  • ARM64 架构目前不支持 VPCU 热添加。
  • 当前的热添加实施涉及 VM 工作负载的实时迁移。
  • CPU 热添加只能修改 sockets。

内存热添加有以下限制：

  • 内存热添加目前仅在 x86_64，arm64 架构上受支持。
  • 完全支持至少运行 Linux v5.8 的 Linux 客户机。
  • Windows 客户机支持已添加到 virtio-win 中，但应将其视为不稳定。
  • 当前的热添加实施涉及 VM 工作负载的实时迁移。
  • VirtualMachine 必须至少有 1GiB 的内存才能支持内存热添加。
# 查看当前特性门控
[root@base-k8s-master-1 ~]# kubectl get kubevirts.kubevirt.io \
    -n kubevirt kubevirt -o jsonpath="{.spec}" | jq
{
  "certificateRotateStrategy": {},
  "configuration": {},
  "customizeComponents": {},
  "imagePullPolicy": "IfNotPresent",
  "workloadUpdateStrategy": {}
}

# 添加特性门控
[root@base-k8s-master-1 ~]# kubectl edit kubevirts.kubevirt.io -n kubevirt kubevirt
# 添加字段如下
spec:
  certificateRotateStrategy: {}
  configuration:
    developerConfiguration:
      featureGates:
      - VMLiveUpdateFeatures        # 新添加
      - HonorWaitForFirstConsumer
      - Snapshot
      - HotplugVolumes
    vmRolloutStrategy: LiveUpdate   # 新添加
  customizeComponents: {}
  imagePullPolicy: IfNotPresent
  workloadUpdateStrategy:
    workloadUpdateMethods:
    - LiveMigrate                   # 新添加

# 查看当前配置
[root@base-k8s-master-1 ~]# kubectl get kubevirts.kubevirt.io \
    -n kubevirt kubevirt -o jsonpath="{.spec}" | jq
kubectl get kubevirts.kubevirt.io \
> -n kubevirt kubevirt -o jsonpath="{.spec}" | jq
{
  "certificateRotateStrategy": {},
  "configuration": {
    "developerConfiguration": {
      "featureGates": [
        "VMLiveUpdateFeatures",
        "HonorWaitForFirstConsumer",
        "Snapshot",
        "HotplugVolumes"
      ]
    },
    "vmRolloutStrategy": "LiveUpdate"
  },
  "customizeComponents": {},
  "imagePullPolicy": "IfNotPresent",
  "workloadUpdateStrategy": {
    "workloadUpdateMethods": [
      "LiveMigrate"
    ]
  }
}

修改后需要重新创建虚拟机，已有的虚拟机不会生效。（应该是这样吧，我没有去做测试）

虚拟机 YAML 见 TroubleShooting。

# 创建并查看虚拟机状态
[root@base-k8s-master-1 kubevirt]# kubectl apply -f vm-dv.yml
virtualmachine.kubevirt.io/vm-dv created
[root@base-k8s-master-1 kubevirt]# virtctl start vm-dv
VM vm-dv was scheduled to start
[root@base-k8s-master-1 kubevirt]# kubectl get vmi vm-dv -o json | jq '.status.currentCPUTopology, .status.memory'
{
  "cores": 1,
  "sockets": 1,
  "threads": 2
}
{
  "guestAtBoot": "1Gi",
  "guestCurrent": "1Gi",
  "guestRequested": "1Gi"
}
[root@base-k8s-master-1 kubevirt]# virtctl console vm-dv
Successfully connected to vm-dv console. The escape sequence is ^]

vm-dv login: fedora
Password:
Last login: Thu Apr  3 14:32:57 on ttyS0
[fedora@vm-dv ~]$ uname -r
6.11.4-301.fc41.x86_64
[fedora@vm-dv ~]$ lscpu | grep CPU
CPU op-mode(s):                       32-bit, 64-bit
CPU(s):                               2
On-line CPU(s) list:                  0,1
CPU family:                           6
NUMA node0 CPU(s):                    0,1
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT Host state unknown
[fedora@vm-dv ~]$ free -h
               total        used        free      shared  buff/cache   available
Mem:           886Mi       274Mi       577Mi       740Ki       160Mi       612Mi
Swap:          885Mi          0B       885Mi

# 修改配置，增加 CPU
[root@base-k8s-master-1 ~]# kubectl patch vm vm-dv --type merge \
    -p '{"spec": {"template": {"spec": {"domain": {"cpu": {"sockets": 4}}}}}}'
virtualmachine.kubevirt.io/vm-alpine patched

# 可以看到虚拟机发生了热迁移
[root@base-k8s-master-1 ~]# kubectl get pod
NAME                             READY   STATUS      RESTARTS        AGE
virt-launcher-vm-dv-4zhps        2/2     Running     0               21s
virt-launcher-vm-dv-wshqw        0/2     Completed   0               8m7s

# 检查 CPU 配置
[root@base-k8s-master-1 ~]# kubectl get vmi vm-dv -o json | jq '.status.currentCPUTopology'
{
  "cores": 1,
  "sockets": 4,
  "threads": 2
}
[root@base-k8s-master-1 kubevirt]# virtctl console vm-dv
Successfully connected to vm-dv console. The escape sequence is ^]

[fedora@vm-dv ~]$ lscpu | grep CPU
CPU op-mode(s):                       32-bit, 64-bit
CPU(s):                               8
On-line CPU(s) list:                  0,1
Off-line CPU(s) list:                 2-7
CPU family:                           6
NUMA node0 CPU(s):                    0,1
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT Host state unknown

# 修改配置，增加内存
[root@base-k8s-master-1 ~]# kubectl patch vm vm-dv --type merge \
    -p '{"spec": {"template": {"spec": {"domain": {"memory": {"guest": "2Gi"}}}}}}'
virtualmachine.kubevirt.io/vm-dv patched

# 检查内存配置
[root@base-k8s-master-1 ~]# kubectl get vmi vm-dv -o json | jq '.status.memory'
{
  "guestAtBoot": "1Gi",
  "guestCurrent": "2Gi",
  "guestRequested": "2Gi"
}
[root@base-k8s-master-1 kubevirt]# virtctl console vm-dv
Successfully connected to vm-dv console. The escape sequence is ^]

[fedora@vm-dv ~]$ free -h
               total        used        free      shared  buff/cache   available
Mem:           1.9Gi       364Mi       1.5Gi       744Ki       162Mi       1.5Gi
Swap:          885Mi          0B       885Mi

TroubleShooting

测试 LiveUpdate 的虚拟机文件

---
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: fedora-from-das
spec:
  storage:
    storageClassName: csi-rbd-sc
    resources:
      requests:
        storage: 5Gi
  source:
    http:
      url: "https://download.fedoraproject.org/pub/fedora/linux/releases/41/Cloud/x86_64/images/Fedora-Cloud-Base-Generic-41-1.4.x86_64.qcow2"
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: vm-dv
  namespace: default
spec:
  runStrategy: Halted
  template:
    metadata:
      labels:
        kubevirt.io/domain: vm-dv
        kubevirt.io/size: small
    spec:
      architecture: amd64
      domain:
        memory:
          guest: 1Gi
        cpu:
          cores: 1
          sockets: 1
          threads: 2
        devices:
          disks:
          - disk:
            name: containerdisk
          - disk:
            name: cloudinitdisk
          interfaces:
          - masquerade: {}
            name: default
        machine:
          type: q35
        resources: {}
      networks:
      - name: default
        pod: {}
      volumes:
      - dataVolume:
          name: fedora-from-das
        name: containerdisk
      - cloudInitNoCloud:
              userData: |-
                #cloud-config
                chpasswd: { expire: False }
                hostname: vm-dv
                fqdn: vm-dv.example.com
                password: fedora
                ssh_authorized_keys:
                  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqK9cDuDDHO8NNIDhXvxcIAUDrzC+RY8ANhrXJ3fe7PGbslBNXBiPpNZNUahIUez4qrz92MXDOhyywf0OTjIYFsWqmof0ytCRzZeBRpeE0uEQIj793YN52yKIbQ797mThqmOCFAFx3ES9+/HtB6H/PWYGMvhyqmiFYNu46ttEKlu8TGQ6nh79eSc1AK+YT1UMVRm06aGp5mfhp2AuL5KTPsPHuJ9XXgPeam2rnaE1erde912qxX6k4PWtQknZ7ZSzlzgfbO2vVheNYoG9SLTZQHbvOws75pfHdI1T2SF9N3WeNRaIySBCGqxhPihavHa+Lg94kxFxLpqMdyKJzKpfT5NSpZZN1qV62JQbRWDzyOcio4ZLmUalF7IXaiD62wvYN64vS90zvUYxr4Gfzkrx9/Cp4+qPBS00KlJGcVU1vfQzdUv57HqVlYQN4Y0tqT7sYAa9/9tnSs/fYPol0NBYbuqxatmRaJltMbs46bOO6plYQJm6RCYbdanh7BqMcg4M= [email protected]
                runcmd:
                  - "sudo echo data > /root/install.log"
        name: cloudinitdisk

CPU 和内存不支持热减少

只支持热添加，减少需要 virtctl restart 重启：

[root@base-k8s-master-1 ~]# kubectl get vm vm-dv -o yaml
...output omitted...
status:
  conditions:
...output omitted...
  - lastProbeTime: null
    lastTransitionTime: "2025-03-05T13:44:45Z"
    message: Reduction of CPU socket count requires a restart
    status: "True"
    type: RestartRequired
...output omitted...

[root@base-k8s-master-1 ~]# kubectl get vm vm-dv -o yaml
...output omitted...
status:
  conditions:
...output omitted...
  - lastProbeTime: null
    lastTransitionTime: "2025-04-03T15:09:18Z"
    message: memory updated in template spec to a value lower than what the VM started
      with
    status: "True"
    type: RestartRequired
...output omitted...

出现 RestartRequired 后，就没法热添加 CPU 和内存了，除非 virtctl restart 重启虚拟机。

virtctl migrate 没有用，只能重启。

KubeVirt 虚拟机 CPU 和内存的热添加
https://www.linuxstudynotes.com/2025/04/03/k8s/kubevirt-%e8%99%9a%e6%8b%9f%e6%9c%ba-cpu-%e5%92%8c%e5%86%85%e5%ad%98%e7%9a%84%e7%83%ad%e6%b7%bb%e5%8a%a0/
KubernetesKubeVirtKVM
暂无评论

发送评论 编辑评论 


				

			

						

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
							
															
													

					

				

			

			
			

				

					
				

			

				

											

							
							
						

																

							
							
						

															
					
											
						

	

		
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
（╯‵□′）╯︵┴─┴
￣﹃￣
(/ω＼)
∠( ᐛ 」∠)＿
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ｀)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ(￣∇￣o)
ヾ(´･ ･｀｡)ノ"
( ง ᵒ̌皿ᵒ̌)ง⁼³₌₃
(ó﹏ò｡)
Σ(っ °Д °;)っ
( ,,´･ω･)ﾉ"(´っω･｀｡)
╮(╯▽╰)╭
o(*////▽////*)q
＞﹏＜
( ๑´•ω•) "(ㆆᴗㆆ)
😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣
Source: github.com/k4yt3x/flowerhd
	

	

		
颜文字
Emoji
小恐龙
花!
	


									

			

			
			
		

	






上一篇
下一篇