Kubevirt 简介
Kubernetes 是一个强大的容器编排工具，在 Kubevirt 之前，K8S 都是容器编排平台，在 Kubevirt 之后 Kubernetes 也可以做虚拟机的管理，简单讲 KVM 虚拟机本质就是一个进程，而容器就是运行进程的，所以容器跑虚拟机完全没有问题，麻烦的就是怎么对容器里跑的虚拟机进行管理，这个管理 Kubevirt 实现了，具体的原理总结好的再细说。
部署 Kubevirt
安装前注意代理问题，如果 api-server 有代理的话需要设置 pod 网络、service 网络，node 网络和 dns 搜索域均在 no_proxy 下，代理问题我也在 TroubleShooting 写了。
# 部署 Kubevirt Operator
[root@base-k8s-master-1 ~]# export VERSION=$(curl -s https://storage.googleapis.com/kubevirt-prow/release/kubevirt/kubevirt/stable.txt)
[root@base-k8s-master-1 ~]# echo $VERSION
v1.4.0
[root@base-k8s-master-1 ~]# kubectl create -f "https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml"
namespace/kubevirt created
customresourcedefinition.apiextensions.k8s.io/kubevirts.kubevirt.io created
priorityclass.scheduling.k8s.io/kubevirt-cluster-critical created
clusterrole.rbac.authorization.k8s.io/kubevirt.io:operator created
serviceaccount/kubevirt-operator created
role.rbac.authorization.k8s.io/kubevirt-operator created
rolebinding.rbac.authorization.k8s.io/kubevirt-operator-rolebinding created
clusterrole.rbac.authorization.k8s.io/kubevirt-operator created
clusterrolebinding.rbac.authorization.k8s.io/kubevirt-operator created
deployment.apps/virt-operator created
# 部署 Kubevirt CR
[root@base-k8s-master-1 ~]# kubectl create -f "https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml"
kubevirt.kubevirt.io/kubevirt created默认情况下，KubeVirt 将部署 6 个 Pod、4 个服务、1 个 daemonset、3 个部署应用、3 个副本集。
[root@base-k8s-master-1 ~]# kubectl get all -n kubevirt
Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2.
NAME READY STATUS RESTARTS AGE
pod/virt-api-768454998c-49tzb 1/1 Running 0 24m
pod/virt-api-768454998c-bwrx8 1/1 Running 0 24m
pod/virt-controller-7466b4d5f-lclvh 1/1 Running 0 24m
pod/virt-controller-7466b4d5f-tzsnx 1/1 Running 0 24m
pod/virt-handler-gcswj 1/1 Running 0 24m
pod/virt-handler-vx4c4 1/1 Running 0 24m
pod/virt-operator-5b5f954844-8ghx5 1/1 Running 0 25m
pod/virt-operator-5b5f954844-gcgsd 1/1 Running 0 25m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubevirt-operator-webhook ClusterIP 10.96.126.107 <none> 443/TCP 24m
service/kubevirt-prometheus-metrics ClusterIP None <none> 443/TCP 24m
service/virt-api ClusterIP 10.96.248.230 <none> 443/TCP 24m
service/virt-exportproxy ClusterIP 10.96.218.53 <none> 443/TCP 24m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/virt-handler 2 2 2 2 2 kubernetes.io/os=linux 24m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/virt-api 2/2 2 2 24m
deployment.apps/virt-controller 2/2 2 2 24m
deployment.apps/virt-operator 2/2 2 2 25m
NAME DESIRED CURRENT READY AGE
replicaset.apps/virt-api-768454998c 2 2 2 24m
replicaset.apps/virt-controller-7466b4d5f 2 2 2 24m
replicaset.apps/virt-operator-5b5f954844 2 2 2 25m
NAME AGE PHASE
kubevirt.kubevirt.io/kubevirt 25m Deployed安装 virtctl
virtctl 是 Kubevirt 的虚拟机管理工具，比方说虚拟机开关机。
VERSION=$(kubectl get kubevirt.kubevirt.io/kubevirt -n kubevirt -o=jsonpath="{.status.observedKubeVirtVersion}")
ARCH=$(uname -s | tr A-Z a-z)-$(uname -m | sed 's/x86_64/amd64/') || windows-amd64.exe
echo ${ARCH}
curl -L -o virtctl https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/virtctl-${VERSION}-${ARCH}
chmod +x virtctl
sudo install virtctl /usr/local/bin测试创建虚拟机
[root@base-k8s-master-1 ~]# kubectl apply -f https://kubevirt.io/labs/manifests/vm.yaml
virtualmachine.kubevirt.io/testvm created
[root@base-k8s-master-1 ~]# kubectl get vms
NAME AGE STATUS READY
testvm 13s Stopped False
[root@base-k8s-master-1 ~]# virtctl start testvm
VM testvm was scheduled to start
[root@base-k8s-master-1 ~]# kubectl get vms
NAME AGE STATUS READY
testvm 96s Running True
[root@base-k8s-master-1 ~]# kubectl get vmi
NAME AGE PHASE IP NODENAME READY
testvm 61s Running 10.100.171.22 base-k8s-worker-2.example.com True
[root@base-k8s-master-1 ~]# virtctl console testvm
Successfully connected to testvm console. The escape sequence is ^]
login as 'cirros' user. default password: 'gocubsgo'. use 'sudo' for root.
testvm login: cirros
Password:
$ sudo -i
# id
uid=0(root) gid=0(root) groups=0(root),10(wheel)
# cat /etc/os-release
NAME=Buildroot
VERSION=2015.05-g31af4e3-dirty
ID=buildroot
VERSION_ID=2015.05
PRETTY_NAME="Buildroot 2015.05"TroubleShooting
代理影响 Kubevirt 安装
K8S 集群有代理会影响安装，特别是 api-server。
问题情况如下：
[root@base-k8s-master-1 ~]# kubectl describe -n kubevirt kubevirts.kubevirt.io kubevirt
...output omitted...
Internal error occurred: failed calling webhook "virtualmachineclusterinstancetype-validator.instancetype.kubevirt.io": failed to call webhook: Post "https://virt-api.kubevirt.svc:443/virtualmachineclusterinstancetypes-validate?timeout=10s": EOF
[root@base-k8s-master-1 ~]# kubectl apply -f https://kubevirt.io/labs/manifests/vm.yaml
Failed calling webhook, failing closed kubevirt-update-validator.kubevirt.io: failed calling webhook "kubevirt-update-validator.kubevirt.io": failed to call webhook: Post "https://kubevirt-operator-webhook.kubevirt.svc:443/kubevirt-validate-update?timeout=10s": EOF可以看到都是对 .kubevirt.svc 访问有问题，而且相关报错可以在 api-server 的 pod 下看到。
[root@base-k8s-master-1 ~]# kubectl logs -n kube-system kube-apiserver-base-k8s-master-1.example.com
Failed calling webhook, failing closed kubevirt-update-validator.kubevirt.io: failed calling webhook "kubevirt-update-validator.kubevirt.io": failed to call webhook: Post "https://kubevirt-operator-webhook.kubevirt.svc:443/kubevirt-validate-update?timeout=10s": EOF原因就在于 api-server 有代理。
[root@base-k8s-master-1 ~]# kubectl get pod -n kube-system kube-apiserver-base-k8s-master-1.example.com -o jsonpath='{.spec.containers[*].env[?(@.name=="no_proxy")].value}{"\n"}'
localhost,127.0.0.1,.example.com,10.96.0.0/16,10.100.0.0/16可以看到 .svc 没有被排除在外（节点地址影不影响没确认，反正我都加上了）。
修改 api-server 配置。
[root@base-k8s-master-1 ~]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
# 找到如下字段修改 no_proxy，添加 .svc 和 192.168.50.0/24（节点地址）
env:
- name: https_proxy
value: http://192.168.50.199:10809
- name: http_proxy
value: http://192.168.50.199:10809
- name: no_proxy
value: localhost,127.0.0.1,.svc,192.168.50.0/24,.example.com,10.96.0.0/16,10.100.0.0/16修改后 api-server 应该会自动重启。
重新安装无问题。