Correct way to move kvm vm

I have a problem to do live migration between two host computers via virt manager. It is a permission issue but I don’t have time to fig it out. It is not a big deal. It is ok to move KVM vms offline.

  1. stop VM from gui or cli or guest console
  2. dump guest configuration as xml
    virsh dumpxml VMNAME > domxml.xml
  3. copy the guest images to another server with same path
  4. define a VM from the dump xml file
    virsh define domxml.xml
  5. Check the configuration and start VM on new host. Usually need to check the network configuration, CPU, and memory.


Using docker and zfs to create limited size mysql database

Idea: Docker is run mysql server. Mysql data is stored on zfs volume and set the quota by zfs utility. Assume zpool name is zpool-mysql and mount point is /mysqldata

  1. Create volume for mysql data (test_quota)

    sudo zfs create -o quota=1.4gb -o mountpoint=/mysqldata/test_quota zpool-mysql/test_quota
  2. Create mysql instance. Port forwarding is from host 8306 to guest 3306. User can access database using host port 8306.

    docker run --name test_quota-db -e MYSQL_ROOT_PASSWORD=rootpassword -e MYSQL_DATABASE=my_test_quota -e MYSQL_USER=dbusername -e MYSQL_PASSWORD=dbuserpassword -v /mysqldata/test_quota:/var/lib/mysql -d  -p 8306:3306 mysql:latest
  3. Create systemd service for the container (test_quota-db.service)

    Description=Mysql with quota setting
    ExecStart=/usr/bin/docker start -a test_quota-db
    ExecStop=/usr/bin/docker stop -t 2 test_quota-db


    sudo systemctl enable test_quota-db.service
  4. Using phpMyAdmin to manage database

    docker run --name test_quota-phpmyadmin --link test_quota-db:db -e MYSQL_ROOT_PASSWORD=rootpassword -d -p 8080:80 phpmyadmin/phpmyadmin:latest


    Description=phpMyAdmin for test_quota-db service
    ExecStart=/usr/bin/docker start -a test_quota-phpmyadmin
    ExecStop=/usr/bin/docker stop -t 2 test_quota-phpmyadmin


    sudo systemctl enable test_quota-phpmyadmin.service

KVM access guest from outside host on CentOS

For default virbr0, it provides a way to help guest to access host (VM<–>host). But the guest cannot be accessed from outside host. But we can use the following commands to enable it temporally.

# iptables -D  FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
# iptables -D  FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable

The best way is to create another bridge for guest.

  1. create new bridge xml file (routeNetwork.xml)

      <bridge name="virbr100" />
      <forward mode="route" />
      <ip address="" netmask="" />
  2. create new bridge
    # virsh net-create routeNetwork.xml
  3. edit the bridge to enable dhcp (I think if we define DHCP at the first step, no need this one. If we don’t do this step, the persistent state is no. Not sure what the impact is.)
    # virsh net-edit routenetwork
      <forward mode='route'/>
      <bridge name='virbr100' stp='on' delay='0'/>
      <mac address='52:54:00:cc:3b:aa'/>
      <ip address='' netmask=''>
          <range start='' end=''/>
  4. Set the bridge autostart
    # virsh net-autostart routenetwork
  5. Check virtual networks
    # virsh net-list
     Name                 State      Autostart     Persistent
     default              active     yes           yes
     routenetwork         active     yes           yes
  6. add masquerade to firewalld
    # firewall-cmd --permanent --add-masquerade
  7. change guest network type
    # virsh --connect qemu:///system
    virsh # edit <VM's name>
    <interface type='bridge'>
      <mac address='52:54:00:ea:98:1a'/>
      <source bridge='virbr100'/>
      <model type='e1000'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
  8. shutdown and start the guest again
  9. add route on your router
    # sudo route -n add <host ip>

    Now the guest can access from your network via it ip 10.10.120.x.

    Other virsh commands used in managing virtual networks are:

    • virsh net-list — list virtual networks
    • virsh net-autostart [network name] — Autostart a network specified as [network name]
    • virsh net-create [XML file] — Generates and starts a new network using a preexisting XML file
    • virsh net-define [XML file] — Generates a new network from a preexisting XML file without starting it
    • virsh net-destroy [network name] — Destroy a network specified as [network name]
    • virsh net-name [network UUID] — Convert a specified [network UUID] to a network name
    • virsh net-uuid [network name — Convert a specified [network name] to a network UUID
    • virsh net-start [name of an inactive network] — Starts a previously undefined inactive network
    • virsh net-undefine [name of an inactive network] — Undefine an inactive network
    • virsh net-dumpxml [network name] — Dump network as xml file

Docker security

Detaching from the container without stopping Ctrl-P Ctrl-Q

Create docker user

$ sudo useradd dockeradmin
$ sudo passwd dockeradmin
$ sudo usermod -aG docker dockeradmin

1. Users are not namespaced. Root in container is root on host. Create a user in Dockerfile. Change to the user via USER or su/sudo/gosu

RUN groupadd -r user && useradd -r -g user user
USER user

2. Set container FS to read-only

$ docker run --read-only debian touch x
touch: cannot touch 'x': Read-only file system

3. Set Volumes to read-only/Use Data Volume Containers

$ docker run -v $(pwd)/secrets:/secrets:ro debian touch /secrets/x
touch: cannot touch '/secrets/x': Read-only file system

$ docker run --volumes-from my-secret-container myimage

4. Drop capabilities

$ docker run --cap-drop SETUID --cap-drop SETGID myimage
$ docker run --cap-drop ALL --cap-add ...


$ docker run -d myimage
$ docker run -d -c 512 myimage

6. Set Memory limits

$ docker run -m 512m myimage

7. Defang setuid/setgid binaries

// to find them
$ docker run debian \
   find / -perm +6000 -type f -exec ls -ld {} \; 2> dev/null

// to defang them
FROM debian:wheezy
RUN find / -perm +6000 -type f -exec chmod a-x {}; \; || true


8. Auditing (Immutable infrastructure, Audit images, not containers)

$ docker diff ...
$ scalock
$ twistlock
$ clair



Using KVM on CentOS7

1. Install CentOS7 with Virtualization Host feature

The Virtualization Host doesn’t install the virt-install and virt-manager. Run the following command to install them.

$ sudo yum install virt-install virt-manager

Also you can install KVM by following command if you didn’t install Virtualization Host feature.

$ sudo yum install kvm virt-manager libvirt virt-install qemu-kvm xauth dejavu-lgc-sans-fonts

2. check kvm module installation

$ lsmod|grep kvm
kvm_intel             162153  110
kvm                   525409  1 kvm_intel

3. Setup X server and run virt-manager

$ export DISPLAY
$ sudo virt-manager

5. KVM cli examples

// check cpu info
$ egrep -c '(vmx|svm)' /proc/cpuinfo
// list templates
$ osinfo-query os
// list VMs
$ sudo virsh --connect qemu:///system list
$ sudo virsh --connect qemu:///system list --all

// show guest infomration
$ sudo virsh dominfo Fedora24
Id:             -
Name:           Fedora24
UUID:           d1e8dd90-54fb-46ee-92af-dad8ec914b2e
OS Type:        hvm
State:          shut off
CPU(s):         2
Max memory:     4194304 KiB
Used memory:    0 KiB
Persistent:     yes
Autostart:      disable
Managed save:   no
Security model: selinux
Security DOI:   0

// shutdown
$ sudo virsh --connect qemu:///system shutdown Fedora24
// force stop
$ sudo virsh --connect qemu:///system destroy Fedora24
// start
$ sudo virsh --connect qemu:///system start Fedora24

// delete guest
$ sudo virsh --connect qemu:///system destroy Fedora24
$ sudo virsh --connect qemu:///system undefine Fedora24
$ sudo rm -f /var/lib/libvirt/images/Fedora24.img
$ sudo virsh pool-refresh default

6. Autostart guest

// autostart guest
$ sudo virsh --connect qemu:///system autostart Fedora24
$ sudo virsh --connect qemu:///system dominfo Fedora24|grep Auto

Set auto start from GUI


7. Issues

a) Using samba share file failed. Report permission denied. Copy install iso images to local and it works fine

b) After clone a VM from GUI, cannot start and report missing the folder such as Fedora24-template which is the source VM name. Created a tool to check the folder.  If the folder is gone, the tool creates it immediately.


Securing Linux SSH

$ sudo vi /etc/ssh/sshd_config
# Run ssh on a non-standard port:
Port 2345 # Change the port as you want 

# Disable protocol 1
# Protocol 2,1
Protocol 2

# Prevent root logins:
PermitRootLogin no

# Limit user logins:
AllowUsers alice bob

# Disable password authentication forcing use of keys
PasswordAuthentication no

How to generate your own private key

$ ssh-keygen -t rsa

Customize DNS nameservers with DHCP setting by NetworkManger on CentOS 7

By default, Network Manager always changes the resolv.conf file if the interface is using DHCP. The NetworkManger puts the records from DHCP server on top and the customize DNS servers at bottom. It causes private DNS setting cannot be found. To fix it, just change the PEERDNS=no in /etc/sysconfig/network-scripts/ifcfg-eth0 and restart network

firewall-cmd examples

firewall-cmd --get-default-zone
firewall-cmd --get-zones

firewall-cmd --list-interfaces
firewall-cmd --add-interface=<interface>

firewall-cmd --add-service=http

firewall-cmd --add-port=443/tcp
firewall-cmd --permanent --add-port=443/tcp

firewall-cmd --add-masquerade
firewall-cmd --add-service=dns --add-service=dhcp
firewall-cmd --runtime-to-permanent

firewall-cmd --permanent --direct --get-all-rules