Differences

This shows you the differences between two versions of the page.

--- unix:virtualization:kvm:server [2020/10/16 23:19] – created rodolico
+++ unix:virtualization:kvm:server [2024/03/30 00:04] (current) – rodolico
@@ Line 1: / Line 1: @@
-====== KVM on server with libvirt ======
+====== libvirt installation (virsh) ======
 **This is a work in progress, 20201015**
+===== Location of Files =====
+Just a note, the files created by virsh and virt-install are stored in **/etc/libvirt/qemu/**. However, they should not be hand-edited; only edit with virsh.
 ===== Install and Configure =====
@@ Line 13: / Line 17: @@
 You should see either vmx or svm in the output.
-Now, install the basic packages needed, and nothing else
+Now, install the basic packages needed, a couple of utilities, but not all the extra crud. //netcat-openbsd// is only needed if you're going to cluster and migrate virtuals from one hypervisor to another, from what I can tell. **virt-top** is a nice little //top// for seeing what is running and what resources they are using in real time.
 <code bash>
-apt install -y --no-install-recommends qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils libguestfs-tools genisoimage virtinst libosinfo-bin virt-top
+apt install -y --no-install-recommends qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils libguestfs-tools genisoimage virtinst libosinfo-bin virt-top netcat-openbsd
 reboot # brings libraries online
 </code>
@@ Line 39: / Line 43: @@
 ===== Defining Network =====
+==== Setting up bridges ====
+For your network, you need bridges for the outside world.
+=== Simple ===
+This is a basic setup that will work for a single interface as per the Debian documentation. It sets up one bridge off of eth0 and gives it a static IP.
+<code bash interfaces>
+auto lo
+iface lo inet loopback
+# The primary network interface
+auto eth0
+#make sure we don't get addresses on our raw device
+iface eth0 inet manual
+iface eth0 inet6 manual
+#set up bridge and give it a static ip
+auto br0
+iface br0 inet static
+        address 192.168.1.2
+        netmask 255.255.255.0
+        network 192.168.1.0
+        broadcast 192.168.1.255
+        gateway 192.168.1.1
+        bridge_ports eth0
+        bridge_stp off
+        bridge_fd 0
+        bridge_maxwait 0
+        dns-nameservers 8.8.8.8
+</code>
+=== Real World ===
+I'm hoping, if you're reading this article, you know how to set up bonding and vlans. The following  is a more real world scenario and sets up three bridges, br_lan, br_dmz and br_wan for LAN, DMZ and public interface respectively. br_wan and br_dmz are set to dummy IP's with a netmask of 255.255.255.255, meaning they can't respond to anything but themselves. br_lan gets its IP from DHCP, so only the LAN interface can be accessed on the hypervisor. Modify it as you want.
 <code bash interfaces>
@@ Line 74: / Line 117: @@
    vlan-raw-device bond0.30
+# the public interface on vlan 10
 auto br_wan
 iface br_wan inet static
@@ Line 83: / Line 127: @@
    bridge_maxwait 0
+# the DMZ on vlan 20
 auto br_dmz
 iface br_dmz inet static
@@ Line 92: / Line 137: @@
         bridge_maxwait 0
+# the private (LAN) interface on vlan 30
 auto br_lan
 iface br_lan inet dhcp
@@ Line 100: / Line 146: @@
 </code>
+==== Adding network to virt-lib ====
+In order to use a network with vir-lib, you need to define it. The best way is to create a few XML files, then use virsh to define them into the system.
+=== One at a time ===
+Create one XML file per interface as follows:
+<code xml br_wan.xml>
+<network>
+  <name>br_wan</name>
+  <forward mode="bridge"/>
+  <bridge name="br_wan"/>
+</network>
+</code>
+Then, import it into the system with virsh, then set it to autostart on boot
+<code bash>
+# import the network xml file
+virsh net-define --file br_wan.xml
+# set to autostart on boot
+virsh net-autostart br_wan
+</code>
+=== Lazy Approach ===
+I'm lazy, so I just created all three, then imported them all at one time.
+<code bash import_bridge.sh>
+#! /usr/bin/env bash
+# create the xml definitions. The br is prepended.
+# Add/remove interfaces if needed
+for interface in wan lan dmz
+do
+cat << EOF > br_$interface.xml
+       <network>
+         <name>br_$interface</name>
+         <forward mode="bridge"/>
+         <bridge name="br_$interface"/>
+       </network>
+EOF
+done
+# uncomment this if you want to view your xml files but
+# not process them
+# exit
+# find all xml files and do the net-define
+for interface in `ls *.xml`
+do
+   virsh net-define --file $interface
+done
+# since the bridge name is followed by xml, simply remove that
+# and set to autostart and start it
+for interface in `ls *.xml | cut -d'.' -f1`
+do
+   virsh net-autostart $interface
+   virsh net-start $interface
+done
+# show me the list of network names
+virsh net-list
+</code>
+This script assumes your network names are of the form br_//something// and it creates the file name as br_//something//.xml. It then looks for all XML files (so, you don't want any others in the current directory), the processes them.
+The last loop assumes there are no periods in the network name. Be warned.
+===== Using Storage =====
+In our example, we are going to use LVM2 to grab a piece of the disk for a new virtual. You can also use a file (File Backed Device, or FBD) by running //fallocate//, or set up access to an iSCSI. virt-install will, by default, create an FBD in its default location, so if you're happy with that, ignore this whole section.
+==== File Backed Device =====
+By default, libvirt uses File Backed Devices (FBD's) from a pool defined internally. You can manually override this by defining a new pool, or by creating a file in your location using fallocate. For example, to create a 10G file in /srv/images named test.disk, you would use:
+<code>fallocate -l 10G /srv/images/test.disk</code>
+and use that when you create the virtual. However, it is better (easier) if you define a pool (or use the default). I have not researched this; see //man virt-install//.
+==== LVM2 ====
+Just create an LV the way you always do.
+<code bash>lvcreate -L 10G -n test.disk vg0</code>
+==== iSCSI ====
+Ok, if you're using iSCSI, I'm guessing you know how to set it up. Just make sure it is available, then use the correct path when you create the image.
+===== Doing the Install =====
+You can probably create a virtual by manually creating the XML file, but why do that when virt-install is your friend. Sure, there are a bunch of parameters, but they are very, very well documented, and will create your system for your rapidly
+This example creates a virtual installing the opnSense firewall/router.
+<code bash>
+virt-install \
+   --hvm \
+   --connect    qemu:///system \
+   --name       router-a \
+   --memory     4096 \
+   --vcpus      4 \
+   --disk       path=/dev/vg0/router-a.disk0,bus=virtio,target=sda \
+   --graphics vnc,port=5901 \
+   --noautoconsole \
+   --cdrom     /media/xen-store/OPNsense-20.1-OpenSSL-dvd-amd64.iso \
+   --os-variant freebsd11.1 \
+   --metadata uuid=d9510e01-e461-461f-9aa8-3cee223cb4a0,name=router-a,title=router-a,description='Primary Router' \
+   --boot     hd,cdrom,menu=on \
+   --network  bridge=br_wan,mac=00:16:3e:bd:26:70,model=virtio \
+   --network  bridge=br_dmz,mac=00:16:3e:bd:26:71,model=virtio \
+   --network  bridge=br_lan,mac=00:16:3e:bd:26:72,model=virtio
+</code>
+Other useful options are
+  * --dry-run # don't really do it
+  * --controller virtio-scsi # this is specific to KVM, but faster there
+  * osinfo-query os # this shows the OS's available for the --os-variant flag
+most of the parameters are self evident, but I'll quickly talk about why I did some of them.
+  * --graphics vnc,port=5901 - Our servers are headless and have no GUI. I want to be able to connect during install using VNC. By defining the port here, it is not auto-selected (and changeable) across boots
+  * --noautoconsole - When starting the virtual, virt-lib tries to bring up a GUI for virtviewer to take over, or run virsh console to bring up a serial console. Since I don't want either, I disable it here
+  * --os-variant freebsd11.1 - By defining this, virt-install can set up some defaults that work well with the known OS.
+  * --metadata uuid=d9510e01-e461-461f-9aa8-3cee223cb4a0,name=router-a,title=router-a,description='Primary Router'
+    * title, name and description will show up in some reports.
+    * uuid is good so the virtual does not get a new, randomly generated UUID every time it runs. Use the uuidgen command to get a randomly generated one for each new machine
+===== Performing Installation =====
+For most systems, I use VNC to do the install. Since I'm remote, I use the following ssh command:
+<code bash>ssh -L localhost:5910:localhost:5901 server</code>
+where
+  * 5910 is the local VNC port I'll attach to (ie, localhost:5910)
+  * 5901 is the VNC port I told virt-install to use
+  * server is the name of the hypervisor
+Then, when I run the virt-install command, I simply make a VNC connection to localhost:5910 and can do my install.
+===== Setting other parameters =====
+If you want the domain to come up automatically when the hypervisor is turned on, the autostart flag needs to be set. Do this with <code bash>virsh autostart domainname</code>
+===== Links =====
+  * https://www.cyberciti.biz/faq/install-kvm-server-debian-linux-9-headless-server/
+  * https://www.techotopia.com/index.php/Managing_KVM_on_RHEL_6_using_the_virsh_Command-line_Tool
+  * https://computingforgeeks.com/virsh-commands-cheatsheet/
+  * https://serverfault.com/questions/350806/convert-libvirt-xen-configuration-to-native
+  * https://libvirt.org/sources/virshcmdref/html/
+  * https://www.utappia.org/2016/04/how-to-migrate-your-virtual-box.html (migrate virtualbox images to kvm qcow2)
+  * https://serverfault.com/questions/627238/kvm-libvirt-how-to-configure-static-guest-ip-addresses-on-the-virtualisation-ho#627245 (set sticky IP's on KVM)