November 18, 2020 00:20
VIRTUAL MACHINE
VM - Allow multicast packages
Multicast packages are generated by e.g. the avahi daemon or minidlna and is neccessary to use the avahi zeroconf service (needed for media streaming etc).
- Show all running vms:
sudo virsh list
- Edit the xml file of the machine, which should be allowed to send out these packages:
sudo virsh edit X
- Go down to the network interface which should be allowed (e.g. NOT the [LOCALSTORAGENET]) to do that and change the following code
<devices> ... <interface type='XXX'> ... </interface> ... </devices>
to
<devices> ... <interface type='XXX' trustGuestRxFilters='yes'> ... </interface> ... </devices>
- Cold-boot the vm.
VM - Enable TRIM to save image space
This frees the unsuded space inside a vm also on disk. This may accelerates fragmentation, but on zfs
it will take quite some time until that happens.
The steps 1-2 are only needed for vm created before qemu v4.0!
- Change disk type to “SCSI” and set
discard="unmap"
- Change the controller type to “VirtIO SCSI” (
virtio-scsi
) - Enable the trim service inside the vm (on older versions of Debian first run
sudo cp /usr/share/doc/util-linux/examples/fstrim.{service,timer} /etc/systemd/system
):sudo systemctl enable fstrim.timer sudo systemctl start fstrim.timer sudo fstrim -av
The last command just trims the ssd/disks for the first time.
- Cold-boot the vm.
When the fstrim
command exits too fast check with sudo lsblk -o MOUNTPOINT,DISC-MAX,FSTYPE
for any 0B
entries - in that case that disk does not support
TRIM -> you have done something wrong. Also you may want to check with du -h [DISK_IMAGE]
the really used space for the images (they should shrink during the first fstrim
)…
Watchdog & Panic Notifier
Libvirt has a watchdog feature, which can e.g. reboot a vm on crash - other than the “panic notifier” device, which just powers the vm down. How to setup the watchdog:
- Add the Watchdog device into the vm
- Inside the vm:
sudo apt install watchdog sudo systemctl enable watchdog
- Enable the device inside the service config
/etc/watchdog.conf
:watchdog-device = /dev/watchdog realtime = yes priority = 1
- Cold-Boot the vm. If you ever wich to test the watchdog, you may crash the kernel with
sync; echo c > /proc/sysrq-trigger
asroot
!
VM - Install windows support here
SERVER
Setup
sudo apt install libvirt-daemon libvirt-bin qemu-kvm qemu-utils
sudo apt install ebtables firewalld dnsmasq
Setup (Debian Jessie+)
The first command should be:
sudo apt install libvirt-daemon-system libvirt-clients qemu-kvm qemu-utils
Support guest UEFI
sudo apt install ovmf
Allow a user to control the kvm
sudo addgroup [USER] kvm
sudo addgroup [USER] libvirt
Firewalld - MAKE SURE TO FIX THAT BUG (if neccessary)
Add/Replace this to /etc/firewalld/firewalld.conf
CleanupOnExit=no
Otherwise a reboot could take up to several minutes!
Useful commands for firewalld
- Configure:
sudo firewall-cmd --state
firewall-cmd --get-active-zones
- What default zones is active?
firewall-cmd --get-default-zone
- What zones is active on…?
firewall-cmd --get-zone-of-interface [INTERFACE_NAME]
- List all services which are known:
firewall-cmd --get-services
- Add a service for an zone:
sudo firewall-cmd --permanent --add-service=[SERVICE_NAME] --zone=[ZONE_NAME]
- Add a port for an zone:
sudo firewall-cmd --permanent --add-port=[PORT]/tcp --zone=[ZONE_NAME]
- Disable firewall COMPLETLY for an interface:
sudo firewall-cmd --permanent --zone=trusted --change-interface=[INTERFACE_NAME]
- Disable firewall COMPLETLY for ALL interfaces (permanently only):
sudo firewall-cmd --set-default-zone=trusted
Enable automatic freezing of guests at host reboot
- Add a new service
[Unit] Description=VMFreezer - saves / restores all running machines of libvirt from / to disk Requires=libvirtd.service #libvirt-guests.service is in after, because @shutdown this order is inverse! #Add here the required path (maybe to save the states on external disks) from /etc/fstab (slash must be a dash) #MAYBE add mnt-raid01.mount to wait for a specific mount point... After=network.service libvirtd.service libvirt-guests.service #Before= [Service] Type=oneshot #infinity -> make sure we wait for ANY vm! TimeoutSec=infinity RemainAfterExit=true ExecStart=/root/restore.sh ExecStop=/root/save.sh [Install] WantedBy=multi-user.target
- Add the required restore script to
/root/restore.sh
(make sure to change the target path!)#!/bin/bash # Restore all guests from saved state and start cd /mnt/ echo "Working in `pwd`." ls -1 *.state | \ while read GUEST; do echo "Restoring $GUEST..." virsh restore $GUEST --running if [ $? -eq 0 ]; then echo "Removing the old state $GUEST..." rm $GUEST else echo "Start of $GUEST failed. The state will be moved to /tmp/ - so it can manually restored... Eventually..." mv $GUEST /tmp/ fi # Now sleep a shot period of time to make sure, that e.g. dynamic memory has been populated properly... sleep 5 done
- Add the required save script to
/root/save.sh
(make sure to change the target path!)#!/bin/bash # Save (store ram and shutdown) all guests cd /mnt/ echo "Working in `pwd`." virsh list | `#list of running guest` \ tail -n +3 | head -n -1 | sed 's/\ \+/\t/g' | `#strip head and tail, use tab for seperator`\ awk '{print($2)}' | \ while read GUEST; do echo "Saving $GUEST..." virsh save $GUEST $GUEST.state done
- Mark the scripts as executable:
sudo chmod 555 /root/save.sh /root/restore.sh
Install the startup vm service
- Add the
vmfreezer.service
file to/etc/systemd/system
- Add the
save.sh
file to/root
- Add the
restore.sh
file to/root
- Set permissons for them
sudo chmod 500 /root/save.sh /root/restore.sh
- DON’T FORGET to modify the scripts to use the correct path to save and restore the vms!
- Enable the new service with
sudo systemctl enable vmfreezer
Shared folders
KVM
Just add a new mapped shared folder with a new [TARGET_PATH].
To mount it, just insert following line into the guests /etc/fstab
:
[TARGET_PATH] [LOCAL_PATH] 9p trans=virtio,version=9p2000.L,msize=262144 0 0
IF you get emergency boot failures - insert the following into /etc/initramfs-tools/modules
:
9p
9pnet
9pnet_virtio
…and update sudo update-initramfs -u
!
If the listing of much files is too slow, try enabling the cache (copied from here):
cache=mode specifies a caching policy. By default, no caches are used.
none = default no cache policy, metadata and data
alike are synchronous.
loose = no attempts are made at consistency,
intended for exclusive, read-only mounts
fscache = use FS-Cache for a persistent, read-only
cache backend.
mmap = minimal cache that is only used for read-write
mmap. Northing else is cached, like cache=none
Samba
Install server…
sudo apt install samba
Add a virtual isolated network for loopback communication with the host and vm
- Make sure to enable DHCP, so the host will listen to the clients (instead being REALLY isolated).
- Add this interface (e.g. virbr1) to the firewall (trusted zone is okay - because the VMs should have a second interface anyway which is in the same network like the host)…
- Note that the host can contact the VMs ONLY using that networks IPs from this network!
- Because the host is always faster than the other network interfaces you REALLY SHOULD apply the following fix:
- Use the command
sudo virsh net-edit [LOCALSTORAGENET_NAME]
to open the xml-configuration-file of the virtual network. - Add there the following code (if you add any other entry than the one domain=… the host will resolve the request for the client - so don’t be confused if the /etc/resolv.conf specifies then the host as dns provider)…
<network> ... <dns> <forwarder domain='router.domain'/> <forwarder addr='1.1.1.1'/> </dns> ... </network>
…to forward any request to either the real network dns provider or e.g. Cloudflare!
- Save it, restart the network and reboot any vms to apply the fix!
- Use the command
Setup the smb.conf to…
#THIS ALL REQUIRES samba
#This is lacated at /etc/samba/smb.conf
[global]
#Network stuff
workgroup = WORKGROUP
server string = %h
#Following: Set it to the servers local IP (the one from virbr1 / localhost)
#hosts allow = localhost 127.0.0.1 192.168.0.0/24
#hosts deny = 0.0.0.0/0
dns proxy = no
disable netbios = yes
name resolve order = bcast host
#Permissions USE sudo smbpasswd -a USER to add user, USE sudo smbpasswd -x USER to remove user
guest account = nobody
security = user
encrypt passwords = true
invalid users = root
guest ok = no
#Stuff
unix extensions = yes
unix password sync = no
usershare owner only = yes
#Log size in Kb
max log size = 50
#Server role inside the network
server role = standalone server
#Fix the permissions to allow group access!
#force user = [USER (Only if neccessary)]
force group = [FSgroup]
#Following seems to be useless with the following fixes...
#create mask = 770
#FIX permission: File: UPPER bound for the bits
create mode = 770
#FIX permission: File: LOWER bound for the bits
force create mode = 770
#FIX permission: Directory: UPPER bound for the bits
directory mode = 770
#FIX permission: Directory: LOWER bound for the bits
force directory mode = 770
#
#NOTE:
#browseable = no -> Hidden share
#
[Share1]
path = [PATH]
available = yes
#Following to hide it anyways!
browseable = no
guest ok = no
#Following to make read only if no user is in the write list!
writeable = no
valid users = [VirtUsers]
write list = [VirtUsers]
VM - Allow a vm access to a specific share…
Nett2Know: Use sudo pdbedit -L
to get current user list…
- Add an account on the host (nologin, nohome) with
sudo adduser --no-create-home --shell /usr/sbin/nologin --disabled-login [USER]
- Add this account to the FSgroup
sudo adduser [USER] [FSgroup]
- Allow samba to map to this account (now is a good PWD neccessary)
sudo smbpasswd -a [USER]
- Add the account to the shares at the smb.conf
- Add the share to the vm and save the credentials there (next paragraph)
Setup a vm to access and mount a specific share
Add this to fstab (it will mount on first access - this is neccessary, because some (…) systemd instances ignore the _netdev option) //[HOST_LOCALSTORAGENET_IP]/[SHARE_NAME] [TARGET_PATH] cifs noauto,x-systemd.automount,x-systemd.idle-timeout=5m,_netdev,nouser,mapchars,cache=strict,noacl,credentials=[CREDENTIAL_FILE (e.g. /root/creds)],domain=workgroup,uid=root,gid=[VM_SHARED_FOLDER_GROUP],file_mode=0770,dir_mode=0770 0 0
On cd-failures with error -13 you fucked up the password or username!
Use cache=strict to fix ghosting folders (if they still appear use ‘none’ - BUT THIS WILL IMPACT PERFORMACE). When there are no ghosting folders or files you can try to use ‘loose’ to further improve performance.
Setup a vm to make shares available (needed only ONCE)…
- Install cifs
sudo apt install cifs-utils
- Add the host localstorage interface to /etc/network/interfaces:
iface [INTERFACE_NAME] inet dhcp
- Add a group for the shares
sudo addgroup [VM_SHARED_FOLDER_GROUP]
- Add a user to this group
sudo addgroup [USER (e.g. www-data)] [VM_SHARED_FOLDER_GROUP]
- Create the authentication file (e.g. /root/creds):
username=[USERNAME] password=[PASSWORD]
- Set permissons for the credential file
sudo chmod 500 [CREDENTIAL_FILE (e.g. /root/creds)]
CLIENT(S)
Setup management-client
sudo apt install virt-manager spice-client-gtk gir1.2-spiceclientgtk-3.0
Setup viewonly-client
sudo apt install virt-viewer