Proxmox server
How to install proxmox on a single server with 2 disks (SSD and hard disk). This was for Proxmox Virtual Environment 7.1-7.
Installing Proxmox
- install proxmox to the SSD with gateway 192.168.0.1 and DNS 192.168.0.1.
- Log in and
apt install net-tools.
root@herring:~# ifconfig
enp0s31f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 48:4d:7e:dd:d2:dc txqueuelen 1000 (Ethernet)
RX packets 813001 bytes 1231793461 (1.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 272623 bytes 21005681 (20.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 19 memory 0xf7000000-f7020000
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 108 bytes 24649 (24.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 108 bytes 24649 (24.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vmbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.21 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::4a4d:7eff:fedd:d2dc prefixlen 64 scopeid 0x20<link>
ether 48:4d:7e:dd:d2:dc txqueuelen 1000 (Ethernet)
RX packets 354563 bytes 1193302999 (1.1 GiB)
RX errors 0 dropped 61 overruns 0 frame 0
TX packets 271784 bytes 19859101 (18.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Creating ZFS Storage
Here is the proxmox link https://pve.proxmox.com/wiki/ZFS_on_Linux.
First I had to remove an old ZFS image.
root@herring:~# vgremove pve-OLD-93BEA5C7
Do you really want to remove volume group "pve-OLD-93BEA5C7" containing 4 logical volumes? [y/n]: y
Do you really want to remove active logical volume pve-OLD-93BEA5C7/swap? [y/n]: y
Logical volume "swap" successfully removed
Do you really want to remove active logical volume pve-OLD-93BEA5C7/root? [y/n]: y
Logical volume "root" successfully removed
Do you really want to remove active logical volume pve-OLD-93BEA5C7/data? [y/n]: y
Logical volume "data" successfully removed
Do you really want to remove active logical volume pve-OLD-93BEA5C7/lvol0? [y/n]: y
Logical volume "lvol0" successfully removed
Volume group "pve-OLD-93BEA5C7" successfully removed
root@herring:~# pvdisplay
--- Physical volume ---
PV Name /dev/sdb3
VG Name pve
PV Size 334.85 GiB / not usable 2.69 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 85721
Free PE 4094
Allocated PE 81627
PV UUID 1VvXph-hxZl-B4Oh-LiOe-TsFa-dJyW-cHDabE
"/dev/sda3" is a new physical volume of "<9.10 TiB"
--- NEW Physical volume ---
PV Name /dev/sda3
VG Name
PV Size <9.10 TiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID eJCBlf-Iw7L-t5qM-QHqb-WtlD-P2xq-3Y7HeP
root@herring:~# pvremove /dev/sda3
Labels on physical volume "/dev/sda3" successfully wiped.
root@herring:~# wipefs -a /dev/sda
/dev/sda: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 8 bytes were erased at offset 0x9187ffffe00 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sda: calling ioctl to re-read partition table: Success
Now install ZFS on the same disk.
root@herring:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 9.1T 0 disk
sdb 8:16 0 335.4G 0 disk
├─sdb1 8:17 0 1007K 0 part
├─sdb2 8:18 0 512M 0 part /boot/efi
└─sdb3 8:19 0 334.9G 0 part
├─pve-swap 253:0 0 8G 0 lvm [SWAP]
├─pve-root 253:1 0 83.5G 0 lvm /
├─pve-data_tmeta 253:2 0 2.3G 0 lvm
│ └─pve-data 253:4 0 222.8G 0 lvm
└─pve-data_tdata 253:3 0 222.8G 0 lvm
└─pve-data 253:4 0 222.8G 0 lvm
sr0 11:0 1 1024M 0 rom
root@herring:~# zpool create herringpool /dev/sda
root@herring:~# zpool export herringpool
root@herring:~# zpool import -d /dev/disk/by-id/ata-WDC_WD100EFAX-68LHPN0_2YK9EU9D-part1 herringpool
# (If no encyrption)
Then in Datacenter->Storage, add ZFS herringpool
# add encryption
root@herring:~# zpool get feature@encryption herringpool
NAME PROPERTY VALUE SOURCE
herringpool feature@encryption active local
root@herring:~# zfs create -o encryption=on -o keyformat=passphrase herringpool/encrypted
Enter new passphrase:
Re-enter new passphrase:
root@herring:~# pvesm add zfspool encrypted_zfs -pool herringpool/encrypted
root@herring:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
herringpool 888K 8.97T 96K /herringpool
herringpool/encrypted 192K 8.97T 192K /herringpool/encrypted
Following these commands, “encrypted_zfs” should appear under “herring” in the web interface.
To mount the zfs (which you need to do after reboot):
zfs mount -l -a
Create a user
root@herring:~# useradd phwl
root@herring:~# passwd phwl
New password:
Retype new password:
passwd: password updated successfully
Then add the user in the web browser.
Create a Container
- From the web server, select “Datacenter -> Herring -> local (herring) -> CT Templates -> Templates” and upload ubuntu-20004-standard.
- In “Datacenter -> herring -> herringzfs (herring)” select “Create CT”. I created “carp” and asked it to use DNS.
- Select “carp” under herring and in Permissions, add the “phwl” user.
Install ssh
# apt install samba openssh-server
Install samba
Add the following to /etc/samba/smb.conf:
[carp-share]
comment = Samba on carp
path = /srv
read-only = no
browsable = yes
writeable = yes
# smbpasswd -a phwl
# systemctl restart smbd.service
Digression: LUKS on macos under Virtualbox
By the way, the trick on MacOS to mount a LUKS drive is to create a vdmk (see https://www.virtualbox.org/manual/ch09.html#rawdisk).
$ sudo VBoxManage internalcommands createrawvmdk -filename "disk2.vmdk" -rawdisk /dev/disk2
$ sudo chmod 666 /dev/disk2
$ sudo chmod 666 disk2.vmdk
Then you can go to “Storage -> Add New Storage Attachment” and select the vmdk file.
Disk Passthrough
# apt install lshw
# mkdir /mnt/backdrive
# mount /dev/disk/by-id/usb-Seagate_Expansion_NAAR7X06-0:0-part1 /mnt/backdrive
Then “Datacenter -> Storage -> SATA -> Add hard disk” and add disk2.vmdk.
Other Fixes
apt-update doesn’t work
Following https://pve.proxmox.com/wiki/Package_Repositories, comment
out /etc/apt/sources.list.d/pve-enterprise.list
# deb http://enterprise.proxmox.com/debian/pve buster pve-enterprise
and add to /etc/apt/sources.list:
deb http://download.proxmox.com/debian/pve buster pve-no-subscription
Starting from reboot
- Browse to
https://192.168.0.21:8006(herring) and open shell
root@herring:~/bin# zfs mount -l -a
root@herring:~/bin# cryptsetup luksOpen /dev/disk/by-id/usb-TOSHIBA_External_USB_3.0_20200312006320F-0\:0-part1 herringback
Enter passphrase for /dev/disk/by-id/usb-TOSHIBA_External_USB_3.0_20200312006320F-0:0-part1:
root@herring:~/bin# mount /dev/mapper/herringback /srv/herringback/
Then start carp using proxmox interface. /srv is served via samba.
