目录

「PVE」那个背后的存在

PVE虚拟环境安装及配置。

PVE(Proxmox Virtual Environment)

Win10/Win11+TrueNAS

安装

软硬件准备

BIOS

  • ASUS B250M 进BIOS F2/F8(临时)
  1. 使能 Intel Virtualization Technology
  2. 使能 VT-d
  3. Legacy兼容开启(当前PVE版本直通核显到Win10并输出HDMI需要,ESXI目前(6.7)不能直通核显同时输出HDMI1

直通

参考官方文档
PVE 6.3-1, 8.3-1 PASSED;

1. 修改GRUB
vi /etc/default/grub
# `GRUB_CMDLINE_LINUX_DEFAULT` 字段增加如下内容,其中`iommu=pt`用于优化非直通设备性能,非必须
intel_iommu=on iommu=pt

update-grub
2. 添加模块
vi /etc/modules
# 增加内容如下
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

update-initramfs -u -k all
# 重启物理机
3. 检查
# 需IOMMU开启
dmesg | grep -e DMAR -e IOMMU -e VT-d
# 需有设备列出
find /sys/kernel/iommu_groups/ -type l
  • IOMMU 开启状态
    https://store.yirami.xyz/review/pve/check_iommu_status.png
  • IOMMU 设备
    https://store.yirami.xyz/review/pve/check_iommu_groups.png
4. 核显(直通)

参考博客2
参考教程3
参考教程4,尚未成功

# 编辑GRUB文件(/etc/default/grub),增加如下
intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction video=efifb:off,vesafb:off,simplefb:off,vesa:off
# 创建并编辑黑名单(/etc/modprobe.d/blacklist.conf),屏蔽驱动
blacklist nvidiafb
blacklist snd_hda_intel
blacklist snd_hda_codec_hdmi
blacklist i915
# 查询核显、声卡ID
lspci -n | grep -E "0300|0403"
# 输出为
 # 00:02.0 0300: 8086:5912 (rev 04)
 # 00:1f.3 0403: 8086:a2f0
# 绑定核显、声卡到vfio
echo "options vfio-pci ids=8086:5912,8086:a2f0" > /etc/modprobe.d/vfio.conf
# 更新配置
update-initramfs -u -k all
# 重启物理机

WEB-GUI

1. 禁用订阅弹出通知 No valid subscription
# 1)
cp /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js.bak
vim /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js  # use / to search `No valid subscription`
# nano /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js  # use ctrl+w to search `No valid subscription`
# change !== to ===, so notice will pop up when active

# 2) prior
sed -Ezi.bak "s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js && systemctl restart pveproxy.service
2. 更新软件源
mv /etc/apt/sources.list.d/pve-enterprise.list /etc/apt/sources.list.d/pve-enterprise.list.bak
# 屏蔽订阅企业源
wget https://mirrors.ustc.edu.cn/proxmox/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
echo "deb https://mirrors.ustc.edu.cn/proxmox/debian bookworm pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list
# 使用 Proxmox 非企业版中科大源
sed -i 's|^deb http://ftp.debian.org|deb https://mirrors.ustc.edu.cn|g' /etc/apt/sources.list
sed -i 's|^deb http://security.debian.org|deb https://mirrors.ustc.edu.cn/debian-security|g' /etc/apt/sources.list
# 将 Debian 官方源替换为中科大源
echo "deb https://mirrors.ustc.edu.cn/proxmox/debian/ceph-quincy bookworm no-subscription" > /etc/apt/sources.list.d/ceph.list
# 替换 Ceph 源
sed -i 's|http://download.proxmox.com|https://mirrors.ustc.edu.cn/proxmox|g' /usr/share/perl5/PVE/APLInfo.pm
# 替换 CT 镜像下载源(重启 PVE 才能生效)
apt update 
# 更新源列表
apt upgrade 
# 升级软件包
apt dist-upgrade 
# 升级系统
apt install openvswitch-switch
# 安装OVS

ACME

参考博客5实现基于 CloudFlare 和 Let’s Encrypt 的 ACME 搭建,基于 Alibaba Cloud 的搭建可参考这篇博客6

  1. 如下图,依次点击 Datacenter -> ACME -> Add(Accounts) https://store.yirami.xyz/review/pve/add_acme_account.png
  2. 如下图,填写账户名及邮箱,并完成注册
    https://store.yirami.xyz/review/pve/add_acme_account_info.png
  3. 注册成功的输出像这样:
    Generating ACME account key..
    Registering ACME account..
    Registration successful, account URL: 'https://acme-v02.api.letsencrypt.org/acme/acct/2321569307'
    TASK OK
  4. 回到 ACME 选项卡下,点击 Add(Challenge Plugins)
  5. 如下图,填写插件名、选择 CloudFlare 作为 DNS 管理 API,并填入 TokenZone ID7 https://store.yirami.xyz/review/pve/add_acme_plugin.png
  6. 如下图,为 PVE 服务器绑定管理插件 https://store.yirami.xyz/review/pve/bind_acme_plugin.png
  7. 激活 PVE 服务器的证书 https://store.yirami.xyz/review/pve/active_ssl_cert.png

网络唤醒

BIOS 设置

ASUS

PVE 设置

PVE 7.4-16, 8.3-1 PASSED;

Turn on WOL
ip addr # 查询待唤醒网卡号,如 `enp4s0`
ethtool enp4s0  # 关注 Wake-on 一栏,`d` 表示禁用,`g` 表示启用
ethtool -s enp4s0 wol g  # (单次)启用 WOL

Tips: 这种启用是临时的,当系统重启或网卡重新加载驱动时,WOL 可能会恢复默认值,即 d,因此需要配合开机重设任务。

Turn on after start
by rc.local

较新的 Linux 内核已经不再提供 rc.local 用于设置开机启动脚本8,但目前仍有对应服务存在9

cat /lib/systemd/system/rc-local.service  # 确认服务存在
systemctl status rc-local  # 确认服务状态(默认关闭)
cat <<EOF > /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

ethtool -s enp4s0 wol g

exit 0
EOF

chmod +x /etc/rc.local
systemctl start rc-local
[推荐] by service
sh -c 'cat > /etc/systemd/system/enable-wol.service <<EOF
[Unit]
Description=Enable Wake-on-LAN (WOL) for enp4s0
After=network.target

[Service]
Type=oneshot
ExecStart=/sbin/ethtool -s enp4s0 wol g
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF'

chmod 644 /etc/systemd/system/enable-wol.service
systemctl daemon-reload
systemctl enable enable-wol.service
# systemctl start enable-wol.service  # 立即运行一次

容器

Debian

Debian 11, 12 PASSED;

  • 下载 LXC 容器模板 https://store.yirami.xyz/review/pve/pve_v8_debian_lxc_template.png
  • Options 设置(注意添加 Featureshttps://store.yirami.xyz/review/pve/pve_v8_debian_lxc_options.png
内核升级 & 固件检查 (PVE 7.4-16)
# 低版本需更新内核
# apt update && apt install pve-kernel-5.19 -y

cd /lib/firmware/i915/ && ls ehl_guc*.bin && ls ehl_huc*.bin && ls icl_dmc*.bin && cd ~
SSH 端口修改

高版本可能使用 systemd 的套接字激活机制 ssh.socket,它工作中不读取 sshd_config,因此在该配置中修改默认端口可能不生效。

  • 禁用 ssh.socket
    sudo systemctl stop ssh.socket
    sudo systemctl disable ssh.socket
    sudo systemctl mask ssh.socket  # 可选,防止未来被启用
    sudo systemctl restart ssh
LXC 权限映射

参考 B站视频博客 进行 LXC 容器安装及基本配置。 注意:在创建 LXC 容器时,需要取消勾选非特权容器(Unprivileged container)

  • 查看核显权限
# 宿主机执行,以查看核显权限
ls -l /dev/dri

https://store.yirami.xyz/review/pve/pve_v8_gpu_dri.png

  • 映射
nano /etc/pve/lxc/103.conf

# append those lines
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
# lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
xc.apparmor.profile: unconfined
  • 开启 GUC
# HD630 核显可能不支持 enable_guc=3
echo "options i915 enable_guc=2" > /etc/modprobe.d/i915.conf

https://store.yirami.xyz/review/pve/pve_v7_lxc_enable_guc_2.png

update-initramfs -u -k all
# 重启物理机
Docker 方案部署 Jellyfin

需要开启硬解,主机不直通显卡、声卡到其它虚拟机,因此如已屏蔽宿主机的相关驱动,需撤回。

# 检查输出
journalctl -b -o short-monotonic -k | egrep -i "i915|dmr|dmc|guc|huc"

nyanmisaka/jellyfin 集成了驱动,无需手动安装驱动 安装核显驱动

# 使用NFS挂载(使用 SAMBA 挂载然后映射到 docker 中总是无法启动 jellyfin 应用)
apt update && apt install nfs-common -y

# make mount dirs
mkdir -p /mnt/share/nfs/jellyfin && \
mkdir -p /mnt/share/nfs/general && \
mkdir -p /mnt/share/nfs/av/official && \
mkdir -p /mnt/share/nfs/anim/official

# test mount
mount -t nfs aio.nas.yirami.xyz:/mnt/YDisk/YArk/APPS/jellyfin /mnt/share/nfs/jellyfin
mount -t nfs aio.nas.yirami.xyz:/mnt/YDisk/YArk/General /mnt/share/nfs/general
mount -t nfs aio.nas.yirami.xyz:/mnt/YDisk/YArk/Restricted/AdultVideo/official /mnt/share/nfs/av/official
mount -t nfs aio.nas.yirami.xyz:/mnt/YDisk/YArk/Restricted/Animation/official /mnt/share/nfs/anim/official

# umount them
umount /mnt/share/nfs/jellyfin
umount /mnt/share/nfs/general
umount /mnt/share/nfs/av/official
umount /mnt/share/nfs/anim/official

# create service file to /etc/systemd/system/

# reload service
systemctl daemon-reload

# auto start after boot
systemctl enable mnt-share-nfs-jellyfin.mount
# systemctl disable mnt-share-nfs-jellyfin.mount

# start mount serivce
systemctl start mnt-share-nfs-jellyfin.mount
# systemctl stop mnt-share-nfs-jellyfin.mount
systemctl start run-jellyfin.service
systemctl stop run-jellyfin.service
systemctl restart run-jellyfin.service

Jellyfin 生成证书10

openssl pkcs12 -password pass:<cert_password> -export -out /mnt/share/nfs/jellyfin/jellyfin.aio.yirami.xyz.p12 -inkey /etc/letsencrypt/live/jellyfin.aio.yirami.xyz/privkey.pem -in /etc/letsencrypt/live/jellyfin.aio.yirami.xyz/cert.pem -certfile /etc/letsencrypt/live/jellyfin.aio.yirami.xyz/fullchain.pem

不过对于内网服务,上述证书部署方案过于麻烦,也不方便集中管理,可以参考部署 Traefik 服务统一对内网服务提供证书保护。

更多关于 Jellyfin 服务的运维笔记可参考这篇

虚拟机

https://store.yirami.xyz/review/pve/pve_v7_pcie_devices.png

Windows 10

  • Handware 设置(注意匹配 BIOS 方式)
    • UEFI 方式直通:USB 控制器、系统盘、万兆光口网卡、声卡 https://store.yirami.xyz/review/pve/pve_v8_windows10_vm_handware.png
  • Options 设置(注意勾选 QEMU Guest Agenthttps://store.yirami.xyz/review/pve/pve_v8_windows10_vm_options.png
虚拟CPU标志同步

对齐虚拟机CPU标志到实体CPU以解决指令集不支持等问题

WebGUI中设置Hardware -> Processors -> Edit -> Typehost

影响:

  1. 默认的kvm64具备通用的CPU标志以便于在任何物理设备上进行虚拟机实时迁移
  2. 采用host则使虚拟CPU具备与实体CPU一致的标志从而获得最接近实体CPU的性能

案例:

  1. 虚拟Windows 10中安装Adobe Photoshop 2022提示需要SSE 4.2指令集支持
VirtIO 驱动
  1. 上传驱动镜像文件到 PVE https://store.yirami.xyz/review/pve/pve_iso_upload.png
  2. Windows 虚拟机设置中挂载该镜像
  3. 启动 Windows 虚拟机并从设备管理器安装相应驱动
  4. 安装 guest-agent 目录下 qemu-ga-x86_64.msc

Tips: 若 Windows 提示安全设置已更改,需要修改 PIN,且唯有联网登录微软账户验证后才能修改,可以先将虚拟机网卡修改为 E1000(该网卡无需额外安装驱动),修改完 PIN 后再改回网卡并安装驱动。

虚拟机时间同步

Windows 默认会将硬件时钟(BIOS/UEFI 时钟)解释为 本地时间(Local Time),而不是 UTC(协调世界时)。而 Linux(包括 Proxmox VE/PVE)默认将硬件时钟视为 UTC,然后根据时区转换为本地时间。

  1. 使 Windows 其将硬件时钟视为 UTC
    # 管理员方式运行 PowerShell
    reg add "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation" /v RealTimeIsUniversal /t REG_DWORD /d 1 /f
  2. 关闭 Windows 时间同步服务以避免冲突(可选)
    net stop w32time
    sc config w32time start=disabled
直通核显

参考博客2及论坛11

# 1. WebGUI中新增显卡直通PCI设备(不勾选Primary GPU)
# 2. 启动虚拟机,此时HDMI应无输出
# 3. 删除前述显卡PCI设备
# 4. 编辑虚拟机配置文件
vi /etc/pve/qemu-server/100.conf
 # 第一行增加`args: -device vfio-pci,host=00:02.0,addr=0x18,x-igd-opregion=on`
 # 倒数第二行配置为`vga: std`或`vga: none`,对应是否开启VNC输出
直通PCIE硬盘
直通SATA硬盘(全盘映射)
# 查找需要全盘映射的磁盘
ls /dev/disk/by-id
 # 如:ata-Samsung_SSD_850_EVO_120GB_S21VNSAG104135K
# 挂载映射,其中`100`为PVE中VM ID;sata0为目的sata号,支持sata0~sata5
qm set 100 -sata0 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_120GB_S21VNSAG104135K
直通SATA控制器
直通USB设备
# WebGUI直接选择添加单个USB口或整个USB控制器

Windows 11

物理机模式安装

当前硬件状态已不满足微软官方的安装要求:

  1. 不支持 i5 7500
  2. 主板不支持 TPM 2.0
  3. 找不到驱动 RST(Rapid Storage Technology)

因此安装需要做一些额外操作

TPM 跳过
  1. 准备官方 Windows 10Windows 11 镜像
  2. 使用 Windows 10source/appraiserres.dll 文件替换 Windows 11 下同名文件
  3. 将改动后的 Windows 11 安装文件重新封装为可启动的镜像文件

PS:

  1. 该方法可以跳过 TPM 检测
  2. 但在本机试验(使用UEFI原生安装)时会因为没有 RST 驱动而找不到硬盘
  3. 推荐使用下面基于 PE 的安装方法
「推荐」PE 下灌装
  1. 使用 Windows 11 原版镜像
  2. 进入 PE,将镜像文件解压到本地磁盘(如 D盘
  3. 如果有驱动程序(包括 PVE 的虚拟化驱动),可以一并准备,并拷贝到本地磁盘
  4. 启动 微PE 桌面的 Windows 安装器,并选择包含 Windows 11 的安装模式
  5. Windows 安装文件 指向到 第2步 中拷贝的路径下的 install.wim 文件
  6. 引导驱动器 指向到待安装磁盘的 EFI 分区
  7. 安装磁盘 指向到待安装磁盘的系统分区
  8. 选择所需的 Windows 版本
  9. 确认后开始安装
PVE 虚拟机相关配置

此部分主要参考前述 Windows 10 的配置。

https://store.yirami.xyz/review/pve/pve_v7_windows11_vm.png

要点:

  • BIOS选择开源的UEFI方案 OVMF
  • 为了方便,网络设备的MAC地址最好手动指定
  • 必须创建 EFI 分区
  • 必须创建虚拟的 TPM 模块
  • 目前直通的设备是
    • VMVE SSD 250G
    • USB 控制器
    • 声卡
    • Intel 82599ES 10G 光口网卡
    • 核显直通

TrueNAS

CORE

https://store.yirami.xyz/review/pve/pve_v6_uefi_truenas_vm.jpg

SCALE
  • Handware 设置(注意匹配 BIOS 方式)
    • Legacy 方式直通:SATA 控制器、系统盘、万兆光口网卡 https://store.yirami.xyz/review/pve/pve_v8_truenas_vm_handware.png
  • Options 设置 https://store.yirami.xyz/review/pve/pve_v8_windows10_vm_options.png

详情

直通SATA硬盘(全盘映射)
# 查找需要全盘映射的磁盘
ls /dev/disk/by-id
 # 如:ata-ShineDisk_M667_32G_AA000000000000001188
# 挂载映射,其中`101`为PVE中VM ID;sata0为目的sata号,支持sata0~sata5
qm set 101 -sata0 /dev/disk/by-id/ata-ShineDisk_M667_32G_AA000000000000001188
维护
监测硬盘温度
#! /bin/sh
echo "CPU temp :"
sysctl -a | egrep -E "cpu\.[0-9]+\.temp"
echo
echo "HDD temp :"
for i in $(sysctl -n kern.disks)
do
        DevTemp=`smartctl -a /dev/$i | awk '/Temperature_Celsius/{print $0}' | awk '{print $10 "C"}'`
        DevSerNum=`smartctl -a /dev/$i | awk '/Serial Number:/{print $0}' | awk '{print $3}'`
        DevName=`smartctl -a /dev/$i | awk '/Device Model:/{print $0}' | awk '{print $3}'`
        echo $i $DevTemp $DevSerNum $DevName
done

Debian 10

  • Handware 设置(注意匹配 BIOS 方式) https://store.yirami.xyz/review/pve/pve_v8_debian_vm_handware.png
  • Options 设置 https://store.yirami.xyz/review/pve/pve_v8_debian_vm_options.png
SSH
Static IP

vim /etc/network/interfaces

SAMBA
apt update
apt install -y cifs-utils
mkdir /mnt/share/yirami
# pre config YDocker:=/mnt/YDisk/Private/Docker in TrueNAS
mount -t cifs //aio.nas.yirami.xyz/YDocker /mnt/share/yirami -o username=ycloud,password=ycloud
NFS
apt update
#apt install -y nfs-utils
apt install -y nfs-common
# 1. mount nfs version 3 without user/password
mkdir /mnt/share/yirami/DL
mount -t nfs aio.nas.yirami.xyz:/mnt/YDisk/Public/DL /mnt/share/yirami/DL
# 2. detach this mount
umount /mnt/share/yirami/DL
# 3. mount nfs version 4 without user/password
mkdir /mnt/share/yirami/public
mount -t nfs4 aio.nas.yirami.xyz:/mnt/YDisk/Public /mnt/share/yirami/public
# 4. mount nfs version 4 with user/password
#nfs does not, and has never supported user/pass authentication, this was always implemented via kerberos
Docker 方案部署 nextcloud

Docker 方案遇到一个问题,即无法在docker中挂载外部SAMBA

mysql12
docker run -d --name mysql \
-v /mnt/share/yirami/mysql:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=y@nxtcloud \
-e MYSQL_DATABASE=nextcloud \
-e MYSQL_USER=yirami \
-e MYSQL_PASSWORD=y@nxtcloud \
-p 3306:3306 \
mysql
# 临时方案
docker run -d --name mysql \
-v /root/mysql:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=y@nxtcloud \
-e MYSQL_DATABASE=nextcloud \
-e MYSQL_USER=yirami \
-e MYSQL_PASSWORD=y@nxtcloud \
-p 3306:3306 \
mysql
nextcloud
docker run -d --name nextcloud \
-v /mnt/share/yirami/nextcloud:/var/www/html \
--link mysql:mysql \
-p 8080:80 \
nextcloud
# 临时方案
docker run -d --name nextcloud \
-v /root/nextcloud:/var/www/html \
--link mysql:mysql \
-p 8080:80 \
nextcloud
# web 登录页中 localhost 改为 mysql
LNMP 方案部署 nextcloud
# LNMP
wget http://soft.vpser.net/lnmp/lnmp1.7.tar.gz -cO lnmp1.7.tar.gz && tar zxf lnmp1.7.tar.gz && cd lnmp1.7 && ./install.sh lnmp
# MySQL@8.0.20->y@nxtcloud->InnoDB@Y->PHP@7.4.12->Allocator@N
# vhost
lnmp vhost add
# aio.cloud.yirami.xyz->rewrite@y->pathinfo@y->log@n->MySQL@y->`y@nxtcloud`->databasename@`aio_cloud`->databaseusername@`aio_cloud`->`/root/aio.cloud.yirami.xyz.ssl/aio.cloud.yirami.xyz.pem`->`/root/aio.cloud.yirami.xyz.ssl/aio.cloud.yirami.xyz.key`
# Nextcloud
wget https://download.nextcloud.com/server/installer/setup-nextcloud.php
mv setup-nextcloud.php /home/wwwroot/aio.cloud.yirami.xyz/
## pre-download if network is not good
## 1. https://download.nextcloud.com/server/releases/nextcloud-21.0.0.zip
## 2. mv nextcloud-21.0.0.zip /home/wwwroot/aio.cloud.yirami.xyz/nc.zip
# open website then setup
http://aio.cloud.yirami.xyz/setup-nextcloud.php
# install_dir@`.`->user@`yirami`->pw@common
# samba
apt update
apt install -y php-dev libsmbclient libsmbclient-dev php-smbclient smbclient
pecl install smbclient
LNMP 方案部署 cloudreve
lnmp
wget http://soft.vpser.net/lnmp/lnmp1.7.tar.gz -cO lnmp1.7.tar.gz && tar zxf lnmp1.7.tar.gz && cd lnmp1.7 && ./install.sh lnmp
## MySQL@8.0.20->y@nxtcloud->InnoDB@Y->PHP@7.4.12->Allocator@N
lnmp vhost add
# aio.cloud.yirami.xyz->rewrite@y->pathinfo@y->log@n->MySQL@y->`y@nxtcloud`->databasename@`aio_cloud`->databaseusername@`aio_cloud`->`/root/aio.cloud.yirami.xyz.ssl/aio.cloud.yirami.xyz.pem`->`/root/aio.cloud.yirami.xyz.ssl/aio.cloud.yirami.xyz.key`
cloudreve
## https://docs.cloudreve.org/getting-started/install
samba
apt update
apt install -y cifs-utils
# mount -t cifs //aio.nas.yirami.xyz/YDisk /mnt/YDisk -o username=yirami
# mount -t cifs //aio.nas.yirami.xyz/YDisk/Private/CLOUDREVE/db /usr/local/mysql/var/ -o username=ycloud,password=ycloud,uid=mysql,gid=mysql
# mount -t cifs //aio.nas.yirami.xyz/YDisk/Private/CLOUDREVE/files /home/wwwroot/aio.cloud.yirami.xyz/uploads -o username=ycloud,password=ycloud,uid=root,gid=root
vim /etc/fstab
# 可能有启动顺序问题,导致mysql启动失败
#//aio.nas.yirami.xyz/YDisk/Private/CLOUDREVE/db /usr/local/mysql/var/ cifs defaults,rw,uid=mysql,gid=mysql,username=ycloud,password=ycloud 0 0
//aio.nas.yirami.xyz/YDisk/Private/CLOUDREVE/files /home/wwwroot/aio.cloud.yirami.xyz/uploads cifs defaults,rw,uid=root,gid=root,username=ycloud,password=ycloud 0 0
power onstart service
vim /usr/lib/systemd/system/cloudreve.service
# add
[Unit]
Description=Cloudreve
Documentation=https://docs.cloudreve.org
After=network.target
After=mysqld.service
Wants=network.target

[Service]
WorkingDirectory=/home/wwwroot/aio.cloud.yirami.xyz
ExecStart=/home/wwwroot/aio.cloud.yirami.xyz/cloudreve
Restart=on-abnormal
RestartSec=5s
KillMode=mixed

StandardOutput=null
StandardError=syslog

[Install]
WantedBy=multi-user.target
# then
systemctl daemon-reload
systemctl start cloudreve
systemctl enable cloudreve

温度监测

CPU及主板

# 安装
sudo apt update
sudo apt install -y lm-sensors
# 配置,一般全部`yes`
sensors-detect
# 监测
sensors

硬盘

# 安装
sudo apt install -y hddtemp
# 监测
hddtemp /dev/sd?
# 给权限,由于 PVE 的 web 服务以用户 www-data 身份运行,需要修改 hddtemp 的权限
sudo chmod +s /usr/sbin/hddtemp

WEB GUI 显示

参考博客13

vim /usr/share/perl5/PVE/API2/Nodes.pm
# 定位到`my $dinfo = df('/', 1);`,其上插入如下
$res->{temperatures} = `sensors`;  # 添加此行以获取 CPU 与主板温度
$res->{hdd_temperatures} = `hddtemp /dev/sd?`;  # 添加此行以获取硬盘温度
vim /usr/share/pve-manager/js/pvemanagerlib.js
# 1. 定位到`gettext('Status') + ': ' + zpool`,修改`height`值为`460`
# 2. 定位到`widget.pveNodeStatus`,修改`height`值为`360`,此两处修改规则为+N*20(此处N为数据行数,如一行主板&CPU、两行硬盘则N为3)
# 3. 定位到`PVE Manager Version`,增加如下字段
systemctl restart pveproxy
// ...[OMITTED]...
	{
	    itemId: 'version',
	    colspan: 2,
	    printBar: false,
	    title: gettext('PVE Manager Version'),
	    textField: 'pveversion',
	    value: ''
	},
	{
	    itemId: 'temperatures',
	    colspan: 2,
	    printBar: false,
	    title: gettext('Temperatures'),
	    textField: 'temperatures',
	    renderer: function(value) {
	        value = value.replace(/Â/g, '');
	        let data = [];
	        let cpus = value.matchAll(/^coretemp-isa-(\d{4})$\n.*?\n((?:Package|Core)[\s\S]*?^\n)+/gm);
	        for (const cpu of cpus) {
	  	        let cpuNumber = parseInt(cpu[1], 10);
	  	        data[cpuNumber] = {
	  	     	    packages: [],
	  	     	    cores: []
	  	        };
	        
	  	        let packages = cpu[2].matchAll(/^Package id \d+:\s*([^°]+).*$/gm);
	  	        for (const package of packages) {
	  	     	    data[cpuNumber]['packages'].push(package[1]);
	  	        }
		    
	            let cores = cpu[2].matchAll(/^Core \d+:\s*([^°]+).*$/gm);
	            for (const core of cores) {
	                data[cpuNumber]['cores'].push(core[1]);
	            }
	        }
		    
	        let output = '';
	        for (const [i, cpu] of data.entries()) {
	            output += `CPU ${i}: `;
	            if (cpu.packages.length > 0) {
	                for (const packageTemp of cpu.packages) {
	                    output += `${packageTemp}°C `;
	                }
	            }
	            if (cpu.cores.length > 0) {
	                output += '(';
	                for (const coreTemp of cpu.cores) {
	                    output += `${coreTemp}, `;
	                }
	                output = output.slice(0, -2);
	                output += ')°C';
	            }
	            output += ' | ';
	        }

	        let boardTemp = value.match(/^acpitz-acpi-\d+$\n.*?\n^temp1:\s*([^°]+).*$/m);
	        if (boardTemp.length > 0) {
	            output += `Board: ${boardTemp[1]}°C | `
	        }
	        output = output.slice(0, -3);
		    
	        return output;
	    }
	},
	{
	    itemId: 'hdd-temperatures',
	    colspan: 2,
	    printBar: false,
	    title: gettext('HDD Temperatures'),
	    textField: 'hdd_temperatures',
	    renderer: function(value) {
	        value = value.replace(/Â/g, '');
	        return value.replace(/\n/g, '<br>')
	    }
	}

// ...[OMITTED]...

参考功耗

模式选项典型功率/瓦瞬时峰值/瓦14备注
待机-515--
开机满载80100硬盘顺序加电,会有较大起伏
TrueNAS+Windows10本地4K播放90120-
TrueNAS+Windows10scrub task110130-
Windows10空闲4550硬盘阵列在线
Windows10浏览器@2K5060硬盘阵列在线
Windows10浏览器@4K6580硬盘阵列在线
Windows10浏览器@8K7590硬盘阵列在线
开机硬盘阵列离线3560-
Windows10空闲3540硬盘阵列离线
Windows10浏览器@2K5060硬盘阵列离线
Windows10浏览器@4K6066硬盘阵列离线
Windows10浏览器@8K6580硬盘阵列离线

参考


  1. ESX 6.7 with HDMI pass through ↩︎

  2. PVE 直通核显 ↩︎ ↩︎

  3. iGPU Passthrough to VM (Intel Integrated Graphics) ↩︎

  4. Full iGPU passthrough for Jellyfin hardware acceleration (Alderlake / UHD 770) ↩︎

  5. Proxmox Valid SSL With Let’s Encrypt and Cloudflare DNS ↩︎

  6. Proxmox VE(PVE)自动获取证书 ↩︎

  7. CloudFlareZone IDOverview 页面的右侧下方查询 ↩︎

  8. PVE/Debian11设置开机自启动项 ↩︎

  9. PVE 局域网唤醒支持 ↩︎

  10. 建议使用 Traefik 代理局域网内 HTTP 流量,并使用其 ACME 自动申请并部署 Let's Encrypt 的证书以开启 HTTPS ↩︎

  11. PVE安装及Win10核显直通HDMI无输出 ↩︎

  12. https://zhuanlan.zhihu.com/p/107820215 ↩︎

  13. Proxmox VE web 界面添加硬件温度显示 ↩︎

  14. 瞬时峰值仅表明当时的观测 ↩︎

  15. 此功率较低可能不准。测量值为上电功率,即打开开关后的测量值,下电功率测量值近似为零。 ↩︎