* [PVE-User] Force reclaiming space on a vdisk...
@ 2022-01-19 8:19 Marco Gaiarin
2022-01-19 8:49 ` Daniel Plominski
0 siblings, 1 reply; 6+ messages in thread
From: Marco Gaiarin @ 2022-01-19 8:19 UTC (permalink / raw)
To: pve-user
Situation: VM with a disk on a ZFS storage, 2TB disk; the disk have some
'system' partitions, and two big partition /dev/sda7 and /dev/sda8 for /home
and /srv.
I've added two more disk (/dev/sdb and /dev/sdc), created one partition per
disk and moved data from old partition to the ones. After that, i've deleted
/dev/sda7 and /dev/sda8, so now /dev/sda is a 2TB disk with roughly 100GB of
data in.
Space get not reclaimed. All disks have 'discard=1'.
I've tried to move /dev/sda to another ZFS storage (that have 500GB of
free space), and move fail.
How can i reclaim the free space on /dev/sda?! I need to create one
partition on the free space, format it, fstrim it?
Thanks.
--
Siamo circondati da troppa gente piena di sé. E a quelli pieni di sé,
io preferisco le persone piene di se, di ma, di forse. (Tonio Dell'Olio)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PVE-User] Force reclaiming space on a vdisk...
2022-01-19 8:19 [PVE-User] Force reclaiming space on a vdisk Marco Gaiarin
@ 2022-01-19 8:49 ` Daniel Plominski
2022-01-20 10:47 ` Marco Gaiarin
0 siblings, 1 reply; 6+ messages in thread
From: Daniel Plominski @ 2022-01-19 8:49 UTC (permalink / raw)
To: Proxmox VE user list
Hello Marco,
enable ZFS set compression=lz4 on the zvolume and perform a memory limit
dd run (inside the vm).
dd if=/dev/zero of=/mountpoint_sda7/CLEANUP bs=99M count=xxx
(this will release the storage space)
Mit freundlichen Grüßen
DANIEL PLOMINSKI
Leitung IT | Head of IT
Telefon 09265 808-151 | Mobil 0151 58026316 | dpl@ass.de
PGP Key: https://pgp.ass.de/dpl@ass.de.asc
PGP Fingerprint: 74DBC06BD9F63187C4DF8934C96585A89CFC10B3
Company Logo
ASS-Einrichtungssysteme GmbH
ASS-Adam-Stegner-Straße 19 | D-96342 Stockheim
Geschäftsführer: Matthias Stegner, Michael Stegner, Ralph Müller
Amtsgericht Coburg HRB 3395 | Ust-ID: DE218715721
Bottom_Line
From tsabolov@t8.ru Wed Jan 19 12:22:38 2022
Return-Path: <tsabolov@t8.ru>
X-Original-To: pve-user@lists.proxmox.com
Delivered-To: pve-user@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 2F80E6202A
for <pve-user@lists.proxmox.com>; Wed, 19 Jan 2022 12:22:38 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 24D7F7B21
for <pve-user@lists.proxmox.com>; Wed, 19 Jan 2022 12:22:38 +0100 (CET)
Received: from relay162.nicmail.ru (relay162.nicmail.ru [91.189.117.6])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id 216C77B0D
for <pve-user@lists.proxmox.com>; Wed, 19 Jan 2022 12:22:37 +0100 (CET)
Received: from [10.28.138.149] (port"082 helo=[192.168.8.155])
by relay.hosting.mail.nic.ru with esmtp (Exim 5.55)
(envelope-from <tsabolov@t8.ru>) id 1nA92r-0007Xi-9f
for pve-user@lists.proxmox.com; Wed, 19 Jan 2022 14:22:30 +0300
Received: from [62.105.41.93] (account tsabolov@t8.ru HELO [192.168.8.155])
by incarp1102.int.hosting.nic.ru (Exim 5.55)
with id 1nA92r-0004wk-S3 for pve-user@lists.proxmox.com;
Wed, 19 Jan 2022 14:22:29 +0300
To: Proxmox VE user list <pve-user@lists.proxmox.com>
From: =?UTF-8?B?0KHQtdGA0LPQtdC5INCm0LDQsdC+0LvQvtCy?= <tsabolov@t8.ru>
Message-ID: <58d43866-6451-c266-7985-07ff89c0e05a@t8.ru>
Date: Wed, 19 Jan 2022 14:22:29 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
Thunderbird/78.14.0
MIME-Version: 1.0
Content-Language: en-US
X-KLMS-AntiSpam-Auth: dkim=none
X-MS-Exchange-Organization-SCL: -1
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.665 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
HTML_MESSAGE 0.001 HTML included in message
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/,
no trust RCVD_IN_MSPIKE_H2 -0.001 Average reputation (+2)
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
Subject: [PVE-User] Unexpected reboot on of 6 node
X-BeenThere: pve-user@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE user list <pve-user.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>,
<mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-user/>
List-Post: <mailto:pve-user@lists.proxmox.com>
List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>,
<mailto:pve-user-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 19 Jan 2022 11:22:38 -0000
Hi,
Like in this old thread
https://forum.proxmox.com/threads/unexpected-reboots-help-need.34310/ I
have similar problem.
In cluster I have 7 node.
root@pve-1: pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.143-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-helper: 6.4-8
pve-kernel-5.4: 6.4-7
pve-kernel-5.4.143-1-pve: 5.4.143-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
ceph: 15.2.15-pve1~bpo10
ceph-fuse: 15.2.15-pve1~bpo10
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve4~bpo10
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.22-pve1~bpo10+1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-4
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.13-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.3-2
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.6-pve1~bpo10+1
I need to disable complete the IPMI Watchdog or Dell IDrac (module
"ipmi_watchdog") For Dell IDrac, please desactivate the Automated System
Recovery Agent in IDrac configuration.
https://pve.proxmox.com/wiki/High_Availability_Cluster_4.x#IPMI_Watchdog_.28module_.22ipmi_watchdog.22.29
Sergey TS
The best Regard
_______________________________________________
pve-user mailing list
pve-user@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PVE-User] Force reclaiming space on a vdisk...
2022-01-19 8:49 ` Daniel Plominski
@ 2022-01-20 10:47 ` Marco Gaiarin
[not found] ` <qmMMMDznVErHvtLDASvXi93ArFgsrEDwDgYRRb1tvWRv7rmQTETQI8owpr6NthnRc4Ti1mgm4XjK-QciST-hyaWdPwjq2OpIc_7yEM5_b0Q=@protonmail.ch>
[not found] ` <mailman.129.1642678747.436.pve-user@lists.proxmox.com>
0 siblings, 2 replies; 6+ messages in thread
From: Marco Gaiarin @ 2022-01-20 10:47 UTC (permalink / raw)
To: Daniel Plominski; +Cc: pve-user
Mandi! Daniel Plominski
In chel di` si favelave...
> enable ZFS set compression=lz4 on the zvolume
Seems just enabled:
root@ctpve1:~# zpool get all rpool | grep lz4
rpool feature@lz4_compress active local
root@ctpve1:~# zpool get all rpool-data | grep lz4
rpool-data feature@lz4_compress active local
> and perform a memory limit dd run (inside the vm).
> dd if=/dev/zero of=/mountpoint_sda7/CLEANUP bs=99M count=xxx
> (this will release the storage space)
OK. But i've just deleted the partitions. I have to create a new partition,
format them, create a 'dd-zero' file in them and then the space will be
released?
Really?!
I'm asking because i supposed that was the 'trim/thin' feature of ZFS to
permit to shrink a disk (eg, 'don't save the unallocated space'), not the
compressione feature (eg, 'don't save a bunch of consecutive zero, compress
it').
Speaking more clearly, i hope: i've perfectly clear that 'zeroing' a portion
of a disk permit the compression feature of zfs to compress it, but i
supposed that was the management of allocated spaces that make their
business here... i'm only a bit puzzled.
I hope someone can clarify, thanks.
PS: this server have a 'Proxmox VE Community Subscription 1 CPU/year'
currently active on, but i prefere if possible to use mailing list for this
support question. FYI.
--
Ognuno vada dove vuole andare, ognuno invecchi come gli pare
ma non raccontate a me che cos'e` la LIBERTA`. (F. Guccini)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PVE-User] Force reclaiming space on a vdisk...
[not found] ` <qmMMMDznVErHvtLDASvXi93ArFgsrEDwDgYRRb1tvWRv7rmQTETQI8owpr6NthnRc4Ti1mgm4XjK-QciST-hyaWdPwjq2OpIc_7yEM5_b0Q=@protonmail.ch>
@ 2022-01-20 12:23 ` Daniel Plominski
0 siblings, 0 replies; 6+ messages in thread
From: Daniel Plominski @ 2022-01-20 12:23 UTC (permalink / raw)
To: Proxmox VE user list
Hello,
we have had very bad experience with trim, due to a virtio driver error,
a special release, the trim resulted in a corrupt Windows NTFS file
system.
The bug is fixed in the latest stable VirtIO drivers, but we only use
the legacy method using targeted "dd" (zero) cleanup on Linux / BSD VMs
and on Windows using "fsutil".
From the perspective of the HOST system, a zvolume formatted with
filesystem and stored data is simply a "data container" with
unstructured data (depending on the structure, however, this can be
compressed well).
If now within the VM an area with zeros is written, this zvolume area is
stimulated to overwrite the contained data, the process is compressed
and the occupied blocks are released.
The block free up should also work with the ZFS algorithm "zle".
How often the cells of an SSD / NVME memory are overwritten depends on
the internal intelligence of the firmware / controller of the respective
memory.
TRIM "Discard" (if supported by the hardware, the hypervisor, the
hypervisor driver and the guest OS) can be used to better control this
remapping.
ASS - Der Bildungseinrichter GmbH
Mit freundlichen Grüßen
DANIEL PLOMINSKI
Leitung IT | Head of IT
Telefon 09265 808-151 | Mobil 0151 58026316 | dpl@ass.de
PGP Key: https://pgp.ass.de/dpl@ass.de.asc
PGP Fingerprint: 74DBC06BD9F63187C4DF8934C96585A89CFC10B3
Company Logo
ASS-Einrichtungssysteme GmbH
ASS-Adam-Stegner-Straße 19 | D-96342 Stockheim
Geschäftsführer: Matthias Stegner, Michael Stegner, Ralph Müller
Amtsgericht Coburg HRB 3395 | Ust-ID: DE218715721
Bottom_Line
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PVE-User] Force reclaiming space on a vdisk...
[not found] ` <mailman.129.1642678747.436.pve-user@lists.proxmox.com>
@ 2022-01-21 10:03 ` Marco Gaiarin
2022-01-25 10:44 ` Marco Gaiarin
0 siblings, 1 reply; 6+ messages in thread
From: Marco Gaiarin @ 2022-01-21 10:03 UTC (permalink / raw)
To: Arjen via pve-user; +Cc: pve-user
Mandi! Arjen via pve-user
In chel di` si favelave...
> When discard is enabled on the virtual disk and also the OS inside the VM executes trim commands for deleted data, then it should inform ZFS that the blocks are free. Marking blocks as usused will not work without the help of the OS inside the VM.
OK. So, speking practically:
a) i've do the wrong things, deleting the partitions, because the guest OS
had not the opportunity to reclaim free space; because trim/discard need
the cooperation of all the 'chain', probably it was needed to delete all the
file in the partition, the trim them, then delete them.
b) if the guest OS does not support trim, or support wrongly/buggy, the same
result will be achieved (creating and) zeroing the partition, if
conpression are enabled. This does not involve trim/discard, but only
compression, acheving the same result indeed.
> Was trimming enabled for the OS inside the VM? Otherwise, you need to create a partition and trim it inside the VM.
Guest os support fstrim, so i'll try a).
Thanks.
--
Con Windows sei in vacanza: ti diverti senza pensare a ciò che fai,
ma dopo un po' finisce. In Linux entri nella vita reale:
Devi tirar fuori le palle! (Alain Modolo)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PVE-User] Force reclaiming space on a vdisk...
2022-01-21 10:03 ` Marco Gaiarin
@ 2022-01-25 10:44 ` Marco Gaiarin
0 siblings, 0 replies; 6+ messages in thread
From: Marco Gaiarin @ 2022-01-25 10:44 UTC (permalink / raw)
To: Marco Gaiarin; +Cc: pve-user
> Guest os support fstrim, so i'll try a).
For the archive: a) worked.
Also, formatting (i've used mke2fs) trigger trim, so i was not forced to
format and thet trim the disk, only format.
Thanks.
--
Utopia aveva una sorella maggiore,
che si chiamava Verita` senza errore (Nomadi)
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-25 20:10 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-19 8:19 [PVE-User] Force reclaiming space on a vdisk Marco Gaiarin
2022-01-19 8:49 ` Daniel Plominski
2022-01-20 10:47 ` Marco Gaiarin
[not found] ` <qmMMMDznVErHvtLDASvXi93ArFgsrEDwDgYRRb1tvWRv7rmQTETQI8owpr6NthnRc4Ti1mgm4XjK-QciST-hyaWdPwjq2OpIc_7yEM5_b0Q=@protonmail.ch>
2022-01-20 12:23 ` Daniel Plominski
[not found] ` <mailman.129.1642678747.436.pve-user@lists.proxmox.com>
2022-01-21 10:03 ` Marco Gaiarin
2022-01-25 10:44 ` Marco Gaiarin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox