From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id BAC6C1FF16D
for <inbox@lore.proxmox.com>; Mon, 19 Aug 2024 15:37:44 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 5EAFC18BF2;
Mon, 19 Aug 2024 15:38:03 +0200 (CEST)
Date: Mon, 19 Aug 2024 15:37:43 +0200
To: pve-devel@lists.proxmox.com
References: <mailman.277.1723799872.302.pve-devel@lists.proxmox.com>
<173827238.15666.1723819383697@webmail.proxmox.com>
<mailman.301.1724049474.302.pve-devel@lists.proxmox.com>
In-Reply-To: <mailman.301.1724049474.302.pve-devel@lists.proxmox.com>
MIME-Version: 1.0
Message-ID: <mailman.307.1724074682.302.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: sjanssens via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: sjanssens <sjanssens@evolix.fr>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: Re: [pve-devel] DRBD integration but without LINSTOR
Content-Type: multipart/mixed; boundary="===============6027211597320485527=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
This is a multi-part message in MIME format.
--===============6027211597320485527==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <sjanssens@evolix.fr>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 96EA8C51A0
for <pve-devel@lists.proxmox.com>; Mon, 19 Aug 2024 15:38:01 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 7541518BCF
for <pve-devel@lists.proxmox.com>; Mon, 19 Aug 2024 15:38:01 +0200 (CEST)
Received: from pele.evolix.net (reverse.v6.evolix.net [IPv6:2a01:9500:37:1:d6ae:52ff:fe79:5382])
(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Mon, 19 Aug 2024 15:37:59 +0200 (CEST)
Received: from [192.168.4.141] (octo.evolix.net [192.168.4.141])
by pele.evolix.net (Postfix) with ESMTPSA id 1D8E27F490
for <pve-devel@lists.proxmox.com>; Mon, 19 Aug 2024 15:37:53 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=evolix.fr; s=dkim2018;
t=1724074673; bh=Lzh/j4yfwd6vRywPx42DZpBJRlSLES2bVkcvo5pu49E=;
h=Date:Subject:To:References:From:In-Reply-To;
b=pBMnnVHJBwAbYZCEHmj9oo5yqusi2M7aL3vMoP+YvQ9vR0oD/9s6RglXM9SMfv9St
B4mZU3w94HqiwiqRbcSwYXE1kzf6wCNKeA1BqRmI7cIz3LRFWPhQCVIkNrtMFj/opg
yoVJ9Pt8Epwbv2scIexxqO9LLCd4//tu3dXMVfe8=
Message-ID: <6b8329fd-1fd3-409a-aeb6-7d341dc0ee5d@evolix.fr>
Date: Mon, 19 Aug 2024 15:37:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [pve-devel] DRBD integration but without LINSTOR
To: pve-devel@lists.proxmox.com
References: <mailman.277.1723799872.302.pve-devel@lists.proxmox.com>
<173827238.15666.1723819383697@webmail.proxmox.com>
<mailman.301.1724049474.302.pve-devel@lists.proxmox.com>
Content-Language: en-US
From: sjanssens <sjanssens@evolix.fr>
In-Reply-To: <mailman.301.1724049474.302.pve-devel@lists.proxmox.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.750 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain
DMARC_PASS -0.1 DMARC pass policy
FORGED_SPF_HELO 2.499 -
HTML_MESSAGE 0.001 HTML included in message
SPF_HELO_PASS -0.001 SPF: HELO matches SPF record
T_SCC_BODY_TEXT_LINE -0.01 -
T_SPF_PERMERROR 0.01 SPF: test of record failed (permerror)
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
I am about to complete my internship at Evolix, during which I worked on
a project to integrate DRBD into Proxmox. This involved manipulating
DRBD resources and using basic drbd-utils tools. I have now completed
the project and uploaded it to Gitea. I am wondering if there is any
interest in sharing this with the community.
Here are the repository links:
* https://gitea.evolix.org/evolix/pve-drbd (new package)
* https://gitea.evolix.org/evolix/pve-manager/src/branch/evolix-drbd
(fork of pve-manager)
* https://gitea.evolix.org/evolix/pve-docs/src/branch/debian (fork of
pve-docs)
* https://gitea.evolix.org/evolix/pve-storage (fork of pve-storage)
I am still working on the repositories (particularly to implement
gitbuildpackage).
I am including a part of the README below for you to review if you have
time:
1. Introduction
Objective
This documentation aims to detail the modifications made to the Proxmox
source code for integrating DRBD. It provides information on the
modifications of existing packages as well as the creation of the new
pve-drbd package.
Background
The DRBD (Distributed Replicated Block Device) module has been
integrated into Proxmox to enhance resource and disk management for VMs.
This integration allows the manipulation of DRBD files and resources
through the Proxmox VE UI, facilitating the deployment of VMs on DRBD
devices.
Compatibility
This patch is compatible with both drbd8 and drbd9. Therefore, the
kernel module must be available, as well as the drbd-utils package (>=
9.22). You can find an installation guide for drbd9 on the Evolix Wiki.
2. Overview of Modifications
Summary
The modifications include updating the existing pve-manager,
pve-storage, and pve-docs packages, and creating a new pve-drbd package.
You can find the list of packages in the official Proxmox git: Proxmox Git.
Modified Packages
pve-manager: Management of Proxmox functions and user interfaces.
Complete user interface with EXTJS (JavaScript)
Base API calls (Perl), redirecting to the target packages.
pve-storage: Management of storage functionalities.
Definition and operation of 'storages' (for VM disk creation)
pve-docs: Updating documentation to reflect new features.
Documentation available with the API, and auto-generated for
the CLI (man pve-drbd)
pve-drbd: New package to integrate DRBD with Proxmox.
Management of API calls related to DRBD
CLI pve-drbd
3. Features
Integration of DRBD Storage Type for Proxmox VMs
We have added support for DRBD storage type, allowing seamless
integration of DRBD usage for virtual machines on Proxmox. DRBD is a
kernel-level block replication solution that ensures high availability
and data resilience by replicating data in real-time between cluster nodes.
Simplified Management of DRBD Resources via Proxmox UI
The management of DRBD resources is fully integrated into the Proxmox
user interface, facilitating the creation, modification, and management
of DRBD resources. This integration allows administrators to link DRBD
disks to VMs directly from the UI, providing a consistent and simplified
user experience. Users can quickly configure and manage their replicated
storage environments intuitively.
Command Line Interface (CLI) pve-drbd
In addition to the graphical interface, a dedicated CLI is also
implemented for advanced management of DRBD resources across nodes. The
pve-drbd CLI allows administrators to execute commands to create,
delete, modify, and monitor DRBD resources across all cluster nodes,
offering maximum flexibility for advanced users and automation scripts.
This CLI is particularly useful for repetitive tasks and large-scale
deployments where quick and efficient management is essential.
Custom Features
Hot migration of VMs based on DRBD is also integrated. It
automatically adds the target node to the resources linked to a VM and
transfers it using qm.
For drbd8, the DRBD dance is implemented. It allows transferring a
VM to another pair of nodes, replacing the old hosts with the newly
chosen ones in the resources.
4. Installation Instructions
Download the git proxmoxDRBD Git.
Download the source packages (pve-manager, pve-storage, pve-docs,
and pve-drbd) into packages (skip this step if you downloaded the git
with submodules).
Compile and install pve-docs, pve-drbd, pve-storage, and then
pve-manager (in the given order). Dependencies will be required.
5. Detailed Description of Modifications and Functionality
pve-manager
The pve-manager package consists of two parts:
Interface:
Modifications:
Utils.js: Added the button for creating DRBD storage and defining
its type (Datacenter -> storage -> add) as well as task messages.
storage/DRBDEdit.js: Storage creation window with its attributes
(linked resource, etc.)
NOTE: The API URL for storage creation is not explicitly written.
The DRBDInputPanel inherits from PVE.panel.StorageBase, which calls the
API by sending the type (here drbd), redirecting the calls to the
pve-storage package (see pve-storage functionality).
dc/DRBD*.js: All windows for DRBD (management of files, resources,
etc.)
dc/Config.js: Added the windows in the dropdown menu.
API Calls:
Modifications:
API2.pm: Added redirection to the API2/Drbd.pm file for
api2/json/drbd API calls.
API2/Drbd.pm: Redirects all DRBD requests to the appropriate Perl
modules, along with some utility calls.
pve-storage
Modifications:
Storage/DRBDPlugin.pm: New file adding support for DRBD storage.
Storage.pm: Added DRBD storage type in the central management of
storage types.
pve-docs
Modifications:
pvedrbd.adoc: Added documentation related to DRBD, accessible from
the Proxmox UI.
pve-drbd
Functionality:
New package for specific DRBD management. The package contains all
API calls and functions to manage resources, files, and utilities for
DRBD integration.
Main Files:
PVE/CLI/pve-drbd: CLI for DRBD management. Each command makes an
API call, processes the response (e.g., formatting it), and writes it to
standard output.
PVE/DRBD/: Contains all necessary functions for task execution.
PVE/API2/: Contains all methods for API calls.
On 8/19/24 08:37, Roland Kammerer via pve-devel wrote:
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--
Samuel Janssens<sjanssens@evolix.fr> - SysAdmin support Evolix
Evolix - Hébergement et Infogérance Open Source
Marseille (37 rue Guibal, Pôle Média, 13003) / Paris / Montréal
http://evolix.com | Twitter: @Evolix @EvolixNOC |http://blog.evolix.com
--===============6027211597320485527==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============6027211597320485527==--