Re: [pve-devel] RE : pve-devel Digest, Vol 132, Issue 53

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

On 24.05.21 23:45, wb wrote:
> However, since I am starting on a new installation, I am surprised to get this kind of answer.
> « Your cluster fs is not working (pmxcfs). See you run on a broken installation. »
> Or 
> « You need a working PVE installation before doing any API calls... »
> 
> With the following command, I have the process up!
> 
> ps aux | grep pmxcfs
> 

running does not mean working...

What's the output/status of:

# systemctl status pve-cluster 
# touch /etc/pve/foo
# findmnt /etc/pve

> 
> I think I have enough knowledge about SAML and Perl to do it, however, the support of a dev would be ideal at least on the lock part.
> 

Nobody questioned that..

> I'm trying to implement a new api so that Proxmox authentication works with SAMLv2.

Yes, as you stated in the initial mail..

> 
> I would have preferred to have more info on the following part :
> # this is just a readonly copy, the relevant one is in status.c from pmxcfs
> # observed files are the one we can get directly through IPCC, they are cached
> # using a computed version and only those can be used by the cfs_*_file methods
> 

I'd suggest ignoring the pmxcfs internal optimized cache-using part, you do not need
that for a start, just use the common file_get_content / file_set_content helper from
the PVE::Tools module, you could do everything with those for now and only then
migrate to a optimized cfs_*_{read,write} helper.

> To try to bring a little more element, I added a file to the following list in the PVE::Cluster file
> my $observed = {
>     'request.tmp' => 1,
> 
> Still in the PVE::Cluster file, It is well in the following part that it blocks :
> 
> 
> If I take the error message from the first email,
> «  error during cfs-locked \'file-request_tmp\' operation: pve cluster filesystem not online /etc/pve/priv/lock. »
> If I test the dir /etc/pve/priv/lock, it exists!

Existence is not a problem, pmxcfs is a clustered realtime configuration filesystem,
it either may not be mounted (and again, running is not always a 100% guarantee that
it is still mounted) or in a cluster (or thinking that's in a cluster due to
`/etc/corosync/corosync.conf` and/or `/etc/pve/corosync.conf` existing) but has no
quorum, i.e., read-only

> 
> Do the files we add in PVE::Cluster file need to be listed in /var/lib/pve-cluster/config.db, if so, any spec please?

no, that's the backing DB, I'd heavily recommend not modifying that one directly if
unsure. Those files get always created on the FUSE VFS layer (besides the very barebone
initial one we create with a small helper).

Note: you need the correct permissions in your service, it must be in www-data group
to be able to read/test directory existance and run as root for writing.