Sys Admin Pocket Survival Guide - Warewulf
Home linUx Solaris HP-UX AIX NetApp EMC(NAS) EMC(SAN) Veritas LDAP

warewulf

baremetal provision tool. common in hpc environment pxe boot machine, most often run stateless nodes.

Ref

Good intro/overview
https://warewulf.readthedocs.io/en/latest/about/architecture.html

google group eg:
https://groups.google.com/a/lbl.gov/g/warewulf/c/Yx9OMqkyJKU

warewulf backup

warewulf has a mysql db, running daily dbdump is recommended.
This stores the node and provision config.
MySQL/MariaDB config is stored in /etc/warewulf/database*conf 


mysqldump --add-drop-database --opt $DB 
Where $DB is one of 
mysql 
warewulf
/var/chroot contents for the vnfs is NOT in the db, and separate store for that is necessary
(eg, vnfs build server, pullvnfs script presumed can run again to repopulate this).
(alt, tar up the existing chroot, xfer the file and untar)
db restore
Restore command (run as root):

bunzip2 -c mysql_backup_mysql_MMDD.sql.bz2    | mysql mysql
bunzip2 -c mysql_backup_warewulf_MMDD.sql.bz2 | mysql warewulf


A few commands to check the restored databases:

mysql -e "describe host" mysql
mysql -e "describe datastore" warewulf
mysql -e "SELECT COUNT(*) FROM user" mysql
mysql -e "SELECT COUNT(*) FROM datastore" warewulf

network troubleshooting


If nodes are not PXE booting:

Jul  9 10:40:50 wwulf dhcpd[3849]: DHCPDISCOVER from ff:7b:25:fa:4f:7e via enp0s3: network 10.15.4.0/24: no free leases
Jul  9 10:40:54 wwulf dhcpd[3849]: DHCPDISCOVER from ff:7b:25:fa:4f:7e via enp0s3: network 10.15.4.0/24: no free leases

then try
wwsh -v dhcp update
wwsh dhcp restart
wwsh pxe  update

write to /etc/dhcp/dhpcd.conf
if config has:

   # Evaluating Warewulf node: c0000 (DB ID:887)
   # Skipping c0000: Not on boot network (10.15.14.0)

then:
check /etc/warewulf/provision.conf
the NIC device defined there need to have a network that is within the IP range defined by the wwulf nodes.

Also see:
https://groups.google.com/a/lbl.gov/g/warewulf/c/ERzlDkDw2tY/m/rg-fvAoOacQJ

wwsh bootstrap rebuild
	# recreate files needed by pxe boot, stored in /srv/warewulf/bootstrap
Notes/Ref
  • VirtualBox iPXE ROM does not support bzImage, but required by some OS like CoreOS (and CentOS?) cuz initramfs is a cpio.bz image. Thus generating ipxe err 23008001 https://github.com/coreos/tectonic-installer/issues/932
  • Have VBox use EFI instead of bios mode, see https://www.makeuseof.com/set-up-efi-linux-virtual-machine-virtualbox/
  • virt manager would be closer to hardware, but a time sink to setup
    proxmox vm should also work fine.

    Files on boot server

    ensure these files are present. UEFI may need additional config. 
    
/var/lib/tftpboot/warewulf/
/var/lib/tftpboot/warewulf/ipxe
/var/lib/tftpboot/warewulf/ipxe/bin-i386-pcbios
/var/lib/tftpboot/warewulf/ipxe/bin-i386-pcbios/undionly.kpxe
/var/lib/tftpboot/warewulf/ipxe/bin-i386-pcbios/undionly.kpxe-old
/var/lib/tftpboot/warewulf/ipxe/bin-x86_64-efi
/var/lib/tftpboot/warewulf/ipxe/bin-x86_64-efi/snp.efi
/var/lib/tftpboot/warewulf/ipxe/bin-i386-efi
/var/lib/tftpboot/warewulf/ipxe/bin-i386-efi/snp.efi


/srv/warewulf/initramfs/x86_64/capabilities/provision-adhoc
/srv/warewulf/initramfs/x86_64/capabilities/provision-files
/srv/warewulf/initramfs/x86_64/capabilities/provision-selinux
/srv/warewulf/initramfs/x86_64/capabilities/provision-vnfs
/srv/warewulf/initramfs/x86_64/capabilities/setup-filesystems
/srv/warewulf/initramfs/x86_64/capabilities/transport-http
/srv/warewulf/initramfs/x86_64/capabilities/setup-ipmi

# wwsh bootstrap rebuild # should generate a series of files like:
/srv/warewulf/bootstrap/x86_64/6/kernel
/srv/warewulf/bootstrap/x86_64/6/cookie
/srv/warewulf/bootstrap/x86_64/6/initfs.gz

/srv/warewulf/ipxe/cfg/ac:1f:6b:a5:9c:f6

    special object config

    custom object 
    

wwsh object modify -s LASTOCTET=5   s00
wwsh object modify -s LASTOCTET=100 n00
wwsh object modify -s LASTOCTET=101 n01

wwsh object print -p :all | egrep 'node|name|LASTOCTET'

wwsh --debug file sync # debug mode pring lot of perl code state. eg grep ERROR or WARNING

    config troubleshooting

    wwsh file sync barf like:
WARNING in Warewulf::DataStore::SQL::BaseClass->persist()/861:  Unable to execute set lookup query: Data too long for column 'value' at row 1

or

wwsh file import  /etc/warewulf/files/node_exporter.service --mode=0644 --path=/etc/systemd/system/multi-user.target.wants/node_Exporter.service
wwsh file set node_exporter.service --path=/etc/systemd/system/multi-user.target.wants/node_Exporter.service

WARNING:  Unable to execute set lookup query: Data too long for column 'value' at row 1

it is cuz Destination PATH is limited to at most 65 chars!  use a shorter PATH :-\

    tftp troubleshooting

    tftp (and dhcpd) needs to be running on the warewulf server. see general_unix.html#tftp for troubleshooting info.

    TBD int ref

    httpd troubleshooting

    Once PXE boot complete via tftp, the image invoke HTTP GET to fetch files from the warewulf servers. these files are expected to be stored in: /srv/warewulf


    Typewriter monospaced fonts in here. This is another line.



    [Doc URL: http://tin6150.github.io/psg/warewulf.html ]
    Last Updated: 2021-07-20
    (cc) Tin Ho. See main page for copyright info.

    hoti1
    sn5050
    psg101 sn50 tin6150