Ceph install Prepare the host sudo zypper in podman Install cephadm SUSE The suse’s build-in ceph is out of date, so use the curl install instead CEPH_RELEASE=18.2.0 # replace this with the active release curl --silent --remote-name --location https://download.ceph.com/rpm-${CEPH_RELEASE}/el9/noarch/cephadm chmod +x cephadm RHEL sudo dnf install --assumeyes centos-release-ceph-reef sudo dnf install --assumeyes cephadm Bootstrap the cluster #cephadm bootstrap --mon-ip *<mon-ip>* --cluster-network *<secondary ceph network>* sudo ./cephadm bootstrap --mon-ip 10.240.1.101 --cluster-network 10.

The Mellanox SX6036 is a 36 ports switch that support both infinitband and price is very good on ebay. (I bought mine at $200 + $50 for the ethernet activation) Manual: https://delivery04.dhe.ibm.com/sar/CMA/XSA/MLNX-OS_VPI_v3_4_3002_UM.pdf reset the device in case the previous password is still there Press the RST key and wait. (Note, it takes almost 3 minutes to reset mine instead of 15s as memtioned in the manual). After the reset, the password would be admin/admin.

The following command can be used to cross flash the Mellanox MCX314A-BCCT (Ethernet only) to Mellanox MCX354A-FCCT (VPI). download and unzip the firmware https://network.nvidia.com/support/firmware/connectx3proib/ wget http://www.mellanox.com/downloads/firmware/fw-ConnectX3Pro-rel-2_42_5000-MCX354A-FCC_Ax-FlexBoot-3.4.752.bin.zip unzip fw-ConnectX3Pro-rel-2_42_5000-MCX354A-FCC_Ax-FlexBoot-3.4.752.bin.zip install flashing tool opensuse: sudo zypper install mstflint get the pcie id sudo lspci | grep Mellanox 03:00.0 Network controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] This 03:00.0 should be passed into the mstflint’s -d parameter. flash the firmware sudo mstflint -nofs --ignore_dev_data --guid=000000000000050 --use_image_ps --allow_psid_change -d03:00.