Originally I had tried a software update on a Dell 2950 III server from Rocky Linux 8 to 9, only to end up with rocky linux , “glibc error: cpu does not support x86-64-v2”. Basically I fried my entire system as this CPU cannot support these new CPU calls.
Today I am going to walk you through a complete Rocky Linux 2022 install, which now replaces Centos from original Centos creator. Today I put in three different USB sticks containing Oracle Linux, Rocky Linux and Centos Stream Linux to see if I could install over my existing partitions as to not wipe out my FreeBSD KVM on the LVM. Well today I am sad to say, the installer does not see the LVM partitions, so I am forced to reinstall Rocky Linux as well as my guests all over again.
I am going to walk you through a safer way to install Centos(Rocky Linux), to future proof yourself in case of reinstalls or problems.
Rocky Linux 8 vs 9:
Run the following shell script:
pico glibc_check.sh (add following)
#!/usr/bin/awk -f
BEGIN {
while (!/flags/) if (getline < "/proc/cpuinfo" != 1) exit 1
if (/lm/&&/cmov/&&/cx8/&&/fpu/&&/fxsr/&&/mmx/&&/syscall/&&/sse2/) level = 1
if (level == 1 && /cx16/&&/lahf/&&/popcnt/&&/sse4_1/&&/sse4_2/&&/ssse3/) level = 2
if (level == 2&&/avx/&&/avx2/&&/bmi1/&&/bmi2/&&/f16c/&&/fma/&&/abm/&&/movbe/&&/xsave/) level = 3
if (level == 3 && /avx512f/&&/avx512bw/&&/avx512cd/&&/avx512dq/&&/avx512vl/) level = 4
host:~ # ./glibc_check.sh CPU supports x86-64-v1 host:~ #
Now if you get like the above with only version 1, you can only install Rocky Linux 8, otherwise if you have version 2 or higher you can install Rocky Linux 9. I believe reason RHEL made this change was to make glibc calls faster.
INSTALL Rocky Linux:
Download latest Rocky Linux ISO and install it with rufus to a USB stick. I am not going to cover a simple graphics installer, but I want to cover partitioning your drive. Let’s go to “Installation Destination” on installer. Now to prevent any future wipeouts of LVMs with installers, we are going to use standard partitions and only install our LVM afterwards manually. This way if we every run into situation again where installer cannot see into our LVM install, it will definitely see our standard partitions and we won’t have to wipe out our LVMs ever again.
Here are my recommendations for /boot, / and swap. For /boot from experience 1 GB is not enough like they say, after you start installing enough kernels, or start adding custom kernels for supporting things like ZFS or Ksmbd, it adds up quickly. So my recommendation for /boot is 3 GB. For swap it is 20% of your memory, I have 32 GB on this server, but I am going to go 8GB, as I rarely like going under that these days. For / partition we will want at least 80-100 GB. At some point you are going to run out of space un-tarring enough kernel sources, or doing 4k file tests, or space for your ISOs, you need some breathing room on your main OS!
Save with this setup and finish your install finishing up your install packages, network, password and so on. What we are going to do is setup sda4 manually after the install for our LVM. Double check everything, make sure they are all ext4 partitions, and there is no LVM anywhere!!!
Post Install Tasks — setting up for LVM, KVM, wireguard:
Let us start by upgrading the system, setting up for KVM, and upgrading the kernel from stock default. Remember we cannot run a lot of thing with less than a 5.15.x kernel, and if you start getting into things like ZFS we would need to be exactly on 5.15.x kernel currently. For our purposes we will just use kernel-ml and can downgrade to 5.15.x for ZFS later if we choose to manually compile our own kernels
dnf update
shutdown -r now (reboot with updated system)
cat /proc/cpuinfo | egrep "vmx|svm" (check we have virtualization enabled)
dnf install @virt virt-top libguestfs-tools virt-install virt-manager xauth virt-viewer
systemctl enable --now libvirtd (everything working? ifconfig -a)
#get rid of virbr0 in ifconfig
virsh net-destroy default
virsh net-undefine default
service libvirtd restart
(check https://www.elrepo.org/ for below install command for rocky 8 or 9)
#we will not be able to install wireguard with less than 5.15 kernel
yum install https://www.elrepo.org/elrepo-release-8.el8.elrepo.noarch.rpm
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
dnf makecache
dnf --enablerepo="elrepo-kernel" install -y kernel-ml
#let's setup our br0 bridge before we reboot
cd /etc/sysconfig/network-scripts
nano ifcfg-enp10s0f0 (your <device name>
#add "BRIDGE=br0" at end of this file
nano ifcfg-br0
#add something like this:
#should match a lot of your file above
STP=no
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.0.2
GATEWAY=192.168.0.1
PREFIX=24
DNS1=192.168.0.3
DNS2=8.8.8.8
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
NAME=br0
UUID=36e9e2ce-47h1-4e02-ab76-34772d136a21
DEVICE=br0
ONBOOT=yes
AUTOCONNECT_SLAVES=yes
IPV6INIT=no
#change UID to something different above and put your own IPs in
shutdown -r now (reboot with new kernel)
ifconfig -a (check br0 is there, then all good)
Ok now we have new kernel and br0 is setup for KVM guests, let’s move on to LVM.
Creating our LVM for our KVM guests:
#make sure we have right device
fdisk -l /dev/sda
fdisk /dev/sda
p
<enter> (should have partition 4 and adding rest of disk space to it)
t (toggle partition 4 - change from linux to LVM)
8e (change to LVM)
w (write changes to partition tables)
partprobe /dev/sda (inform OS of partition changes)
pvcreate /dev/sda4 (now we have it as a LVM-can check with "pvs")
vgcreate vps /dev/sda4 (creating our volume group - "vgdisplay")
#now we are all setup, we can create as many KVM guests as we want
#for example give 70G to one guest and give remaining space to a devel guest
lvcreate -n cappy -L 70G vps (create a 70G guest - "lvdisplay")
lvcreate -n devel -l 100%FREE vps(give remaining space to this guest)
#can always delete it later
pvdisplay (check we used up all the space on vps)
#let's make sure guests can suspend and resume on host reboots:
pico /etc/sysconfig/libvirt-guests
"ON_SHUTDOWN=suspend"
systemctl start libvirt-guests
systemctl enable libvirt-guests
Well my two marine batteries died I had used as a backup for servers. What I had done for my servers at home was modified a UPS and hooked up two 100ah marine batteries in parallel for 200ah of capacity. While this was a great cheap solution at the time, 2 marine batteries were 100 dollars each and a modified UPS to run off them was a great idea at the time.
New tech in batteries – LifePo4(Lithium Iron Phosphate)
I was using alkaline 12V batteries that long ago, but now with the new contender on the market Lifepo4, short for Lithium Iron Phosphate we really need to look at why they are so much better for our homes than the old alkaline batteries. I had to watch countless YouTube videos on electrical engineering to get to bottom of this. To basically sum it up, let’s say we have one 12 volt car battery at 100ah(amp hours), and we have one 12 volt 100ah Lifepo4 battery. Alkaline batteries need to be charged once they reach 50% capacity, Lifepo4 batteries can be completely discharged and still run loads needed.
The recommendation is to charge Lifepov4 batteries by time it hits 20% so you don’t drain it completely. Also alkaline batteries die after about 500 cycles, where the new Lifepov4 standard can last as long as 6000 cycles! So I have found Lifepov4 batteries will not only last you longer, but you can use them longer before they are completely discharged, and for an added bonus, they are safe in your house because they won’t cause fires like other batteries will!
This basically all means then a 200ah alkaline battery would equal the performance of capacity of a 100ah Lifepov4 battery, and will last longer, not only 500 cycles! Definitely then worth twice the cost of an alkaline battery.
Current pricing of Lifepov4:
Alright that is all nice and all, but let’s look at real world prices of replacing my alkaline 200ah setup. The current best priced 100ah Lifepov4 battery on amazon is:
This one
WOW $570 + tax CDN@!!! So what $650 CDN just for a 100ah battery! And this gets worse as we go to 200ah batteries at 1k or more, and 300ah batteries at nearly 2k it seems! Ok this is completely unacceptable, we are being robbed in the USA and Canada on these China made cells in these, what if we build our own?
What are current stats and prices from China?:
Checkout following link from reputable supplier in China on Alibaba:
China sells them by the cell at 3.4 volts and a certain amount of AH(amp hours) for each cell. So a quick lesson in electrical engineering is in order to understand some of this. There are two ways to hookup batteries to each other, in series or in parallel. So if we had 4 of these 3.4 volt China cells in series, you add the voltage together and the amp hours stay the same. So in this case if we took 4 of these and connected them all together we would have the 12 volt battery we need. If we instead hooked them up in parallel, the volts stay the same but the amp hours increase. So in parallel we would only have 3.4 volts, not what we want. We want to start at a 12 volt battery and get the most AH for our money.
So what we need is 4 of these cells to build a 12 volt battery, then in future we can build another one and hook those two up in parallel to increase capacity if we want. For series connection all we do is hook each battery up negative to positive with cables, for parallel all we do is hookup terminals on each battery negative to negative and positive to positive with cables or bus bars, whatever you prefer, easy enough right? China supplies bus bars generally so we good there, we don’t have to go off and buy 2 gauge cables off Amazon or anything.
Another quick lesson, the total Power of a battery would always be the formula:
Watts = Volts x Amps , so to get any of those values all we need is 2 of them: ie: to get amps knowing watts and volts, we would just do watts/volts=amps and so on.
Grade A vs Grade B cells from China:
People all over the internet will argue over Grade A vs Grade B EVE cells, basically Grade A someone put in the effort to fully test them, discharge and charge and will charge you double for that. With Grade B they do same tests, but don’t go through effort of fully discharging and charging them, but it is from same production line, so for cost effectiveness we want to go with Grade B obviously and take our chances. They should all be same volts when we get them, only thing they didn’t test what capacity with fully charging and discharging them, so that means we could get a better battery to lol. So if it is for home use, use Grade B for price alone, if for a business go for Grade A.
Let’s pick a cell from China:
If you look at that Alibaba webpage from above, we know Amy is reputable, may take us two months to get our batteries, but cost wise it will be better. Let’s take a quote from the forum link above about someone’s experience:
“I placed my order of 4 x EVE LF280K with Amy on June 7th. Shipment was two boxes (2 batteries per box) and box#1 arrived July 28th and box#2 followed the next day on July 29th (52 days delivery time to Toronto Canada).
The cells were described by Amy as “LF280K, brand new. Grade B —the voltage and internal resistance are matched, the capacity is not. The actual capacity is 275AH-284AH. QR code has B stamp , $111/pcs”.
Shipping was $140 and final delivery made via UPS. Tracking number was provided when order was placed, but package could not be tracked until it arrived in Canada (by sea).
Transaction was super smooth and went without a hitch. Thanks to members of this forum for recommending Amy as a reliable and trustworthy source. Will definitely order from her again.
I’m a degenerate gambler, so I’m looking forward to running capacity tests on each cell to see if I got a good batch or not “
Wow he paid a mere $650 CDN total for a 280AH battery! That would cost us 2k on amazon or anywhere else!
Is that all or do we need more?:
Technically you could just do that and be fine but there are actually two more things needed for a Lifepov4 battery: a BMS and a charger.
A BMS(Battery management system) is a card you buy online that has little wires hooked up to all the battery terminals, think of it as something that protects the battery from overcharging, over discharging, temperature cutoffs and so forth, it is basically a good thing to put on the battery. They run cheap crappy ones, and hundred or 2 for the good ones that even come with a Bluetooth app for your phone giving you all detailed stats about your new Lifepov4 battery you put together. It is a good idea and would recommend it. Last thing we need is a charger meant for Lifepov4, and we are set.
In all honesty, I think a good quality BMS, charger, and perhaps a box to put the battery in when your finished is a good idea. If BMS dies, charger dies, or battery cell dies, it’s as easy as just pulling it out and replacing it, unlike a normal battery where they put so much glue and stuff on it to prevent you from opening it that you’d have to toss the whole thing out.
Putting it all together:
Ok this is definitely not the faint of heart, in order to now change our setup from modified UPS using 2 alkaline batteries we have to rethink our whole setup for Lifepo4. So the UPS will need to be removed, we will need the following in CDN dollars:(take off 25% if USD)
a) Inverter(1500 watts or more) $400
b) Automatic Transfer Switch $200
c) 280 AH Lifepov4 battery $650-800 + $2-300 for BMS card and charger
So is it even worth it? Our upfront costs are already $1500-2k to replace our simple setup, why not just go to Costco and pay $400 for 2 more batteries that will die on us again. Also why not just get a UPS that supports Lifepov4 instead of going inverter/transfer switch route.
Here is why, a UPS that supports lifepov4 is very expensive, I believe Tripp Lite makes one, I think their top model supports around 750 watts and your going to pay 1k for something that can’t even handle a microwave oven or many servers running at same time? What if the power grid goes out, I would definitely need to be able to run around 1000 watts safely for servers, routers, internet, TV etc in livingroom. Of course you could get a top of the line one that supports more, but then your paying thousands for a rackmount option, not worth it.
How about we keep our inverter separate from rest of system that way we can upgrade it anytime we want, also add more lifepov4 batteries in parallel in the future if we want more capacity. This one 12V battery at 280 AH is like 6 alkaline batteries in parallel and is lighter to boot! $200 a piece from Costco = $1200 in alkaline batteries that would just die on you again!
Keep in mind we will stay 12V with our 1500 watt inverter or even a 2000 watt inverter, it is only when you start scaling up to 3000 watt inverters we would have to start scaling our batteries to 24V or even 48V if you wanted to run a whole house.
I left the best thing for last:
This new setup would allow you to do so much more, we could toss in a solar charge controller for a couple hundred dollars and some used solar panels after to the mix, and get free energy now if we wanted. We could switch our our automatic transfer switch to use the batteries when they are full from solar energy, and switch back when our batteries hit 20% discharge! I bet that sounds like fun, paying less in electricity bills just because you did a proper setup. Also you can check battery status from your phone at all times vs buying one off amazon that includes nothing!
Let’s get to some links for our parts list already!:
Let’s start with 1500 watt inverter, this will be cheapest we will find for good quality, even if we go new still be cheaper than amazon. There are cheaper 1200 watt inverters on amazon, but I’d feel more comfortable with 1500 watts. If you have a lot of money go with an expertpower expensive inverter, they have transfer switch and charger built into them already, but I don’t think it’s worth it, what if your inverter dies, that is an expensive replacement then, but if money is no issue get a 3000 watt inverter from them. Why did I pick this one? Because you find a 1500 watt inverter with a pure sine wave not modified sine wave(the former protects your electronics) for a better price and quality parts 🙂
b) Automatic Transfer Switch:
This will allow us to use the grid and switch to inverter batteries if grid goes down on us so our servers/PC’s stay on and whatever else you want for up to 1500 watts of the inverter we picked. Keep in mind on this you have to change a jumper setting for an instant switch as it’s set for a generator with a time delay when you get it. Also they have a pre-wired version so you don’t have to hack up extension cords if you choose on US amazon site. I believe this one will support up to a 2000 watt inverter, they have a 50 amp version as well.
This is just a smart choice, if our inverter ever dies or we want to upgrade to 2000 watts, it is a quick swap out, and we don’t have to pay for an expensive inverter with a transfer switch already built it saving us money again. For reference $300CDN vs $7-800 is a big deal especially it it dies one day, I’d rather put that money towards an inverter that has more watts.
Talk to Amy on Alibaba about some Grade B cells, also explore her links maybe get a JBD BMS(I hear the overkill BMS is just that one anyways) and a battery box to put everything in. Lot’s of YouTube videos showing how to hookup BMS to these cells, pick one you like, preferably with a phone app, treat yourself here.
What if you want to expand the setup to solar?
Swap out the Automatic Transfer Switch above with this instead:
Then get yourself a solar controller charger(stick to one with same watts as your inverter) and some used solar panels your good to go! This thing will feed off your batteries when solar has charged them up, then switch back to grid when your batteries are low. Free energy, why not put the battery you built to use 🙂
I’ll add more links and information to this setup once I get all these parts, but this is how I would go about it for best setup for your home. Start small with 1500 watt inverter, scale up as needed, maybe one day you want to run your fridge, deepfreeze and air conditioner as well, that would be a good time to scale up to say a 3000 watt inverter and solar charge controller.
I would recommend you build your own battery with BMS and try a setup like this, not only will you be happier in the end, not just the cost savings, just the pure knowledge you will gain from learning how to set it all up from YouTube videos will make you a much better person going green but at same time give you the confidence to go completely off grid one day if you choose, that knowledge is invaluable.
