There are a lot of good LVM guides on the net. This one is written for myself, dealing with issues that I forget.
For the beginner, some of the disadvantages are that the syntax can be a little tricky, and the popular gparted doesn't (yet) work with it. Also, legacy grub, still used by many systems, including Fedora as of F15, will require a separate /boot partition. (Grub2 is able to boot from an LVM).
Assuming you have decided to use LVM, either during installation, or when adding disks afterwards, the procedure is relatively straightforward. One first creates a physical volume, then a volume group inside the physical volume. Next, one creates the logical volume inside the volume group. Lastly, one creates a filesystem for the logical volume.
Note that most of this will require root privilege.
In the workplace, this might mean adding a disk in a SAN or NAS. In any case where the disk has been added and the machine hasn't been restarted, it may not see the new space.
(For the newcomer, if any of the terms below are unfamiliar, don't worry. Briefly iscsi is a method of linking, by IP address, to storage devices. HBA stands for Host Bus Adapter, and is used with fibre channel storage.).
One can run fdisk -l to see if the new disk has been detected. If not, and it is an iscsi server then
iscsiadm -m node -R |
will have the iscsi daemon rescan all sessions and should show the space. If this doesn't work, (though it should), you can also try
echo "- - -" > /sys/class/scsi_host/hostX/scan |
The X in hostX refers to whatever host number you have in /sys/class/scsi_host, so the command might go
echo "- - -" > /sys/class/scsi_host/host1/scan |
Also, note that there is a space between each dash.
With HBA's I usually use (if your browser doesn't show it, this should all be one line)
for name in $(ls /sys/class/scsi_device);do echo 1 > /sys/class/scsi_device/$name/device/rescan;done |
I haven't done this often enough to really keep track of what works with what (that is SAN vs. NAS), but one of the three steps should show the new disk, and running fdisk -l should show a new, unpartitioned disk.
A couple of other things to note with NAS and SAN. The new disk might not show up where you expected it. In other words, if you have sda, sdb and sdc, you might find that you suddenly have sda, sdb, sdc, and sdd, but that the unpartitioned disk isn't, as you would expect, sdd, but is actually sdb with the former sdb now being sdc, etc.
When using HBA's, if there's more than one port, you might see that you suddenly have 5 new disks, all the same size. You can run the partitioning and volume creating commands on any of them, and it will work--that is, you will wind up with one disk, and suddenly, sdd, sde, sdf and so on will all show the results of your partitioning.
fdisk /dev/sdd |
I have an old page about fdisk that is, hopefully, still useful, explaining what we're doing here in a bit more detail. For purposes of this page, it's assumed that the user has some idea about what fdisk does. We'll create a new partition, make it primary, as it will be the only partition on the new disk, change its filesystem to LVM and then write it to disk. So first one types
n |
For a new partition. It will give you a choice of e for extended or p for primary. Type
p |
for primary. When asked for the partition number, type 1. It will then ask for the first cylinder. Just hitting enter accepts the default of the beginning of the drive. It will then ask for the last cylinder, again giving a default, this time the end of the drive. Hit enter again to accept it. Now type
t |
to toggle the filesystem type. It will tell you that you can type L for a list of hex codes. We want 8e for LVM, so type that in. At this point, I like to type p as in print, to make sure that it's done what I wanted. Then type w as in write, and it will give a message that it's writing to disk. It may also say that the kernel is using the old partition table, rebooting is necessary.
However, one can avoid the reboot by running the command
partprobe |
In RH based systems at least, partprobe is provided by the parted
package.
You should now see, if you do ls /dev/sd* that you have a /dev/sdd1.
pvcreate /dev/sdd1 |
You should see a message that the volume was successfully created. (I'm using a CentOS 5.6 machine, with an extra 500 GB drive. You'll note that the sizes listed are somewhat smaller. The system reserves some disk space.)
If you run the command pvdisplay, you'll see something like
"dev /sdd1" is a new physical volume of "465.76 GB" PV Name /dev/sdd1 VG Name PV Size 465.76 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID 8t0c12-Q8p9-IzRs-2Zaj-i0QQ-G09V-ukR5J4 |
The next step is to make a volume group on the physical volume. You can run pvdisplay to see what you have--it should show a physical volume which is about the size of the disk. I like to give volumegroups, as well as LVMs, distinctive names--it will make it easier if I have to resize one, and I can be sure that I'm working on the right thing. In this case, I'll call it testvol
vgcreate testvol /dev/sdd1 |
Running pvdisplay now shows me
--- Physical volume --- PV Name /dev/sdc1 VG Name testvol PV Size 465.76 GB / not usable 1.50 MB Allocatable yes PE Size (KByte) 4096 Total PE 119234 Free PE 119234 Allocated PE 0 |
Running vgdisplay shows me
VG Name testvol System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 465.76 GB PE Size 4.00 MB Total PE 119234 Alloc PE / Size 0 / 0 Free PE / Size 119234 / 465.76 GB VG UUID La9WTg-gZK4-pHA3-c9lH-YpVL-oM32-vpJSCB |
The important thing is that I see how many free PE (physical Extents) that I have, for I'll use that in the next step, creating a logical volume on the this volume group. I'll call the logical volume test.
I'm going to use the number of extents. Although it also gives the size in GB, I've found that using that result almost always tells me there isn't enough free space, that it will be a few extents short. Since it's easy to see the number of free extents with the vgdisplay command, I might as well use them. One can also use a percentage--in this case, I am going to use the entire partition, so I can use percent instead of extents. To make the logical volume, named test, I run
lvcreate -n test -l 100%VG testvol |
To do it with extents I would have used
lvcreate -n test -l 119324 testvol |
So, the syntax is lvcreate -n for a name -l for the size in extents or percent and finally, the name of the volume group that you're using. The l is a lower case letter L, not the numeral one. To give the size in MB, GB, etc, use -L rather than -l, M for Megabytes, G for Gigabytes, etc. The default, if you don't put a letter, is Megaybes so -L 100 would be 100MB.
If you now look at your /dev directory, you'll see that there's an entry /dev/testvol/test which is a symbolic link to /dev/mapper/testvol-test.
The last step is to create a file system on the LVM. In this case, as I'm on CentOS 5.6, I'll use ext3.
mkfs.ext3 /dev/testvol/test |
It will go through creating a file system on the logical volume. When done it will tell you that the drive will be checked every 39 mounts or 180 days, whichever comes first and that you can use tune2fs to change this. If you do want to change the behavior, then
tune2fs -c 0 /dev/testvol/test |
Now that it has a file system, you can create a mount point and mount the new drive. For example
mkdir lvm_mount mount /dev/testvol/test /lvm_mount |
To make the mount permanent, add an entry to your /etc/fstab. Nothing special has to be done, the entry can be like any other. For example
/dev/testvol/test /lvm_mount ext3 defaults 0 0 |
umount /lvm_mount |
Note that the command is umount, NOT unmount.
Remove the logical volume.
lvremove /dev/testvol/test |
You'll be asked if you're sure, hit y for yes.
Deactivate the volume group, then remove it.
vgchange -a n testvol vgremove testvol |
Now the physical volume
pvremove /dev/sdd1 |
Once this is done, you should then run fdisk and repartition the drive. If you don't know what you're planning to do with it, just remove the partition. In this case, it's simple, as there's only one partition.
fdisk d w |
The d is for delete--as there's only partition, it won't ask which partition to delete, then w for write, as we did before. Afterwards, one should run partprobe to make sure the system reads the new partition table.
So we have an LVM /dev/testvol/test that is 20GB in size. We now either add a new partition with fdisk, or perhaps expand the drive if it's an external SAN or NAS. In any case, one might have to follow the same steps given in adding a new disk to get the system to see it.
Once again, we'll use /dev/sdd. In this case, the steps are to add a new partition, give it a physical volume (I find this to be easier and less prone to my own errors than using pvresize), expand the volume group into it, then expand the logical volume. So first run fdisk as before, add another 20 GB partition. For example fdisk /dev/sdd, hit n for new, p for primary, 2 for partition number. Take the default starting cylinder and to add another 20GB use +20480M. (A GB is 1024 megabytes, so that's 1024*20). Use t to toggle it, choose partition 2, choose 8e as the partition type for LVM, hit w for write, and then if it says the system must be rebooted, run partprobe.
Now we create the new physical volume.
pvcreate /dev/sdd2 |
Running pvdisplay should now show us a new physical volume with something like
"/dev/sdd2" is a new physical volume of "20.91 GB" --- NEW Physical volume --- PV Name /dev/sd22 VG Name PV Size 20.91 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID 22ayk4-Ghhq-8yns-uNHh-nDdf-OGhc-p0G106 |
umount /lvm_mount |
This is up to the individual--in production, I always unmount the volume first, at home, it depends. I have lost data once while extending a logical volume, but I've also frequently resized them without losing anything. Whether the volume is now mounted or not, the next steps are the same.
We will now extend the volumegroup testvol into the new physical volume.
vgextend testvol /dev/sdd2 |
If we now run vgdisplay testvol we'll see that we have 5353 extents free. We can, once again, either use -l 5353 or -l 100%VG.
lvextend -l 100%VG /dev/testvol/test |
(I believe that using lvextend -lr will resize the file system, but, to be honest, haven't tried it. These days, I'm usually using zfs instead.)
Lastly we have to resize the filesystem as well. (But see above). One can unmount the partition, first, but usually, it's not necessary. If one runs df -H at this point, you'll still see the old size of 20GB. If using the RedHat and offshoots default of an XFS file system, use xfs_growfs, otherwise, use resize2fs.
resize2fs /dev/testvol/test |
You may get a message like please run 'e2fsck -f /dev/testvol/test' first. I have never tried running e2fsck on a mounted logical volume, it's probably a bad idea. At any rate, run it. Hopefully, it completes without problems.
If the volume is mounted, you will see a message that it is performing an online resizing. If you run df -H again, you'll see that the /lvm_mount partition is now 20 GB larger.
Note that online resizing can be done with ext3fs but not with ext2fs. Although it is ext3 the command is still resize2fs.
In this default installation of ScientificLinux 6 (SL6), I use a 15GB disk and let it use a default partitioning scheme. Running pvdisplay afterwards shows me that 14.51 GB are used, with none free. Running lvdisplay shows me that the swap partition is 3.97 GB, more than I want to give it. Running ls /dev/mapper shows me that I have two logical volumes. Assuming I gave the machine a name like myhost.mydomain.com, the name will be something like /dev/mapper/vg_myhost-lv_root and vg_myhost-lv_swap.
First I want to completely delete the swap partition. I turn off swap first, (with any other sort of partition, one would unmount it before removing or reducing. With swap, one just turns it off.)
swapoff /dev/mapper/vg_myhost-lv_swap |
Next I remove the logical volume. (NOT THE VOLUME GROUP). It will ask
me if I'm sure, giving me a y/n option, and I will type y for yes.
lvremove /dev/mapper/vg_myhost-lv_swap |
Logical volumes are shown two ways in /dev. One is the one I use here, /dev/mapper/vg_myhost-lv_swap and also as /dev/vg_myhost/lv_swap. Use either one. Both are actually symlinks to /dev/dm-X, (starting with /dev/dm-0).
Now I want to see how much space I have.
vgdisplay |
This will give me various information, including Free PE / Size. PE is a physical extent, usually about 4MB, but size should give it, in this case, in GBs. In my case this line reads
Free PE / Size 1016 / 3.97 GiB |
Out of this 3.97 GB, I decide that I want to extend the root partition 2.5 GB.
lvextend -L +2.5G /dev/mapper/vg_myhost-lv_root |
-L is used when using sizes like G or M for gigabytes and megabytes, -l (lower case L) is used when using physical extents.
You should see a message that the volume was successfully resized.
Now to check how much space I have left, I run vgdisplay again. This time I see that I now I have 1.47 GB of free space. It will also show me the number of free extents. As the GB measurement usually isn't exact, it's easier to use extents. For example, if I actually use -L +1.47G I'll get a message that it's 4MB short. (The message will be something like Insufficient space, 377 PE required only 376 available.) So, as vgdisplay showed me that I have 376 PE left, I'll use that. I'll use the -n option to give it the same name that it had before.
lvcreate -l 376 -n lv_swap vg_myhost |
In other words using extents, we use a lower case L, the -n is the name. This can confuse people. If you remember, the name was vg_myhost-lv_swap. However, if you put in that full name, you will have /dev/mapper/vg_myhost-vg_myhost-lv-swap. So just put in the last part of the name, in other words, whatever you would expect to see AFTER vg_myhost-. You can name it anything you wish, in this case, I'm staying with the original naming scheme.
Lastly, give the name of the volume group, the vg_myhost.
You should see that the logical volume was created. Now you will have to make it a swap partition and turn the swap on.
mkswap /dev/mapper/vg_myhost-lv_swap swapon /dev/mapper/vg_myhost-lv_swap |
Running the free command (just type the word "free", without quotes, at a command prompt), should now show you that swap is working.
Lastly, we have to resize the filesystem on the root logical volume. If you run df -H right now, you'll see that the / partition is still at the old size.
resize2fs /dev/mapper/vg_myhost-lv_root |
You'll see a message that it is performing an online resize. Running df -H will now show you the new size for the / partition.
I just find this method a bit simpler than reducing the swap partition. The user may prefer the more common method of using lvreduce, but this, at least for me, is somewhat quicker.