Monday, August 27, 2018

LVM Snapshots

This is a basic overview of how to work with lvm snapshots, I wrote this article to help me remember the procedure and for later reference.

Snapshots provide a point in time snapshot of a volume, allowing you to undertake some operations that you may want to roll back later, i.e. testing new software, testing deployment procedures, upgrading a system etc.

NOTE: To restore a snapshot a volume needs to be activated, if the volume is currently mounted this will involve unmounting, deactivating, activating then finally mounting the volume. If a snapshot is taken of the root logical volume, then the system will need to be restarted to finish the process as you can’t unmount the root volume.

This example requires that we have a volume group with some free space, I've set up a CentOS 7 virtual machine for this example with 5GB of free space in the centos volume group: The disk layout on the virtual machine looks like this:
# lsblk
sda               8:0    0   45G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0   44G  0 part 
  ├─centos-root 253:0    0 35.1G  0 lvm  /
  └─centos-swap 253:1    0  3.9G  0 lvm  [SWAP]
# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda2  centos lvm2 a--  <44.00g 5.00g
# vgs
  VG     #PV #LV #SN Attr   VSize   VFree
  centos   1   2   0 wz--n- <44.00g 5.00g
# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root centos -wi-ao---- <35.12g                                                    
  swap centos -wi-ao----  <3.88g                   
To begin lets create a logical volume called datavol01:
# lvcreate -L 1G -n datavol1 centos
  Logical volume "datavol1" created.
Create a filesystem on new logical volume:
mkfs.xfs /dev/centos/datavol1
meta-data=/dev/centos/datavol1   isize=512    agcount=4, agsize=65536 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=262144, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
Mount the new logical volume:
# mount /dev/centos/datavol1 /mnt/
Create a file in the mount for demonstration purposes:
# touch /mnt/this
Check that the file was created as expected:
# ls -lah /mnt
total 0
drwxr-xr-x.  2 root root  30 Aug  9 22:20 .
dr-xr-xr-x. 17 root root 224 Aug  8 01:05 ..
-rw-r--r--.  1 root root   0 Aug  9 22:20 this
Creating snapshots
Let's create a snapshot of datavol1 using 1GB of free space in the volume group:
# lvcreate -L 1G -s -n snap01 /dev/centos/datavol1 
  Logical volume "snap02" created.
1GB was chosen for testing while writing this article, in practice you would use a size that’s appropriate for the task, see note at the bottom about auto extending snapshots.

If more space is required for the snapshot then it can be extended using lvextend:
# lvextend -L +1G /dev/centos/snap01
  Size of logical volume centos/snap01 changed from 1.00 GiB (256 extents) to 2.00 GiB (512 extents).
  Logical volume centos/snap01 successfully resized.
Create snapshot with 100% of the free space in a volume group Alternatively a snapshot can be created using all the available space in volume group:
# lvcreate -l100%FREE -s -n snap01 /dev/centos/datavol1
Removing snapshots
Now that we have a snapshot create another file:
# touch /mnt/that
Removing the snapshot retains changes since taking the snapshot:
# lvremove /dev/centos/snap01
Do you really want to remove active logical volume centos/snap01? [y/n]: y
  Logical volume "snap01" successfully removed
List the files in /mnt/, notice they are both still there?
# ls -lah /mnt
total 0
drwxr-xr-x.  2 root root  30 Aug  9 22:20 .
dr-xr-xr-x. 17 root root 224 Aug  8 01:05 ..
-rw-r--r--.  1 root root   0 Aug  9 22:20 that
-rw-r--r--.  1 root root   0 Aug  9 22:20 this
Now let's recreate a snapshot to demonstrate rolling back (merging):
# lvcreate -L 1G -s -n snap01 /dev/centos/datavol1
Create another file to demonstrate the merge process:
# touch /mnt/me
To further demonstrate the process touch the first file we create to change its modified date:
# touch /mnt/this
List /mnt again to see how it looks now:
# ls -lah /mnt
total 0
drwxr-xr-x.  2 root root  40 Aug  9 22:22 .
dr-xr-xr-x. 17 root root 224 Aug  8 01:05 ..
-rw-r--r--.  1 root root   0 Aug  9 22:22 me
-rw-r--r--.  1 root root   0 Aug  9 22:20 that
-rw-r--r--.  1 root root   0 Aug  9 22:22 this
Merging snapshots
Merging a snapshot will undo changes since the snapshot was created restoring the volume to its pre-snapshot state. Before merging the snapshot, the volume should be unmounted and deactivated, this can be done after:
# umount /mnt
# lvchange -an /dev/centos/datavol1 
Merge the snapshot to restore to the pre-snapshot state:
# lvconvert --merge /dev/centos/snap01
  Delaying merge since origin is open.
  Merging of snapshot centos/snap01 will occur on next activation of centos/datavol1.
Reactivate and remount the volume:
# lvchange -ay /dev/centos/datavol1 
# mount /dev/centos/datavol1 /mnt/
Do an ls of /mnt/ again, notice that "me" is gone and "this" has reverted to its earlier timestamp?
# ls -lah /mnt/
total 0
drwxr-xr-x.  2 root root  30 Aug  9 22:20 .
dr-xr-xr-x. 17 root root 224 Aug  8 01:05 ..
-rw-r--r--.  1 root root   0 Aug  9 22:20 that
-rw-r--r--.  1 root root   0 Aug  9 22:20 this
We can now remove the datavol1 logical volume:
# umount /mnt
# lvremove /dev/centos/datavol1
Auto Extend Snapshot
Be aware that if a snapshot is filled there is potential for data loss, to prevent this lvm can be configured to auto extend the snapshot volume by changing values in the /etc/lvm/lvm.conf, the default CentOS conf file has snapshot_autoextend_threshold set to 100 which disables the auto extend feature, change it to a value lower than 100.
# grep snapshot_autoextend /etc/lvm/lvm.conf
# Configuration option activation/snapshot_autoextend_threshold.
# Also see snapshot_autoextend_percent.
# snapshot_autoextend_threshold = 70
snapshot_autoextend_threshold = 100
# Configuration option activation/snapshot_autoextend_percent.
# snapshot_autoextend_percent = 20
snapshot_autoextend_percent = 20
After changing the config the lvm monitor process will need to be restarted:
# lvm2-monitor.service

1 comment:

  1. It amazes me how no one talks about how a snapshot's chunksize tremendously affects performance during writes, specially with merges. Having tested a snapshot with lots of writes (OLTP database), merging with a chunksize of 4kb (default) or 8kb takes HOURS, with a chunksize of 512kb it takes 3 minutes.
    Tested on a Xeon Gold 6246 with 6 SSD in RAID 10 an RHEL 6.9.


Raspberry Pi 4 heatsink testing

With all the talk of how hot the Raspberry Pi 4 runs I thought I’d do some testing to see how different heatsinks perform. I had a few heats...