LvmMaintenance - Le Wiki Koumbit

Contents

Manipulations du LVM

Manipulations du LVM

création

Création du volume physique

# pvcreate /dev/mapper/crypt-data3

Création du volume groupe

# vgcreate SSDdata /dev/mapper/crypt-data3

Trouver l'espace total occupé par le volume groupe

# vgdisplay SSDdata | grep "Total PE"
  Total PE              30525

Création du volume logique

lvcreate -l 30525 SSDdata -n aegir-disk

Activer des volumes lvm

Un simple /etc/init.d/lvm start

# /etc/init.d/lvm start
Setting up LVM Volume Groups...
  Reading all physical volumes.  This may take a while...
  Found volume group "vg01" using metadata type lvm2
  Found volume group "vg00" using metadata type lvm2
  1 logical volume(s) in volume group "vg01" now active
  4 logical volume(s) in volume group "vg00" now active

On peut vérifier l'état des volumes avec la commande

# lvscan -v
    Finding all logical volumes
  ACTIVE            '/dev/vg01/test' [1.46 GB] inherit
  ACTIVE            '/dev/vg00/home' [1000.00 MB] inherit
  ACTIVE            '/dev/vg00/tmp' [1000.00 MB] inherit
  ACTIVE            '/dev/vg00/usr' [2.93 GB] inherit
  ACTIVE            '/dev/vg00/var' [2.57 GB] inherit

Création de vserver-data

romulus:/home/anarcat# pvcreate /dev/md2
  Physical volume "/dev/md2" successfully created
romulus:/home/anarcat# vgcreate vserver-data /dev/md2
  Incorrect metadata area header checksum
  Incorrect metadata area header checksum
  Incorrect metadata area header checksum
  Volume group "vserver-data" successfully created
romulus:/home/anarcat# lvcreate -L1500 -ntestlv vserver-data
  Incorrect metadata area header checksum
  Logical volume "testlv" created
romulus:/home/anarcat# mke2fs -j /dev/vserver-data/testlv 
mke2fs 1.37 (21-Mar-2005)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
192000 inodes, 384000 blocks
19200 blocks (5.00%) reserved for the super user
First data block=0
12 block groups
32768 blocks per group, 32768 fragments per group
16000 inodes per group
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912

Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Destruction d'une partition

lvremove vserver-data/backup

Si vous voyez:

  Can't remove open logical volume "backup"

c'est que le volume est ouvert par un process quelconque.

Il est parfois possible de le resizer directement quand meme, ou de trouver quels processes l'utilisent avec chcontext --ctx 1 lsof ou fuser.

Adding disks

Very, simple, although the manual lacks some details:

cfdisk /dev/hdc # make one partition and mark it as `8E` (Linux LVM)
pvcreate /dev/hdc1
vgextend rhea /dev/hdc1

S'il s'agit d'un array RAID, on n'a pas à créer une partition avec cfdisk, ça ressemblerait plutôt à:

pvcreate /dev/mapper/mdX_crypt
vgextend vgName /dev/mapper/mdX_crypt

Migrating volumes between disks

Don't forget to make sure there is enough space on the device you wish to migrate to. Use the command pvs to do so.

Once you have added a disk, you can choose to move logical volumes around, say if one disk performs better or if you want to remove one. Use pvmove:

pvmove -n example.com-disk /dev/md1 /dev/md2

This will move the example.com VM between those two arrays.

A big +, we don't need to shutdown the running VM while moving it.

The above command works like a charm for moving parts of an LV when it is spread out across different PVs (maybe this situation happened because we lacked space at some point). For example, to move part of an LV that is on md1 to md2 where you want it to reside completely with the other parts:

first ensure that you have enough space on the destination PV with the command in the section #Displaying_which_device_a_volume_is_on and pvs.
Then use pvmove -n example.com-disk /dev/md1 /dev/md2

Taking a physical volume out of a volume group

This operation is especially necessary when we are using LVM to transfer data from one pair of disks to another one.

This operation will only work if the physical disk that is removed does not contain any logical volumes anymore. So you will need to either transfer them all out or get rid of the excess (remove the remaining LVs).

Prior to removing the disks that correspond to a physical volume, run the following to remove it from the volume group. Here the volume group is data:

vgreduce -v data /dev/md1

Displaying which device a volume is on

The lvs command can be used to display on which physical volume a logical volume is stored, in the above example the LV will be shown as being on /dev/md2:

lvs -o lv_name,lv_attr,lv_size,seg_size,vg_name,devices

Resizing a partition

Shrinking

Voici une procédure qui a très bien fonctionné avec un système de fichier ext4.

Attention, ça peut être très long s'il y a beaucoup de données. Passer 82G de données dans une partition de 200GB vers une partition de 150GB a nécessité 4h de downtime. référence 17109

# first check of the lvm
e2fsck -f  /dev/mapper/data-zenon.m2014.net--disk

# Shrink ext4 and then the LV to the desired size
resize2fs -p /dev/mapper/data-zenon.m2014.net--disk 180G

lvreduce -L 180G /dev/mapper/data-zenon.m2014.net--disk

# Before continuing, run e2fsck. If it bails because the partition
# is too small, don't panic! The LV can still be extended with
# lvextend until e2fsck succeeds, e.g.:
# lvextend -L +1G /dev/mapper/vg0-foo
e2fsck -f /dev/mapper/data-zenon.m2014.net--disk

# Resize the filesystem to match the LVs size, check and mount it
resize2fs -p /dev/mapper/data-zenon.m2014.net--disk

resize2fs 1.41.12 (17-May-2010)
Le système de fichiers a déjà 2621440 blocs. Rien à modifier !

e2fsck -f /dev/mapper/data-zenon.m2014.net--disk
e2fsck 1.41.12 (17-May-2010)
Passe 1 : vérification des i-noeuds, des blocs et des tailles
Passe 2 : vérification de la structure des répertoires
Passe 3 : vérification de la connectivité des répertoires
Passe 4 : vérification des compteurs de référence
Passe 5 : vérification de l'information du sommaire de groupe
/dev/mapper/data-...--disk : 53954/655360 fichiers (0.5% non contigus), 1126618/2621440 blocs

référence: http://pubmem.wordpress.com/2010/09/16/how-to-resize-lvm-logical-volumes-with-ext4-as-filesystem/

Growing

Before doing anything below here, stop the xen box first, then umount !

On older setups where xen VMs were controlled by puppet, this also means disabling puppet to ensure that the VM won't get restarted in the middle of the operation!

# Did we already mention that before doing that, the Xen box using the LV should be stopped and/or the partition should be un-mounted? 
# Failing to ensure this pre-condition could lead to data corruption in the partition!!

fsck  -f -y /dev/mapper/visite102-home

(optional) If you're adding a new drive, then format some space on it. You shouldn't do that by hand and if you're doing this on a production server, there should be some RAID (or we should be thinking about getting some RAID there).
- ```
cfdisk /dev/sdb
pvcreate /dev/sdb1
vgextend visite102 /dev/sdb1
```

In all cases (here we're adding 8Gb to the disk):

vgdisplay 
lvextend -L +8G visite102/home 
resize2fs -p /dev/mapper/visite102-home
fsck -f /dev/mapper/visite102-home

Alternative way of growing

Another run that was successful in 12582 and 23535

pvcreate /dev/mapper/md2_crypt
vgextend crypto /dev/mapper/md2_crypt
umount /dev/mapper/crypto-srv
fsck.ext4 -C0 -f /dev/mapper/crypto-srv
lvresize -l1181338 /dev/mapper/crypto-srv
resize2fs -p /dev/mapper/crypto-srv
fsck.ext4 -C0 -f /dev/mapper/crypto-srv
mount /srv

See upstream docs.

Snapshots

Créer un snapshot à partir du lvm "alternc.koumbit.net-disk" sur le volume group "data"

lvcreate -L1G -s -n alternc-backup data/alternc.koumbit.net-disk

Effacer le snapshot "alternc-backup" du volume group "data"

lvremove data/alternc-backup

Rollback les changements d'un disque à l'état sauvegardé dans le snapshot (s'assurer que c'est umount comme du monde avant)

lvconvert --merge /dev/data/alternc-backup

Retirer des disques ou déplacer vers d'autres disques

LVM permet de déplacer du data sur d'autres disques de façon complètement transparante. La documentation est là: http://www.tldp.org/HOWTO/LVM-HOWTO/removeadisk.html

J'ai utilisé cette technique pour migrer rhea.office.koumbit.net d'un setup /dev/hda + RAID-1/LVM vers un setup RAID-1/LVM complet, en créant un nouvel array raid... Voir l'aventure dans 44642.

plus de inodes!

On doit pas juste vérifier l'espace disque sur un volume lvm ou une partition, on doit aussi avoir assez de inode. Heureusement, munin fait le suivi pour nous. Selon la documentation, on ne peut pas ajouter des inodes a une système de fichier existant. On doit le reformater en modifiant les configurations préalablement du fichier /etc/mke2fs.conf. Pour ajouter des inodes, on doit baisser le ratio, on passe donc la value de 16384 à 8192.

référence http://serverfault.com/questions/396768/ext4-file-system-max-inode-limit-can-anyone-please-explain

CategoryMaintenance

Wiki

Page

User