Linux Tipps und Tricks: Difference between revisions
From Lolly's Wiki
Jump to navigationJump to search
No edit summary |
|||
| (6 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
This is the hard way to kick your kernel into void. No filesystem sync is done, just and ugly fast direkt reboot! | This is the hard way to kick your kernel into void. No filesystem sync is done, just and ugly fast direkt reboot! | ||
You should never do this... | You should never do this... | ||
< | <syntaxhighlight lang=bash> | ||
# echo 1 > /proc/sys/kernel/sysrq | # echo 1 > /proc/sys/kernel/sysrq | ||
# echo b > /proc/sysrq-trigger | # echo b > /proc/sysrq-trigger | ||
</ | </syntaxhighlight> | ||
First line enables sysrq, second line sends the reboot request. | First line enables sysrq, second line sends the reboot request. | ||
| Line 12: | Line 12: | ||
==Scan all SCSI buses for new devices== | ==Scan all SCSI buses for new devices== | ||
< | <syntaxhighlight lang=bash> | ||
# for i in /sys/class/scsi_host/host*/scan ; do echo "- - -" > $i ; done | # for i in /sys/class/scsi_host/host*/scan ; do echo "- - -" > $i ; done | ||
</ | </syntaxhighlight> | ||
==Scan all FC ports for new devices== | ==Scan all FC ports for new devices== | ||
| Line 22: | Line 22: | ||
!!!BUT the connection might get lost for a moment!!! | !!!BUT the connection might get lost for a moment!!! | ||
The softer way is [[#Scan all SCSI buses for new devices|to scan the SCSI buses]]. | The softer way is [[#Scan all SCSI buses for new devices|to scan the SCSI buses]]. | ||
< | <syntaxhighlight lang=bash> | ||
# for i in /sys/class/fc_host/*/issue_lip ; do echo "1" > $i ; done | # for i in /sys/class/fc_host/*/issue_lip ; do echo "1" > $i ; done | ||
</ | </syntaxhighlight> | ||
==Rescan a device (for example after changing a VMDK size)== | ==Rescan a device (for example after changing a VMDK size)== | ||
< | <syntaxhighlight lang=bash> | ||
# device=sda | # device=sda | ||
# echo 1 > /sys/class/block/${device}/device/rescan | # echo 1 > /sys/class/block/${device}/device/rescan | ||
</ | </syntaxhighlight> | ||
This is for device sda after changing the VMDK from 20GB to 25GB: | This is for device sda after changing the VMDK from 20GB to 25GB: | ||
< | <syntaxhighlight lang=bash> | ||
# device=sda | # device=sda | ||
# echo $[ 512 * $( | # echo "$[ 512 * $(</sys/block/${device}/size) / 1024 ** 3 ] GB" | ||
20 | 20 GB | ||
# echo 1 > /sys/class/block/${device}/device/rescan | # echo 1 > /sys/class/block/${device}/device/rescan | ||
# echo $[ 512 * $( | # echo "$[ 512 * $(</sys/block/${device}/size) / 1024 ** 3 ] GB" | ||
25 | 25 GB | ||
# parted /dev/${device} "print free" | # parted /dev/${device} "print free" | ||
Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 10485760 blocks) or | Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 10485760 blocks) or | ||
| Line 54: | Line 54: | ||
1 1049kB 21,5GB 21,5GB zfs | 1 1049kB 21,5GB 21,5GB zfs | ||
21,5GB 26,8GB 5369MB Free Space | 21,5GB 26,8GB 5369MB Free Space | ||
</ | </syntaxhighlight> | ||
I want to put the free space into partition 1 and resize the rpool: | I want to put the free space into partition 1 and resize the rpool: | ||
< | <syntaxhighlight lang=bash> | ||
# parted /dev/${device} "resizepart 1 -1" | # parted /dev/${device} "resizepart 1 -1" | ||
# parted /dev/${device} "print free" | # parted /dev/${device} "print free" | ||
| Line 88: | Line 88: | ||
rpool 24,9G 1,69G 23,2G - 11% 6% 1.00x ONLINE - | rpool 24,9G 1,69G 23,2G - 11% 6% 1.00x ONLINE - | ||
# zpool set autoexpand=off rpool | # zpool set autoexpand=off rpool | ||
</ | </syntaxhighlight> | ||
Done. | Done. | ||
| Line 96: | Line 96: | ||
Be careful! Like in this example the lowest SCSI-ID is not always the lowest device name! | Be careful! Like in this example the lowest SCSI-ID is not always the lowest device name! | ||
Check it with <i>lsscsi</i> from the Ubuntu package lsscsi: | Check it with <i>lsscsi</i> from the Ubuntu package lsscsi: | ||
< | <syntaxhighlight lang=bash> | ||
# lsscsi | # lsscsi | ||
[2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0 | [2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0 | ||
[32:0:0:0] disk VMware Virtual disk 1.0 /dev/sdb | [32:0:0:0] disk VMware Virtual disk 1.0 /dev/sdb | ||
[32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda | [32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda | ||
</ | </syntaxhighlight> | ||
Then check it is not longer in use: | Then check it is not longer in use: | ||
| Line 110: | Line 110: | ||
Then delete it: | Then delete it: | ||
< | <syntaxhighlight lang=bash> | ||
# echo 1 > /sys/bus/scsi/drivers/sd/32\:0\:0\:0/delete | # echo 1 > /sys/bus/scsi/drivers/sd/32\:0\:0\:0/delete | ||
</ | </syntaxhighlight> | ||
The 32:0:0:0 is the number reported from the lsscsi above. | The 32:0:0:0 is the number reported from the lsscsi above. | ||
Et voila: | Et voila: | ||
< | <syntaxhighlight lang=bash> | ||
# lsscsi | # lsscsi | ||
[2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0 | [2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0 | ||
[32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda | [32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda | ||
</ | </syntaxhighlight> | ||
==Copy a GPT partition table== | ==Copy a GPT partition table== | ||
Copy partition table of sdX to sdY: | Copy partition table of sdX to sdY: | ||
< | <syntaxhighlight lang=bash> | ||
# sgdisk /dev/sdX --replicate=/dev/sdY | # sgdisk /dev/sdX --replicate=/dev/sdY | ||
# sgdisk --randomize-guids /dev/sdY | # sgdisk --randomize-guids /dev/sdY | ||
</ | </syntaxhighlight> | ||
Or with: | Or with: | ||
< | <syntaxhighlight lang=bash> | ||
# sgdisk --backup=sdX.table /dev/sdX | # sgdisk --backup=sdX.table /dev/sdX | ||
# sgdisk --load-backup=sdX.table /dev/sdY | # sgdisk --load-backup=sdX.table /dev/sdY | ||
# sgdisk -G /dev/sdY | # sgdisk -G /dev/sdY | ||
</ | </syntaxhighlight> | ||
<pre> | <pre> | ||
| Line 154: | Line 154: | ||
===Correct the GPT partition table=== | ===Correct the GPT partition table=== | ||
< | <syntaxhighlight lang=bash> | ||
root@mariadb:~# parted /dev/sdb | root@mariadb:~# parted /dev/sdb | ||
GNU Parted 3.2 | GNU Parted 3.2 | ||
| Line 171: | Line 171: | ||
Number Start End Size File system Name Flags | Number Start End Size File system Name Flags | ||
1 1049kB 6442MB 6441MB zfs | 1 1049kB 6442MB 6441MB zfs | ||
</ | </syntaxhighlight> | ||
===Resize the partition=== | ===Resize the partition=== | ||
< | <syntaxhighlight lang=bash> | ||
root@mariadb:~# parted /dev/sdb | root@mariadb:~# parted /dev/sdb | ||
GNU Parted 3.2 | GNU Parted 3.2 | ||
| Line 202: | Line 202: | ||
(parted) q | (parted) q | ||
Information: You may need to update /etc/fstab. | Information: You may need to update /etc/fstab. | ||
</ | </syntaxhighlight> | ||
===Optional: Resize the ZPool in it=== | ===Optional: Resize the ZPool in it=== | ||
Check the actual values: | Check the actual values: | ||
< | <syntaxhighlight lang=bash> | ||
root@mariadb:~# zpool list MYSQL-DATA | root@mariadb:~# zpool list MYSQL-DATA | ||
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT | NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT | ||
| Line 213: | Line 213: | ||
NAME PROPERTY VALUE SOURCE | NAME PROPERTY VALUE SOURCE | ||
MYSQL-DATA autoexpand off default | MYSQL-DATA autoexpand off default | ||
</ | </syntaxhighlight> | ||
Now inform ZPool to grow to the end of the partition. | Now inform ZPool to grow to the end of the partition. | ||
Set autoexpand to on: | Set autoexpand to on: | ||
< | <syntaxhighlight lang=bash> | ||
root@mariadb:~# zpool set autoexpand=on MYSQL-DATA | root@mariadb:~# zpool set autoexpand=on MYSQL-DATA | ||
</ | </syntaxhighlight> | ||
Send an online to the already onlined device to force a recheck in the ZPool to resize it without export/import: | Send an online to the already onlined device to force a recheck in the ZPool to resize it without export/import: | ||
< | <syntaxhighlight lang=bash> | ||
root@mariadb:~# zpool online MYSQL-DATA /dev/sdb1 | root@mariadb:~# zpool online MYSQL-DATA /dev/sdb1 | ||
</ | </syntaxhighlight> | ||
Et voila: | Et voila: | ||
< | <syntaxhighlight lang=bash> | ||
root@mariadb:~# zpool list MYSQL-DATA | root@mariadb:~# zpool list MYSQL-DATA | ||
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT | NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT | ||
MYSQL-DATA 50,0G 994M 49,0G - 5% 1% 1.00x ONLINE - | MYSQL-DATA 50,0G 994M 49,0G - 5% 1% 1.00x ONLINE - | ||
rpool 19,9G 3,36G 16,5G - 19% 16% 1.00x ONLINE - | rpool 19,9G 3,36G 16,5G - 19% 16% 1.00x ONLINE - | ||
</ | </syntaxhighlight> | ||
Set autoexpand to off if you want prevent to autoexpand if partition grows: | Set autoexpand to off if you want prevent to autoexpand if partition grows: | ||
< | <syntaxhighlight lang=bash> | ||
root@mariadb:~# zpool set autoexpand=off MYSQL-DATA | root@mariadb:~# zpool set autoexpand=off MYSQL-DATA | ||
</ | </syntaxhighlight> | ||
===Optional: Resize the LVM physical volume=== | ===Optional: Resize the LVM physical volume=== | ||
Check the values: | Check the values: | ||
< | <syntaxhighlight lang=bash> | ||
# parted /dev/${device} "print free" | # parted /dev/${device} "print free" | ||
Model: VMware Virtual disk (scsi) | Model: VMware Virtual disk (scsi) | ||
| Line 255: | Line 255: | ||
PV VG Fmt Attr PSize PFree | PV VG Fmt Attr PSize PFree | ||
/dev/sda1 vg-root lvm2 a-- <35.00g 0 | /dev/sda1 vg-root lvm2 a-- <35.00g 0 | ||
</ | </syntaxhighlight> | ||
OK, we need to resize the physical volume | OK, we need to resize the physical volume | ||
< | <syntaxhighlight lang=bash> | ||
# pvresize /dev/sda1 | # pvresize /dev/sda1 | ||
Physical volume "/dev/sda1" changed | Physical volume "/dev/sda1" changed | ||
1 physical volume(s) resized / 0 physical volume(s) not resized | 1 physical volume(s) resized / 0 physical volume(s) not resized | ||
</ | </syntaxhighlight> | ||
Check the values: | Check the values: | ||
< | <syntaxhighlight lang=bash> | ||
# pvs | # pvs | ||
PV VG Fmt Attr PSize PFree | PV VG Fmt Attr PSize PFree | ||
/dev/sda1 vg-root lvm2 a-- <45.00g 10.00g | /dev/sda1 vg-root lvm2 a-- <45.00g 10.00g | ||
</ | </syntaxhighlight> | ||
<syntaxhighlight lang=bash> | |||
# lvextend -l +100%FREE /dev/vg-root/log | |||
</syntaxhighlight> | |||
Done. | Done. | ||
==Find open but deleted files== | |||
Sometimes you have a full filesystem, but cannot see files with ls.<br> | |||
And the output of <i>du -sh <mountpoint></i> and <i>df -h <mountpoint></i> differ, because <i>du</i> just sums the files by traversing the directory.<br> | |||
Then it is time to look for files that are open by any process but deleted in the filesystem.<br> | |||
You can investigate the /proc kernel filesystem: | |||
<syntaxhighlight lang=bash> | |||
# find /proc/*/fd -ls | grep '(deleted)' | |||
91565697 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/7 -> /tmp/ibNhVEnm\ (deleted) | |||
91565698 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/8 -> /tmp/ibhSEF8n\ (deleted) | |||
91565699 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/9 -> /tmp/ibADGDrl\ (deleted) | |||
91565703 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/13 -> /tmp/ibtl5efn\ (deleted) | |||
</syntaxhighlight> | |||
Or you can use <i>lsof</i>: | |||
<syntaxhighlight lang=bash> | |||
# lsof +aL1 | |||
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME | |||
mysqld 2118 mysql 7u REG 0,27 0 0 32780 /tmp/ibNhVEnm (deleted) | |||
mysqld 2118 mysql 8u REG 0,27 0 0 32782 /tmp/ibhSEF8n (deleted) | |||
mysqld 2118 mysql 9u REG 0,27 0 0 32786 /tmp/ibADGDrl (deleted) | |||
mysqld 2118 mysql 13u REG 0,27 0 0 32796 /tmp/ibtl5efn (deleted) | |||
</syntaxhighlight> | |||
Or for a specific running command: | |||
<syntaxhighlight lang=bash> | |||
# lsof +aL1 -c mariadbd | |||
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME | |||
mariadbd 2821 mysql 7u REG 0,36 0 0 50 /tmp/#50 (deleted) | |||
mariadbd 2821 mysql 8u REG 0,36 0 0 51 /tmp/#51 (deleted) | |||
mariadbd 2821 mysql 10u REG 0,36 5420 0 52 /tmp/#52 (deleted) | |||
mariadbd 2821 mysql 13u REG 0,36 0 0 53 /tmp/#53 (deleted) | |||
</syntaxhighlight> | |||
To truncate the files you can use the filedescriptor of the process that has the file open:<br> | |||
<b>ATTENTION! Do this ONLY if you exactly know what you are doing!!! Restarting the process is often the much safer solution!!!</b> | |||
<syntaxhighlight lang=bash> | |||
# : > "/proc/<pid>/fd/<fd>" | |||
</syntaxhighlight> | |||
Latest revision as of 14:00, 19 November 2024
Hard reboot
This is the hard way to kick your kernel into void. No filesystem sync is done, just and ugly fast direkt reboot! You should never do this...
# echo 1 > /proc/sys/kernel/sysrq
# echo b > /proc/sysrq-trigger
First line enables sysrq, second line sends the reboot request.
For more look at kernel.org!
Scan all SCSI buses for new devices
# for i in /sys/class/scsi_host/host*/scan ; do echo "- - -" > $i ; done
Scan all FC ports for new devices
!!!Be CAREFUL!!! This command line issues a Loop Initialization Protocol (LIP). This is a bus reset hat means that removed devices in the fabric will disappear and new ones will appear. !!!BUT the connection might get lost for a moment!!! The softer way is to scan the SCSI buses.
# for i in /sys/class/fc_host/*/issue_lip ; do echo "1" > $i ; done
Rescan a device (for example after changing a VMDK size)
# device=sda
# echo 1 > /sys/class/block/${device}/device/rescan
This is for device sda after changing the VMDK from 20GB to 25GB:
# device=sda
# echo "$[ 512 * $(</sys/block/${device}/size) / 1024 ** 3 ] GB"
20 GB
# echo 1 > /sys/class/block/${device}/device/rescan
# echo "$[ 512 * $(</sys/block/${device}/size) / 1024 ** 3 ] GB"
25 GB
# parted /dev/${device} "print free"
Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 10485760 blocks) or
continue with the current setting?
Fix/Ignore? F
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 26,8GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
2 17,4kB 1049kB 1031kB bios_grub
1 1049kB 21,5GB 21,5GB zfs
21,5GB 26,8GB 5369MB Free Space
I want to put the free space into partition 1 and resize the rpool:
# parted /dev/${device} "resizepart 1 -1"
# parted /dev/${device} "print free"
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 26,8GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
2 17,4kB 1049kB 1031kB bios_grub
1 1049kB 26,8GB 26,8GB zfs
26,8GB 26,8GB 983kB Free Space
# zpool list rpool
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 19,9G 1,68G 18,2G - 14% 8% 1.00x ONLINE -
# zpool set autoexpand=on rpool
# zpool status rpool
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
sda1 ONLINE 0 0 0
# zpool online rpool sda1
# zpool list rpool
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 24,9G 1,69G 23,2G - 11% 6% 1.00x ONLINE -
# zpool set autoexpand=off rpool
Done.
Remove a SCSI-device
Let us say we want to remove /dev/sdb.
Be careful! Like in this example the lowest SCSI-ID is not always the lowest device name! Check it with lsscsi from the Ubuntu package lsscsi:
# lsscsi
[2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0
[32:0:0:0] disk VMware Virtual disk 1.0 /dev/sdb
[32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda
Then check it is not longer in use:
- mount
- pvs
- zpool status
- etc.
Then delete it:
# echo 1 > /sys/bus/scsi/drivers/sd/32\:0\:0\:0/delete
The 32:0:0:0 is the number reported from the lsscsi above.
Et voila:
# lsscsi
[2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0
[32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda
Copy a GPT partition table
Copy partition table of sdX to sdY:
# sgdisk /dev/sdX --replicate=/dev/sdY
# sgdisk --randomize-guids /dev/sdY
Or with:
# sgdisk --backup=sdX.table /dev/sdX
# sgdisk --load-backup=sdX.table /dev/sdY
# sgdisk -G /dev/sdY
-R, --replicate=second_device_filename
Replicate the main device's partition table on the specified second device. Note that the replicated partition table is an exact
copy, including all GUIDs; if the device should have its own unique GUIDs, you should use the -G option on the new disk.
-G, --randomize-guids
Randomize the disk's GUID and all partitions' unique GUIDs (but not their partition type code GUIDs). This function may be used
after cloning a disk in order to render all GUIDs once again unique.
Resize a GPT partition
The partition was resized in VMWare from ~6GB to ~50GB.
In the VM I did Remove a SCSI-device for the resized device and then Scan all SCSI buses for new devices after that parted saw the new size.
Correct the GPT partition table
root@mariadb:~# parted /dev/sdb
GNU Parted 3.2
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an extra 92274688 blocks) or continue with the
current setting?
Fix/Ignore? F <-- ! choose F
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 53,7GB <-- ! the new size is reported now
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 6442MB 6441MB zfs
Resize the partition
root@mariadb:~# parted /dev/sdb
GNU Parted 3.2
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 53,7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 6442MB 6441MB zfs
(parted) resizepart 1
End? [6442MB]? 53,7GB <-- ! Put new size here
(parted) p <-- ! Control if it worked
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 53,7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 53,7GB 53,7GB zfs
(parted) q
Information: You may need to update /etc/fstab.
Optional: Resize the ZPool in it
Check the actual values:
root@mariadb:~# zpool list MYSQL-DATA
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
MYSQL-DATA 5,97G 994M 5,00G 44G 47% 16% 1.00x ONLINE -
root@mariadb:~# zpool get autoexpand MYSQL-DATA
NAME PROPERTY VALUE SOURCE
MYSQL-DATA autoexpand off default
Now inform ZPool to grow to the end of the partition. Set autoexpand to on:
root@mariadb:~# zpool set autoexpand=on MYSQL-DATA
Send an online to the already onlined device to force a recheck in the ZPool to resize it without export/import:
root@mariadb:~# zpool online MYSQL-DATA /dev/sdb1
Et voila:
root@mariadb:~# zpool list MYSQL-DATA
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
MYSQL-DATA 50,0G 994M 49,0G - 5% 1% 1.00x ONLINE -
rpool 19,9G 3,36G 16,5G - 19% 16% 1.00x ONLINE -
Set autoexpand to off if you want prevent to autoexpand if partition grows:
root@mariadb:~# zpool set autoexpand=off MYSQL-DATA
Optional: Resize the LVM physical volume
Check the values:
# parted /dev/${device} "print free"
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 48.3GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
32.3kB 1049kB 1016kB Free Space
1 1049kB 48.3GB 48.3GB primary boot
48.3GB 48.3GB 999kB Free Space
# pvs
PV VG Fmt Attr PSize PFree
/dev/sda1 vg-root lvm2 a-- <35.00g 0
OK, we need to resize the physical volume
# pvresize /dev/sda1
Physical volume "/dev/sda1" changed
1 physical volume(s) resized / 0 physical volume(s) not resized
Check the values:
# pvs
PV VG Fmt Attr PSize PFree
/dev/sda1 vg-root lvm2 a-- <45.00g 10.00g
# lvextend -l +100%FREE /dev/vg-root/log
Done.
Find open but deleted files
Sometimes you have a full filesystem, but cannot see files with ls.
And the output of du -sh <mountpoint> and df -h <mountpoint> differ, because du just sums the files by traversing the directory.
Then it is time to look for files that are open by any process but deleted in the filesystem.
You can investigate the /proc kernel filesystem:
# find /proc/*/fd -ls | grep '(deleted)'
91565697 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/7 -> /tmp/ibNhVEnm\ (deleted)
91565698 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/8 -> /tmp/ibhSEF8n\ (deleted)
91565699 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/9 -> /tmp/ibADGDrl\ (deleted)
91565703 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/13 -> /tmp/ibtl5efn\ (deleted)
Or you can use lsof:
# lsof +aL1
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
mysqld 2118 mysql 7u REG 0,27 0 0 32780 /tmp/ibNhVEnm (deleted)
mysqld 2118 mysql 8u REG 0,27 0 0 32782 /tmp/ibhSEF8n (deleted)
mysqld 2118 mysql 9u REG 0,27 0 0 32786 /tmp/ibADGDrl (deleted)
mysqld 2118 mysql 13u REG 0,27 0 0 32796 /tmp/ibtl5efn (deleted)
Or for a specific running command:
# lsof +aL1 -c mariadbd
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
mariadbd 2821 mysql 7u REG 0,36 0 0 50 /tmp/#50 (deleted)
mariadbd 2821 mysql 8u REG 0,36 0 0 51 /tmp/#51 (deleted)
mariadbd 2821 mysql 10u REG 0,36 5420 0 52 /tmp/#52 (deleted)
mariadbd 2821 mysql 13u REG 0,36 0 0 53 /tmp/#53 (deleted)
To truncate the files you can use the filedescriptor of the process that has the file open:
ATTENTION! Do this ONLY if you exactly know what you are doing!!! Restarting the process is often the much safer solution!!!
# : > "/proc/<pid>/fd/<fd>"
