Linux Tipps und Tricks: Difference between revisions
From Lolly's Wiki
Jump to navigationJump to search
| (2 intermediate revisions by the same user not shown) | |||
| Line 273: | Line 273: | ||
Done. | Done. | ||
==Find open but deleted files== | |||
Sometimes you have a full filesystem, but cannot see files with ls.<br> | |||
And the output of <i>du -sh <mountpoint></i> and <i>df -h <mountpoint></i> differ, because <i>du</i> just sums the files by traversing the directory.<br> | |||
Then it is time to look for files that are open by any process but deleted in the filesystem.<br> | |||
You can investigate the /proc kernel filesystem: | |||
<syntaxhighlight lang=bash> | |||
# find /proc/*/fd -ls | grep '(deleted)' | |||
91565697 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/7 -> /tmp/ibNhVEnm\ (deleted) | |||
91565698 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/8 -> /tmp/ibhSEF8n\ (deleted) | |||
91565699 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/9 -> /tmp/ibADGDrl\ (deleted) | |||
91565703 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/13 -> /tmp/ibtl5efn\ (deleted) | |||
</syntaxhighlight> | |||
Or you can use <i>lsof</i>: | |||
<syntaxhighlight lang=bash> | |||
# lsof +aL1 | |||
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME | |||
mysqld 2118 mysql 7u REG 0,27 0 0 32780 /tmp/ibNhVEnm (deleted) | |||
mysqld 2118 mysql 8u REG 0,27 0 0 32782 /tmp/ibhSEF8n (deleted) | |||
mysqld 2118 mysql 9u REG 0,27 0 0 32786 /tmp/ibADGDrl (deleted) | |||
mysqld 2118 mysql 13u REG 0,27 0 0 32796 /tmp/ibtl5efn (deleted) | |||
</syntaxhighlight> | |||
Or for a specific running command: | |||
<syntaxhighlight lang=bash> | |||
# lsof +aL1 -c mariadbd | |||
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME | |||
mariadbd 2821 mysql 7u REG 0,36 0 0 50 /tmp/#50 (deleted) | |||
mariadbd 2821 mysql 8u REG 0,36 0 0 51 /tmp/#51 (deleted) | |||
mariadbd 2821 mysql 10u REG 0,36 5420 0 52 /tmp/#52 (deleted) | |||
mariadbd 2821 mysql 13u REG 0,36 0 0 53 /tmp/#53 (deleted) | |||
</syntaxhighlight> | |||
To truncate the files you can use the filedescriptor of the process that has the file open:<br> | |||
<b>ATTENTION! Do this ONLY if you exactly know what you are doing!!! Restarting the process is often the much safer solution!!!</b> | |||
<syntaxhighlight lang=bash> | |||
# : > "/proc/<pid>/fd/<fd>" | |||
</syntaxhighlight> | |||
Latest revision as of 14:00, 19 November 2024
Hard reboot
This is the hard way to kick your kernel into void. No filesystem sync is done, just and ugly fast direkt reboot! You should never do this...
# echo 1 > /proc/sys/kernel/sysrq
# echo b > /proc/sysrq-trigger
First line enables sysrq, second line sends the reboot request.
For more look at kernel.org!
Scan all SCSI buses for new devices
# for i in /sys/class/scsi_host/host*/scan ; do echo "- - -" > $i ; done
Scan all FC ports for new devices
!!!Be CAREFUL!!! This command line issues a Loop Initialization Protocol (LIP). This is a bus reset hat means that removed devices in the fabric will disappear and new ones will appear. !!!BUT the connection might get lost for a moment!!! The softer way is to scan the SCSI buses.
# for i in /sys/class/fc_host/*/issue_lip ; do echo "1" > $i ; done
Rescan a device (for example after changing a VMDK size)
# device=sda
# echo 1 > /sys/class/block/${device}/device/rescan
This is for device sda after changing the VMDK from 20GB to 25GB:
# device=sda
# echo "$[ 512 * $(</sys/block/${device}/size) / 1024 ** 3 ] GB"
20 GB
# echo 1 > /sys/class/block/${device}/device/rescan
# echo "$[ 512 * $(</sys/block/${device}/size) / 1024 ** 3 ] GB"
25 GB
# parted /dev/${device} "print free"
Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 10485760 blocks) or
continue with the current setting?
Fix/Ignore? F
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 26,8GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
2 17,4kB 1049kB 1031kB bios_grub
1 1049kB 21,5GB 21,5GB zfs
21,5GB 26,8GB 5369MB Free Space
I want to put the free space into partition 1 and resize the rpool:
# parted /dev/${device} "resizepart 1 -1"
# parted /dev/${device} "print free"
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 26,8GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
2 17,4kB 1049kB 1031kB bios_grub
1 1049kB 26,8GB 26,8GB zfs
26,8GB 26,8GB 983kB Free Space
# zpool list rpool
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 19,9G 1,68G 18,2G - 14% 8% 1.00x ONLINE -
# zpool set autoexpand=on rpool
# zpool status rpool
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
sda1 ONLINE 0 0 0
# zpool online rpool sda1
# zpool list rpool
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 24,9G 1,69G 23,2G - 11% 6% 1.00x ONLINE -
# zpool set autoexpand=off rpool
Done.
Remove a SCSI-device
Let us say we want to remove /dev/sdb.
Be careful! Like in this example the lowest SCSI-ID is not always the lowest device name! Check it with lsscsi from the Ubuntu package lsscsi:
# lsscsi
[2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0
[32:0:0:0] disk VMware Virtual disk 1.0 /dev/sdb
[32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda
Then check it is not longer in use:
- mount
- pvs
- zpool status
- etc.
Then delete it:
# echo 1 > /sys/bus/scsi/drivers/sd/32\:0\:0\:0/delete
The 32:0:0:0 is the number reported from the lsscsi above.
Et voila:
# lsscsi
[2:0:0:0] cd/dvd NECVMWar VMware SATA CD00 1.00 /dev/sr0
[32:0:1:0] disk VMware Virtual disk 1.0 /dev/sda
Copy a GPT partition table
Copy partition table of sdX to sdY:
# sgdisk /dev/sdX --replicate=/dev/sdY
# sgdisk --randomize-guids /dev/sdY
Or with:
# sgdisk --backup=sdX.table /dev/sdX
# sgdisk --load-backup=sdX.table /dev/sdY
# sgdisk -G /dev/sdY
-R, --replicate=second_device_filename
Replicate the main device's partition table on the specified second device. Note that the replicated partition table is an exact
copy, including all GUIDs; if the device should have its own unique GUIDs, you should use the -G option on the new disk.
-G, --randomize-guids
Randomize the disk's GUID and all partitions' unique GUIDs (but not their partition type code GUIDs). This function may be used
after cloning a disk in order to render all GUIDs once again unique.
Resize a GPT partition
The partition was resized in VMWare from ~6GB to ~50GB.
In the VM I did Remove a SCSI-device for the resized device and then Scan all SCSI buses for new devices after that parted saw the new size.
Correct the GPT partition table
root@mariadb:~# parted /dev/sdb
GNU Parted 3.2
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an extra 92274688 blocks) or continue with the
current setting?
Fix/Ignore? F <-- ! choose F
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 53,7GB <-- ! the new size is reported now
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 6442MB 6441MB zfs
Resize the partition
root@mariadb:~# parted /dev/sdb
GNU Parted 3.2
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 53,7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 6442MB 6441MB zfs
(parted) resizepart 1
End? [6442MB]? 53,7GB <-- ! Put new size here
(parted) p <-- ! Control if it worked
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 53,7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 53,7GB 53,7GB zfs
(parted) q
Information: You may need to update /etc/fstab.
Optional: Resize the ZPool in it
Check the actual values:
root@mariadb:~# zpool list MYSQL-DATA
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
MYSQL-DATA 5,97G 994M 5,00G 44G 47% 16% 1.00x ONLINE -
root@mariadb:~# zpool get autoexpand MYSQL-DATA
NAME PROPERTY VALUE SOURCE
MYSQL-DATA autoexpand off default
Now inform ZPool to grow to the end of the partition. Set autoexpand to on:
root@mariadb:~# zpool set autoexpand=on MYSQL-DATA
Send an online to the already onlined device to force a recheck in the ZPool to resize it without export/import:
root@mariadb:~# zpool online MYSQL-DATA /dev/sdb1
Et voila:
root@mariadb:~# zpool list MYSQL-DATA
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
MYSQL-DATA 50,0G 994M 49,0G - 5% 1% 1.00x ONLINE -
rpool 19,9G 3,36G 16,5G - 19% 16% 1.00x ONLINE -
Set autoexpand to off if you want prevent to autoexpand if partition grows:
root@mariadb:~# zpool set autoexpand=off MYSQL-DATA
Optional: Resize the LVM physical volume
Check the values:
# parted /dev/${device} "print free"
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 48.3GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
32.3kB 1049kB 1016kB Free Space
1 1049kB 48.3GB 48.3GB primary boot
48.3GB 48.3GB 999kB Free Space
# pvs
PV VG Fmt Attr PSize PFree
/dev/sda1 vg-root lvm2 a-- <35.00g 0
OK, we need to resize the physical volume
# pvresize /dev/sda1
Physical volume "/dev/sda1" changed
1 physical volume(s) resized / 0 physical volume(s) not resized
Check the values:
# pvs
PV VG Fmt Attr PSize PFree
/dev/sda1 vg-root lvm2 a-- <45.00g 10.00g
# lvextend -l +100%FREE /dev/vg-root/log
Done.
Find open but deleted files
Sometimes you have a full filesystem, but cannot see files with ls.
And the output of du -sh <mountpoint> and df -h <mountpoint> differ, because du just sums the files by traversing the directory.
Then it is time to look for files that are open by any process but deleted in the filesystem.
You can investigate the /proc kernel filesystem:
# find /proc/*/fd -ls | grep '(deleted)'
91565697 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/7 -> /tmp/ibNhVEnm\ (deleted)
91565698 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/8 -> /tmp/ibhSEF8n\ (deleted)
91565699 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/9 -> /tmp/ibADGDrl\ (deleted)
91565703 0 lrwx------ 1 mysql mysql 64 Nov 19 12:55 /proc/2118/fd/13 -> /tmp/ibtl5efn\ (deleted)
Or you can use lsof:
# lsof +aL1
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
mysqld 2118 mysql 7u REG 0,27 0 0 32780 /tmp/ibNhVEnm (deleted)
mysqld 2118 mysql 8u REG 0,27 0 0 32782 /tmp/ibhSEF8n (deleted)
mysqld 2118 mysql 9u REG 0,27 0 0 32786 /tmp/ibADGDrl (deleted)
mysqld 2118 mysql 13u REG 0,27 0 0 32796 /tmp/ibtl5efn (deleted)
Or for a specific running command:
# lsof +aL1 -c mariadbd
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
mariadbd 2821 mysql 7u REG 0,36 0 0 50 /tmp/#50 (deleted)
mariadbd 2821 mysql 8u REG 0,36 0 0 51 /tmp/#51 (deleted)
mariadbd 2821 mysql 10u REG 0,36 5420 0 52 /tmp/#52 (deleted)
mariadbd 2821 mysql 13u REG 0,36 0 0 53 /tmp/#53 (deleted)
To truncate the files you can use the filedescriptor of the process that has the file open:
ATTENTION! Do this ONLY if you exactly know what you are doing!!! Restarting the process is often the much safer solution!!!
# : > "/proc/<pid>/fd/<fd>"
