Table of Contents
I read this article and cringed at it.
Text bellow is my attempt to rewrite it, to [hopefully] make it more redable and somewhat educational.
I ultimately suck at writing educational stuff, but who's stop me from trying?
Intro
Assume we're in type two hyperwisor, QEMU for example, to justify things happening here.
Storage Device (aka Volume)
Let's examine initial layout:
lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0 squashfs 4.0 0 100% /snap/core20/1974
loop1 squashfs 4.0 0 100% /snap/core20/1950
loop2 squashfs 4.0 0 100% /snap/lxd/24322
loop3 squashfs 4.0 0 100% /snap/snapd/19361
loop4 squashfs 4.0 0 100% /snap/snapd/19457
sda
├─sda1 ext4 1.0 cloudimg-rootfs 7226c063-0abb-4b72-a414-b7395a2d5e26 39.1G 19% /
├─sda14
└─sda15 vfat FAT32 UEFI 134.209.201.134.209.201.16216247BE-4CB6 98.3M 6% /boot/efi
We have a couple of loop devices and sd device with some partitions on it. This is almost standard layout for a servers runninUbuntu Server or omrthung like that. For Desctop ti will be more complicated. loop devcises is irrelevant for now, so i'll skip them in fruther listings, and focus on other devices. It has three partitions on it. One formatted with ex4 file system, one formatted with fat32, and last - formatted with .....
Naming:
loop - a pseudo-device, allows access to files as a block devices
hd - IDE device
sd - SATA, SAS, USB or SCASI device
nvme0 - NVME device
mmcblk0 - SD Card or eMMC device
For example:
hda - first HDD, sda - first SSD, nvme0n1 - first NVME, mmcblk0p1 - first SD scard and so on.
Hypervisor just granted us a new clean volume. lsblk output will be slightly defferrent this time:
lsblk -f
NAME FSTYPE FSVER LABEL UUIIf a new storage device was added to our configurationD FSAVAIL FSUSE% MOUNTPOINTS
...
sda
├─sda1 ext4 1.0 cloudimg-rootfs 7226c063-0abb-4b72-a414-b7395a2d5e26 39.1G 19% /
├─sda14
└─sda15 vfat FAT32 UEFI 134.209.201.134.209.201.16216247BE-4CB6 98.3M 6% /boot/efi
sdb
Notice another device named sdb appeared. This is our new device. It won't have any patrtitions from the beginning, but it will change soon.
You can call sda a root volume, because it contain system root partition, and essential for VM to boot.
Added device - sdb can just be called a volume.
I use term volume here because none of those devices are real disks.
The Theory
The data is not a file. Data is a bunch of ones and zeroes scattered around storage medium. If we're writing all of the data to a storage just like that, we're risk losing where it was written. Another record is needed to store where data is written. Pair data on the disk and some table with address records of beginnig each file and its length and bam! We got primitive filesystem. Ok, not quite, I used this approach when recorded my first programms for ZX Spectrum to audio tape, but you get a point.
In order to be usable for OS storage should be formatted with one of many file systems. Although it can be formated directly, it's not reconmmended. (#TODO rewrite) More convinient way is to create a partition table first, which allow convinient partition placement and sizing.
(#TODO rewrite) After its creation, file system should be mounted into operating system, to access the data, and should be unmounted prior any low level operations or shutdown, to prevent data corruption or loss.
The Excercize Plan
For sdb device:
- Create a partition table;
- create a partition using all free space available on a device;
- format created partition with EXT4 file system;
- mount partition inside a OS and play with files;
- ensure file system will be re-mounted automatically, if OS is restared;
- scale a device up, then make sure all storage space can be used by the OS;
Double and triple check things from now.
You don't want to mess with wrong volume or a partition.
Even in hypervisor.
Especially - if you're not.
There is a lots of software suited for this task. GUI tools, like GParted especially useful, because they can do almost all operations you may want, and present patrition layout as convinient diagramm. Unfortunately, we're stuck in conlole for now, so no fancy diagramms.
Partition table
#TODO ILLUSTRATION WITH EMPTY NOT FORMATTED STORAGE VOLUME
There's currently two primary types of partition tables MBR and GPT.
MBR or Master Boot Record is an older partition scheme introduced in 1983. It uses first sector of a disk to store information about how partitions are organized. It compatible with a wide range [older] BIOS systems. It has some limitations in terms of partitions (4 partitions) count and maximum storage size (2 TB maximum).
GPT or GUID Partition Table is a newer standard. Part of UEFI initiative. More flexble than MBR. Suports storage of any size and can contain up to 128 partitions.
All resonably recent hardware supports GPT, so it's kinda no-braier what to choose now.
In terms of software we have two options: fdisk and parted. Parted may not be installed by default, so Fdisk it is.
Before doing anything let's check what we've got.
sudo fdisk -l /dev/sdb
Disk /dev/sdb: 100 GiB, 107374182400 bytes, 209715200 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Physical and logical sector size is the same, so we nodt need to align a new patririon. Good.
(#TODO change wording) Now launch fdisk utility and tell it to work with sdb device (#TODO volume ???).
sudo fdisk /dev/sdb
Welcome to fdisk (util-linux 2.37.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x77fd0371.
We are now in fdisk interactive mode. m key, followed by Enter can be used to show all options available. q to exit.
I'll echo fdisk here:
Any changes made - made only in memory and will not be written to device, unless you specifically ask for this.
We're gonna need next set of commands:
m - to display menu;
g - to create a GPT partition table;
n - to create a new partition;
p - to display partition info;
w - to write changes to a device;
Partition table
Now, it's time to actually do something.
First of all - partition table. Use g for that. fdisk will think a little and give you something like this in response:
Created a new GPT disklabel (GUID: 7B391787-5E44-034A-B797-904A4FDB8EB0).
Done! Now's onto partition...
Partition
Use n to create a new a new partition. fdisk will ask you a bunch of questions, primerly about desired partition geometry. If you leave answers blank it will create a single partition that fill all free space. Exactly what's needed.
Partition number (1-128, default 1):
First sector (2048-209715166, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-209715166, default 209715166):
Created a new partition 1 of type 'Linux filesystem' and of size 100 GiB.
And partiton is ready...
Verify and write
For now, a new disk laoyout was stored in memory. Before writing it to a volume lets check again, how it looks. Use p command to see partition info:
Disk /dev/sdb: 100 GiB, 107374182400 bytes, 209715200 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7B391787-5E44-034A-B797-904A4FDB8EB0
Device Start End Sectors Size Type
/dev/sdb1 2048 209715166 209713119 100G Linux filesystem
Looks fine by me.
Next action is irreversable.
Just saying.
Use w command to write the layut to a volume. fdisk will exit, dropping us back to the shell.
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
Another lsblk call will show what we are accomplitied:
$ lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
...
sda
├─sda1 ext4 1.0 cloudimg-rootfs 7226c063-0abb-4b72-a414-b7395a2d5e26 39.1G 19% /
├─sda14
│
└─sda15
vfat FAT32 UEFI 47BE-4CB6 98.3M 6% /boot/efi
sdb
└─sdb1
We have a partition! Yay!
But still need a File System.
The File system
ext4 (fourth extended filesystem) is a journaling file system for Linux, developed as the successor to ext3.
...
ext4 is the default file system for many Linux distributions including Debian and Ubuntu.
https://en.wikipedia.org/wiki/Ext4
For my purpose I will use ext4 file system, but you do you.
sudo mkfs -t ext4 /dev/sdb1
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done
Creating filesystem with 26214139 4k blocks and 6553600 inodes
Filesystem UUID: 2c2bd3fd-4028-4102-b807-60e45393a915
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done
Writing inode tables: done
Creating journal (131072 blocks):
done
Writing superblocks and filesystem accounting information: done
Now add a label to a partition, to mark its purpose - storing data:
$ sudo e2label /dev/sdb1 DATA
# or
$ sudo tune2fs –L DATA /dev/sdb1
It's not required to add labels, but it's a good practice.
You'll thank yourself later.
Final result
To examine the final result use lsblk again:
$ lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0 squash 4.0 0 100% /snap/core20/1950
loop1 squash 4.0 0 100% /snap/core20/1974
loop2 squash 4.0 0 100% /snap/lxd/24322
loop3 squash 4.0 0 100% /snap/snapd/19361
loop4 squash 4.0 0 100% /snap/snapd/19457
sda
├─sda1 ext4 1.0 cloudimg-rootfs 7226c063-0abb-4b72-a414-b7395a2d5e26 39.1G 19% /
├─sda14
│
└─sda15
vfat FAT32 UEFI 47BE-4CB6 98.3M 6% /boot/efi
sdb
└─sdb1 ext4 1.0 2c2bd3fd-4028-4102-b807-60e45393a915
Note that now sdb volume has one partition on it - sdb1.
Mount
It is the last step in our list, and it's quite easy. No operations that can wipe all your data or something.
Let's mount new partition to the OS:
$ sudo mkdir /mnt/data
$ sudo mount /dev/sdb1 /mnt/data
$ df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 392M 1.6M 390M 1% /run
/dev/sda1 49G 9.2G 40G 19% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda15 105M 6.1M 99M 6% /boot/efi
tmpfs 392M 4.0K 392M 1% /run/user/1000
/dev/sdb1 98G 24K 93G 1% /mnt/data
To see mount in action, you can do something like this:
# write the file to the partition
$ touch /mnt/data/test
# check the file
$ ls /mnt/data/test
# unmount directory
$ sudo umount /mnt/data
# check the file again
$ ls /mnt/data/test
Feel free to mount it again and check the file - it will be there.
You can check mount status in various ways: findmnt, df or even cat /proc/mounts.
Current mount will be active until next OS reboot. To get FS mounted after reboot we'll need to instruct OS to do so. There is a file /etc/fstab used exactly for this purpose. Open it with your favorite text editor and add something like this:
/dev/sdb1 /mnt/data ext4 defaults 0 0
Now, when OS knows how and where to mount our partition we can use mount -a to mount it in automated manner. Also, it will be automatically mounted after reboot.
Bonus: non-interactive mode
You made it here? Nice!
A little bonus for patience.
$ sudo parted /dev/sdb --script mklabel gpt mkpart primary ext4 0% 100%
$ sudo mkfs -t ext4 /dev/sdb1
$ sudo e2label /dev/sdb1 DATA
$ sudo mkdir /mnt/data
$ echo "/dev/sdb1 /mnt/data ext4 defaults 0 0" | sudo tee -a /etc/fstab
$ sudo mount -a
Bonus 2: same but with parted
add about change partition name