Linux Services Organization

Our goal, introduce Linux services to the enterprise world.
Contact us in contact@linuxsv.org

Linux Services Organization : Filesystem Linux Server

The filesystem concept explains how files and directories are stored in a system. The different formats used in order to store files and directories define the filesystem types that can be divided in two main categories : 'standard' and 'journaling' filesystem.

Journaling FileSystem

As data storage requirements grow in size, Linux has started to use filesystems with journaling because of the advantages that present 'journaling' filesystem compared with 'standard' filesystem :

* Storage check in the boot process is faster in journaling than in standard filesystem.

* In case of storage crash a journaling filesystem has a log (the journal) that can be used to restore the data. The 'standard' filesystem does not have this functionality.

The following are the most used journaling filesystems used in Linux :

GFS2
Global Filesystem 2 is commonly used as a cluster filesystem on a RHEL6 system cluster. It uses distributed metadata and multiple journaling.

ext4
Default filesystem for RHEL6. It is an improved version of ext3: supports larger files, faster and more efficient allocation of disk space, more robust journaling, etc.

ext3
Default filesystem for RHEL5. It is basically ext2 + journaling.

JFS
Journaled filesystem owned by IBM.

ReiserFS
Reiser FileSystem is resizable and supports fast journaling based on the concept of "balanced trees".

xfs
Journaling filesystem developer by Silicon Graphics specialized in very large files.

Standard FileSystem

All filesystems without journaling features are standard filesystems. The following is a list of the most common standard filesystems :


ext
The first Linux filesystem, used a long time ago ...

ext2
Linux Second Extended filesystem, the foundation for ext3 without journaling.

CIFS
The Common Internet File System (CIFS), an evolution of Samba/Server Message Block (SMB) system based on Microsoft and IBM network protocols. In Linux it is used to share files and printers with Microsoft Windows systems.

NFS
Network File System, commonly used to share files between Linux/Unix computers.

ISO 9660
The filesystem used to store data in CD-ROMs.

/proc
Linux virtual filesystem, used to provide information on kernel configuration and device status.

swap
The Linux swap filesystem, used to substitute the physical memory when not more memory is available.

MS-DOS and VFAT
Filesystem used to store MS-DOS-formatted data.

NTFS
Microsoft Windows NT/2000/XP/2003 filesystem designed for account security.

A mentioned ext3 is equal to ext2 + journaling so in order to migrate a ext2 filesystem to ext3 filesystem only journaling over ext2 is needed. The command tune2fs can be used to migrate ext2 filesystem on /dev/hda2 to ext3 :


$ tune2fs -j /dev/hda2

Partition creation with fdisk

Just before a filesystem is created on a partition we need create that partition. Linux provides several tools for creating disk partition, and one of this tools is fdisk command. In the following example we are going to create one partition on a disk and create an ext4 filesystem over it :

* First steep is list all disks/partitions attached on the system :

$ fdisk -l

Disk /dev/sda: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device      Boot      Start      End      Blocks      Id      System
/dev/sda1      *            1          13      104391      83      Linux
/dev/sda2                 14          783    6185025    8e      Linux LVM

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

We can see to disks on the system sda and sdb . The sda disk has two partitions : sda1 Linux partition used for /boot and sda2 that allocates and LVM partition. The second disk sdb has not any partition so lets go to create a new partition on it.

* Second steep is create a new partition on disk sdb :

$ fdisk /dev/sdb

Command (m for help): n
Create a new partition : 'n'
Command action
e extended
p primary partition (1-4)
p
Primary partition : 'p'
Partition number (1-4): 1
Partition number '1' -> sdb1 : '1'
First cylinder (1-130, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-130, default 130): +300M
Partition sdb1 size : '300M'

Command (m for help): p
Print the result before writing changes to partition table : 'p'

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device      Boot      Start      End      Blocks      Id      System
/dev/sdb1                  1            37      297171      83      Linux

Command (m for help): w
The partition table has been altered!
Write the changes to partition table : 'w'

Calling ioctl() to re-read partition table.
Syncing disks.
$ partprobe

Force the Kernel to read the new partition table : 'partprobe' command

As usual fdisk can do more : delete partition, change partition attributes, etc. For more information 'man fdisk'.

Filesystem creation with mkfs

Once a partition has been created using fdisk the next step is create a filesystem into the partition using mkfs. For example to create an ext4 filesystem on /dev/sdb1 :


Be careful, all data will be lost on /dev/sdb1 if this command is executed


mkfs.ext4 /dev/sdb1
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
74296 inodes, 297168 blocks
14858 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
37 block groups
8192 blocks per group, 8192 fragments per group
2008 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.


Now an ext4 filesystem of 300M is ready to be used once mounted on the system.

With this command several filesystem types can be created (ext2,ext3,ext4,vfat,msdos,cramfs) using the corresponding mkfs.XXX command. If we need to re-create a filesystem on an existing partition with data first we need to backup the data on a separate partition then create the new filesystem on the partition with mkfs command and finally restore the data on the new formatted partition. Keep in mind that when a filesystem is created on a partition all the data in the partition is lost.

Ext filesystem attributes

Linux ext2/ext3/ext4 filesystems attributes can be managed with lsattr command that lists file/directory filesystem attributes and chattr command that changes file/directory filesystem attributes. One common example of the use of this command is set the filesystem parameter 'immutable' to a file, that means that nobody (including root user) can remove that file :


$ ls -lrt /etc/fstab
-rw-r--r-- 1 root root 914 nov 11 2009 /etc/fstab

Root can remove /etc/fstab

$ lsattr /etc/fstab
------------- /etc/fstab

There is not filesystem attribute set

$ chattr +i /etc/fstab
Immutable filesystem parameter set, nobody including root can remove the file

$ lsattr /etc/fstab
----i-------- /etc/fstab

Now immutable parameter is displayed : 'i'

$ rm -rf /etc/fstab
rm: can not delete /etc/fstab : Operation not allowed

Mounting filesystem

Once the filesystem has created on a partition the next step is mount the filesystem into the system using mount in order to get access to files and directories on the filesystem :


$ mount partition mountpoint -t filesystemtype -o options

$ mount /dev/hda3 /mnt -t ext4 -o rw,noexec
It mounts /dev/hda3 partition which contains an ext4 filesystem in /mnt in read-write mode (rw). Binary files executions are not allowed on this filesystem (noexec).

$ mount
...
/dev/hda3 on /tmp type ext4 (rw,noexec)
...

Typing just mount lists all mounted partitions on the system.


$ umount /mnt
Just in case to umount the partition execute umount mountpoint|partition : that means that 'umount /dev/hda3' also works.

Mount options (-o option1,option2 ) control the way the filesystem is acceded by the system. This options can be the following :

atime
File inode is updated each time the file is accessed. With noatime option the file inode is not updated when the file is acceded.

auto
Searches through /etc/filesystems for the appropriate filesystem type. With noauto option en explicit mount execution is required.

defaults
Default mount options : rw, exec, suid, dev, auto, nouser, and async.

dev
Allows access to character devices and block access to devices such as drives. With nodev option access to the character devices is not allowed.

exec
Allows binaries to be executed on the filesystem. With noexec option binaries executions are not allowed.

remount
Re-mounts a currently mounted filesystem.

ro
Mounts the filesystem in read-only mode.

rw
Mounts the filesystem in read/write mode.

suid
Allows setuid or setgid file permissions on this filesystem. With nosuid setuid or setgid permission are not allowed on the filesystem.

sync
Writes and reads are done synchronously on this filesystem. With async read-write process is done asynchronously.

user
Allows nonroot users to mount this filesystem. Options noexec, nosuid, and nodev are included in this option. With nouser option only root is allowed to mount the filesystem.

/etc/fstab

Linux automates the filesystem mounting via /etc/fstab file which contains the information used on the boot process to mount the appropriate filesystems on system directories as /, /boot, /tmp, etc. :


$ cat /etc/fstab

/dev/hda2     /                   ext4 defaults 1 1
/dev/hda1     /boot          ext4 defaults 1 2
tmpfs            /dev/shm   tmpfs defaults 0 0
devpts         /dev/pts      devpts gid=5,mode=620 0 0
sysfs           /sys              sysfs defaults 0 0
proc             /proc            proc defaults 0 0
/dev/hda3    swap          swap defaults 0 0


The following are the six fields beginning on left

Label
Device to be mounted. It can be set up with LABEL (for example LABEL=/) if the physical partition has been labelled with e2label command.

Mount Point
System directory where the filesystem will be mounted.

Filesystem Type
Valid filesystem types are ext, ext2, ext3, ext4, msdos, vfat, devpts, proc, tmpfs, udf, iso9660, nfs, smb, and swap.

Mount Options
The same options as -o in mount command.

Dump Value
0 or 1. 1 means that data is automatically saved to disk by the dump command when you exit Linux.

Filesystem Check Order
Defines the order that filesystems are checked by fsck during the boot process. The root directory (/) filesystem should be set to 1, and other filesystems should be set to 2.

When the command 'mount -a' is executed the system verifies that all mount points on /etc/fstab are mounted as described on this file. If any mounting is missing then the system automatically mounts it :

$ mount -a

Autofs

Autofs daemon is responsible to mount temporarily configured directories as needed. The relevant configuration files are /etc/sysconfig/autofs, /etc/auto.master, /etc/auto.misc, and /etc/auto.net.

Default service configuration is on /etc/sysconfig/autofs :

$ cat /etc/sysconfig/autofs

...
# If nothing happens on automount within 300s the share is umounted
DEFAULT_TIMEOUT=300

#Mounts are not browseable
DEFAULT_BROWSE_MODE="no"
...

/etc/auto.master

The autofs master configuration file /etc/auto.master is the starting point of mount configurations. It includes another configuration files for specific mountings :

$ cat /etc/auto.master

...
# Mounts /misc following the instructions on /etc/auto.misc
/misc /etc/auto.misc

# Mounts /home following the instructions on /etc/auto.home
/home /etc/auto.home

# Mounts on /net a directory for each IP/hostname that is exporting any NFS directory
/net -hosts
+auto.master

/etc/auto.misc

$ cat /etc/auto.misc
...

# This line mounts the CDROM /dev/cdrom on /misc/cd when somebody tries to access to /misc/cd
cd -fstype=iso9660,ro,nosuid,nodev :/dev/cdrom

# This line mounts NFS share ftp.example.org:/pub/linux on /misc/linux when somebody tries to access to /misc/linux
linux -ro,soft,intr ftp.example.org:/pub/linux
...

/etc/auto.home

$ cat /etc/auto.home
...

# With the use of '*' and '&' each NFS exported home directory from fileserver is mounted in /home. Exported /home/user1 is mounted on /home/user1
* fileserver.example.com:/export/home/&
...

/etc/auto.net

The /etc/auto.net script lists and mounts all exported NFS shares on the host where autofs is running. It mounts on /net/nfs_share_ip all NFS shares exported from nfs_share_ip. For example the server 192.168.10.223 is exporting some NFS shares and the shares are accessible from 'server' host where 'autofs' service is running

[server]$ /etc/init.d/autofs restart
stopping automount [OK]
starting automount [OK]
Make sure that autofs is running

[server]$ cd /net/192.168.10.223
[server]192.168.10.223$ ls -lrt

drwxr-xr-x 2 root root 0 oct 31 20:51 centos55x64
drwxr-xr-x 2 root root 0 oct 31 20:51 clusterdisk
dr-xr-xr-x 3 root root 0 oct 31 20:51 diskless
drwxr-xr-x 2 root root 0 oct 31 20:51 home
drwxr-xr-x 2 root root 0 oct 31 20:51 hometest
dr-xr-xr-x 3 root root 0 oct 31 20:51 mnt
drwxr-xr-x 2 root root 0 oct 31 20:51 rhel4
dr-xr-xr-x 3 root root 0 oct 31 20:51 rhome
dr-xr-xr-x 3 root root 0 oct 31 20:51 var


As can be seen autofs mounts all exported NFS shares from 192.168.10.223 in 'server' host on /net/192.168.10.223 directory

Autofs initialization

Once autofs is configured modifying the configuration files the next step is start the service :

$ /etc/init.d/autofs restart
stopping automount [OK]
starting automount [OK]
Now the service is up and ready to be used

To be sure that the service will be active at boot time the following command must be used :

$ chkconfig --level 345 autofs on
It activates autofs services on runlevels 3, 4 and 5

Questions

1.- The fdisk command can be used to create a filesystem on a disk partition (true/false)

2.- Autofs daemon is responsible to mount all essentials filesystems as /boot and / on boot (true/false)

3.- Like fdisk, parted is a valid tool to create disk partitions (true/false)

4.- Which command must be used in order to label sda2 partition as '/' ?

5.- Which command mounts /dev/sda2 partition on /mnt in read-only mode ?

6.- Which command creates an vfat filesystem on /dev/sda3 ?

7.- Which line must be added to /etc/fstab file in order to mount /dev/sda3 partition that contains an ext4 filesystem on /mnt directory on boot ?

8.- Which command shows all mounted partitions on a system ?

9.- Which of the following commands must be used to make sure than even root can not remove /etc/rsyncd.conf file ?
A - lsattr +i /etc/rsyncd.conf
B - chattr -i /etc/rsyncd.conf
C - chattr +i /etc/rsyncd.conf
D - lsattr -i /etc/rsyncd.conf

10.- Which of the following is a journaling filesystem ?
A - ext2
B - ext4
C - ntfs
D - reiserfs

Labs

1.- Create a new 100M partition on disk sdb, create an ext4 filesystem on it and mount it on /mnt automatically at boot. Make sure that binary files execution on it is not allowed on this filesystem.

2.- Configure autofs to mount automatically on /misc/cdrom the CDROM driver. Make sure that autofs starts on boot.

3.- Create a new 50M partition on disk sdb , create an ext4 filesystem on it and mount it on /boot2 directory. Copy the content of /boot on /boot2 and mount /boot2 automatically on boot in read-only mode.

-- This page is part of Linux Server online tutorial --