Formatting a partition from command line

When you need to format a partition from the command line, a few commands are all you need. Here’s the typical workflow.

First, create the filesystem:

mkfs.ext4 /dev/sdxy

If you want to reserve less space for the root user:

tune2fs -m 0 /dev/sdxy

Check the filesystem for errors before mounting:

fsck.ext4 -fy /dev/sdxy

Then mount it where you need it:

mount /dev/sdxy /wherever/I/want

Replace /dev/sdxy with your actual partition and /wherever/I/want with your mount point.

Enjoy!

Accelerate downloads with lftp

lftp is a versatile command-line file transfer tool. What makes it stand out is its ability to open multiple connections for a single download, which can significantly improve transfer speeds on servers that limit bandwidth per connection.

Parallel downloads with pget

The command I use most is pget with the -n option for parallel connections:

lftp -e 'pget -n 3 http://example.com/file.zip'

If the server limits each connection to 100KB/s, three connections will give you roughly 300KB/s.

Resuming interrupted downloads

If a download fails halfway through, pget continues from where it left off when you run the same command again:

lftp -e 'pget -c -n 3 http://example.com/file.zip'

Mirroring directories

lftp can mirror entire directory structures:

lftp -e 'mirror --parallel=3 /remote/path /local/path'

Queue management

lftp http://example.com
queue pget file1.zip
queue pget file2.zip
run

Enjoy!

Logical Volume Management basics

I have explained how to activate an LVM logical volume before, but never talked much about LVM itself. Here is a brief introduction.

I have been playing with LVM for about a couple of years now and I must say I have found it very practical, even for managing simple home desktops and servers. The main feature I use is the ability to resize volumes online, but LVM is useful for other things too: creating snapshots, combining multiple disks into a single volume group, and adding redundancy through mirroring.

Physical Volumes, Volume Groups, Logical Volumes

LVM works in three layers:

  • Physical Volume (PV) — a disk or partition that LVM can use. Initialised with pvcreate.
  • Volume Group (VG) — a pool of storage made from one or more physical volumes.
  • Logical Volume (LV) — a virtual partition carved out of a volume group. This is what you format and mount.

Start by preparing a partition (use fdisk to set the type to 8e — Linux LVM):

sudo pvcreate /dev/sdb1
sudo vgcreate my_volume_group /dev/sdb1
sudo lvcreate -L 10G -n my_volume my_volume_group

Now format and mount it:

sudo mkfs.ext4 /dev/my_volume_group/my_volume
sudo mount /dev/my_volume_group/my_volume /mnt

Resizing

Need more space? If you have free space in the volume group:

sudo lvextend -L +5G /dev/my_volume_group/my_volume
sudo resize2fs /dev/my_volume_group/my_volume

The filesystem is resized online — no unmounting required.

Snapshots

LVM snapshots let you take a point-in-time copy of a volume. Useful for backups:

sudo lvcreate -L 1G -s -n my_snapshot /dev/my_volume_group/my_volume

The snapshot starts small and grows as changes are made to the original volume. Remove it when no longer needed:

sudo lvremove /dev/my_volume_group/my_snapshot

Enjoy!

How to run a system command in background from PHP

If you are at the command line and want a process to run in the background, you add an & at the end of the command:

sleep 10 && echo "Done!" &

The same approach works from PHP when you need to run a long process without making the user wait:

exec("tar czf /backup/site.tar.gz /var/www > /dev/null 2>&1 &");

The > /dev/null 2>&1 discards output so PHP doesn’t wait for it. The & sends the process to the background.

PHP functions compared

  • exec() — returns last line of output, does not block if output is redirected
  • shell_exec() — returns full output as a string
  • system() — outputs directly to the browser
  • passthru() — for binary output

Using nohup and PIDs

exec("nohup tar czf /backup/site.tar.gz /var/www > /dev/null 2>&1 & echo $!", $output);
$pid = $output[0];

nohup keeps the process alive after PHP exits. The PID lets you monitor or kill it later.

Enjoy!

Install NX server/client on Ubuntu Lucid

My favourite NX server is Freenx.
FreeNX is a GPL implementation of the NX server/client that allows you to graphically access your machine over the Internet.
FreeNX runs natively on SSH, it’s open source and, above all, fast.

So, first of all you need to add the free-nx repository:

sudo add-apt-repository ppa:freenx-team

and then update and install freenx:

sudo apt-get update
sudo apt-get install freenx

The client I use instead, is from NoMachine.
The Linux version can be download from here.
If you use a different Ubuntu version you can follow the Freenx documentation.

Enjoy!

How to fix GRUB 2 with Ubuntu live cd

When GRUB fails and your system won’t boot, the errors can be intimidating. But fixing it with a live CD is straightforward.

Boot from an Ubuntu live CD or USB. Once the desktop loads, open a terminal.

First, mount your root partition:

sudo mkdir /mnt/newroot
sudo mount /dev/sdxy /mnt/newroot

Bind the virtual filesystems:

sudo mount --bind /dev /mnt/newroot/dev
sudo mount --bind /proc /mnt/newroot/proc
sudo mount --bind /sys /mnt/newroot/sys

Chroot into your system:

sudo chroot /mnt/newroot

Reinstall GRUB:

sudo grub-install /dev/sdx
sudo update-grub

Exit the chroot, unmount everything, and reboot. Remove the live CD when prompted.

For UEFI systems, the command changes slightly:

sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi

If you can still reach a grub rescue prompt, try booting manually:

set root=(hd0,msdos1)
set prefix=(hd0,msdos1)/boot/grub
insmod normal
normal

Enjoy!

Mount Windows share on Linux

Sometimes I still have to deal with Windows. :-/ This time it’s an office shared disk. I was able to mount it automatically but only root could write in it, which meant I couldn’t paste files from Nautilus.

To overcome this I added the following entry in /etc/fstab:

//192.168.0.1/deviceName$ /mountpoint cifs
domain=hq,username=myUsername,password=myPassword,
_netdev,uid=username,gid=username 0 0

The _netdev option ensures the share is mounted only when the network is available, which is particularly useful on laptops that switch networks.

Once you have changed the fstab, unmount the network drive if it was already mounted:

sudo umount /mountpoint

and then mount it again:

sudo mount -a

If you only need occasional access to a Windows share, smbclient is a quick alternative:

smbclient //192.168.0.1/deviceName$ -U myUsername

This opens an FTP-like interface for browsing and transferring files.

Hope it works for you!

Linux: what is partition UUID and how to use it

If you have worked with Linux disks, you have seen device names like /dev/sda1. They work fine until you add or remove a disk — then the names can shift, breaking your mount points and scripts. UUIDs solve this because they are permanent and tied to the filesystem itself.

Instead of writing /dev/sda1 in fstab, use the UUID:

UUID=65b14742-9610-4289-8ea4-af8edcc5d8da /media/data ext4 defaults 0 2

There are a few ways to find the UUID of a partition:

sudo vol_id -u /dev/sda2
sudo blkid
ls -l /dev/disk/by-uuid/

On older systems vol_id was the standard tool, while blkid is the modern equivalent available on virtually all current distributions. You can also see the PARTUUID with blkid, which identifies the partition table entry rather than the filesystem — this is useful for booting with UEFI.

If UUIDs feel unwieldy, you can label a filesystem and use the label instead:

sudo e2label /dev/sda1 mydata
LABEL=mydata /media/data ext4 defaults 0 2

Labels are easier to read but not guaranteed to be unique. UUIDs are the safer choice for production systems.

Hope it works!

Load LVM partition manually

Sometimes you have to load a logical volume that is not part of the main filesystem. Other times you might have to mount a volume from a live CD, perhaps to recover data from a system that is no longer booting.

In both cases it needs to be done manually.

Install the LVM tools

If lvm2 is not already installed:

sudo apt-get install lvm2

Scan for and activate volumes

If you can see the disk with fdisk but LVM does not recognise the volumes, the kernel may not have scanned for them yet. Start by scanning for physical volumes and volume groups:

sudo pvscan
sudo vgscan
sudo lvscan

pvscan detects LVM physical volumes, vgscan finds the volume groups, and lvscan lists the logical volumes within them.

Activate and mount

If the volume group is inactive (common after booting from a live CD or moving a disk to another machine), activate it:

sudo vgchange -ay

This activates all available volume groups. Now mount the logical volume:

sudo mount /dev/VolumeGroupName/LogicalVolumeName /mnt

Replace VolumeGroupName and LogicalVolumeName with the names shown by lvscan. If you are not sure what the volume is called, check /dev/mapper/ for device-mapper entries.

Troubleshooting

If lvscan shows your volumes but vgchange reports “Volume group not found”, check the output of vgscan carefully. Sometimes the volume group name contains a hyphen (e.g. my-server) which can confuse the command — try using the full path from /dev/mapper/ instead:

sudo mount /dev/mapper/my--server-myvol /mnt

Note the double hyphen — LVM represents the hyphen in the volume group name as — in the mapper path.

Enjoy!

Linux: How to forward system emails

By default, most Linux systems deliver system emails — cron output, monitoring alerts, security notices — to the local root mailbox. If you never check it, those messages pile up and you might miss something important.

The simplest way to deal with this is to forward them to an external address. The most common method is the /etc/aliases file. Add a line like:

root: you@example.com

Then run newaliases to rebuild the alias database. Local mail will be forwarded to the external address via your MTA.

For per-user forwarding, create a .forward file in the user’s home directory with your email address in it. This doesn’t need an alias rebuild and works even if you don’t have sudo access.

Enjoy!