Author: Andrea

Remove all non-ascii characters with perl

Sometimes when creating one-liners, the source contains non-ASCII characters that make parsing harder. To strip them all out, use Perl:

cat dirty-source.txt | perl -pe 's/[^[:ascii:]]//g' > clean-output.txt

This replaces every character that is not in the ASCII set with nothing, leaving only clean ASCII text. The same approach works for Windows line endings or other invisible characters by adjusting the character class.

If you prefer a sed alternative for the same task:

sed 's/[^[:print:]]//g' dirty-source.txt > clean-output.txt

Hope it helps!

Converting spreadsheets to text files with ssconvert

ssconvert is a utility shipped with Gnumeric, a command-line spreadsheet converter. It is useful when you have data in a spreadsheet and want to parse it from a script.

ssconvert -O 'separator=;' spreadsheet.ods spreadsheet.csv

This converts an ODS file to CSV using a semicolon as separator. ssconvert supports ODS, XLS, XLSX, CSV, HTML, and more.

You can also export specific sheets:

ssconvert -O 'separator=|;format=raw' --export-type=Gnumeric_stf:stf_csv spreadsheet.ods sheet2.csv

The format=raw option prevents ssconvert from reformatting numbers.

Enjoy!

Firefox – prevent websites from refreshing

Fed up with websites and tabs continuously reloading themselves and using resources unnecessarily? Me too!

In Firefox, you can disable this from the about:config page. Type about:config in the address bar, accept the warning, and search for:

accessibility.blockautorefresh

Set it to true. Problem solved — pages can no longer auto-refresh without your permission.

You can also keep an eye on which tab uses the most resources by typing about:performance in the address bar. Firefox shows a list of open tabs with their CPU and memory usage, making it easy to find and close the resource hogs.

Enjoy!

How to update all docker base images with one command

To pull the latest version of all your Docker images at once:

for docker in $(docker images | awk '{print $1,$2}' | sed 's/ /:/g' | grep -v REPOSITORY); do docker pull $docker; done

This lists all images, extracts repository and tag, formats as repository:tag, and pulls each one.

After updating, old image versions stay on disk. To clean them up:

docker image prune -a

This removes all unused images and frees up space.

Enjoy!

How to update all pip python packages with one command

To update all your pip-installed Python packages at once:

sudo pip install -U `pip list -o | cut -d ' ' -f1 | tr '\n' ' '`

This lists outdated packages with pip list -o, extracts their names, joins them with spaces and passes them to pip install -U. A one-liner that saves you from updating each package individually.

NB: One-liners like this are potentially destructive. Read what they do before running them, especially with sudo.

For a safer alternative on modern systems, use pip-review:

pip install pip-review
pip-review --auto

This prompts you before each update.

Enjoy!

Proxmox – Passthrough card/device to openVz container

I wanted to passthrough a dual tuner TV card (Hauppauge WinTV Nova-T-500) to an openVz container in order to use tvHeadEnd. The procedure should be the same for any PCI device.

First, identify the device you want to passthrough:

tree /dev/dvb/

This shows the device nodes under /dev/dvb/. Then mount the host device into the container by editing the container configuration file (/etc/vz/conf/CT_ID.conf) and adding:

DEVICES="c:212:*:rw"

Replace 212 with the major device number of your device (visible with ls -l /dev/dvb/…). After a restart of the container, the device should be accessible from within it.

You can verify with:

ls -l /dev/dvb/

Enjoy!

Use Remmina to satisfy all your remote connection needs

Remmina is a remote desktop client that supports multiple network protocols — RDP, VNC, NX, XDMCP and SSH — in a nice and very user-friendly GUI.

In its basic usage, you only need to know the protocol, the server address, username and password.

Remmina basic connection tab

This works perfectly if you are in the same LAN. For connections over the internet, Remmina can tunnel through SSH, configured in the SSH tab:

Remmina SSH tunnel settings

It supports RDP for Windows machines, VNC for Linux desktops, and NX for remote sessions. All saved in a single, clean interface.

Enjoy!

Test email from command line

When you set up email forwarding or configure a new MTA, you will want to confirm it is working before relying on it for alerts. A quick test is all you need.

Send a test message

mail -s Test myEmail@gmail.com

If the mail command is not available, install it with apt-get install mailutils. Then check your inbox.

If it does not arrive

Run mailq to see if the message is stuck in the queue. The mail log at /var/log/mail.log will also show the delivery attempt and any errors.

Enjoy!

PHP: remove duplicate elements from a multi-dimensional array

This post is over 3 years old, so please keep in mind that some of its content might not be relevant anymore.

PHP’s array_unique() is great for removing duplicates from a flat array, but it chokes on multi-dimensional arrays. If you have nested data and need to remove duplicates, here’s a clean one-liner that does the job.

The trick: serialize, unique, unserialize

The idea is straightforward: flatten the array temporarily by serialising each element, let array_unique() do its thing on the flat list, then unserialise everything back to its original structure.

$uniqueArray = array_map("unserialize", array_unique(array_map("serialize", $array)));

Working from the inside out:

  • array_map("serialize", $array) — turns each nested element into a string representation
  • array_unique() — removes duplicate strings (which means duplicate nested structures)
  • array_map("unserialize", ...) — restores each string back to its original PHP value

The result is a multi-dimensional array with all duplicate entries removed, regardless of how deep the nesting goes. You keep the original structure and no external libraries needed.

Hope it helps!

Ban offending IP addresses – fail2ban

If you run a public SSH server, you will see failed login attempts in your logs constantly. Fail2ban monitors those logs and adds firewall rules to block offending IPs.

Here is what you see in /var/log/auth.log from a machine that has been online for a few days:

Nov 20 06:30:43 serverName sshd[63467]: Address 91.191.166.95 maps to 91-191-166-95.netdirekt.com.tr
Nov 20 00:47:45 serverName sshd[24617]: Invalid user fluffy from 221.179.34.37
Nov 20 00:47:55 serverName sshd[24617]: Failed password for invalid user fluffy from 221.179.34.37

Install and configure:

apt-get install fail2ban
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit the .local file. A typical SSH jail:

[DEFAULT]
ignoreip = 127.0.0.1 192.168.1.0/24
bantime = 600
maxretry = 3

[ssh]
enabled = true

Check banned IPs:

iptables -L
iptables -L --line-numbers

Remove a specific ban:

iptables -D fail2ban-ssh

On modern systems, use fail2ban-client for an easier overview:

sudo fail2ban-client status sshd

Set it up once and it will quietly block thousands of attempts without you thinking about it.

Hope it helps!