Information and reviews on Linux, UNIX and BSD operating systems
Unix is a multi-user, multi-tasking operating system that was initially developed by Bell Labs in the 1970s. Various similar operating systems that are POSIX compliant (ie compatible with Unix) have since been developed (POSIX is an abbreviation for Portable Operating System Interface for UNIX). These include the following: the BSD family (OpenBSD, FreeBSD, NetBSD, Mac OS/X), Solaris, IRIX, AIX, SunOS, UnixWare, OpenServer and Linux.
Software written for POSIX compliant operating systems is generally able to be compiled for all POSIX platforms with little or no platform-specific changes. This makes the commands and utilities available for this extended family of operating systems very similar, and there are few differences between administering them.
The articles in this section are about useful command line utilities and commonly used application software on POSIX compliant operating systems. For Mac OSX specific articles see my OSX section.
Post sort order: Post Date (Newest First) | Post Date (Oldest First) | Alphabetical | Date Updated
Using the "ls" command to list directory contents from the command line with the -l flag will include the date and time down to the minute, but not show the seconds. This post shows how to display the seconds as well with bash on Linux and OSX.
Top is a useful command line tool for showing processes running on Linux such as how much CPU and memory they've been using and how long they've been running, and also showing the system load, CPU and memory usage as a whole. But what if you only one to show output for one process or command?
I'm sure you've done it before: added a commit message with git but typed it in wrong and need to change it. If you haven't pushed the commit yet then it's as simple as 'git commit --amend -m "New commit message"'.
I needed a really cheap standalone virtual server to test some stuff, on the internet and not running behind a VDSL router at my office, and got the cheapest RamNode server for $13.50 for a whole year.
If you need to use nslookup on a Debian server and you're getting an "nslookup: command not found" then use apt-get to install dnsutils.
The base64 command is used to encode and decode data to and from base64. This post shows how to decode base64 data in a file and when copying and pasting.
If it was nice and intuitive to install Java on Debian (and other Debian based distros such as Ubuntu), I wouldn't need to write this post because it would just be "apt-get install java". But no, I always forget what I need to install, hence this post...
SAN Certificates (Subject Alternative Names Certificates) allow you to have multiple domain names on a secure single certificate, which means you can serve up multiple secured domains on a single IP address without using SNI (Server Name Indication). This post shows how to generate the CSR (Certificate Signing Request) for a SAN Certificate.
You may need to use different identity files when logging into ssh/sftp servers, so this post shows how to specify a different one to use from the command line, and then always using that key by default in the config file.
A user on Linux belongs to a primary group, which is specified in the /etc/passwd file, and can be assigned to multiple supplementary groups, which are specific in the /etc/group file. The usermod command can be used after creating to user to assign them to additional groups(s).
When you go to install PHP on a Debian server, it will attempt to install a whole bunch of Apache packages instead. This is not very useful if you want to install a different web server, or no web server at all.
The ls command is used to list directory contents when using the bash shell (and other shells), and has a variety of flags to choose what to display and how to format. This post shows how to list just the directories using ls.
cURL is an extremely useful command line tool for making HTTP requests and can be used for diagnosing errors, downloading content and so. But what if you just want to see the response headers to see, for example, if a page is doing the right sort of redirect?
When testing some website redirects with curl from the command line, I got the error message "curl: (3) [globbing] bad range specification in column 120"
This is a quick reference page for some tools for configuring web server settings and testing secure certificates.
Use the ssh-keygen command with the -p flag to change or remove the passphrase for an SSH RSA private key file.
The CVE-2015-0235 "GHOST" exploit exposes a buffer overflow in glibc gethostbyname. This post shows how you can tell if your Debian Squeeze or Lenny install is affected and how to patch it.
Debian releases are named after characters from Toy Story and are frequently referred to with the name rather than the version. This post is for my own quick reference for the version-to-name of each Debian release.
I have a Debian virtual server running on VMWare Fusion and there's no RAID setup configured. However, I kept getting emails with the subject "info: mpt raid status change on debian" so needed to switch the service off.
This post looks at the error message "mysql-server : Depends: mysql-server-5.1 but it is not going to be installed" when attempting to install MySQL on a fresh install of Debian 6, when Debian 7 is the current up to date distro.
I recently reinstalled the operating system on my MacBookPro and discovered when trying to SSH into a new server for the first time I was getting the error "Failed to add the host to the list of known hosts (/Users/chris/.ssh/known_hosts)." This post shows what I needed to do to fix them problem.
Rsync is a useful command line utility for synchronising files and directories across two different file systems. I recently needed to use rsync to only copy over files that did not already exist at the other end, so this post documents how to do this.
I very occasionally need to support people with websites on servers other than those I manage. If the server is only accessible using FTP and I need to get a copy of the entire website, then I use the ncftpget command line tool to recursively download all the files and directories in one go.
When migrating a mono application / web service running on one server to a new server I got an error telling me that the "Argument cannot be null" and "Parameter name: path". This post shows the full error message and the additional configuration options that needed to be set to fix it.
Lynx is a command line web browser which I often use for checking the behaviour of redirection headers and content types in a web page's http headers. The previous post showed how to get the headers with Lynx and this post how to set the user agent.
Lynx is a command line web browser which I often use for checking the behaviour of redirection headers and content types in a web page's http headers. This first post shows how to get the headers and the second post how to set the user agent.
If you're a web developer like me who needs to test sending emails from a development version of a website, but your ISP blocks port 25 outbound, then it's possible to set up a "tunnel" via SSH. You then send mail to port 25 on localhost and it actually sends the mail from the server at the other end.
The uname -a command will tell you which kernel version of Linux is running but doesn't give you any information about the distribution or distro version number. This quick tip shows how to do this for CentOS / Red Hat Enterprise Linux.
If you have a lot of files to upload to a website and the only access is via FTP then you can use a nice GUI tool to just drag and drop the directory, or you can use the command line ncftpput tool.
So you need to put a tab on the command line when using BASH on Linux, Mac, BSD? How to do it? The tab character is usually used for command completion etc so typing in a tab won't render one on the command line.
I recently needed to set up a CentOS 5 box and completely forgot how to set the timezone. There's apparantly a nice easy text GUI for doing it in RHEL but I couldn't find the same thing on the CentOS box so here's how to do it. It should also work for most Linux systems.
A little while ago I posted how to use the Linux/Unix/OSX find command to locate files bigger or smaller than a particular size but didn't cover how to find zero length files. This is shown here.
While trying to troubleshoot why email wasn't being delivered to one of my mail aliases on a new server this morning, I discovered a nifty little trick for testing the deliverability of an email address using exim from the command line.
When entering commands in a bash shell each command is stored in the history which is written to the .bash_history file when logging out. This post shows how to clear the history for the current session, and how to clear all commands completely from the history.
The find command is really useful for locating files and directories etc on *nix and Mac. I recently needed to find all files greater than 1MB but it wasn't all that clear how to do this from the find man page so I've written this post to show how to do it.
In the past to download a sequence of files (e.g named blue00.png to blue09.png) I've used a for loop for wget but there's a simpler and more powerful way to do the same thing with curl.
I recently posted a Subversion Command Line Script to export changed files and in response to a comment on that page have posted a new version here, which writes out the current revision number to a file and only exports from that revision when the script is run again.
Sometimes a script may need to only be run as root or using sudo, or run ensuring that it is not being run by root or using sudo. This quick post shows how to check if it's the root user and exit the script.
A couple of days ago I posted how to export just the added/modified files from a subversion repository between two revisions using TortoiseSVN, the Windows Explorer SVN plugin. This post has a command line script which is used to achieve the same thing and means it can be run from UNIX based systems from the CLI.
This post is more of a reference for myself rather than anything else to solve an issue I was having with DNS lookups on a CentOS machine. It's how to prevent the /etc/resolv.conf file being overwritten when the system restarts if you have a custom configuration you don't want changed.
There may be times when you need to know the actual location a BASH script is located within the script. This can be done with a combination of the $0 value and the dirname command.
As part of looking at geotargeting with PHP and GeoIP the other day I posted how to enable the PHP GeoIP functions on Debian 5. Unfortunately there doesn't seem to be any way to update the GeoIP.dat file automatically with more up to date IP address mapping so this post shows how it can be done manually, and provides a BASH script for automation.
If you call the curl_init function in PHP and do not have the curl module installed you will get the error message "Fatal error: Call to undefined function curl_init() in /path/to/script.php on line XY". This post shows how to install the PHP curl module on Debian Linux and the instructions should also work on all Debian derived distros such as the Ubuntu family.
If you call the ImageCreateFromPNG function (and similar image processing functions) in PHP and do not have the GD module installed you will get the error message "Fatal error: Call to undefined function ImageCreateFromPNG()" error message. This post shows how to install the PHP GD module on Debian Linux and the instructions should also work on all Debian derived distros such as the Ubuntu family.
If you are getting the "Fatal error: Call to undefined function imap_open()" error in PHP when using the IMAP functions then they are not installed with your installation of PHP. This post looks at how to install the PHP IMAP functions on Debian and should also work for Ubuntu/Kubuntu/etc and other Debian derived Linux distros.
The bzip2 command line tool decompresses .bz compressed files. By default it will delete the original file, naming it the same as the original filebut without the extension and keep it in the same directory as the original file. This post looks at how to write it out to a different directory and/or with a different filename.
Samba is a free re-implementation of the SMB/CIFS networking protocol which creates a compatible way of sharing files with Windows computers on Linux and BSD computers. This post looks at how to tell which version of samba is running on the server from the command line.
If you are getting the "Fatal error: Call to undefined function imap_open()" error in PHP when using the IMAP functions then they are not installed with your installation of PHP. This post looks at how to install the PHP IMAP functions on CentOS and should also work for Red Hat Enterprise Linux (RHEL), Fedora and other derived Linux distros.
When trying out the Nagios server monitoring software on a CentOS machine using the Fedora Quickstart documentation to install it, I was getting "The requested URL /nagios/cgi-bin/trends.cgi was not found on this server" errors when accessing the Reporting -> Trends function. This post looks at the solution.
This post looks at how to list the installed packages with YUM from the command line for YUM based Linux distributions, such as CentOS and Fedora.
Subversion (SVN) is a version control system. This post looks at how to install subversion on CentOS (the process is similar for other Linux distros) and the setting up a repository.
Cron is a time based scheduling service on Linux and Unix computers which allows you to run process at specific times for example once a day, once every hour and so on. This brief post looks at how to run a cron command every 15 minutes.
Putty is an SSH/telnet/etc client which allows you to connect to remote servers such as a Linux or BSD webserver. When you change directories etc in a terminal shell the window title in Putty is likely to change, depending on the shell's settings. This post looks at how to change the Putty window title to be what you want and how to prevent it changing as you change directories.
Man pages on Linux and other *NIX operating systems document how commands work. I've often noticed in a default install on Red Hat based distros that odd UTF characters display such as Â and â and these can obscure the meaning of the information in the manpage. This post looks at how to make these weird UTF8 characters display normally in Linux manpages.
If you've ever accidentally hit Ctrl+S in the Bash shell it appears to come inactive and no matter what you do no output is echoed and nothing appears to be happening. This post looks at how to make the shell active again.
I needed to add some monitoring of one of my customer's mail servers and compiled sysmon on a CentOS Linux machine to do the monitoring. When compiling sysmon I got the error message "undefined reference to `yywrap'". This post looks at the full error message and the solution.
If you do a lot of command line stuff in Linux, sooner or later you'll come across the "argument list too long" error when copying, moving, deleting, listing etc files in a directory that contains a lot of files. This post looks at a couple of ways of solving this problem.
A long running process running on a Unix/Linux command line may stop running if you lose your SSH session. Using the "nohup" command will allow the command to keep running if you lose your connection or need to logout. This post looks at how to do this.
Often when shell scripting with BASH you need to test if a file exists (or doesn't exist) and take appropriate action. This post looks at how to check if a file exists with the BASH shell.
So you have a file that starts with a dash/hyphen/- and need to delete it. But when you try to do so, the "rm" command complains that you have passed an "invalid option --". This post looks at the simple solution to deleting a file that starts with a dash/hyphen.
logwatch is a daily process for reporting and analyzing log files and I recently started getting errors on one of the CentOS Linux machines I manage and no daily report. The particular machine gets thousands of bounced emails per day and the mail log files get very large. logwatch had been reporting to me every single bounced email message so the report emails had got very large before stopping altogether. This post looks at the error messages I got and what I did to fix the problem.
logwatch is a system log analyzer and reporter which emails daily reports about information in the system log files in /log. On CentOS and Red Hat Enterprise Linux this is usually configured by default to email the root user on a daily basis with the results from the log analysis. This post looks at how to stop logwatch from reporting on a particular service, for example sendmail and postfix.
When you connect to a host using SSH or SFTP it does a series of checks to ensure you are connecting to the host you are expecting to connect to. One of these is a reverse lookup on the IP address to check the hostname is the same as the hostname you are connecting to. If it's not, you'll get an error message like "reverse mapping checking getaddrinfo for ... POSSIBLE BREAK-IN ATTEMPT!". The post looks at a solution to this message.
I recently installed the VSFtpd FTP Server onto a CentOS box and have a fairly tight firewall setup script using iptables. When I logged in to test it and issued an "ls -l" command it took a really long time for the driectory listing to come back. At first I thought it wasn't going to show the directory listing at all but it finally did. This post looks at the solution to the problem.
One of my Linux CD Mall customers emailed me the other day with a Linux annoyance: "The computer always booted into Linux by default if I wasn't quick enough to stop it. Most annoying. I would have preferred the computer to wait until I told it where to boot but could not find a way of doing that. All of the Linux distros that I tried just take over without being asked." This post looks at how you can change the grub boot loader's settings to solve this issue using a text editor. And I agree with him, it is annoying how the boot loader is often set up by default to boot into Linux too quickly.
While creating some ISO images for backup purposes earlier this week, I got the error message "Incorrectly encoded string" and "Possibly creating an invalid Joliet extension" when using genisoimage/mkisofs to create the image.
The date daylight savings starts and ends changed in New Zealand last year, but unfortunately there was only a few months between when the decision was made and when it happened and the updates to the timezone databases on various Linux distributions wasn't done in time in many cases. This affected many servers and services which displayed the wrong time for the couple of weeks between when daylight savings actually started and when it would have in previous years.
After re-partitioning a hard drive by removing all existing partitions and creating a single primary parition which takes up the entire disk, I formatted it as ext3, mounted it at /mnt/hdh and copied all the ISO images across from the exsting ISO drive. I rebooted, pausing to swap the disk over to a different cable, and got the error message "hdb: unknown partition table".
I create CDs and DVDs for my Linux CD Mall website on a CentOS 4 machine, and create the new ISO images for Debian using jigdo. This is the first of several posts which will look at jigdo, and looks at how to install jigdo on CentOS and Red Hat Enterprise Linux, because the necessary package is not included in the default software repositories.
The Dag RPM Repository contains a collection of RPM packages for Red Hat Enterprise Linux, CentOS, Fedora and others which are not included in the base installation/libraries for those Linux distributions. If you get an error message similar to the one shown below when attempting to install an RPM package from the Dag RPM Repository, then you need to install the Dag GPG keys:
warning: rpmts_HdrFromFdno: V3 DSA signature: NOKEY, key ID 6b8d79e6 Public key for jigdo-0.7.3-1.el4.rf.i386.rpm is not installed
I burn CDs and DVDs for my Linux CD Mall website on a CentOS 4 machine and use md5sum checksums to ensure that the ISO images have downloaded correctly from the http or ftp server or via bittorrent. I store each ISO files's md5sum in a separate .md5 file and use the md5sum command's -c flag to check the contents are valid.
The default command line text editor on Linux (and BSD varaiants) determines what is used when you run commands such as "crontab -e" and is often not what you want to use yourself. Typcially vi/vim is the default text editor; many people prefer emacs or other editors, and I prefer to use nano myself.
wget is a command line utility for downloading files from FTP and HTTP web servers. By default when you download a file with wget, the file will be written to the current directory, with the same name as the filename in the URL. However, it is possible to save the downloaded file with a different filename using wget.
The ext3 filesystem is a Linux filesystem with journalling capabilities. I'm always forgetting the command I need to run to format a partitition with the ext3 file system so have posted this article so I always have a easy to find reference for it... The example used here is done on the command line, and not using a GUI tool.
I needed to install the MSSQL libraries for PHP on a CentOS 5 server and ran into a few issues when installing some of the required dependencies, getting "Transaction Check Error" error messages which indicated conflicts between files in packages.
An ISO image or .iso file contains the disk image of an ISO 9660 file system, which is used for storing data on CD-ROMs and DVD-ROMs. It is possible to take an ISO image and record it onto a CD or DVD using various applications for doing so, and it is also possible to mount the ISO image using Linux, so that you can access the files without having to actually burn it to disk first.
The Unix/Linux shell command "find" is really useful for finding files on your computer. There are also graphical utilities for finding files, but sometimes it is easier to use the command line, or it's simply not possible, eg if logged in to a remote server using SSH. This post looks at how to use the find utility to locate files based on the modification time eg looking for files that have been modified in the last 24 hours.
A useful command line tool is
date, which is typically used for displaying the current system date, or setting it. The default format of the date and time displayed will be the system default, eg "Fri Dec 7 16:41:08 NZDT 2007", but it is possible to apply your own formatting, and also to specify a different date to use, without adjusting the system clock.
When trying to install something on a CentOS 5 machine using Yum today, I got the error message "Metadata file does not match checksum" when Yum was trying to update the repository metadata. My first instinct was to run the error message through Google but the first couple of answers looked a little complex. I then remembered I'd had issues with Yum in an earlier post when I got the database disk image is malformed error message, so decided to try out the same solution before following the advice from the other sites.
openSUSE has an excellent system management tool called Yast, but the software package management tool in Yast is really slow to run. It is possible to install and use Yum instead on openSUSE and all of a sudden the slowness of running Yast's package management goes away. I have a pretty fast machine with a lot of memory, but it can take 5 to 10 minutes to open up the Yast software management application.
If the hostname setting is incorrect on your CentOS/RedHat/Fedora machine, it's really easy to change the hostname from the command line. There are also GUI tools for doing this but we'll just look at the CLI tools for doing this.
From the BASH shell it's possible to have auto-completion where you start to type in part of a command, and then use a keystroke sequence, such as PageUp or PageDown, to then cycle through the history for commands which started with the first text you have entered.
Security-Enhanced Linux, also know as SELinux, implements various security policies on Linux and additional levels of access crontrol. It was originally developed by the U.S. National Security Agency to adhere to the "Orange Book" guidelines. On CentOS 5 it is enabled by default, but there may be circumstances where you don't need SELinux's additional security and may want to disable it.
I've just started using a VPS (Virtual Private Server) with CentOS 5.0 i386 as the virtual server operating system, but it has a pretty minimal operating system install. When I went to run the nslookup command to look up an IP address I got the error message
nslookup: command not found.
I set up a VPS (Virtual Private Server) with Net24 today with CentOS 5.0 i386 as the virtual server operating system. After logging in as root I discovered the operating system install was pretty minimal and didn't include yum for package management, so I had to manually install yum with rpm.
AWStats provides a useful overview of website statistics from your Apache log files. There is no automatic way to install AWStats on CentOS using yum, so this article looks at how to install AWStats on CentOS. The instructions below should also work on other Linux distributions that do not have an automatic way of installing AWStats.
I process logs on my webservers using AWStats, and use the GeoIP plugin to translate IP addresses into country codes. This article looks at how to install the Geo I PurePerl plugin, which is used by AWStats, and also how to download and install the Geo IP database files which are required by this plugin. The server I installed this onto was a CentOS 5 server, but the instructions here should work equally well on other Linux distributions.
I run AWStats on one of my servers and it copies files over from some of the other servers I manage every 15 minutes and processes them. However, I had a server issue yesterday with the machine that AWStats runs on, and it meant the auto process which copies the files over and runs AWStats stopped running for about 12 hours. The end result was some lost information and needing to regenerate stats for the month to date. I needed to override the AWStats LogFile configuration directive on the command line to be able to do this.
When you connect to an OpenSSH sshd server, it is configured by default to do a hostname lookup on your IP address. If there are any issues with the DNS configuration on the host machine, or with the DNS server it is using, this can lead to a delay when logging in using ssh for around 30 seconds. It is very easy to switch this host name lookup function off in the sshd_config file.
Management of sendmail is done on FreeBSD by changing to sendmail's directory (/etc/mail) and running "make" followed by the appropriate command. This either needs to be done as root or using sudo.
This article looks at how to mount a USB flash card reader and flash cards plugged into it with Linux. Also covered is how to make it easy to mount with a desktop icon in KDE and how to scan the SCSI bus to locate the device name and number.
This article looks at how to use the sg_scan and sg_map utilities to scan the SCSI and USB bus to determine the device name and number for mounting USB devices on Linux.
Private and public RSA keys can be generated on Unix based systems (such as Linux and FreeBSD) to provide greater security when logging into a server using SSH. The
ssh-keygen command allows you to generate, manage and convert these authentication keys.
MD5 is a one-way hash algorithm which can be used for security or to verify the integrity of a file as it creates a digital signature for a file.