The Shell

A few tips for how to use the Unix shell. This article assumes a working knowledge of Unix – you know how to log in and run commands.

This is not a complete guide to shell use and shell scripting, but just a bit to wet your appetite (see further reading at end). Also, it provides some answers to common problems that beginners (and more experienced users) face.

The Unix Shell

Introduction

The Unix shell accepts your typed commands, and executes programs. It has been around since the 1970's, but is still a very versatile tool.

Apart from interactive use, it can also read commands from a file, this file (with the right modification) can be run as a separate executable program – a shell script.

This text discusses bash, the GNU Bourne-Again SHell, unless otherwise noted. This is by far the most common shell in use at UiO today – perhaps also world-wide.

Starting the shell

When you log in on a text terminal (eg. with ssh), you start an login shell. If you log in to a graphical environment (KDE, Gnome, MacOS,...) you don't run the shell directly, but you can start up terminal windows where you run an interactive shell – but not a login shell.

When the shell is ready to accept commands, it will show a "prompt" to signify so – a short text string. The prompt can be tuned by the user, but this article uses the convention with the dollar sign '$' for the normal prompt, and angle bracket '>' for the secondary prompt (in multi-line commands). Don't type these characters in the examples below!

When the shell starts, it typically reads a few setup files to modify its behaviour. Some of these are in your home directory (~/), and you can edit them yourself:

.bash_login is read by login shells.
.bashrc is read by interactive shells, but not by a login shell.

These are text files, and you can modify them as you wish. If they don't exists, you can create them. Make sure you are in your home directory, and type eg.

$ nano .bashrc

You can of course use any editor you like.

(The shell will actually look for more files than the two mentioned above. Consult the bash manual for the details.)

Tip:

Place most of you modifications in ~/.bashrc, and have the login shell read that file as well! Insert the following lines at the end of .bash_login:

# Execute ordinary shell setup:
source .bashrc

The line starting with the '#' is a comment. The next line will make the shell read and interpret the commands in the .bashrc file.

Silence is golden

If you use ssh to run remote commands, the ssh server (sshd) will use your shell to start the command. Even if this is a non-interactive shell, it will still execute your .bashrc. This is (implicitly) the case with scp, sftp, x2go and other remote connection types.

If the commands in your .bashrc gives output – including error messages – you might get weird problems with remote connections. So make sure all commands in .bashrc are silent!

Tip:

Apart from modifying your PATH and a few other variables (see below), other things in your .bashrc will make little sense in a non-interactive (remote) setting. So terminate your .bashrc early with a statement like

# If not running interactively, exit here.
[ -z "$PS1" ] && return

After this, you can place other settings and commands that could potentially cause output.

An interactive login shell, on the other hand, doesn't care about output. So if you e.g. like a little word of wisdom when you log in, you can place the command

fortune

in your .bash_login . This will still give an error message if you log in to a machine where this program isn't installed, so you might want to check if the command is present:

if type -p fortune > /dev/null ; then
  fortune
fi

Less is more

A good rule of thumb is to keep your setup files as small as possible. Apart from error messages, doing things in your .bashrc or .bash_login that depends on other resources can make your shell behave erratically, or not at all. In particular, accessing (eg. with the source command) files on a different disk might cause your shell to hang if this disk is unavailable at login time. Also, loading modules in your shell setup files is error-prone — and goes against the very purpose of using modules.

Alternately, you can lump such statements together in a shell function. In .bash_login, or in the interactive block of .bashrc, put something like this:

my_project_setup()
{
   source /uio/lagringshotell/geofag/projects/.../setup_bash
   module load whatever
}
# Bind the F12 key to call this function:
bind '"\e[24~":"my_project_setup\n"'

When you now start a new shell, you will have a command "my_project_setup" (check with the type command) that will do whatever you place in the body of the function. The name is of course arbitrary, choose one that makes sense to you. As seen in the example, you can also bind a function key to call this shell function.

Environment variables

The shell maintains an environment that is available to the programs it runs. Most of this environment consists of variables, with a "name=value" structure. For example:

# Set a default editor
$ EDITOR=nano
$ export EDITOR

The "export" statement means that the variable will be visible for programs the shell starts, not just the shell itself. With bash, you can merge these commands, eg:

$ export EDITOR=nano  # same as above (bash)

Make sure there are no space characters around the '='.

Variable names are customarily given in capital letters. You can look up (print) the value of a variable by preceding the name with a dollar sign ('$'). For example, to see what shell you are using, try the following command:

$ echo $SHELL
/bin/bash

You can list the variables with the "env" command:

$ env | grep PAGER    # search for "PAGER" in all your variables
$ echo $PAGER         # Look up PAGER, likely yielding a similar result.

You can create variables for your own use, but many of them have meaning also for other commands. For example, many programs that will need you to edit a piece of text will run the command defined in $EDITOR to let you do so. Programs that will show text output on the terminal will typically run it through $PAGER to accommodate scrolling, searching and so on ("less" is a suggested value here).

Sometimes it is convenient to trap the output of some command and assign it to a variable. This can be done by running the command in backticks (`), like this

NOW=`date +%T`

The command is then first executed in a subshell, and the output assigned to the variable.

PATH

What happens when you type eg. "date" and press Enter? First, the shell will check if "date" matches a defined alias or function, or a shell internal command. If none were found, it will look in the file system for an executable file called "date". It will search the directories in the PATH variable, which is a colon-separated list of directories:

$ echo $PATH                   # inspect the list of program directories to search
$ PATH=$PATH:/site/bin         # Add a directory to the search path

Note that you don't need to "export" the PATH in the second command above, as it is (most likely) already exported.

The "type" command can be used to show what the shell thinks of any given command:

$ type date export
date is /usr/bin/date
export is a shell builtin

Test

Have you ever created a test program called "test", and been frustrated because it didn't do what you thought it should do?

$ gcc -o test test.c
$ test                 # nothing happens

This is because the name "test" is used already:

$ type -a test
test is a shell builtin
test is /usr/bin/test

Both of these do approximately the same (test file properties, among other things), but to run your newly-compiled program, specify the directory:

$ ./test

It's possible to add the period "." to your PATH to make the shell look for programs in whatever directory you are in. This is not recommended, but if you choose to do so, make sure you put it at the end of your PATH.

Scripts

A shell script is a text file with a bunch of shell commands. You can create scripts to do complicated tasks, or just to collect commands you use frequently in a convenient «shorthand».

There are many dialects of the the shell, and as such many possible types of script. The Bourne shell ('/bin/sh') is universally available, and it is suggested you stick to that for your shell scripts. On Linux, /bin/sh is just the Bourne Again shell in sheep's clothing, and syntax is largely the same.

Example

Here is an example of a shell script:

#!/bin/sh

# Print the date and a short greeting:
date
echo Good Morning!

This script will print out the date and time (command "date"), and then print "Good Morning!" with the "echo" command.

Empty lines are ignored, as are comment lines starting with the hash sign ('#').

Running the Script

The first line starts with a '#', so it is ignored by the shell. But the '#!' combination tells the operating system that this file should be interpreted by whatever follows on the line ('/bin/sh'), if the script is used as a stand-alone command. So if the above script is in an executable file called "myscript" in the current directory, it can be executed in two ways:

sh myscript
./myscript

You can use the same #!-mechanism to create eg. python-scripts or perl-scripts, but that is outside the scope of this article.

The shell running your script will be a non-interactive shell (even if the script itself is interactive), so it won't read commands from your .bashrc file.

Creating

As it is a text file, a script can be created with any text editor you are comfortable with, eg. emacs, pico, vim. It is common to name a shell script with the '.sh' suffix, but that is not required. It will, however, help some systems (and humans) to recognize the file as a shell script.

Some editors will for instance do syntactic highlighting, if they recognize the file. Here is how emacs can render the above script.

Even if emacs doesn't recognize the script, you can make it do so with "M-x shell-script-mode" (that's ESC x shell-script-mode).

When the file is first created, you should make it executable, eg:

$ chmod +x myscript

Inheritance

Your shell's environment (variables) can be changed in the running shell, and it it is inherited by other processes started by that shell. So if you modify eg. your PATH and run a shell script, the script will experience your modified PATH (unless it explicitly sets PATH itself).

This inheritance only goes one way. A script (or other process) cannot modify the environment of the calling shell, at least not without some trickery.

One trick is to use the "source" command on your shell script, as shown above. This will make your running shell execute the script directly, rather than starting a new shell to do the job. Obviously, this only works on shell scripts with compatible syntax.

Another way is the "eval" shell internal command, which will evaluate an expression in the context of the running shell. You could for instance have a rather complicated shell script "setfoo" that figured out how to set some variable you needed:

#!/bin/sh
# Compute the correct value for FOOBAR,
# we skip the details here:
FOOBAR=whatever
echo FOOBAR=\"$FOOBAR\"
echo export FOOBAR

Running this would output

FOOBAR="whatever"
export FOOBAR

so running

$ eval `setfoo`

would first execute setfoo in a subshell (because of the backticks), and the output FOOBAR="whatever" and export FOOBAR will be interpreted in the current shell. The quote characters (") would be necessary if the variable value contained space (' ') or other funny characters.

A Final Warning

To learn shell scripting, you must write shell scripts, and you must study other shell scripts. Take care if you download shell scripts from untrusted sources! If you run them, they can potentially delete all your files or open back doors to your computer. Scripts are every bit as dangerous as compiled code in this respect.

The nice thing is that it is much easier to read and understand a shell script compared to binary code.