Software Carpentry

This lecture covers the basics of using the Unix command shell, which is still one of the most efficient ways to interact with a computer.

Requires: Access to Linux or Unix, or Cygwin on Windows.

Introduces: The Unix Shell

Motivating Question:

1. My research requires work on a server where there is no graphical interface. How do I work efficiently?
2. How can I automate tasks that involve more than one step or program?

Lectures:

1. Introduction (4:09) (pdf, ppt)
   • The shell has been an important interface for 50 years.
   • Today, shells are used to interact with servers or with programs that have a command line interface.
   • Shells also allow users to quickly combine several tools to minimize programming effort.
2. Files and Directories (9:55) (pdf, ppt)
   • Logging in to the shell and running basic utilities.
   • Investigating the file and directory structure from the command line using the commands pwd, ls, and cd.
3. Creating and Deleting (6:23) (pdf, ppt)
   • Creating directories and files using mkdir and a simple editor called nano.
   • Moving and copying files.
   • Deleting files and directories.
4. Pipes and Filters (9:11) (pdf, ppt)
   • The real strength of the shell is its ability to combine simple utilities in useful ways.
   • Introduction to a few wild-card characters.
   • Redirection operators > and >>.
   • The pipe operator, |, which passes output directly to another program.
5. Permissions (10:54) (pdf, ppt)
   • The shell has tools to control who can see, edit, and run files.
   • For directories, the shell controls who can enter and traverse the directory.
   • The world can be divided into users, groups, and everyone else.
   • Using ls to view the 9 character permission flag for a file.
   • Using chmod to change permissions for a file or directory.
6. Finding Things (9:22) (pdf, ppt)
   • Use grep to match text in a file or files.
   • grep can also match patterns specified by a code called a regular expression.
   • Use find to identify files whose names match a given pattern.
   • Both grep and find have extensive options available through flags.
7. Job Control (5:37) (pdf, ppt)
   • The command ps can be used to list processes, called jobs, that are running.
   • Appending a & to the end of a command makes that command run in the background.
   • Jobs can be moved to and from the background using fg and bg.
8. Variables (6:49) (pdf, ppt)
   • Like any program, the shell uses a set of variables to control its operation.
   • Two important variables are PATH and HOME
   • Variables can be created and made available to programs the shell calls using the commands set and export.
9. Secure Shell (SSH) (13:04)
   • Remote logins can be used to connect to distant computers and perform command line tasks on them.
   • The secure shell command is ssh, and uses the syntax “ssh username@computer” to connect.
   • scp can be used to copy files to/from a remote computer using the syntax “scp user@computer:path/to/file user@computer:path/to/destination/file“
   • ssh can be made to use public-private keys in order to silently authenticate into a remote computer.
10. Exercises