The Art Of Lazy Programming

In your time as a Computer Scientist at Rice, you will learn many theoretical aspects of Computer Science and the way to apply programming to complex topics. However, another crucial step towards becoming a great Computer Scientist is efficiency as a developer. The Art of Lazy Programming is a one-credit course where students will be taught precisely that: students will learn vital information every programmer should know, tips on learning complex but time-saving tools, and most importantly, the answer to “how does one master their programming environment and gain the knowledge needed to be an experienced programmer”?

The intention of this class is to teach students various tools and topics such as grep, docker, and sshfs — that will make their life easier by introducing efficiency. While students may learn some of these tools in classes and spend dozens of hours using them, their core benefits are never explicitly taught. Yet, mastering these tools will allow students to solve large-scale problems that seem impossibly complex. Additionally, upon completion of the course, students will also develop problem-solving skills that can be applicable outside Computer Science.

Finally, I advocate for the importance of education. If you are not a Rice student, but would like to learn more about the topics covered in this course, please feel free to reach out to me. I am more than happy to share my knowledge with you.

Syllabus

Week 0 Notes

Week 0 is the first week of the course, where I introduce the course to the class. Watching the video is a great way to skip straight to the content and expectations of the class.

Intro

As computer scientists, we know that computers are great at aiding in repetitive tasks. However, far too often, we forget that this applies just as much to our use of the computer as it does to the computations we want our programs to perform. We have a vast range of tools available at our fingertips that enable us to be more productive and solve more complex problems when working on any computer-related problem. Yet, many of us utilize only a small fraction of those tools; we only know enough magical incantations by rote to get by, and blindly copy-paste commands from the internet when we get stuck.

This class is an attempt to address this and close a gap between theory and practice. I intend to teach how to make most of the tools you know, show new tools to add to your toolbox, and instill in you some excitement for exploring (and perhaps building) more tools to be a true lazy programmer.

Demo

Basic Unix Commands

There are many bash commands, but this short guide is meant to cover the essentials, either as an introduction for those unfamiliar with Unix-based terminals or as a refresher for those more experienced.

Basic Keystrokes and Characters

Foundations

Navigating the terminal requires familiarity with basic commands for directory manipulation, file interaction, and process monitoring. To delve deeper into any command, consult the Unix manuals by entering man command_name.

File and Navigation Fundamentals

• cd: Change directories (folders).
• mkdir: Create a new directory.
• ls: List all files in a given directory.
• cp: Copy a file from one location to another.
• scp: Copy files between two machines over SSH, similar to cp.
• rsync: A more sophisticated method for copying files either locally or over a network, combining the functionalities of cp and scp.
• mv: Move a file from one location to another.
• rm: Delete a file. Use -r to remove directories.
• pwd: Display the current working directory’s path.
• chmod: Change file permissions.
• chgrp: Change the group associated with a file or directory.

Probing File Contents

• cat: Display the entire contents of a file to the command line, useful for small files or as part of a command chain.
• wc: Count the words in a file, or use -l to count lines.
• head and tail: Display the beginning or end of a file, respectively.
• less: Efficiently view large files by loading only a portion at a time. Use arrow keys to navigate, G to jump to the end, g to return to the beginning, / to search within the file, and q to exit.

Process Basics

• ps: List currently running processes.
• kill: Terminate a process.
• htop: Launch a sophisticated process manager, akin to a text-based version of Task Manager on Windows or Activity Monitor on Mac.

Other Useful Commands

• du -h: Check disk usage with output in human-readable format.
• wget: Download files from the internet using a URL.
• curl: Fetch or send data using URL syntax, offering more options than wget. For differences, refer to curl vs wget.

Rust replacements

Here are several well-known Rust projects that serve as modern alternatives or enhancements to classic Unix commands:

Rust Projects as Unix Command Alternatives

• Exa (ls): Exa is a modern replacement for ls, the traditional Unix command to list directory contents. It features better defaults and is rich in features like git integration and color-coded output. Github Link
• Ripgrep (grep): Ripgrep is a line-oriented search tool that recursively searches your current directory for a regex pattern. It is much faster than traditional grep due to its efficient use of memory and CPU, and its ability to ignore binary files and hidden directories by default. Github Link
• Bat (cat): Bat is a clone of cat with syntax highlighting, line numbers, and git integration. It also integrates with other tools like ripgrep to provide a seamless viewing experience. Github Link
• Fd (find): Fd is a simple, fast, and user-friendly alternative to find. It helps you find files in the file system quickly and efficiently. Github Link
• Dust (du): Dust is a more intuitive version of du. It shows disk usage and free space in a more comprehensible way by automatically sorting folders by size and visualizing the output in a tree format. Github Link
• Zoxide (cd): Zoxide is a smarter cd command, learning your habits and allowing you to jump to frequently and recently used directories with fewer keystrokes. Github Link
• Procs (ps): Procs is a modern replacement for ps. It provides detailed and colored output to make the information more comprehensible. Github Link
• Skim (fzf/ctrl-r): Skim is a fuzzy finder similar to fzf but written in Rust. It can be used to quickly find files, directories, or even history commands. Github Link

These projects not only replace traditional Unix commands but also enhance them with modern features and better performance, thanks to Rust’s efficiency and safety features.

Tips

• If you wish to use vim keybinds, you can by setting readline to set -o vi in your .bashrc. This allows you to use vim-like editing commands directly in the command line.
• If you wish to “pause” a command, you can use <C-z> to set the current process to the background. Then when you are ready to unpause, you can use fg to continue your progress.
• To search through your command history quickly, use <C-r> for a reverse search. Start typing part of the command you’re looking for, and press <C-r> repeatedly to cycle through past commands that match.
• Customize your prompt to show helpful information like the current directory, git branch, or other useful status by modifying the PS1 variable in your .bashrc.
• Use alias to shorten commands that you use frequently. For example, you can add alias ll='ls -alF' to your .bashrc to create a shortcut for a detailed listing format.
• When scripting, use set -e to make your script exit on the first error. This can help prevent errors from cascading and make debugging easier.
• To preserve your session’s state between logins, consider using a terminal multiplexer like tmux or screen. This is especially useful for remote sessions over SSH.
• If you find yourself needing the same command frequently but with slight variations, use history | grep <search-term> to quickly find previously used commands without needing to retype them completely.
• For more efficient navigation, use pushd and popd to work with directories as a stack. This allows you to “push” directories onto your stack as you navigate and “pop” them off to return to previous locations.

Video For Week 0

There is also the talk written in markdown format: Week 0 Talk. Hint it’s on the same repo.

Week 0 Homework

Installation

Please follow the instructions in the Week 0 Installation Guide to install the necessary software and tools for this week.

GitHub Repository Access

To gain access to the team’s repository, please either fill out this form or send me your GitHub username directly. Access the repository sections here:

• Week 0

Notes

For instructions on using the command line interface (CLI), refer to the Week 0 Notes. Please note that while the basics of bash will be covered in our next lesson, you are encouraged to experiment on your own in the meantime.

Homework Overview

This week’s homework consists of three parts, focusing more on reading and exercises than coding. Parts 2 and 3 are graded based on completion. All submissions should be made via GitHub to your specific section’s repository.

Submitting Homework

• Repository Setup: Submit all homework through your section’s repo using the git submodule setup described in the week0 installation guide.
• Submodule Naming: Name your submodule lastname-firstname.
• Committing: Commit all your work to your own repo. I will update the main repo during grading. You do not need to commit to the main repo once your submodule is set up.
• Time Limit: If any assignment takes more than an hour, feel free to skip it.

Part 1: Observation and Installation

1. Observation Task: Note any differences between your setup and a clear session. Upload your observations to the Week 0 Notes under the filename “part_1_notes.txt”.
2. Installation Task: Install neofetch and capture its output:

   $ git clone https://github.com/dylanaraps/neofetch
   $ cd neofetch
   $ make PREFIX=./ install
   

   Upload the output screenshot to the Week 0 Notes as “neofetch.img”. Example output:

   git add part_1_notes.txt neofetch.img
   git commit -m "part 1"
   git push
   

Part 2: Video Analysis

• Task: Watch a video segment from this video from 5:00 to 19:00. Take notes and submit them as “part_2_notes.txt”. If possible, watching the entire video is recommended.

  git add part_2_notes.txt
  git commit -m "part 2 notes"
  git push
  

Part 3: Article Review

• Task: Read this article by Paul Graham and note anything significant. Submit your notes as “part_3_notes.txt”.

  git add part_3_notes.txt
  git commit -m "part 3 notes"
  git push
  

Completion: Submitting your assignments will contribute positively to your grade. Ensure you follow the guidelines to ensure your submissions are recorded correctly.

Week 1 Notes - Shell Scripting

Week 1 builds on the Unix tools from Week 0 by showing you how to combine them into reusable, powerful scripts. Instead of typing the same commands repeatedly, you’ll learn to write small programs that automate your workflow.

Introduction

So far you’ve executed individual commands and chained them with pipes. But what if you want to:

• Run the same sequence of commands every day?
• Make decisions based on conditions?
• Process multiple files with the same logic?
• Save your workflows for teammates to use?

This is where shell scripting comes in. Shell scripts are the next step in complexity: they’re programs optimized for performing shell-related tasks. They make creating command pipelines, saving results, and reading input easier than general-purpose programming languages.

We’ll focus on bash scripting since it’s the most common, but the concepts apply to other shells too.

Basic Syntax

Variables

To assign a variable in bash:

foo=bar

To access it, use $:

echo $foo
# Output: bar

Critical gotcha: Spaces matter!

foo=bar      # ✓ Correct
foo = bar    # ✗ Wrong - tries to run 'foo' with arguments '=' and 'bar'

In shell scripts, space is the argument separator. This is the #1 source of beginner errors.

Strings: Single vs Double Quotes

Quotes change how bash interprets your strings:

foo=bar

echo "$foo"   # Output: bar (variable substituted)
echo '$foo'   # Output: $foo (literal string)

Rules:

• "double quotes" - Variables are expanded, escape sequences work
• 'single quotes' - Everything is literal, no substitution

Best practice: Use double quotes for variables, single quotes for literal strings.

name="world"
echo "Hello, $name!"     # Hello, world!
echo 'Hello, $name!'     # Hello, $name!

Comments

# This is a comment
echo "This runs"  # This is also a comment

Control Flow

If Statements

if [[ condition ]]; then
    # do something
elif [[ other_condition ]]; then
    # do something else
else
    # default case
fi

Example:

if [[ -f "myfile.txt" ]]; then
    echo "File exists"
else
    echo "File not found"
fi

Common test conditions:

Pro tip: Use [[ ]] (double brackets) instead of [ ] (single brackets). Double brackets are:

• More powerful (support &&, ||, pattern matching)
• Safer (better handling of empty variables)
• More forgiving with syntax

For Loops

Iterate over a list:

for item in apple banana cherry; do
    echo "I like $item"
done

Iterate over files:

for file in *.txt; do
    echo "Processing $file"
    wc -l "$file"
done

Iterate over command output:

for user in $(cat users.txt); do
    echo "Hello, $user"
done

C-style for loop:

for ((i=0; i<10; i++)); do
    echo "Number $i"
done

While Loops

count=0
while [[ $count -lt 5 ]]; do
    echo "Count: $count"
    ((count++))
done

Read from file line by line:

while IFS= read -r line; do
    echo "Line: $line"
done < input.txt

Case Statements

For pattern matching (like switch in other languages):

case "$1" in
    start)
        echo "Starting..."
        ;;
    stop)
        echo "Stopping..."
        ;;
    restart)
        echo "Restarting..."
        ;;
    *)
        echo "Unknown command: $1"
        ;;
esac

Functions

Functions let you organize and reuse code:

greet() {
    echo "Hello, $1!"
}

greet "Alice"  # Output: Hello, Alice!
greet "Bob"    # Output: Hello, Bob!

Example: Create directory and cd into it:

mcd() {
    mkdir -p "$1"
    cd "$1"
}

mcd my_project  # Creates and enters my_project/

Functions with multiple arguments:

backup_file() {
    local source="$1"
    local dest="$2"
    cp "$source" "${dest}.bak"
    echo "Backed up $source to ${dest}.bak"
}

backup_file config.txt config

Special Variables

Bash uses special variables to access script metadata and arguments:

Example script using special variables:

#!/bin/bash

echo "Script name: $0"
echo "First argument: $1"
echo "All arguments: $@"
echo "Number of arguments: $#"
echo "Process ID: $$"

Running it:

$ ./myscript.sh hello world
Script name: ./myscript.sh
First argument: hello
All arguments: hello world
Number of arguments: 2
Process ID: 12345

Exit Codes and Error Handling

Every command returns an exit code (0 = success, non-zero = error):

grep "pattern" file.txt
echo $?  # 0 if found, 1 if not found

Conditional Execution

Use exit codes to chain commands:

# AND operator (&&) - run second command only if first succeeds
mkdir mydir && cd mydir

# OR operator (||) - run second command only if first fails
grep "error" log.txt || echo "No errors found"

# Semicolon (;) - run commands sequentially regardless
cd /tmp; ls; pwd

Examples:

# If backup succeeds, delete original
tar -czf backup.tar.gz mydir && rm -rf mydir

# Try command, show error if it fails
./deploy.sh || echo "Deployment failed!"

# Try multiple fallbacks
command1 || command2 || command3 || echo "All failed"

Set Error Handling

Make scripts safer with these flags:

#!/bin/bash
set -e  # Exit on any error
set -u  # Exit on undefined variable
set -o pipefail  # Exit if any command in pipeline fails

# Combine them:
set -euo pipefail

Example:

#!/bin/bash
set -e

# Script stops if any command fails
cd /important/directory
rm *.tmp
./critical_task.sh

Explicit Error Checking

if ! command; then
    echo "Command failed!"
    exit 1
fi

Or:

command
if [[ $? -ne 0 ]]; then
    echo "Error occurred"
    exit 1
fi

Command Substitution

Capture command output in a variable:

# Modern syntax (preferred)
current_date=$(date)
files=$(ls *.txt)

# Old syntax (still works)
current_date=`date`

Example:

#!/bin/bash

echo "Starting program at $(date)"
echo "Running on $(hostname)"
echo "Current directory has $(ls | wc -l) items"

In conditionals:

if [[ $(whoami) == "root" ]]; then
    echo "Running as root"
fi

Process Substitution

Treat command output as a file:

diff <(ls dir1) <(ls dir2)

This executes both ls commands and provides their outputs as temporary files to diff.

Example: Compare remote and local files:

diff <(ssh server 'cat /etc/config') /etc/config

Redirection

Standard Streams

• stdin (0) - Standard input
• stdout (1) - Standard output
• stderr (2) - Standard error

Redirect Output

# Redirect stdout to file (overwrite)
command > output.txt

# Redirect stdout to file (append)
command >> output.txt

# Redirect stderr to file
command 2> error.txt

# Redirect both stdout and stderr
command &> all_output.txt
command > output.txt 2>&1

# Discard output
command > /dev/null
command 2> /dev/null
command &> /dev/null

Redirect Input

# Read from file
command < input.txt

# Here document (multi-line input)
cat << EOF
This is line 1
This is line 2
EOF

# Here string (single line input)
grep "pattern" <<< "search this string"

Example: Script with all redirections:

#!/bin/bash

# Save output and errors separately
./my_command > output.log 2> error.log

# Append to existing logs
./another_command >> output.log 2>> error.log

# Run silently
./quiet_command &> /dev/null

Globbing and Wildcards

Bash expands patterns before running commands:

Basic Wildcards

Examples:

ls *.txt           # All .txt files
rm file?.log       # file1.log, fileA.log, etc.
cp [A-Z]*.py src/  # Files starting with uppercase

Brace Expansion

Create multiple similar strings:

# Expand to: file1.txt file2.txt file3.txt
echo file{1,2,3}.txt

# Expand to: file1.txt file2.txt ... file10.txt
echo file{1..10}.txt

# Convert images
convert image.{png,jpg}  # Same as: convert image.png image.jpg

# Create directory structure
mkdir -p project/{src,test,docs}

# Backup with timestamp
cp config.json config.json.{bak,$(date +%Y%m%d)}

Combine with variables:

prefix="test"
touch {$prefix}_{1..5}.txt  # test_1.txt through test_5.txt

Advanced Globbing

Enable with shopt -s globstar:

# Match files in subdirectories
ls **/*.py  # All .py files in any subdirectory

Script Template

Here’s a good template to start your scripts:

#!/usr/bin/env bash

# Script: my_script.sh
# Description: What this script does
# Usage: ./my_script.sh [options] arguments

set -euo pipefail  # Exit on error, undefined vars, pipe failures

# Default values
VERBOSE=false
OUTPUT_DIR="./output"

# Functions
usage() {
    cat << EOF
Usage: $0 [-v] [-o OUTPUT_DIR] input_file

Options:
    -v              Verbose mode
    -o OUTPUT_DIR   Output directory (default: $OUTPUT_DIR)
    -h              Show this help message

Example:
    $0 -v -o /tmp data.txt
EOF
    exit 1
}

log() {
    if [[ $VERBOSE == true ]]; then
        echo "[$(date +'%Y-%m-%d %H:%M:%S')] $*" >&2
    fi
}

# Parse arguments
while getopts "vo:h" opt; do
    case $opt in
        v) VERBOSE=true ;;
        o) OUTPUT_DIR="$OPTARG" ;;
        h) usage ;;
        *) usage ;;
    esac
done

shift $((OPTIND-1))  # Remove parsed options

# Check for required arguments
if [[ $# -lt 1 ]]; then
    echo "Error: Missing input file" >&2
    usage
fi

INPUT_FILE="$1"

# Validate input
if [[ ! -f "$INPUT_FILE" ]]; then
    echo "Error: File not found: $INPUT_FILE" >&2
    exit 1
fi

# Main logic
log "Processing $INPUT_FILE"
mkdir -p "$OUTPUT_DIR"

# Your code here

log "Done!"

Best Practices

1. Quote Variables

Always quote variables to handle spaces and special characters:

# BAD
rm $file

# GOOD
rm "$file"

# BAD
for f in $(ls *.txt); do

# GOOD
for f in *.txt; do

2. Use [[ Instead of [

# Prefer this
if [[ -f "$file" ]]; then

# Over this
if [ -f "$file" ]; then

3. Check for Errors

# Don't assume commands succeed
mkdir "$dir" || exit 1

# Or use set -e
set -e
mkdir "$dir"

4. Use shellcheck

Install and run shellcheck on your scripts:

shellcheck myscript.sh

It catches common mistakes like:

• Unquoted variables
• Unreachable code
• Syntax errors
• Portability issues

5. Make Scripts Executable

chmod +x myscript.sh

And use a shebang:

#!/usr/bin/env bash

Use #!/usr/bin/env bash instead of #!/bin/bash for portability.

6. Use Functions

Break complex scripts into functions:

setup() {
    mkdir -p "$WORK_DIR"
    cd "$WORK_DIR"
}

process_file() {
    local file="$1"
    # ... processing logic
}

cleanup() {
    rm -rf "$TEMP_DIR"
}

# Main execution
setup
for file in *.txt; do
    process_file "$file"
done
cleanup

7. Handle Cleanup with Traps

Ensure cleanup happens even on errors:

cleanup() {
    rm -rf "$TEMP_DIR"
    echo "Cleaned up temporary files"
}

trap cleanup EXIT  # Run cleanup when script exits

TEMP_DIR=$(mktemp -d)
# ... do work ...
# cleanup runs automatically

Debugging Scripts

Enable Debug Mode

# Show commands as they execute
bash -x script.sh

# Or in script
set -x
# ... commands to debug
set +x

Use echo Liberally

echo "DEBUG: variable value is $var"
echo "DEBUG: about to run critical command"

Check Exit Codes

command
echo "Exit code: $?"

Validate Early

# Check preconditions
[[ -f "$input" ]] || { echo "Input file missing"; exit 1; }
[[ -w "$output_dir" ]] || { echo "Output dir not writable"; exit 1; }

Common Patterns

Process Files Safely

# Handle filenames with spaces
find . -name "*.txt" -print0 | while IFS= read -r -d '' file; do
    process "$file"
done

Retry Logic

retry() {
    local max_attempts=5
    local attempt=1
    
    until "$@"; do
        if [[ $attempt -ge $max_attempts ]]; then
            echo "Failed after $max_attempts attempts"
            return 1
        fi
        echo "Attempt $attempt failed. Retrying..."
        ((attempt++))
        sleep 2
    done
}

retry curl -f https://example.com/api

Parallel Processing

# Process files in parallel
for file in *.txt; do
    process_file "$file" &  # & runs in background
done
wait  # Wait for all background jobs

Lock Files

LOCKFILE="/tmp/myscript.lock"

if [[ -f "$LOCKFILE" ]]; then
    echo "Script already running"
    exit 1
fi

touch "$LOCKFILE"
trap "rm -f '$LOCKFILE'" EXIT

# ... do work ...

Scripts vs Functions

When to use a script:

• Reusable across different projects
• Can be any language (Python, Ruby, etc.)
• Needs to be invoked from cron or other tools
• Standalone tool

When to use a function:

• Specific to your shell workflow
• Needs to modify shell environment (change directory, set variables)
• Faster to load (loaded once with shell config)
• Part of your personal productivity toolkit

Example function for your .bashrc:

# Quickly find and cd to a directory
f() {
    local dir=$(find . -type d -name "*$1*" | head -1)
    if [[ -n "$dir" ]]; then
        cd "$dir"
    else
        echo "Directory not found"
    fi
}

Resources

• Bash Manual
• ShellCheck - Linter for shell scripts
• Bash Guide for Beginners
• Advanced Bash-Scripting Guide
• explainshell.com - Explains shell commands

Summary

Shell scripting is about:

• Automation - Don’t repeat yourself
• Composition - Combine small tools
• Robustness - Handle errors gracefully
• Readability - Future you will thank present you

Start small: automate one annoying task. Build from there. The goal isn’t to become a bash expert—it’s to make your daily work easier.

Video for week 1

WIP.

Week 1 Homework - Shell Scripting

Time estimate: 1.5 hours maximum

Submission: Create a directory week1-[netid] with your solutions and add it as a submodule to the class repo.

Setup

Create your homework directory:

mkdir week1-[netid]
cd week1-[netid]

All scripts should be executable (chmod +x script.sh) and include a proper shebang (#!/usr/bin/env bash).

Exercise 1: Custom ls (15 minutes)

Read man ls and write an ls command that lists files with the following properties:

• Includes all files, including hidden files
• Sizes are in human-readable format (e.g., 454M instead of 454279954)
• Files are ordered by recency (most recently modified first)
• Output is colorized

Save your command in a file called ex1_command.txt.

Sample output:

-rw-r--r--   1 user group 1.1M Jan 14 09:53 baz
drwxr-xr-x   5 user group  160 Jan 14 09:53 .
-rw-r--r--   1 user group  514 Jan 14 06:42 bar
-rw-r--r--   1 user group 106M Jan 13 12:12 foo
drwx------+ 47 user group 1.5K Jan 12 18:08 ..

Hint: Look for flags related to “all”, “human-readable”, “time”, and “color” in the man page.

Exercise 2: Semester Setup Script (20 minutes)

Write a bash script called semester that creates a directory structure for organizing coursework.

Requirements:

The script should:

1. Take a semester name as an argument (e.g., fall2025)
2. Create the following directory structure:

   fall2025/
   ├── comp140/
   ├── comp182/
   ├── math212/
   └── projects/
   

3. Create a README.md file in the root directory with:
   • The semester name
   • The current date (use date)
   • A placeholder for course descriptions

Example usage:

./semester fall2025

Expected output:

Created semester directory: fall2025
Created course directories: comp140, comp182, math212, projects
Created README.md

Bonus: Make the course names configurable via additional arguments.

Exercise 3: marco and polo (20 minutes)

Write two bash functions that work together for directory navigation:

• marco - Saves the current working directory
• polo - Returns you to the directory where you last executed marco

Requirements:

1. Create a file called navigation.sh
2. Implement both functions
3. The saved directory should persist across terminal sessions (hint: save to a file)
4. If polo is called before marco, print a helpful error message

Example usage:

$ source navigation.sh
$ cd /usr/local/bin
$ marco
Saved current directory
$ cd ~
$ cd Documents
$ pwd
/home/user/Documents
$ polo
$ pwd
/usr/local/bin

Hint: You’ll need to save the directory to a file in your home directory (like ~/.marco_location).

Exercise 4: Retry Script (25 minutes)

Write a bash script called retry.sh that runs a command until it fails, then reports statistics.

The script should:

1. Take another script as an argument
2. Run it repeatedly until it exits with a non-zero status
3. Capture both stdout and stderr to separate files
4. Report:
   • Number of successful runs before failure
   • The final error message
   • Total time elapsed

Test it with this script (save as flaky.sh):

#!/usr/bin/env bash

n=$(( RANDOM % 100 ))

if [[ $n -eq 42 ]]; then
   echo "Something went wrong"
   >&2 echo "The error was using magic numbers"
   exit 1
fi

echo "Everything went according to plan"

Example usage:

$ ./retry.sh ./flaky.sh
Run 1: success
Run 2: success
Run 3: success
...
Run 47: FAILED

Statistics:
- Successful runs: 46
- Failed on run: 47
- Total time: 2.3 seconds

Error output:
The error was using magic numbers

Standard output from failed run:
Something went wrong

Hints:

• Use a while loop with a counter
• Check $? after each run
• Use date +%s to track time
• Redirect output: command > stdout.log 2> stderr.log

Exercise 5: HTML Archive Creator (20 minutes)

Write a command (can be a one-liner or script called archive_html.sh) that:

1. Recursively finds all HTML files in the current directory
2. Creates a tar.gz archive containing them
3. Handles filenames with spaces correctly
4. Names the archive with today’s date: html_backup_YYYY-MM-DD.tar.gz

Requirements:

• Must work even if files have spaces in their names
• Should use find combined with xargs or find -exec
• Should create the archive in the current directory

Test setup:

mkdir -p test/web/{pages,styles}
touch test/web/index.html
touch test/web/pages/"about us.html"
touch test/web/pages/contact.html
touch test/web/styles/main.css

Expected result:

$ ./archive_html.sh
Found 3 HTML files
Created archive: html_backup_2025-01-20.tar.gz

Hints:

• find . -name "*.html" finds HTML files
• Use find -print0 with xargs -0 for handling spaces
• Or use find -exec tar ...
• Use date +%Y-%m-%d for the date format

Exercise 6: Log Analyzer (20 minutes)

Write a script called analyze_logs.sh that processes a log file and produces a summary report.

Given this sample log file (save as server.log):

2025-01-20 10:23:15 INFO Server started on port 8080
2025-01-20 10:23:18 INFO Connected to database
2025-01-20 10:24:32 WARNING High memory usage: 85%
2025-01-20 10:25:01 ERROR Failed to connect to redis
2025-01-20 10:25:15 INFO User login: alice@rice.edu
2025-01-20 10:26:42 ERROR Database timeout
2025-01-20 10:27:03 INFO User login: bob@rice.edu
2025-01-20 10:28:15 WARNING Disk space low: 5% remaining
2025-01-20 10:29:01 ERROR Connection refused
2025-01-20 10:30:12 INFO User login: charlie@rice.edu

Your script should output:

$ ./analyze_logs.sh server.log

Log Analysis Report
===================
Total lines: 10
INFO messages: 5
WARNING messages: 2
ERROR messages: 3

Most recent error:
2025-01-20 10:29:01 ERROR Connection refused

Unique users logged in: 3

Requirements:

• Use grep, wc, and other text processing tools
• Extract and count different log levels
• Find the most recent error
• Count unique user logins

Hints:

• grep -c "pattern" counts matches
• tail -1 gets the last line
• Use cut or awk to extract specific fields
• sort | uniq | wc -l counts unique values

Exercise 7 (Bonus/Advanced): Recent File Finder (10 minutes)

Write a script or one-liner that finds and lists the most recently modified files in a directory tree.

Requirements:

1. Recursively search from current directory
2. Find the 10 most recently modified files
3. Display: modification time, file size (human-readable), and file path
4. Exclude hidden files and directories

Example output:

$ ./recent_files.sh
Jan 20 14:32  1.2M  ./reports/analysis.pdf
Jan 20 14:15  456K  ./data/results.csv
Jan 20 13:58   12K  ./scripts/process.py
...

Hints:

• Use find with -type f and -mtime
• Combine with ls -lht or use find -printf
• Use head to limit results
• May need to exclude paths matching */.*

Submission Checklist

Your week1-[netid] directory should contain:

week1-netid/
├── ex1_command.txt
├── semester
├── navigation.sh
├── retry.sh
├── flaky.sh (provided)
├── archive_html.sh
├── analyze_logs.sh
├── server.log (provided)
└── recent_files.sh (bonus)

Before submitting:

1. Make all scripts executable: chmod +x *.sh semester
2. Test each script works as described
3. Add comments explaining tricky parts
4. Run shellcheck if available: shellcheck *.sh

Submit by: Adding your directory as a git submodule and pushing to your repo

Grading

• Exercises 1-4: Required (90%)
• Exercises 5-6: Required (10%)
• Exercise 7: Bonus (5% extra credit)

Remember: If any exercise takes more than the estimated time, stop and ask for help in office hours. The goal is learning, not frustration!

Tips

• Test incrementally - don’t write the whole script at once
• Use echo statements to debug
• Check exit codes: echo $?
• Quote your variables: "$var" not $var
• Read error messages carefully
• Use shellcheck to catch common mistakes

Good luck, and remember: the best programmers are lazy programmers who automate everything!

Week 2 Notes - Terminal Text Editors & Better Alternatives

For the past two weeks, you’ve probably been using nano or VS Code with the Remote SSH extension. Both are fine for getting started, but there’s a better way: a way that lets you edit at the speed you think, without reaching for the mouse, and works on every machine you’ll ever SSH into.

This week, we’re learning Vim.

Why Learn a Terminal Editor?

The Reality of Programming

You spend most of your programming time:

• Reading code (not writing)
• Navigating between files
• Making small edits (not writing novels)
• Working on remote servers (where GUIs don’t exist)

A good terminal editor optimizes for these activities.

Why Not Just Use nano?

nano is great for quick edits, but:

• No efficient navigation (you hold arrow keys like it’s 1995)
• No powerful search/replace
• No text objects or motions
• No macros or automation
• Limited to what your hands can physically type

nano is fine for editing a config file once. Vim is for editing code all day.

Why Not Just Use VS Code Remote SSH?

VS Code Remote SSH is excellent, but:

• Requires stable network connection
• Heavy resource usage on remote server
• Not available on all systems (shared servers, restricted environments)
• Slower for quick edits
• You still need to know a terminal editor for when VS Code isn’t an option

VS Code is great. But you should also know Vim.

Why Vim?

1. Available everywhere - Every Linux/Unix system has Vi/Vim installed
2. Incredibly fast - Once you learn it, editing becomes muscle memory
3. Composable - Commands combine like a language
4. Powerful - Text manipulation capabilities you didn’t know were possible
5. No mouse required - Your hands stay on the keyboard
6. Extensible - Plugins, custom commands, integration with everything

Most importantly: Vim isn’t just an editor, it’s a way of thinking about text editing. Once you learn it, you’ll want Vim keybindings in every tool you use.

The Vim Learning Curve

Let’s be honest: Vim has a steep learning curve.

But here’s the timeline:

• Hour 1-2: Painful. You’ll be slower than nano.
• Hours 3-10: Still slower, but you’ll see glimpses of power.
• Hours 10-20: About as fast as your old editor.
• Hour 20+: Faster. Noticeably faster.
• Week 4+: You can’t imagine editing any other way.

Strategy for this class:

• Week 2: Learn the basics, use for homework
• Week 3-4: Force yourself to use it exclusively
• Week 5+: Enjoy the benefits

Philosophy: Modal Editing

Vim’s core insight: Most of your time isn’t spent typing new text.

Think about how you edit:

• Navigate to the right place (most time)
• Make a small change
• Navigate somewhere else
• Repeat

Vim’s solution: Different modes for different tasks.

The Four Essential Modes

The key: You spend most of your time in Normal mode, where every key is a command.

Example: Deleting a Word

In a traditional editor:

1. Click at the start of the word
2. Click and drag to select the word
3. Press Delete

In Vim (Normal mode):

1. Type daw (delete a word)

Three keystrokes. No mouse. No holding down arrow keys.

Getting Started: Survival Guide

Opening Vim

vim filename.txt     # Edit a file
vim                  # Start with empty buffer
vim -                # Read from stdin

The Panic Button

Stuck in Vim? Press <Esc> a few times, then type :q! and press Enter.

• <Esc> - Get back to Normal mode
• :q! - Quit without saving

Essential Commands (Command Mode)

All of these start with : (from Normal mode):

Basic Editing

From Normal mode:

1. Press i to enter Insert mode
2. Type your text (works like any editor)
3. Press <Esc> to return to Normal mode
4. Type :wq to save and quit

Congratulations, you can now use Vim!

(Inefficiently, but it works.)

Movement: The Foundation

In Normal mode, you never use the arrow keys. Here’s why:

     k
     ↑
  h ← → l
     ↓
     j

Why hjkl? Your fingers stay on the home row. Faster than reaching for arrow keys.

Basic Movement

Practice: Open a file and move around using only hjkl. No arrow keys!

Word Movement

Example:

The quick-brown fox jumps
    ^     ^     ^   ^
    w     w     w   w

Line Movement

File Movement

Screen Movement

Search Movement

Example:

/function    " Search for "function"
n            " Next occurrence
N            " Previous occurrence

Character Search (within a line)

Example:

def calculate_total(items, tax_rate):
#              ^                  ^
#              fa                 fa

Editing: Operators (Verbs)

Vim’s power comes from composing operators with motions.

Core Operators

Composing: Operator + Motion

Format: {operator}{motion}

Examples:

See the pattern? Every operator works with every motion.

Entering Insert Mode

Most common: i, a, o

Text Objects: Smart Selections

Text objects let you select logical chunks of text.

Format

{operator}{a/i}{text-object}

• a - “a” (includes surrounding characters like quotes, brackets)
• i - “inner” (excludes surrounding characters)

Common Text Objects

Examples

Cursor on: "Hello, World!"

Code example:

def function(arg1, arg2, arg3):
    #        ^cursor here

Counts: Multipliers

You can prefix most commands with a number.

Format: {count}{operator}{motion}

Examples:

Practical example:

# Instead of:
dd
dd
dd

# Just do:
3dd

Visual Mode: Selecting Text

Visual mode lets you select text to operate on.

Types of Visual Mode

Using Visual Mode

1. Enter visual mode with v, V, or Ctrl-v
2. Move cursor to expand selection
3. Press an operator: d (delete), y (yank), c (change), etc.

Example:

1. Position cursor at start of word
2. Press v
3. Press e (moves to end of word)
4. Press d (deletes selected text)

Visual Block Mode (Powerful!)

Select rectangular regions:

var1 = 10
var2 = 20
var3 = 30
var4 = 40

1. Cursor on 1 in var1
2. Ctrl-v (enter visual block mode)
3. 3j (select down 3 lines)
4. l (select right to include the digit)
5. c (change selected text)
6. Type x
7. <Esc>

Result:

varx = 10
varx = 20
varx = 30
varx = 40

This is magic for editing columnar data!

Undo and Redo

Vim has unlimited undo with a tree structure. You can undo, make changes, and still get back to the original state.

Search and Replace

Basic Search

/pattern      " Search forward
?pattern      " Search backward
n             " Next match
N             " Previous match

Search and Replace (Substitute)

Format: :[range]s/old/new/[flags]

Examples:

:s/foo/bar/        " Replace first 'foo' with 'bar' on current line
:s/foo/bar/g       " Replace all 'foo' with 'bar' on current line
:%s/foo/bar/g      " Replace all 'foo' with 'bar' in entire file
:%s/foo/bar/gc     " Replace all 'foo' with 'bar' in entire file (confirm each)
:5,12s/foo/bar/g   " Replace in lines 5-12
:.,$s/foo/bar/g    " Replace from current line to end of file

Flags:

• g - Global (all occurrences on line)
• c - Confirm each substitution
• i - Case insensitive
• I - Case sensitive

Practical example:

" Change all instances of 'color' to 'colour' in entire file
:%s/color/colour/g

" Change 'TODO' to 'DONE' with confirmation
:%s/TODO/DONE/gc

Repetition: The Dot Command

The . (dot) command repeats the last change.

Example:

word1 word2 word3 word4

1. Cursor on w in word1
2. dw (delete word)
3. Move to word2
4. . (repeat delete)
5. Move to word3
6. . (repeat delete again)

This is incredibly powerful for repetitive edits.

Working with Multiple Files

Buffers

Vim keeps files in “buffers” (think of them as open files).

:e filename        " Edit (open) a file
:ls                " List buffers
:b {number}        " Switch to buffer {number}
:b {name}          " Switch to buffer matching {name}
:bn                " Next buffer
:bp                " Previous buffer
:bd                " Delete (close) buffer

Windows (Splits)

Split your screen to view multiple files:

:sp filename       " Split horizontally
:vsp filename      " Split vertically
Ctrl-w h/j/k/l     " Navigate between windows
Ctrl-w w           " Cycle through windows
Ctrl-w q           " Close current window
Ctrl-w =           " Make windows equal size

Tabs

Tabs contain windows:

:tabnew filename   " Open file in new tab
:tabn              " Next tab
:tabp              " Previous tab
:tabclose          " Close tab
gt                 " Next tab (Normal mode)
gT                 " Previous tab (Normal mode)

Macros: Recording and Replay

Macros let you record a sequence of commands and replay them.

Recording a Macro

1. q{register} - Start recording (register is a-z)
2. Perform your edits
3. q - Stop recording
4. @{register} - Replay macro
5. @@ - Replay last macro
6. {count}@{register} - Replay macro {count} times

Example: Add semicolons to end of lines

Starting with:

var x = 10
var y = 20
var z = 30

1. Cursor on first line
2. qa (start recording into register ‘a’)
3. A; (append semicolon at end of line)
4. <Esc>j (escape to Normal mode, move down)
5. q (stop recording)
6. 2@a (replay macro twice for remaining lines)

Result:

var x = 10;
var y = 20;
var z = 30;

Common Patterns and Recipes

Delete blank lines

:g/^$/d

Indent/Dedent in Visual mode

1. Select lines with V
2. > to indent, < to dedent
3. . to repeat

Or in Normal mode:

• >> - Indent current line
• << - Dedent current line
• 5>> - Indent 5 lines

Join lines

J       " Join current line with next
3J      " Join next 3 lines

Swap lines

ddp     " Delete line and paste below (swap with next)
ddkP    " Delete line, move up, paste before (swap with previous)

Toggle case

~       " Toggle case of character under cursor
g~w     " Toggle case of word
g~~     " Toggle case of line
gUw     " Uppercase word
guw     " Lowercase word

Increment/Decrement numbers

Ctrl-a  " Increment number under cursor
Ctrl-x  " Decrement number under cursor

Comment multiple lines

Using visual block:

1. Ctrl-v (visual block mode)
2. Select lines (with j)
3. I (insert at start of block)
4. Type # (or // for other languages)
5. <Esc>

Configuration: Your .vimrc

Vim is configured via ~/.vimrc.

Essential Settings

" Basic settings
set number              " Show line numbers
set relativenumber      " Show relative line numbers
set mouse=a             " Enable mouse support
set clipboard=unnamed   " Use system clipboard
set ignorecase          " Case-insensitive search
set smartcase           " Case-sensitive if uppercase present
set incsearch           " Incremental search
set hlsearch            " Highlight search results
set expandtab           " Use spaces instead of tabs
set tabstop=4           " Tab width
set shiftwidth=4        " Indent width
set autoindent          " Copy indent from current line
set smartindent         " Smart indenting
set wrap                " Wrap long lines
set linebreak           " Wrap at word boundaries
syntax on               " Enable syntax highlighting
filetype plugin indent on  " Enable filetype detection

Useful Mappings

" Map jk to escape insert mode
inoremap jk <Esc>

" Clear search highlighting
nnoremap <silent> <leader><space> :noh<CR>

" Quick save
nnoremap <leader>w :w<CR>

" Quick quit
nnoremap <leader>q :q<CR>

" Move between windows
nnoremap <C-h> <C-w>h
nnoremap <C-j> <C-w>j
nnoremap <C-k> <C-w>k
nnoremap <C-l> <C-w>l

Note: <leader> is typically \ by default. You can change it:

let mapleader = " "     " Use space as leader

Starter .vimrc

Create ~/.vimrc with this:

" Senior Mars' Minimal Vim Config

"=== Basic Settings ===
set nocompatible
syntax on
filetype plugin indent on

set number relativenumber
set mouse=a
set hidden
set nowrap
set ignorecase smartcase
set incsearch hlsearch

"=== Indentation ===
set expandtab
set tabstop=4
set shiftwidth=4
set autoindent
set smartindent

"=== UI ===
set showcmd
set wildmenu
set laststatus=2
set cursorline

"=== Leader Key ===
let mapleader = " "

"=== Key Mappings ===
" Easy escape
inoremap jk <Esc>

" Quick save/quit
nnoremap <leader>w :w<CR>
nnoremap <leader>q :q<CR>

" Clear search
nnoremap <leader><space> :noh<CR>

" Window navigation
nnoremap <C-h> <C-w>h
nnoremap <C-j> <C-w>j
nnoremap <C-k> <C-w>k
nnoremap <C-l> <C-w>l

" Visual mode indenting
vnoremap < <gv
vnoremap > >gv

"=== Quality of Life ===
" Return to last edit position when opening files
autocmd BufReadPost *
    \ if line("'\"") > 0 && line("'\"") <= line("$") |
    \   exe "normal! g`\"" |
    \ endif

" Disable arrow keys (force hjkl)
noremap <Up> <Nop>
noremap <Down> <Nop>
noremap <Left> <Nop>
noremap <Right> <Nop>
inoremap <Up> <Nop>
inoremap <Down> <Nop>
inoremap <Left> <Nop>
inoremap <Right> <Nop>

When to Use Vim vs. Other Editors

Use Vim When:

• Editing files on remote servers
• Making quick edits
• Working in the terminal
• Pair programming over SSH
• Your IDE isn’t available
• You need maximum speed and efficiency

Use VS Code (or IDE) When:

• Learning a new codebase (better project navigation)
• Debugging with breakpoints and step-through
• Working with complex refactoring tools
• Using language-specific features (like Jupyter notebooks)
• You need GUI tools (like git GUI, database viewers)

Best approach: Learn both. Use the right tool for the job.

Many developers use Vim keybindings in their IDE (VS Code has excellent Vim extension).

Better Alternatives (and Variants): Neovim + Helix

Vim is the baseline because it’s everywhere. But in 2025, you’ll also see two very common “upgrades”:

• Neovim: Vim, but modernized and highly configurable
• Helix: modal editor with a different philosophy (and great defaults)

You don’t need to switch today. But you should know they exist, what they’re good at, and how to get started if you’re curious.

Neovim (nvim): Vim, but modern

A modern fork of Vim with a strong ecosystem and first-class Lua configuration.

Why people like it:

• Better defaults and smoother modernization than classic Vim
• Excellent plugin ecosystem
• Easier “editor as a tool” workflow: LSP, completion, formatting, git integration
• Config in Lua (less “mystery vimscript” once you get the hang of it)

When to use Neovim:

• You want Vim skills plus IDE-like features in the terminal
• You like customizing your workflow and keeping everything keyboard-driven
• You want consistent editing locally and over SSH

If you learn Vim motions/operators, you already know 80% of Neovim. The difference is mostly “modern features + configuration.”

Helix (hx): modal editing with better defaults

A newer terminal editor inspired by Vim, but designed to be more discoverable.

Helix’s big idea: selection-first editing. Instead of “operator then motion” (like dw), Helix encourages selecting text, then acting on it.

Why people like it:

• Great out-of-the-box experience (less configuration required)
• Built-in LSP support feels natural
• Keybindings are designed to be learnable without decades of historical baggage
• Very good for multi-cursor and structural edits

Helix is not “Vim but better.” It’s “Vim-adjacent with a different editing grammar.” Both are worth knowing.

Practical advice: what should you do?

For this class:

• Learn Vim first (it transfers everywhere)
• If you’re excited, try Neovim as your daily driver
• If you want modern defaults with less setup, try Helix

Best case outcome: you end up bilingual:

• Vim motions as the universal baseline
• One modern editor (Neovim or Helix) for your personal workflow

Optional: Vim keybindings everywhere

If you love the Vim editing model but want to stay in your IDE:

• VS Code: install the Vim extension
• JetBrains IDEs: IdeaVim plugin

This is extremely common in industry: one editing language, many environments.

Learning Resources

Essential

1. vimtutor - Built-in interactive tutorial

   vimtutor
   

   Do lessons 1-4 minimum. Takes ~30 minutes.

2. Vim Help - Comprehensive built-in documentation

   :help
   :help motion
   :help text-objects
   :help :substitute
   

3. Vim Adventures - Game to learn Vim https://vim-adventures.com/

Recommended Reading

• Vim as Language - Best mental model
• Practical Vim - Book by Drew Neil
• Vim Tips Wiki

Practice

• Vim Golf - Solve editing challenges in minimum keystrokes
• OpenVim - Interactive tutorial

Common Mistakes and Misconceptions

“I need to memorize everything”

No. Learn the basics, use them daily, look up what you need as you go. You’ll gradually build muscle memory.

“I have to give up my mouse”

No. You can enable mouse support (set mouse=a). But you’ll naturally use it less as Vim becomes faster.

“Vim makes me a better programmer”

No. Vim makes you a faster editor, not a better programmer. Good code comes from thinking, not typing speed.

“I need tons of plugins”

No. Start with vanilla Vim. Add plugins only when you feel the pain they solve.

“Vim is only for hardcore terminal nerds”

No. Vim keybindings are available in VS Code, IntelliJ, Sublime, even web browsers. Learn once, use everywhere.

The Two-Week Challenge

Week 2: Use Vim for this week’s homework. It will be painful. Do it anyway.

Week 3: Use Vim exclusively for all coding. No fallback to other editors. Reference this guide and use :help.

Week 4+: Notice you’re editing faster. Notice you’re annoyed when Vim keybindings aren’t available.

By Week 4, you won’t want to go back.

Conclusion

The Three Laws of Vim:

1. Stay in Normal mode as much as possible
2. Learn one new command per day
3. If you’re doing something repetitive, there’s a better way

Next week, we’ll use Vim to do serious data wrangling with sed, awk, and regex. You’ll be glad you learned it.

Week 2 Video

WIP.

Week 2 Homework - Terminal Text Editors

Time estimate: 1.5 hours maximum

Submission: Create a directory week2-[netid] with your solutions and add it as a submodule to the class repo.

Important Note

This week will feel slower than Week 1. That’s expected. You’re learning a new way to edit text, and your muscle memory needs time to develop.

If any exercise takes significantly longer than the time estimate, stop and ask for help. The goal is learning, not frustration.

Setup

Create your homework directory:

mkdir week2-[netid]
cd week2-[netid]

Exercise 1: Complete vimtutor (30 minutes)

Required

Run the built-in Vim tutorial:

vimtutor

Complete lessons 1 through 4 minimum. If you have time, do all 7 lessons.

Deliverable: Create a file vimtutor_notes.txt with:

• Which lessons you completed (1-4 minimum, 1-7 ideal)
• Three things you learned that surprised you
• One thing you found confusing or difficult

Save this file using Vim:

1. vim vimtutor_notes.txt
2. Press i to enter insert mode
3. Type your notes
4. Press <Esc> to return to normal mode
5. Type :wq to save and quit

Exercise 2: Set Up Your .vimrc (15 minutes)

Required

Create a basic Vim configuration file.

Steps:

1. Create ~/.vimrc if it doesn’t exist:

   vim ~/.vimrc
   

2. Copy the basic config from the Week 2 notes (the “Starter .vimrc” section)

3. Save and test:

   vim test.txt
   

   Verify you see:

   • Line numbers
   • Relative line numbers
   • Syntax highlighting
   • Your cursor line highlighted

Deliverable: Create a file vimrc_setup.txt answering:

• What settings did you add to your .vimrc?
• Which setting do you think will be most useful?
• Did you customize anything beyond the starter config? If so, what?

Exercise 3: Install a Plugin Manager and Plugin (20 minutes)

Required

Modern Vim users manage plugins with a plugin manager. We’ll use vim-plug.

Part A: Install vim-plug

For Vim:

curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

For Neovim: (or whatever plugin manager you’re using)

sh -c 'curl -fLo "${XDG_DATA_HOME:-$HOME/.local/share}"/nvim/site/autoload/plug.vim --create-dirs \
       https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim'

Part B: Configure plugins in .vimrc

Add this section to your ~/.vimrc (above your other settings):

" Plugins
call plug#begin('~/.vim/plugged')

" Add plugins here
Plug 'junegunn/fzf', { 'do': { -> fzf#install() } }
Plug 'junegunn/fzf.vim'

call plug#end()

Part C: Install the plugins

1. Open Vim: vim
2. Run the command: :PlugInstall
3. Wait for installation to complete
4. Quit and reopen Vim

Part D: Test fzf

fzf is a fuzzy file finder (like Ctrl-P in VS Code).

1. Navigate to a directory with multiple files
2. Open Vim: vim
3. Type :Files and press Enter
4. Start typing a filename - see it filter in real-time
5. Press Enter to open a file

Deliverable: Create plugin_setup.txt with:

• Screenshot or description of fzf working
• What command did you use to install plugins? (:PlugInstall)
• What does fzf do and why is it useful?

Bonus: Add a keybinding to open fzf with Ctrl-P:

" Add to your .vimrc
nnoremap <C-p> :Files<CR>

Exercise 4: Vim Movement Practice (15 minutes)

Required

Download this practice file:

curl -o movement_practice.txt https://raw.githubusercontent.com/iggredible/Learn-Vim/master/ch05_moving_in_file.md

Open it in Vim and complete these tasks using ONLY Vim motions (no arrow keys, no mouse):

1. Jump to the beginning of the file
2. Jump to the end of the file
3. Find the word “vertical” (search forward)
4. Jump to line 50
5. Delete the word under your cursor
6. Delete from cursor to end of line
7. Undo your last change
8. Yank (copy) a line
9. Paste it below

Deliverable: Create movement_log.txt documenting:

• The command you used for each task above
• Which motion felt most natural?
• Which motion felt most awkward?

Example format:

1. Jump to beginning: gg
2. Jump to end: G
...

Exercise 5: Edit a Shell Script with Vim (20 minutes)

Required

Create a script using only Vim and proper Vim workflows.

Task: Write a script word_counter.sh that:

• Takes a filename as an argument
• Counts total words
• Counts unique words
• Shows the 5 most common words

Requirements:

• Use ONLY Vim to create and edit the script
• Practice using o to open new lines
• Use dw to delete words when fixing mistakes
• Use cw to change words
• Use yy and p to copy/paste lines

Starter code (type this in Vim, don’t copy-paste):

#!/usr/bin/env bash
set -euo pipefail

# TODO: Add argument validation

# TODO: Count total words

# TODO: Count unique words  

# TODO: Show top 5 most common words

Expected behavior:

$ ./word_counter.sh sample.txt
Total words: 342
Unique words: 156
Top 5 words:
  45 the
  32 and
  28 to
  22 a
  19 of

Hints:

• Use tr to split words
• Use sort and uniq -c
• Use head -5 for top 5

Deliverable: The completed word_counter.sh script

Exercise 6: Visual Mode Practice (15 minutes)

Required

Create a file visual_practice.py with this messy Python code:

def calculate_total(items):
x = 0
for item in items:
x = x + item
return x

def calculate_average(items):
total = calculate_total(items)
count = len(items)
return total / count

def main():
numbers = [1, 2, 3, 4, 5]
print(calculate_total(numbers))
print(calculate_average(numbers))

Tasks (using Vim):

1. Fix the indentation in calculate_total function

   • Use Visual Line mode (V) to select the poorly indented lines
   • Use > to indent

2. Add a docstring to calculate_total:

   """Calculate the sum of all items."""
   

   • Use o to open a new line below the function definition
   • Type the docstring

3. Comment out the print statements in main

   • Use Visual Block mode (Ctrl-v) to select the start of both lines
   • Use I to insert at the beginning
   • Type #
   • Press <Esc>

4. Duplicate the entire calculate_average function

   • Use V to select the entire function
   • Use y to yank (copy)
   • Navigate below it
   • Use p to paste
   • Change the duplicate to calculate_median using cw

Deliverable: The corrected visual_practice.py file

Exercise 7: Search and Replace (10 minutes)

Required

Create replace_practice.txt with:

TODO: Fix the login bug
TODO: Update documentation
DONE: Add tests
TODO: Refactor database code
DONE: Deploy to staging
TODO: Review pull request

Tasks (using Vim search and replace):

1. Replace all “TODO” with “IN_PROGRESS” globally
2. Replace only the first “DONE” with “COMPLETED”
3. Delete all lines containing “database”

Commands you’ll need:

:%s/old/new/g        " Replace all in file
:s/old/new/          " Replace first on line
:g/pattern/d         " Delete lines matching pattern

Deliverable:

• The modified replace_practice.txt
• A file replace_commands.txt listing the exact commands you used

Exercise 8 (Bonus): Macros (15 minutes)

Optional - Extra Credit

Given this CSV data in grades.csv:

Alice,85,92,88
Bob,78,85,90
Charlie,92,88,95
David,88,90,87
Eve,95,92,98

Task: Use a Vim macro to transform it to:

Student: Alice, Average: 88.33
Student: Bob, Average: 84.33
Student: Charlie, Average: 91.67
Student: David, Average: 88.33
Student: Eve, Average: 95.00

Approach:

1. Record a macro that:
   • Extracts the student name
   • Calculates the average (you can use Python or bc)
   • Formats the output line
2. Replay it for all lines

Hint: You might need to use external commands from Vim:

:.!python3 -c "print(sum([85, 92, 88])/3)"

Deliverable:

• The transformed grades_formatted.txt
• A file macro_explanation.txt explaining your macro

Exercise 9 (Bonus): Vim Golf (10 minutes)

Optional - Extra Credit

Vim Golf is about solving editing challenges in minimum keystrokes.

Challenge: Given this text in golf.txt:

function_name

Transform it to:

def function_name():
    pass

Your score: Number of keystrokes (lower is better)

Deliverable: A file golf_solution.txt with:

• Your keystroke sequence
• Your total count
• Explanation of what each keystroke does

Example solution format:

Keystrokes: Idef <Esc>A():<Esc>opass<Esc>
Count: 23
Explanation:
I - Insert at beginning of line
def  - Type "def "
<Esc> - Return to normal mode
...

Submission Checklist

Your week2-[netid] directory should contain:

Required Files

week2-netid/
├── vimtutor_notes.txt
├── vimrc_setup.txt
├── plugin_setup.txt
├── movement_log.txt
├── word_counter.sh
├── visual_practice.py
├── replace_practice.txt
└── replace_commands.txt

Bonus Files

├── grades_formatted.txt
├── macro_explanation.txt
├── golf_solution.txt
└── golf_explanation.txt

Before submitting:

1. Make all scripts executable: chmod +x *.sh
2. Test that your .vimrc works: vim --clean -u ~/.vimrc
3. Verify plugins load: Open Vim and run :PlugStatus
4. Commit and push to your repo

Grading

• Exercises 1-7: Required
• Exercises 8-9: Bonus

Completion-based grading: If you attempted the exercise honestly and followed the requirements, you get full credit for that exercise.

Tips for Success

Start Small

Don’t try to memorize everything. Focus on:

• hjkl for movement
• i for insert, <Esc> to exit
• dd, yy, p for delete/copy/paste
• :w, :q, :wq for file operations

Use the Help

Vim has amazing built-in help:

:help motion
:help operator
:help visual-mode
:help :substitute

Check Your Progress

After each exercise, verify you can:

1. Open a file in Vim
2. Navigate without arrow keys
3. Make edits efficiently
4. Save and quit without panic

Common Mistakes to Avoid

1. Using arrow keys: Force yourself to use hjkl
2. Staying in Insert mode: Spend most time in Normal mode
3. Not using text objects: Learn diw, ci(, da"
4. Ignoring the dot command: . repeats your last change

When You’re Stuck

1. Press <Esc> several times (get to Normal mode)
2. Type :help followed by what you’re trying to do
3. Check the Week 2 notes
4. Google: “vim how to [what you want]”
5. Come to office hours

Practice Strategy

Week 2: Force yourself to use Vim for all homework
Week 3: Use Vim exclusively (no VS Code fallback)
Week 4+: Enjoy being faster than before

Resources

• Built-in help: :help (from within Vim)
• Interactive practice: https://www.openvim.com/
• Vim Adventures: https://vim-adventures.com/ (game)
• Cheat sheet: https://vim.rtorr.com/
• Our notes: Week 2 notes on course website

Final Thoughts

Learning Vim is like learning to touch type. The first week is painful. The second week is frustrating. The third week you start to see it. By the fourth week, you wonder how you ever lived without it.

Stick with it.

By Week 4, you’ll be editing at the speed you think, and every other editor will feel slow.

Remember:

• Vim is a language for editing
• Operators + motions = powerful combinations
• Modal editing feels weird until it doesn’t
• The best way to learn is to force yourself to use it

Good luck, and see you in Week 3!

Submit by: Next Wednesday via git submodule

Stuck? Ask for help early, don’t suffer in silence

Week 3 Notes - SSH, tmux, and Intro to Servers

This week we move from working locally to working on remote servers. You’ve been SSH’ing into CLEAR since Week 0, but now we’ll understand what servers really are, configure SSH properly, and learn to create persistent workspaces that survive disconnections.

Introduction

For the past three weeks, you’ve been doing all your work on CLEAR. But what is CLEAR, really? It’s not “your computer in the cloud”, but a shared computer(s). Your laptop is just a window into this machine.

The problem: when you close your laptop or your WiFi drops, your connection dies and your running jobs die with it. Today we fix this.

Part 1: Understanding Servers

What is a Server?

A server is simply a computer optimized for specific tasks that runs continuously and is typically accessed remotely. It’s not fundamentally different from your laptop—it’s just a computer you don’t sit at.

Key characteristics:

• Always on (no sleep mode or battery concerns)
• Often shared by multiple users
• Typically has no monitor, keyboard, or mouse attached
• Optimized for specific workloads (computation, storage, etc.)

Servers are not magic, and “the cloud” is just someone else’s computer (often in a data center).

Why Use Servers?

Power:

• More CPU cores
• More RAM
• Better GPUs for machine learning
• Faster storage systems

Reliability:

• 24/7 uptime (no shutting down for battery)
• Professional backups
• Redundant power and cooling
• Maintained by IT staff

Shared Resources:

• Expensive hardware shared across users
• Large datasets stored centrally
• Software licenses (MATLAB, etc.)
• Collaboration (shared files, shared environments)

Accessibility:

• Work from anywhere (home, lab, coffee shop)
• Switch between devices easily
• Always available
• Same environment for entire team

The Rice Server Landscape

CLEAR (Cluster for Learning, Exploration, and Research):

• Linux cluster for teaching and research
• Fedora-based operating system
• Shared by all Rice students and faculty
• Your home directory is networked storage (accessible from all nodes)
• Primary teaching cluster

Other servers you might encounter:

CLEAR Architecture

When you SSH to CLEAR, you don’t connect to a single computer—you connect to a cluster:

Login nodes:

• Where you land when you SSH
• For light work only (editing files, compiling, submitting jobs)
• Shared by many users
• DO NOT run heavy computations here

Compute nodes:

• Where heavy jobs actually run
• Accessed via job scheduler (SLURM)
• More powerful than login nodes
• Your jobs get dedicated resources

Storage:

• Your home directory is networked (accessible from all nodes)
• Changes made on one node appear everywhere
• Backed up regularly
• Has quota limits

The workflow: Log into login node → submit job to scheduler → job runs on compute node → results appear in your home directory.

What Happens When You SSH

Understanding this is crucial:

$ ssh username@ssh.clear.rice.edu

Behind the scenes:

1. Your laptop opens a secure TCP connection to the server
2. Server authenticates you (password or SSH key)
3. Server starts a new shell process (bash/zsh) for you
4. That shell runs ON THE SERVER (not your laptop)
5. Your terminal displays input/output over the encrypted connection

Critical insight: The shell process lives on the server. When your connection drops, the shell dies, and everything running in it dies too.

This is why closing your laptop kills your jobs. The shell process terminates, and all child processes (your running scripts) terminate with it.

Unless you use tmux.

Part 2: SSH Deep Dive

Basic SSH Review

The basic SSH command:

ssh username@hostname

Example:

ssh netid@ssh.clear.rice.edu

This works, but it’s tedious. Let’s make it better.

SSH Config File

The SSH config file (~/.ssh/config) lets you define shortcuts and settings for different servers.

Create ~/.ssh/config:

Host clear
    HostName ssh.clear.rice.edu
    User your_netid
    ServerAliveInterval 60
    ServerAliveCountMax 3

Now instead of typing ssh netid@ssh.clear.rice.edu, you just type:

ssh clear

Config file options explained:

• Host: The shortcut name you’ll use
• HostName: The actual server address
• User: Your username on that server
• ServerAliveInterval: Send keepalive every 60 seconds
• ServerAliveCountMax: Disconnect after 3 failed keepalives

The keepalive options prevent your connection from timing out during idle periods.

Advanced SSH Config

Multiple servers:

Host clear
    HostName ssh.clear.rice.edu
    User netid
    ServerAliveInterval 60

Host davinci
    HostName davinci.rice.edu
    User netid
    ServerAliveInterval 60

Host lab
    HostName lab.cs.rice.edu
    User netid
    ProxyJump clear

ProxyJump (jump host):

• Connects to lab by first connecting through clear
• Useful for servers only accessible from Rice network
• Syntax: ProxyJump intermediate_host

Wildcards for common settings:

Host *.rice.edu
    User netid
    ServerAliveInterval 60
    
Host clear
    HostName ssh.clear.rice.edu
    
Host davinci
    HostName davinci.rice.edu

This applies User and ServerAliveInterval to all Rice hosts.

Port Forwarding: The Game Changer

Port forwarding lets you access services running on the server as if they were running on your laptop.

Local port forwarding (most common):

Basic syntax:

ssh -L local_port:localhost:remote_port user@host

Example - Jupyter notebook:

On server:

jupyter notebook --no-browser --port=8888

On laptop (new terminal):

ssh -L 8888:localhost:8888 clear

In browser:

http://localhost:8888

What’s happening:

• Server runs Jupyter on port 8888
• SSH forwards your laptop’s port 8888 to server’s port 8888
• You access localhost:8888 on your laptop
• Traffic goes through SSH tunnel to server
• Server does computation, laptop just displays

More examples:

Web development server:

# Server runs Flask on port 5000
ssh -L 5000:localhost:5000 clear

Database access:

# Server runs PostgreSQL on port 5432
ssh -L 5432:localhost:5432 clear

Multiple ports:

ssh -L 8888:localhost:8888 -L 5000:localhost:5000 clear

Add to SSH config for convenience:

Host clear-jupyter
    HostName ssh.clear.rice.edu
    User netid
    LocalForward 8888 localhost:8888

Host clear-web
    HostName ssh.clear.rice.edu
    User netid
    LocalForward 8080 localhost:8080

Now:

ssh clear-jupyter
# Automatically sets up port forwarding

Remote port forwarding (less common):

Makes your laptop’s service accessible from server.

ssh -R 8080:localhost:3000 clear

What’s happening:

• Your laptop runs service on port 3000
• Server’s port 8080 forwards to your laptop’s port 3000
• Useful for webhooks, testing, etc.

Dynamic port forwarding (SOCKS proxy):

Turn server into proxy for all connections:

ssh -D 8080 clear

Configure browser to use SOCKS proxy at localhost:8080.

All browser traffic now goes through CLEAR. Useful for:

• Accessing resources only available from Rice network
• Accessing journal articles
• Internal tools
• VPN alternative

SSH Connection Management

Keep connections alive:

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    TCPKeepAlive yes

Reuse connections (multiplexing):

Host *
    ControlMaster auto
    ControlPath ~/.ssh/control-%r@%h:%p
    ControlPersist 10m

Benefits:

• First connection creates a master
• Subsequent connections reuse it (much faster)
• Persists 10 minutes after last connection closes

Compression for slow connections:

Host slow-server
    Compression yes

SSH Security Best Practices

Never share your private key. Ever.

Use passphrases on private keys.

Use strong key types:

• Ed25519 (preferred, modern)
• RSA 4096-bit (if Ed25519 not supported)
• Avoid DSA, ECDSA

Protect private key permissions:

chmod 600 ~/.ssh/id_ed25519

Use different keys for different purposes:

ssh-keygen -t ed25519 -f ~/.ssh/id_clear -C "clear access"
ssh-keygen -t ed25519 -f ~/.ssh/id_github -C "github"

Specify key in config:

Host clear
    IdentityFile ~/.ssh/id_clear
    
Host github.com
    IdentityFile ~/.ssh/id_github

Part 3: tmux - Terminal Multiplexer

The Problem tmux Solves

Without tmux:

$ ssh clear
$ python long_script.py
# Running...
# WiFi drops / laptop closes
# Connection dies
# Script killed
# All progress lost

With tmux:

$ ssh clear
$ tmux new -s work
$ python long_script.py
# Running...
# WiFi drops / laptop closes
# Connection dies
# BUT script still running on server
$ ssh clear
$ tmux attach -s work
# Back to running script!

tmux creates a persistent session on the server that survives disconnections.

What is tmux?

tmux = terminal multiplexer

Think of it as:

• Window manager for your terminal
• Virtual desktop that lives on the server
• Workspace that persists when you disconnect

Three hierarchical concepts:

Session:

• Top level container
• Like a “project” or “workspace”
• Persists on server
• Can have multiple windows

Window:

• Like a tab in your browser
• One per task/file
• Can split into panes

Pane:

• Split window (side-by-side or stacked)
• Each runs its own shell
• Navigate between them

Hierarchy: Session → Windows → Panes

tmux Basics

Install tmux (if needed):

# Usually already installed on servers
sudo apt install tmux   # Ubuntu
sudo yum install tmux   # Fedora/RHEL
brew install tmux       # macOS

Create a session:

tmux new -s session_name

If you don’t name it, tmux assigns a number (0, 1, 2…).

Always name your sessions descriptively:

tmux new -s ml-training
tmux new -s data-pipeline
tmux new -s debugging

Detach from session (keep it running):

Ctrl-b d

Explanation:

• Ctrl-b is the “prefix” key (all tmux commands start with this)
• d means detach

List sessions:

tmux ls

Output:

ml-training: 3 windows (created Tue Jan 21 10:30:15 2025)
data-pipeline: 1 window (created Tue Jan 21 11:15:42 2025)

Attach to session:

tmux attach -t session_name

Or shorthand:

tmux a -t session_name

Kill session (when done):

tmux kill-session -t session_name

Kill all sessions:

tmux kill-server

tmux Key Bindings

All tmux commands start with the prefix: Ctrl-b

Press Ctrl-b, release, then press the command key.

Essential commands:

Session management:

Helpful features:

Creating a Research Workspace

Typical layout:

┌─────────────────────┬──────────────────┐
│                     │                  │
│   Editor            │   Run Script     │
│   (vim/nano)        │   (output)       │
│                     │                  │
├─────────────────────┴──────────────────┤
│   Monitor (htop, logs, tail)           │
└────────────────────────────────────────┘

Create this layout:

# Create session
tmux new -s work

# Split vertically (left and right)
Ctrl-b %

# Select left pane
Ctrl-b arrow-left

# Open editor
vim script.py

# Select right pane
Ctrl-b arrow-right

# Split horizontally (top and bottom)
Ctrl-b "

# Top-right: run script
python script.py

# Bottom: monitor
Ctrl-b arrow-down
htop

Or use tmux commands:

tmux new -s work
tmux split-window -h
tmux split-window -v
tmux select-pane -t 0

tmux Configuration

Create ~/.tmux.conf:

# Change prefix to Ctrl-a (easier to reach than Ctrl-b)
unbind C-b
set -g prefix C-a
bind C-a send-prefix

# Better split commands (| and -)
bind | split-window -h
bind - split-window -v
unbind '"'
unbind %

# Vim-like pane navigation
bind h select-pane -L
bind j select-pane -D
bind k select-pane -U
bind l select-pane -R

# Resize panes with vim-like keys
bind -r H resize-pane -L 5
bind -r J resize-pane -D 5
bind -r K resize-pane -U 5
bind -r L resize-pane -R 5

# Enable mouse support
set -g mouse on

# Start window numbering at 1 (not 0)
set -g base-index 1
setw -g pane-base-index 1

# Renumber windows when one is closed
set -g renumber-windows on

# Increase scrollback buffer
set -g history-limit 10000

# Don't rename windows automatically
set -g allow-rename off

# Reload config file
bind r source-file ~/.tmux.conf \; display "Config reloaded!"

# Better colors
set -g default-terminal "screen-256color"

# Status bar customization
set -g status-position bottom
set -g status-bg colour234
set -g status-fg colour137
set -g status-left '#[fg=colour233,bg=colour245,bold] #S '
set -g status-right '#[fg=colour233,bg=colour245,bold] %Y-%m-%d %H:%M '

# Pane border
set -g pane-border-style fg=colour238
set -g pane-active-border-style fg=colour51

# Window status
setw -g window-status-current-style fg=colour51,bg=colour238,bold

Reload config:

tmux source-file ~/.tmux.conf

Or from within tmux:

Ctrl-b :source-file ~/.tmux.conf

tmux Workflow Patterns

Pattern 1: One session per project

tmux new -s thesis
tmux new -s coursework
tmux new -s research

Switch between them:

Ctrl-b s  # List sessions, choose with arrows

Pattern 2: Persistent development environment

# On server
tmux new -s dev

# Set up panes
Ctrl-b %      # Split vertically
Ctrl-b "      # Split bottom horizontally

# Left: editor
# Top-right: run code
# Bottom: git, tests, etc.

# Detach
Ctrl-b d

# Later, reconnect
tmux attach -t dev

Pattern 3: Long-running job monitoring

tmux new -s training

# Start job with logging
python train_model.py 2>&1 | tee training.log &

# New pane to monitor
Ctrl-b "
tail -f training.log

# Another pane for system resources
Ctrl-b "
htop

# Detach and go to class
Ctrl-b d

# Check later
tmux attach -t training

Pattern 4: Pair programming

Person A (creates session):

ssh clear
tmux new -s pairing

Person B (joins session):

ssh clear
tmux attach -t pairing

Both see the same screen, both can type. Great for:

• Debugging together
• Teaching
• Code review
• Technical interviews

tmux Copy Mode

Scroll back through output:

Ctrl-b [        # Enter copy mode

In copy mode:

• Arrow keys or vi keys (hjkl) to navigate
• Page Up/Down to scroll
• / to search
• q to exit

Copy text (vi mode):

Ctrl-b [        # Enter copy mode
Space           # Start selection
(move cursor)
Enter           # Copy selection
Ctrl-b ]        # Paste

Enable vi mode in ~/.tmux.conf:

setw -g mode-keys vi

tmux and vim Together

Both use similar philosophies and keybindings. Configure them to work together.

In ~/.tmux.conf:

# Smart pane switching with awareness of vim splits
is_vim="ps -o state= -o comm= -t '#{pane_tty}' \
    | grep -iqE '^[^TXZ ]+ +(\\S+\\/)?g?(view|n?vim?x?)(diff)?$'"
bind -n C-h if-shell "$is_vim" "send-keys C-h"  "select-pane -L"
bind -n C-j if-shell "$is_vim" "send-keys C-j"  "select-pane -D"
bind -n C-k if-shell "$is_vim" "send-keys C-k"  "select-pane -U"
bind -n C-l if-shell "$is_vim" "send-keys C-l"  "select-pane -R"

Now Ctrl-h/j/k/l navigate both vim splits and tmux panes seamlessly.

tmux Session Scripts

Automate common setups with scripts:

Create ~/bin/dev-session.sh:

#!/usr/bin/env bash

SESSION=$1

if [ -z "$SESSION" ]; then
    echo "Usage: $0 SESSION_NAME"
    exit 1
fi

# Create session
tmux new-session -d -s "$SESSION"

# Window 1: Editor
tmux rename-window -t "$SESSION":1 'editor'
tmux send-keys -t "$SESSION":1 'vim' C-m

# Window 2: Terminal
tmux new-window -t "$SESSION":2 -n 'terminal'

# Window 3: Logs
tmux new-window -t "$SESSION":3 -n 'logs'
tmux split-window -h -t "$SESSION":3
tmux send-keys -t "$SESSION":3.1 'htop' C-m

# Select first window
tmux select-window -t "$SESSION":1

# Attach to session
tmux attach-session -t "$SESSION"

Usage:

chmod +x ~/bin/dev-session.sh
~/bin/dev-session.sh myproject

tmux Plugins (Optional)

tmux Plugin Manager (TPM):

Install:

git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm

Add to ~/.tmux.conf:

# List of plugins
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-sensible'
set -g @plugin 'tmux-plugins/tmux-resurrect'  # Save/restore sessions
set -g @plugin 'tmux-plugins/tmux-continuum'  # Auto-save sessions

# Initialize TPM (keep at bottom of tmux.conf)
run '~/.tmux/plugins/tpm/tpm'

Install plugins:

Ctrl-b I  # Capital I

Useful plugins:

• tmux-resurrect: Save/restore tmux sessions across reboots
• tmux-continuum: Automatically save sessions
• tmux-yank: Better copy/paste
• tmux-open: Open links/files from tmux

Part 4: The Complete Research Pipeline

The Workflow

Step 1: Configure SSH (one time)

Create ~/.ssh/config:

Host clear
    HostName ssh.clear.rice.edu
    User netid
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60

Generate and copy SSH key:

ssh-keygen -t ed25519
ssh-copy-id clear

Step 2: Connect to server

ssh clear

Step 3: Create persistent session

tmux new -s experiment

Step 4: Set up workspace

# Split vertically
Ctrl-b %

# Left pane: create/edit script
vim experiment.py

# Right pane: will run the script
# (leave empty for now)

# Create bottom monitoring pane
Ctrl-b "

Step 5: Run job with logging

Right pane:

python experiment.py 2>&1 | tee experiment.log

Bottom pane:

tail -f experiment.log

Or monitor system:

htop

Step 6: Detach and disconnect

Ctrl-b d
exit

Close laptop. Go to class. Get coffee.

Step 7: Reconnect later

ssh clear
tmux attach -t experiment

Everything is still running.

Step 8: Transfer results

On your laptop:

rsync -avP clear:~/experiment/results.json ./

Or entire directory:

rsync -avP clear:~/experiment/ ./experiment_results/

Step 9: Clean up

# Kill the session when done
tmux kill-session -t experiment

# Or keep it for later reference

Using tee for Logging

tee writes output to both stdout and a file:

python script.py | tee output.log

Capture stderr too:

python script.py 2>&1 | tee output.log

Append instead of overwrite:

python script.py 2>&1 | tee -a output.log

Better pattern with timestamps:

{
    echo "===== Started at $(date) ====="
    python experiment.py
    echo "===== Finished at $(date) ====="
} 2>&1 | tee "experiment_$(date +%Y%m%d_%H%M%S).log"

Monitoring Long-Running Jobs

Watch a log file update:

tail -f logfile.txt

Or with line numbers:

tail -f logfile.txt | nl

Watch command output refresh:

watch -n 1 'tail -5 progress.log'

Monitor last N lines:

watch -n 1 'ls -lht results/ | head -10'

Check if process is still running:

ps aux | grep python

Or:

pgrep -f experiment.py

Monitor resources:

htop

Filter by user:

htop -u $USER

Part 5: rsync - Smart File Transfer

Why rsync over scp?

scp (secure copy):

• Copies everything every time
• No progress indicator
• Can’t resume interrupted transfers
• No synchronization

rsync:

• Only copies changed files
• Shows progress
• Can resume
• Synchronizes directories
• Handles deletions properly

Basic rsync Usage

From server to laptop:

rsync -avP server:~/path/to/file.txt ./

From laptop to server:

rsync -avP ./file.txt server:~/path/to/

Flags explained:

• -a: Archive mode (preserves permissions, timestamps, symbolic links)
• -v: Verbose (show what’s being transferred)
• -P: Progress + partial (show progress bar, keep partial files for resuming)

rsync Patterns

Sync entire directory:

rsync -avP clear:~/project/ ./project_backup/

Important: The trailing / matters!

# Syncs contents of project into project_backup
rsync -avP clear:~/project/ ./project_backup/

# Syncs project directory itself into project_backup
rsync -avP clear:~/project ./project_backup/
# Result: ./project_backup/project/...

Dry run (see what would happen):

rsync -avPn clear:~/data/ ./data/

The -n flag shows what would be transferred without actually doing it.

Exclude patterns:

rsync -avP --exclude '*.tmp' --exclude '.git' --exclude '__pycache__' \
      clear:~/project/ ./project/

Delete on destination:

rsync -avP --delete clear:~/project/ ./project/

This makes destination exactly match source (deletes files not on server).

Bandwidth limit:

rsync -avP --bw-limit=1000 clear:~/large_file ./

Limits to 1000 KB/s.

Compress during transfer:

rsync -avPz clear:~/data/ ./data/

The -z flag compresses data during transfer (good for text files, unnecessary for already-compressed files).

rsync for Backups

Incremental backups with hardlinks:

#!/usr/bin/env bash
# backup.sh

DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="$HOME/backups"
LATEST="$BACKUP_DIR/latest"

mkdir -p "$BACKUP_DIR"

rsync -avP \
      --link-dest="$LATEST" \
      clear:~/project/ \
      "$BACKUP_DIR/$DATE/"

# Update latest symlink
rm -f "$LATEST"
ln -s "$DATE" "$LATEST"

This creates incremental backups using hardlinks. Unchanged files are hardlinked (take no extra space).

Part 6: Server Hygiene

Check Disk Usage

Your usage:

du -sh ~/*

More detailed:

du -h ~ | sort -h | tail -20

Find large files:

find ~ -type f -size +100M -exec ls -lh {} \;

Or:

find ~ -type f -size +100M -exec du -h {} \; | sort -h

Check quota:

quota -s

On CLEAR:

fs lq

Check filesystem usage:

df -h

Find files by age:

# Files modified in last 7 days
find ~ -type f -mtime -7

# Files not modified in last 90 days
find ~ -type f -mtime +90

Clean Up

Delete carefully:

# NEVER do this blindly
rm -rf ~/old_data/*

# DO THIS instead
ls ~/old_data/*
# Verify what you're deleting
rm ~/old_data/specific_file.txt

Compress instead of delete:

tar -czf old_data_$(date +%Y%m%d).tar.gz old_data/
rm -rf old_data/

Use trash instead of rm:

# Install trash-cli
pip install trash-cli --user

# Use trash instead of rm
trash old_file.txt

# Can recover if needed
trash-list
trash-restore

Clear cache directories:

rm -rf ~/.cache/*
rm -rf ~/.local/share/Trash/*

Process Management

List your processes:

ps aux | grep $USER

Find specific process:

ps aux | grep python

Or:

pgrep -a python

Kill process by PID:

kill 12345

Kill by name:

pkill -f experiment.py

Force kill:

kill -9 12345

Or:

pkill -9 -f experiment.py

Kill all your Python processes:

pkill -u $USER python

Find processes using CPU:

ps aux | sort -nrk 3 | head

Find processes using memory:

ps aux | sort -nrk 4 | head

Background Jobs

Run command in background:

python script.py &

List background jobs:

jobs

Bring job to foreground:

fg %1

Kill background job:

kill %1

Detach running process:

# Press Ctrl-Z to suspend
# Then:
bg  # Continue in background
disown  # Detach from shell

Better: Use tmux or nohup:

nohup python script.py > output.log 2>&1 &

File Permissions

Check permissions:

ls -la

Make script executable:

chmod +x script.sh

Common permission patterns:

chmod 755 script.sh    # rwxr-xr-x (owner: read/write/execute, others: read/execute)
chmod 644 file.txt     # rw-r--r-- (owner: read/write, others: read-only)
chmod 700 private.sh   # rwx------ (owner only)
chmod 600 secret.txt   # rw------- (owner read/write only)

Recursive:

chmod -R 755 directory/

Change ownership:

chown user:group file.txt

Shared Server Etiquette

Don’t run heavy jobs on login nodes:

• Login nodes are shared by all users
• Heavy computation slows everyone down
• Use job scheduler (SLURM) for heavy work

Check load before running:

uptime

Output: load average: 5.23, 4.15, 3.87

If load is high (> number of CPUs), don’t add more.

Use nice for CPU-heavy tasks:

nice -n 19 python script.py

This gives your process lowest priority.

Monitor your resource usage:

htop -u $USER

Clean up temporary files:

rm -rf /tmp/mywork_*

Part 7: Reproducibility and Documentation

Save Environment Information

Create metadata file:

{
    echo "Experiment run at: $(date)"
    echo "Hostname: $(hostname)"
    echo "User: $(whoami)"
    echo "Working directory: $(pwd)"
    echo "Git commit: $(git rev-parse HEAD 2>/dev/null || echo 'N/A')"
    echo "Python version: $(python --version 2>&1)"
    echo "Pip freeze:"
    pip freeze
} > metadata.txt

As a function (add to .bashrc):

save_metadata() {
    local outfile="${1:-metadata.txt}"
    {
        echo "Date: $(date)"
        echo "Host: $(hostname)"
        echo "User: $(whoami)"
        echo "PWD: $(pwd)"
        echo "Git: $(git rev-parse HEAD 2>/dev/null || echo 'not in repo')"
        echo "Python: $(python --version 2>&1)"
        echo "---"
        echo "Environment:"
        pip freeze
    } > "$outfile"
}

Usage:

save_metadata experiment_metadata.txt

Log Everything

Pattern for experiments:

#!/usr/bin/env bash
# run_experiment.sh

EXPERIMENT_DIR="experiment_$(date +%Y%m%d_%H%M%S)"
mkdir -p "$EXPERIMENT_DIR"
cd "$EXPERIMENT_DIR"

# Save metadata
{
    echo "Started: $(date)"
    echo "Host: $(hostname)"
    echo "Git: $(git rev-parse HEAD)"
    python --version
} > metadata.txt

# Run experiment with logging
python ../experiment.py \
    --config config.json \
    2>&1 | tee experiment.log

# Save completion info
echo "Completed: $(date)" >> metadata.txt

Video for week3

WIP

Week 3 Homework - Your First VPS

Time estimate: 1.5 hours maximum

Goal: Create your own cloud server, configure SSH properly, run a persistent web service with tmux, and prove it survives disconnections.

Why This Matters

You’ve been using CLEAR (a shared server). Now you’ll create and configure your own Virtual Private Server (VPS). This is how you’d deploy a real web application, run experiments on cloud hardware, or set up your own development environment.

By the end, you’ll have:

• Your own cloud server (that you can recreate anytime)
• SSH configured properly
• A web server running persistently with tmux
• Understanding of how “the cloud” actually works

Important Notes

Cost: DigitalOcean gives students $200 in free credits (valid 1 year). This homework uses ~$0.10 of credit.

Cleanup: DELETE YOUR DROPLET when done to save credits for future projects.

Time limit: If any part takes more than estimated time, ask for help.

Submission Requirements

Create a directory week3-[netid]/ containing:

Required files:

week3-netid/
├── answers.md           # Answers to questions throughout
├── browser_screenshot.png    # Your web server in browser
├── tmux_screenshot.png      # tmux session with web server running
└── deleted_screenshot.png   # Proof you deleted the droplet

Optional:

└── notes.md            # What broke, how you fixed it

Exercise 1: Get DigitalOcean Student Credit (10 minutes)

I’ll have you do this before class so you have credit ready.

Step 1: GitHub Student Pack

1. Go to https://education.github.com/pack
2. Click “Get your Pack”
3. Sign in with your GitHub account
4. Verify with your Rice email (@rice.edu)
5. Wait for approval (usually instant, sometimes 1-2 days)

If not instant, you can continue with the 60-day free trial and apply credits later.

Step 2: Activate DigitalOcean Credit

1. Once approved, find DigitalOcean in the pack
2. Click “Get access to DigitalOcean”
3. Create a DigitalOcean account (or sign in)
4. Apply the $200 credit

Verify you have credit:

1. Go to https://cloud.digitalocean.com/
2. Click your profile (top right) → Billing
3. You should see $200 credit

Exercise 2: Generate SSH Key (5 minutes)

If you already have an SSH key from Week 3 notes, skip to Step 3.

Step 1: Check for Existing Key

ls ~/.ssh/id_*.pub

If you see id_ed25519.pub or id_rsa.pub, you already have a key. Skip to Step 3.

Step 2: Generate New Key

ssh-keygen -t ed25519 -C "your_rice_email@rice.edu"

• Press Enter for default location
• Enter passphrase (optional but recommended)
• Press Enter again

Step 3: Copy Public Key

Display your public key:

cat ~/.ssh/id_ed25519.pub

Copy the entire output (starts with ssh-ed25519).

Question 1: In answers.md, explain what an SSH key is and why it’s better than passwords.

Exercise 3: Create Your Droplet (10 minutes)

Step 1: Create New Droplet

1. Log into https://cloud.digitalocean.com/
2. Click “Create” → “Droplets”

Step 2: Choose Configuration

Image: Ubuntu 24.04 LTS (under “Marketplace” or “OS”)

Droplet Type: Basic (should be selected)

CPU Options: Regular (should be selected)

Size: $4/month option

• 512 MB RAM
• 1 CPU
• 10 GB SSD

Datacenter: Choose closest to you (e.g., New York or San Francisco)

Step 3: Add SSH Key

Authentication: Select “SSH Key”

Click “New SSH Key”

• Paste your public key from Exercise 2
• Name it: “my-laptop” or similar
• Click “Add SSH Key”

Step 4: Finalize

Hostname: Give it a name like rice-student-server

Tags: (leave empty)

Backups: No (unchecked)

Click “Create Droplet”

Wait 1-2 minutes for it to be created.

Step 5: Note IP Address

Once created, you’ll see your droplet with an IP address like 123.456.78.90

Deliverable: In answers.md, record your droplet’s IP address.

Question 2: In answers.md, explain what a “droplet” is and how it differs from your laptop.

Exercise 4: Connect via SSH (10 minutes)

Step 1: First Connection

ssh root@YOUR_DROPLET_IP

Replace YOUR_DROPLET_IP with the actual IP.

You might see:

The authenticity of host '123.456.78.90' can't be established.
ED25519 key fingerprint is SHA256:...
Are you sure you want to continue connecting (yes/no)?

Type yes and press Enter.

If successful, you’ll see:

Welcome to Ubuntu 24.04 LTS
root@rice-student-server:~#

Step 2: Explore

# Where am I?
hostname

# What OS?
cat /etc/os-release

# How much RAM?
free -h

# Disk space?
df -h

Question 3: In answers.md, answer:

• What’s the hostname?
• How much RAM does your droplet have?
• How much disk space is available?

Step 3: Create Non-Root User (Security Best Practice)

Don’t use root for everything. Create your own user:

# Create user (replace 'yourname' with your netid)
adduser yourname

# Add to sudo group
usermod -aG sudo yourname

# Copy SSH keys to new user
rsync --archive --chown=yourname:yourname ~/.ssh /home/yourname

Test the new user:

su - yourname

If it works, exit back to root:

exit

Exercise 5: Configure SSH (15 minutes)

Step 1: Create SSH Config on Your Laptop

On your laptop (not the server), edit ~/.ssh/config:

vim ~/.ssh/config

Add:

Host droplet
    HostName YOUR_DROPLET_IP
    User yourname
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60

Replace:

• YOUR_DROPLET_IP with your actual IP
• yourname with the username you created

Save and quit (:wq in vim).

Step 2: Test SSH Config

Disconnect from server (if connected):

exit

Now connect using the alias:

ssh droplet

Should connect without asking for IP or username!

Question 4: In answers.md, explain what the SSH config file does and why it’s useful.

Step 3: Disable Password Authentication (Security)

On the server, edit SSH config:

sudo vim /etc/ssh/sshd_config

Find and change these lines:

PasswordAuthentication no
PubkeyAuthentication yes

Save and restart SSH:

sudo systemctl restart sshd

Disconnect and reconnect to verify it still works:

exit
ssh droplet

Exercise 6: Install tmux (5 minutes)

On your droplet:

# Update package list
sudo apt update

# Install tmux
sudo apt install tmux -y

# Verify
tmux -V

Should show: tmux 3.x or similar.

Create basic tmux config:

cat > ~/.tmux.conf << 'EOF'
# Enable mouse
set -g mouse on

# Start numbering at 1
set -g base-index 1

# Better prefix (Ctrl-a instead of Ctrl-b)
unbind C-b
set -g prefix C-a
bind C-a send-prefix
EOF

Test tmux:

tmux new -s test

You should see a green bar at bottom.

Detach:

Ctrl-a d

Reattach:

tmux attach -t test

Kill the test session:

tmux kill-session -t test

Exercise 7: Create a Web Server (20 minutes)

Step 1: Create a Simple Web App

On your droplet, create a directory:

mkdir ~/web-demo
cd ~/web-demo

Create a simple Flask app:

cat > app.py << 'EOF'
from flask import Flask
import socket
import os
from datetime import datetime

app = Flask(__name__)

@app.route('/')
def home():
    return f'''
    <html>
    <head><title>My First VPS</title></head>
    <body style="font-family: Arial; margin: 50px; background: #f0f0f0;">
        <h1>🎉 Hello from the Cloud!</h1>
        <p><strong>Hostname:</strong> {socket.gethostname()}</p>
        <p><strong>Server Time:</strong> {datetime.now()}</p>
        <p><strong>Running on:</strong> {os.uname().sysname} {os.uname().release}</p>
        <hr>
        <p>This server is running in a tmux session!</p>
        <p>Try disconnecting and reconnecting - I'll still be here! 😎</p>
    </body>
    </html>
    '''

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)
EOF

Step 2: Install Flask

# Install pip if not present
sudo apt install python3-pip -y

# Install Flask
pip3 install flask --break-system-packages

Step 3: Test Locally

python3 app.py

You should see:

 * Running on http://0.0.0.0:5000

Test it works:

curl localhost:5000

Should see HTML output.

Stop the server (Ctrl-C).

Exercise 8: Run Server in tmux (15 minutes)

Step 1: Create tmux Session

tmux new -s webserver

Step 2: Set Up Panes

# Split vertically
Ctrl-a %

# Left pane: will run server
# Right pane: will monitor logs

Step 3: Run Server

In left pane:

cd ~/web-demo
python3 app.py 2>&1 | tee server.log

In right pane (Ctrl-a arrow-right):

cd ~/web-demo
tail -f server.log

You should see Flask startup messages in both panes.

Step 4: Test Persistence

1. Detach from tmux:

   Ctrl-a d
   

2. Disconnect from server:

   exit
   

3. Wait 30 seconds

4. Reconnect:

   ssh droplet
   

5. Reattach to tmux:

   tmux attach -t webserver
   

Server should still be running!

Deliverable: Take screenshot showing tmux session with web server running.

Save as: tmux_screenshot.png

Question 5: In answers.md, explain what happened when you disconnected and why the server kept running.

Exercise 9: Access Server from Your Laptop (15 minutes)

Step 1: Set Up Port Forwarding

On your laptop (new terminal), connect with port forwarding:

ssh -L 5000:localhost:5000 droplet

Or add to your SSH config:

Host droplet
    HostName YOUR_DROPLET_IP
    User yourname
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60
    LocalForward 5000 localhost:5000

Then just:

ssh droplet

Step 2: Access in Browser

Open your browser and go to:

http://localhost:5000

You should see your web page with server information!

Deliverable: Take screenshot of browser showing your web page.

Save as: browser_screenshot.png

Question 6: In answers.md, explain how port forwarding works and why you can access a server running on your droplet through localhost on your laptop.

Step 3: Verify It’s Really Running on Droplet

In the browser, note:

• The hostname (should be your droplet name)
• The server time (should match droplet timezone)

Refresh the page a few times - time should update.

In your droplet’s tmux session (right pane with logs), you should see:

127.0.0.1 - - [date] "GET / HTTP/1.1" 200 -

Each browser refresh creates a new log entry.

Exercise 10: Stress Test Persistence (10 minutes)

Goal: Prove the server survives multiple disconnections

Step 1: Initial State

Server is running in tmux. You’re connected via SSH with port forwarding. Browser shows the page.

Step 2: First Disconnect

1. In browser: Refresh page (works)
2. Close SSH connection: exit
3. Wait 1 minute
4. Browser: Try to refresh (should fail - no port forwarding)

Step 3: Reconnect

1. SSH back in with port forwarding:

   ssh -L 5000:localhost:5000 droplet
   

2. Check tmux:

   tmux ls
   

   Should show: webserver: 2 windows...

3. Reattach:

   tmux attach -t webserver
   

4. Browser: Refresh page (works again!)

Step 4: Verify Uptime

In the tmux session (left pane with server), server should show continuous uptime - no restart messages.

Check log file for gaps:

cat ~/web-demo/server.log

Should show timestamps proving server never stopped.

Question 7: In answers.md, explain how this differs from running the server directly in SSH without tmux.

Exercise 11: Cleanup (10 minutes)

IMPORTANT: Delete your droplet to save credits!

Step 1: Stop Server

In tmux session:

# Left pane: Ctrl-C to stop server
# Exit tmux
Ctrl-a d

Kill tmux session:

tmux kill-session -t webserver

Step 2: Delete Droplet

1. Go to https://cloud.digitalocean.com/
2. Click on your droplet
3. Click “Destroy” (top right)
4. Type droplet name to confirm
5. Click “Destroy”

Deliverable: Take screenshot of DigitalOcean dashboard showing droplet is deleted.

Save as: deleted_screenshot.png

Step 3: Verify Deletion

Try to SSH:

ssh droplet

Should fail with connection refused or timeout.

Question 8: In answers.md, answer:

• How much credit did this homework use?
• How much credit do you have remaining?
• How long could you run this $4/month droplet with your remaining credits?

Exercise 12 (Bonus): SSH Config Enhancements (+10%)

Add these to your SSH config and document what each does:

Host droplet-secure
    HostName YOUR_DROPLET_IP
    User yourname
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60
    ServerAliveCountMax 3
    ControlMaster auto
    ControlPath ~/.ssh/control-%r@%h:%p
    ControlPersist 10m
    Compression yes
    ForwardAgent yes

Create ssh_config_explanation.md explaining each option and when you’d use it.

Exercise 13 (Bonus): Automated Setup Script (+10%)

Create a script that automates the server setup:

setup_server.sh:

#!/usr/bin/env bash
set -euo pipefail

# Install packages
sudo apt update
sudo apt install -y tmux python3-pip

# Install Flask
pip3 install flask --break-system-packages

# Create web directory
mkdir -p ~/web-demo
cd ~/web-demo

# Create app (use heredoc)
cat > app.py << 'EOF'
# (your Flask app code)
EOF

# Create tmux startup script
cat > start_server.sh << 'EOF'
#!/bin/bash
tmux new -d -s webserver
tmux split-window -h -t webserver
tmux send-keys -t webserver:0.0 'cd ~/web-demo && python3 app.py 2>&1 | tee server.log' C-m
tmux send-keys -t webserver:0.1 'cd ~/web-demo && tail -f server.log' C-m
tmux attach -t webserver
EOF

chmod +x start_server.sh

echo "Setup complete! Run: ./start_server.sh"

Upload to your droplet and test:

scp setup_server.sh droplet:~/
ssh droplet
./setup_server.sh

Document in automation_notes.md.

Exercise 14 (Bonus): systemd Service (+15%)

Make your web server start automatically on boot using systemd.

Create /etc/systemd/system/webapp.service:

[Unit]
Description=Flask Web App
After=network.target

[Service]
User=yourname
WorkingDirectory=/home/yourname/web-demo
ExecStart=/usr/bin/python3 /home/yourname/web-demo/app.py
Restart=always

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable webapp
sudo systemctl start webapp
sudo systemctl status webapp

Test:

sudo reboot
# Wait, reconnect
curl localhost:5000

Document the process in systemd_notes.md.

Submission Checklist

Your week3-[netid]/ directory must contain:

Required (100%)

• answers.md - All 8 questions answered
• tmux_screenshot.png - Shows tmux with server running
• browser_screenshot.png - Shows webpage in browser
• deleted_screenshot.png - Shows droplet deleted

Bonus (up to +35% extra credit)

• ssh_config_explanation.md (+10%)
• setup_server.sh + automation_notes.md (+10%)
• systemd_notes.md (+15%)
• notes.md (what broke, how you fixed it)

Grading

Required exercises (100%):

• Exercise 1-3: Setup (20%)
• Exercise 4-5: SSH configuration (20%)
• Exercise 6-8: tmux and web server (30%)
• Exercise 9-10: Port forwarding and persistence (20%)
• Exercise 11: Cleanup (10%)

Grading is completion-based: If you attempted each exercise honestly and followed instructions, full credit for that exercise.

Common Issues and Solutions

“Permission denied (publickey)”

Your SSH key isn’t being used.

Solution:

ssh -i ~/.ssh/id_ed25519 root@YOUR_IP

Or check SSH config has correct IdentityFile path.

“Connection refused” on port 5000

Flask isn’t binding to all interfaces.

Check app.py has:

app.run(host='0.0.0.0', port=5000)

Not:

app.run()  # Wrong - only binds to localhost

“Cannot connect after disconnect”

Port forwarding is per-SSH-session.

Solution: Reconnect with -L flag:

ssh -L 5000:localhost:5000 droplet

Flask not found

Install:

python3 -m pip install flask --user

Droplet creation fails

You may need to verify your account (credit card or PayPal).

Alternative: Use free trial, apply student credits later.

Can’t delete droplet

Navigate to droplet page → “Destroy” at very top right.

If still stuck, contact DigitalOcean support.

Questions Template

Copy this into your answers.md:

# Week 3 Homework Answers

## Student Info
- Name: 
- NetID:
- Droplet IP: 

## Exercise Answers

### Question 1
What is an SSH key and why is it better than passwords?

**Answer:** 

### Question 2
What is a "droplet" and how does it differ from your laptop?

**Answer:**

### Question 3
- Hostname: 
- RAM: 
- Available disk space: 

### Question 4
What does the SSH config file do and why is it useful?

**Answer:**

### Question 5
What happened when you disconnected and why did the server keep running?

**Answer:**

### Question 6
How does port forwarding work and why can you access a server running on your droplet through `localhost` on your laptop?

**Answer:**

### Question 7
How does running the server in tmux differ from running it directly in SSH without tmux?

**Answer:**

### Question 8
- Credit used: $
- Remaining credit: $
- How long could you run a $4/month droplet: 

## Notes
(Optional - any problems you encountered and how you solved them)

Tips for Success

Take screenshots as you go - Don’t wait until the end

Read error messages carefully - They usually tell you what’s wrong

Test each step before moving on - Don’t skip ahead if something doesn’t work

Ask for help early - Don’t waste hours stuck on one issue

Save your SSH config - You’ll use these patterns all semester

Delete your droplet! - Seriously, don’t waste $200 of free credits

What You’ve Learned

By completing this homework, you’ve:

• Created and configured a cloud server from scratch
• Set up SSH keys and config for easy access
• Used tmux to create persistent sessions
• Run a web service that survives disconnections
• Used port forwarding to access remote services
• Understood the basics of “the cloud”
• Practiced good security (SSH keys, no root, non-standard user)
• Learned responsible resource management (cleanup)

These skills transfer to:

• AWS EC2
• Google Cloud
• Azure
• Any VPS provider
• Any Linux server

You now understand how servers actually work. Everything in “the cloud” is just this, at scale.

Submit by: Next Monday via git submodule

Questions? Office hours or class forum

Remember: Delete your droplet!

Week 0 Installation Guide

Welcome to Week 0! In this guide, you’ll learn how to log into CLEAR and submit your first assignment. We’ve included specific instructions tailored to different operating systems to ensure everyone can get started smoothly.

Getting Started with Remote Servers

Everyone has their preferred setup for working on remote machines. While you’re encouraged to use what works best for you, we recommend following these guidelines if you’re new to this environment. Be sure to review the basic Unix commands notes to familiarize yourself with the necessary commands. This tutorial is intended for MacOS and/or Linux Users, but instructions for Windows are included.

What is CLEAR?

“CLEAR is a robust and dynamic Linux cluster with exciting features available to Rice students and faculty. The cluster is designed to offer a Linux environment available for teaching and courseware needs.” Specifically, CLEAR is the cluster we will be using for this class, as we’ll all be using Fedora Linux as our environment.

How to Login into CLEAR

To access CLEAR, you need an SSH client (I’ll explain how to install one in the next section). Once you have an SSH client, you can log into CLEAR using the following details:

1. Host Name: Use ssh.clear.rice.edu.
2. Login Credentials: Enter your Rice NetID and password.
3. Direct Access: Your home directory on CLEAR is linked to your desktop home from storage.rice.edu.

Installing an SSH Client

Windows Users

If you are on Windows, it is recommended to install the Windows Subsystem for Linux with Ubuntu (WSL). Windows Powershell is now compatible with OpenSSH on Windows 10, however, for optimal server use for our labs, a *UNIX environment is fully preferred. The Windows Subsystem for Linux brings a full Linux experience to Windows 10 with WSL 2 bringing a full Linux kernel to windows. If your computer/laptop does not support virtualization, then a SSH program like putty may be ideal.

Windows Subsystem for Linux

To install WSL, run the following command in PowerShell as an administrator:

wsl --install

For a detailed guide, watch this installation video.

Linux Users

For Linux users, particularly those using Ubuntu, ensure you have the OpenSSH client installed:

sudo apt-get update && sudo apt-get upgrade
sudo apt install openssh-client

macOS Users

Mac users typically have an SSH client pre-installed. If not, or for a more enhanced experience, consider installing iTerm2. Additionally, enabling remote login and installing Xcode can be beneficial:

sudo systemsetup -setremotelogin on
sudo systemsetup -getremotelogin
xcode-select --install

Logging into CLEAR

To connect to CLEAR, use the following command:

ssh [your Rice NetID]@ssh.clear.rice.edu

Example for Windows: https://imgur.com/a/CyQ2vU4

Doing this successfully will log you into CLEAR. It is important to note that you may need to type your password twice (once for the ssh client and once DUO authentication).

Further Customization and Setup

View the extra notes for more information on how to customize your environment. Specifically, how to log into CLEAR faster and bypass DUO.

Introduction to Git

Git is a version control system that allows you to track changes to your files. Git is a free and open source software distributed under the GPL.

You will be learning about git in this class through the assignments. You may be interested in the guide on First Time Git Setup if you need a refresher on using git in general.

Git and GitHub - Submodule Setup

Think of a submodule as a GitHub repo inside another GitHub repo (i know, woah). Here’s how you can set it up effectively:

1. Create and clone your repo where all your work will be done:

$ git clone git@github.com:charlie/work-test.git
cloning into 'work-test'...
remote: counting objects: 3, done.
remote: total 3 (delta 0), reused 0 (delta 0), pack-reused 0
unpacking objects: 100% (3/3), done.
checking connectivity... done.

2. Clone the main class repo:

 $ git clone git@github.com:mks65/euler.git
cloning into 'euler'...
remote: counting objects: 14, done.
remote: compressing objects: 100% (9/9), done.
remote: total 14 (delta 5), reused 9 (delta 3), pack-reused 0
unpacking objects: 100% (14/14), done.
checking connectivity... done.

3. Navigate to the appropriate directory in the class repo and add your repo as a submodule:

cd class-repo
git submodule add -b "branch" "url to your repository" "required submodule directory name"

Here is an example:

$ cd euler/
socrates: euler charlie$ cd 04/
socrates: euler/04 charlie$ git submodule add -b main git@github.com:charlie/work-test.git cruz-charlie
cloning into '04/cruz-charlie'...
remote: counting objects: 3, done.
remote: total 3 (delta 0), reused 0 (delta 0), pack-reused 0
unpacking objects: 100% (3/3), done.
checking connectivity... done.

4. Commit and push your changes to the main class repo:

git commit -am "Added submodule for lastname-firstname"
git push

Here is an example:

$ git pull
already up-to-date.
socrates:~/desktop/git_demo/euler/04 charlie$ git commit -a -m "added charlie submodule"
    [master f25eeda] added charlie submodule
2 files changed, 4 insertions(+)
    create mode 100644 .gitmodules
    create mode 160000 randomizer/6/cruz-charlie
    socrates:~/desktop/git_demo/euler/04 charlie$ git push
    counting objects: 5, done.
    delta compression using up to 4 threads.
    compressing objects: 100% (5/5), done.
    writing objects: 100% (5/5), 626 bytes | 0 bytes/s, done.
total 5 (delta 0), reused 0 (delta 0)
    to https://github.com/mks65/euler.git
    11dc0c6..f25eeda  master -> master

5. Remove the main repo clone if no longer needed:

$ cd ../../
$ rm -rf euler/

Ensure to follow these steps to maintain a clean and organized repository structure for your coursework.

Week 1 Installation Guide

Prerequisites: You should have completed the Week 0 Installation Guide and have SSH access to CLEAR.

This week focuses on shell scripting, so you’ll need tools for writing, testing, and debugging bash scripts.

Required Tools

1. ShellCheck (Script Linter)

ShellCheck is essential for learning bash. It catches common mistakes and suggests improvements.

macOS

brew install shellcheck

Linux (Ubuntu/Debian)

sudo apt install shellcheck

Windows (WSL)

sudo apt install shellcheck

CLEAR

ShellCheck should already be available. Test with:

shellcheck --version

If not installed, you can download the binary:

# In your home directory on CLEAR
cd ~
wget https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.x86_64.tar.xz
tar -xf shellcheck-stable.linux.x86_64.tar.xz
mkdir -p ~/.local/bin
mv shellcheck-stable/shellcheck ~/.local/bin/

Then add to your ~/.bashrc:

export PATH="$HOME/.local/bin:$PATH"

Verify Installation

shellcheck --version

2. Text Editor

You need a way to write scripts. Choose one of the following:

Option A: nano (Easiest for beginners)

Already installed on most systems. No setup needed.

nano myscript.sh

Basic commands:

• Ctrl+O - Save
• Ctrl+X - Exit
• Ctrl+K - Cut line
• Ctrl+U - Paste

Option B: vim (More powerful, steeper learning curve)

Already installed. We’ll cover vim in Week 3, but you can start learning now.

vim myscript.sh

Survival guide:

• Press i to enter insert mode
• Type your script
• Press Esc to exit insert mode
• Type :wq and press Enter to save and quit
• Type :q! and press Enter to quit without saving

Option C: VS Code with Remote SSH (Recommended for local editing)

If you prefer working on CLEAR with a GUI editor:

1. Install VS Code
2. Install the “Remote - SSH” extension
3. Connect to CLEAR through VS Code

Benefits:

• Syntax highlighting
• Auto-completion
• Integrated terminal
• ShellCheck integration

Optional (But Useful) Tools

bat (Better cat with syntax highlighting)

Useful for viewing scripts with syntax highlighting.

macOS

brew install bat

Linux

sudo apt install bat

Note: On some systems, the command is batcat instead of bat due to naming conflicts.

Usage:

bat myscript.sh  # View script with highlighting

tldr (Simplified man pages)

Quick reference for commands instead of full man pages.

macOS

brew install tldr

Linux

sudo apt install tldr

Usage:

tldr tar    # Quick examples for tar
tldr find   # Quick examples for find

Verification

Run these commands to verify your setup:

# Check bash version (should be 4.0+)
bash --version

# Check shellcheck
shellcheck --version

# Check text editor (choose one)
nano --version
vim --version

# Test script creation
echo '#!/usr/bin/env bash' > test.sh
echo 'echo "Hello, Week 1!"' >> test.sh
chmod +x test.sh
./test.sh

# Test shellcheck
shellcheck test.sh

# Clean up
rm test.sh

Expected output:

$ ./test.sh
Hello, Week 1!

$ shellcheck test.sh
# No output means no errors!

Setting Up Your Environment

Create a scripts directory

Good practice: keep scripts organized.

mkdir -p ~/scripts
cd ~/scripts

Add scripts to PATH (Optional)

If you want to run your scripts from anywhere:

# Add to ~/.bashrc (or ~/.zshrc)
export PATH="$HOME/scripts:$PATH"

Then reload:

source ~/.bashrc

Now scripts in ~/scripts can be run from anywhere (if executable).

ShellCheck Integration

Using ShellCheck

Always check your scripts before running:

shellcheck myscript.sh

Example output:

In myscript.sh line 3:
rm $file
   ^---^ SC2086: Double quote to prevent globbing and word splitting.

Fix the issue:

rm "$file"  # Add quotes

Ignore specific warnings (when you know what you’re doing):

# shellcheck disable=SC2086
rm $file  # Intentionally unquoted

Common Issues

“Permission denied” when running script

Problem:

$ ./myscript.sh
-bash: ./myscript.sh: Permission denied

Solution:

chmod +x myscript.sh

“No such file or directory” with shebang

Problem:

#!/bin/bash

Works on some systems but not others.

Solution: Use the portable version:

#!/usr/bin/env bash

Scripts work locally but not on CLEAR

Problem: Different bash versions or missing tools.

Solution: Always test on CLEAR before submitting:

ssh clear
cd ~/path/to/scripts
./myscript.sh

Editor Configuration

nano Configuration

Create ~/.nanorc:

set autoindent
set tabsize 4
set tabstospaces
include /usr/share/nano/*.nanorc

vim Configuration (Basic)

Create ~/.vimrc:

" Basic settings for shell scripting
syntax on
set number
set autoindent
set tabstop=4
set shiftwidth=4
set expandtab

Testing Your Setup

Create and run this test script:

#!/usr/bin/env bash
# test_setup.sh - Verify Week 1 setup

set -euo pipefail

echo "Testing Week 1 setup..."

# Test 1: Bash version
echo -n "Bash version: "
bash --version | head -1

# Test 2: ShellCheck
echo -n "ShellCheck: "
if command -v shellcheck &> /dev/null; then
    echo "✓ Installed"
else
    echo "✗ Not found - Install shellcheck!"
    exit 1
fi

# Test 3: Text editor
echo -n "Text editor: "
if command -v nano &> /dev/null; then
    echo "✓ nano available"
elif command -v vim &> /dev/null; then
    echo "✓ vim available"
else
    echo "✗ No editor found"
    exit 1
fi

# Test 4: Script permissions
echo -n "File permissions: "
if [[ -x "$0" ]]; then
    echo "✓ Script is executable"
else
    echo "✗ Script not executable (run: chmod +x $0)"
fi

echo ""
echo "Setup complete! You're ready for Week 1."

Save as test_setup.sh, make executable, and run:

chmod +x test_setup.sh
./test_setup.sh

Quick Reference Card

Save this for quick access:

# Make script executable
chmod +x script.sh

# Run script
./script.sh

# Check script for errors
shellcheck script.sh

# Debug script (show commands as they execute)
bash -x script.sh

# Edit script
nano script.sh    # or vim script.sh

# View script with syntax highlighting
bat script.sh     # if installed

What’s NOT Required

You don’t need to install:

• Complex IDEs
• Git GUI tools (command line is fine)
• Docker (not until later weeks)
• Any programming languages besides bash

Keep it simple. You already have everything you need to write powerful scripts.

Next Steps

1. Verify your installation with the test script above
2. Complete the Week 1 homework
3. Experiment with ShellCheck on your scripts
4. Consider learning vim basics (we’ll cover it in Week 3)

Questions? Bring them to office hours or post in the class forum.

Resources

• ShellCheck Wiki
• Bash Guide
• Nano Guide
• Vim Tutorial (interactive)

Remember: The goal is to write working scripts, not to master every tool perfectly. Start simple and build up!

Week 2 Installation Guide

Prerequisites: You should have completed Week 0 and Week 1 installations. This week focuses on terminal text editors.

Note to self: Make sure to mention how to upload files to remote servers using scp or sftp if needed.

Check What You Have

Most Unix systems come with Vim pre-installed. Check first:

vim --version

If you see version 8.0 or higher, you’re good to go. If not installed or very old, follow the instructions below.

Vim Installation

macOS

macOS includes Vim, but it’s often outdated. Install the latest via Homebrew:

brew install vim

Linux (Ubuntu/Debian)

sudo apt update
sudo apt install vim

Linux (Fedora/RHEL)

sudo dnf install vim-enhanced

Windows (WSL)

sudo apt install vim

Setting Up Your .vimrc

Create a basic configuration file:

vim ~/.vimrc

Paste this minimal config (from the notes):

" Basic settings
set nocompatible
syntax on
filetype plugin indent on

set number relativenumber
set mouse=a
set hidden
set ignorecase smartcase
set incsearch hlsearch

" Indentation
set expandtab
set tabstop=4
set shiftwidth=4
set autoindent

" UI
set showcmd
set wildmenu
set laststatus=2

" Leader key
let mapleader = " "

" Key mappings
inoremap jk <Esc>
nnoremap <leader>w :w<CR>
nnoremap <leader>q :q<CR>
nnoremap <leader><space> :noh<CR>

" Window navigation
nnoremap <C-h> <C-w>h
nnoremap <C-j> <C-w>j
nnoremap <C-k> <C-w>k
nnoremap <C-l> <C-w>l

Save and quit: :wq

Test it:

vim test.txt

You should see line numbers and syntax highlighting.

Neovim (Modern Alternative)

Neovim is a modern fork of Vim with better defaults, Lua configuration, and built-in LSP support. It’s compatible with most Vim plugins and commands.

Why Neovim?

• Better performance
• More active development
• Lua configuration (more powerful than Vimscript)
• Built-in LSP (Language Server Protocol)
• Better plugin ecosystem

Installation

macOS

brew install neovim

Linux (Ubuntu/Debian)

sudo apt install neovim

Linux (Fedora/RHEL)

sudo dnf install neovim

Windows (WSL)

sudo apt install neovim

If not available, you can install to your home directory:

cd ~
wget https://github.com/neovim/neovim/releases/latest/download/nvim-linux64.tar.gz
tar xzf nvim-linux64.tar.gz
mkdir -p ~/.local/bin
ln -s ~/nvim-linux64/bin/nvim ~/.local/bin/nvim

Then add to your ~/.bashrc:

export PATH="$HOME/.local/bin:$PATH"

Verify Installation

nvim --version

Should show version 0.9.0 or higher.

Configuration

Neovim uses ~/.config/nvim/init.vim (or init.lua for Lua config).

For Vim compatibility, create a simple init.vim:

mkdir -p ~/.config/nvim
vim ~/.config/nvim/init.vim

Add this to use your existing Vimrc:

set runtimepath^=~/.vim runtimepath+=~/.vim/after
let &packpath = &runtimepath
source ~/.vimrc

Or start fresh with Lua config:

vim ~/.config/nvim/init.lua

-- Basic settings
vim.opt.number = true
vim.opt.relativenumber = true
vim.opt.mouse = 'a'
vim.opt.ignorecase = true
vim.opt.smartcase = true
vim.opt.hlsearch = true
vim.opt.wrap = false
vim.opt.expandtab = true
vim.opt.tabstop = 4
vim.opt.shiftwidth = 4

-- Leader key
vim.g.mapleader = ' '

-- Key mappings
vim.keymap.set('i', 'jk', '<Esc>')
vim.keymap.set('n', '<leader>w', ':w<CR>')
vim.keymap.set('n', '<leader>q', ':q<CR>')
vim.keymap.set('n', '<leader><space>', ':noh<CR>')

Using Neovim

Commands are identical to Vim:

nvim filename.txt    # Instead of vim

Or alias it:

# Add to ~/.bashrc or ~/.zshrc
alias vim='nvim'
alias vi='nvim'

Helix (Different Paradigm)

Helix is a modern modal editor inspired by Kakoune. It has different keybindings than Vim but similar philosophy.

Why Helix?

• Selection-first editing (select, then act)
• Multiple cursors built-in
• No configuration needed (works great out of the box)
• Built-in LSP, tree-sitter syntax highlighting
• Written in Rust (fast and safe)

Differences from Vim

• Different keybindings (not Vim-compatible)
• Selection before action (vs. Vim’s action then motion)
• No modes named Normal/Insert (called Normal/Select)
• Different learning curve

Installation

macOS

brew install helix

Linux (Ubuntu/Debian)

# Add PPA
sudo add-apt-repository ppa:maveonair/helix-editor
sudo apt update
sudo apt install helix

Or build from source:

git clone https://github.com/helix-editor/helix
cd helix
cargo install --path helix-term

Linux (Fedora)

sudo dnf install helix

Other

Build from source to your home directory:

cd ~
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

git clone https://github.com/helix-editor/helix
cd helix
cargo install --path helix-term --locked

Then add to ~/.bashrc:

export PATH="$HOME/.cargo/bin:$PATH"

Verify Installation

hx --version

Basic Usage

hx filename.txt

Key differences from Vim:

• i enters insert mode (same)
• Esc returns to normal mode (same)
• But motion then action: w selects word, then d deletes selection
• :q to quit (same)
• :w to save (same)

Configuration

Helix works great without configuration, but you can customize:

mkdir -p ~/.config/helix
hx ~/.config/helix/config.toml

Basic config:

theme = "onedark"

[editor]
line-number = "relative"
mouse = true
cursorline = true
auto-save = false

[editor.cursor-shape]
insert = "bar"
normal = "block"
select = "underline"

[keys.normal]
# Add custom keybindings if desired

Should You Use Helix?

For this class, we focus on Vim because:

• More widely available
• More resources and community
• Transferable to other tools (Vim mode everywhere)

But Helix is worth trying if you:

• Want a modern editor without configuration
• Like the selection-first paradigm
• Want built-in LSP without setup

You can learn both, but start with Vim for this course.

Running vimtutor

The best way to learn Vim is the built-in tutorial.

For Vim:

vimtutor

For Neovim:

nvim +Tutor

For Helix:

hx --tutor

Complete at least lessons 1-4. Takes about 30 minutes.

Verification Checklist

Test your setup:

# Create test file
echo "Hello, Week 2!" > test.txt

# Edit with Vim
vim test.txt
# Press i, type something, press Esc, type :wq

# Edit with Neovim (if installed)
nvim test.txt

# Edit with Helix (if installed)
hx test.txt

# Clean up
rm test.txt

Check your config works:

# Vim
vim ~/.vimrc
# Should see line numbers and syntax highlighting

# Neovim
nvim ~/.config/nvim/init.vim
# or
nvim ~/.config/nvim/init.lua

Common Issues

“vim: command not found”

Install Vim following the instructions above for your system. It might also be named vi on some systems.

“E325: ATTENTION” message

Vim found a swap file from a previous session. Choose:

• O - Open read-only
• R - Recover and continue editing
• D - Delete swap file (if you’re sure)
• Q - Quit

Colors look wrong

Enable 256 colors in Vim:

set t_Co=256

Or use a better terminal (iTerm2, Alacritty, WezTerm).

Can’t use system clipboard

Check if Vim was compiled with clipboard support:

vim --version | grep clipboard

Should show +clipboard. If it shows -clipboard, reinstall Vim:

# macOS
brew install vim

# Linux
sudo apt install vim-gtk3

Week 3 Installation Guide

Prerequisites: You should have completed Week 0-2 installations. This week requires minimal new installation since most tools are already available or will be installed on your VPS.

Required Tools

1. SSH Client - You should already have this from Week 0.

2. tmux (Local Installation - Optional)

tmux on your laptop is optional but useful for local work.

macOS:

brew install tmux

Linux (Ubuntu/Debian):

sudo apt install tmux

Linux (Fedora/RHEL):

sudo dnf install tmux

Windows (WSL):

sudo apt install tmux

Should show tmux 3.0 or higher.

Note: You’ll also install tmux on your DigitalOcean droplet during homework.

3. rsync

Check if installed:

rsync --version

If missing:

macOS: Pre-installed Linux: sudo apt install rsync Windows (WSL): sudo apt install rsync

4. DigitalOcean Student Credits

Not an installation, but required for homework.

Steps:

1. Go to https://education.github.com/pack
2. Sign up with Rice email
3. Get approved (usually instant)
4. Activate DigitalOcean offer ($200 credit)

You don’t need this before class, but start the GitHub Student Pack application early (can take 1-2 days for approval).

More Tools & Other Recommendations

Customization

Set up SSH Keys

SSH keys provide a convenient and secure way to log into remote machines without typing your password. Here’s how to set up your SSH keys, which act as a form of verification for your machine:

$ ssh-keygen

$ ssh-keygen -p  # Set or change your passphrase

$ /Users/username/.ssh/id_rsa already exists

To manage your private keys (and not have to enter your passphrase every time you SSH), first, start ssh-agent in the background:

$ eval "$(ssh-agent -s)"

Then, add your private key (you will be prompted for your passphrase):

$ ssh-add

To log in to our machines (e.g., risotto) without entering your password every time, you will need to copy your public key to the remote machine. You can do this with the following command, where username is replaced by your Rice net id:

$ ssh-copy-id username@ssh.clear.rice.edu

If ssh-copy-id is unavailable, manually copy the key:

$ cat ~/.ssh/id_rsa.pub | ssh username@ssh.clear.rice.edu "mkdir -p ~/.ssh; cat >> ~/.ssh/authorized_keys"

You know can type in your password once per startup session.

Set up SSH Config File - Bypassing DUO Authentication

Writing an SSH config file can streamline SSH connections and avoid having to repeatedly add flags when connecting to machines. If you do not have a config file you first need to create one in ~/.ssh/config.

To enable shorter ssh names, e.g. accessing clear by typing ssh clear vs ssh username@ssh.clear.rice.edu you need to add additional lines per host to your ssh config (change everything inside the brackets)

Host clear
    User [your_netid]
    HostName ssh.clear.rice.edu
    PasswordAuthentication no
    PreferredAuthentications publickey

With this setup, you can connect to CLEAR by typing:

$ ssh clear

Choose Your Shell

Before you log into our machines, make sure you set your default shell preferences. The default choice is csh, but you can choose between: bash, sh, tcsh, csh, zsh.

We recommend zsh, but you are welcome to choose whatever you feel most comfortable with. To change the default shell for any remote machine, you log into account config (you need to be on the RICE VPN to access this page). Log in with your NetID, then under “Account Maintenance” -> “Shell Management,” you can choose your desired shell environment.

Bash Profiles

Editing shell profiles can save you time and effort by configuring aliases or short cuts for various commands. In addition, it allows for changing the colors of different file and directory types, and adding other convenient functionality.

Since we primarily use bash, we have included some configurations we have found useful that we usually set a priori.

These should be set in your ~/.bashrc file (you may have to create it if it is missing).

Aliases allow you to set shortcuts for frequently used commands. These are some good ones to configure.

alias rm='rm -i'  # Ask before deleting
alias mv='mv -i'
alias ls='ls -G'
alias ll='ls -lthG'  # Detailed listing
alias l.='ls -G -d .*'  # Show hidden files
alias mkdir='mkdir -pv'  # Make parent directories as needed
alias wget='wget -c'  # Continue stopped downloads

Other helpful functions:

# Prevent accidental overwriting
set -o noclobber

# Enhance history features
export HISTCONTROL=ignoredups
export HISTFILESIZE=
export HISTSIZE=
export HISTTIMEFORMAT="[%F %T] "
export HISTFILE=~/.bash_history_unlimited
shopt -s histappend
shopt -s autocd
shopt -s checkwinsize
shopt -s globstar  # Match files with **
PROMPT_COMMAND="history -a; $PROMPT_COMMAND"

# Color settings for man pages
export LESS_TERMCAP_mb=$'\e[1;32m'
export LESS_TERMCAP_md=$'\e[1;32m'
export LESS_TERMCAP_me=$'\e[0m'
export LESS_TERMCAP_se=$'\e[0m'
export LESS_TERMCAP_so=$'\e[01;33m'
export LESS_TERMCAP_ue=$'\e[0m'
export LESS_TERMCAP_us=$'\e[1;4;31m'

# Enable terminal colors
export TERM=xterm-256color

To view your command history with timestamps, use history. Access the history file directly at $HISTFILE for a detailed log.

Certainly! Below is a section on the Fish shell, detailing its features and benefits for users considering an alternative to Bash.

Choose Your Shell: Why Fish Might Be Your Next Best Friend

When setting up your development environment, selecting the right shell can significantly enhance your workflow efficiency. While Bash is the default shell on many systems due to its long-standing presence, Fish (the Friendly Interactive SHell) offers several compelling features that might make it your shell of choice.

What is Fish?

Fish is a smart and user-friendly command line shell for macOS, Linux, and the rest of the family. Unlike traditional shells, which can be daunting for newcomers, Fish is designed with usability in mind, offering powerful features to both new and seasoned users.

Features That Make Fish Stand Out

• Autosuggestions: Fish suggests commands as you type based on history and completions, just like a web browser. Watch as Fish suggests commands in real time, allowing you to repeat previous commands faster or use similar commands without remembering every detail.

• Enhanced Tab Completions: Fish offers advanced completion capabilities. It completes commands, options, and even file paths, reducing the amount of typing and minimizing the guesswork.

• Rich, Out-of-the-box Configuration: Unlike Bash, Fish works impressively right out of the box. It includes a web-based configuration interface for adjusting prompt colors and functions without the need to edit text files.

• Extensible and Easy Scripting: Fish’s scripting syntax is simple and clean. For example, there are no $ signs needed for variable expansion. Conditions and loops are easier to write and read, making script maintenance less cumbersome.

• Web-based Configuration: Fish includes a web interface that allows you to configure it from your browser—something unique that no other shell offers. This makes tweaking Fish’s behavior and appearance easier for users who prefer graphical interfaces over text files.

Switching to Fish

Fish can be installed on most systems with a simple package manager command. For example, on macOS, you can use Homebrew:

$ brew install fish

And on Ubuntu systems:

$ sudo apt install fish

To make Fish your default shell, you can use the chsh command:

$ chsh -s /usr/local/bin/fish  # The path to Fish might differ based on your installation

Integrating Fish into Your Workflow

Transitioning to Fish can be straightforward due to its intelligent design. It reads and executes commands from ~/.config/fish/config.fish, similar to Bash’s .bashrc. You can start by adding custom functions or aliases into this file.

Fish is particularly useful for developers who value a rich, straightforward, and interactive command line experience. Its syntax and features promote efficiency and ease of use, potentially making your command line tasks faster and more enjoyable.

For those interested in exploring the power of Fish or wanting a break from the more traditional Bash shell, Fish offers a refreshing and powerful alternative.

Anyway, choose Fish cuz it’s cool.

Extra

You may be interested in my dotfiles.

Code Review Guide

Code review is not just a duty to help maintain code quality; it’s also a powerful learning tool. Regularly reviewing code helps you discover new techniques and reinforces best practices. Therefore, it’s crucial to approach code reviews diligently and provide constructive, honest feedback.

Core Principles of Effective Code Reviews

Good code should be clear, clean, and efficient. Here are some questions to guide your review:

1. Clarity: Is the code easy to understand at first glance? Can you discern the purpose of each line without extensive investigation?
2. Standards Compliance: Does the code adhere to established coding standards and guidelines? (Refer to our Code Style Guide).
3. Redundancy: Is there any code duplicated more than twice?
4. Function Length: Are there overly long functions or methods that could be refactored to reduce complexity?
5. Efficiency: Do you notice any inefficiencies? Could any parts of the code be optimized for better performance?

Detailed Code Review Checklist

Use this checklist to ensure comprehensive reviews. These items encompass best practices derived from various resources:

Best Practices

• Avoid hard-coding values—use constants and variables.
• Comments should explain why the code exists in its current form, detailing any temporary or complex logic.
• Handle potential errors using try/catch blocks or other exception handling methods.
• Prefer fewer conditional (if/else) blocks.
• Minimize nested loops unless necessary.
• Use early exits (like break in loops or return in functions) to reduce nesting.
• Utilize existing frameworks and libraries that offer the needed functionality instead of custom solutions.
• Credit any external code sources integrated into the application.

Clarity

• Ensure the code is self-explanatory.
• Use descriptive names for variables, functions, and classes.
• Maintain a single responsibility principle for classes and functions.
• Provide user feedback for operations that aren’t instantaneous.

Cleanliness

• Maintain proper use of whitespace and ensure code blocks are easily distinguishable.
• Follow the designated style guide.
• Organize imports and constant declarations at the beginning of files.
• Remove any code that is commented out to keep the codebase clean.

Efficiency and Reusability

• Avoid repeating the same code across the project.
• For variable data or settings, use hooks like command-line arguments to facilitate adjustments without modifying the code.
• Choose the most efficient data structures and algorithms.
• Ensure the code runs within a reasonable time frame given the context.
• For code involving randomness, provide mechanisms to set and document the seed for reproducibility.

Code Style Guidelines

• Python: Follow the PEP 8 style guide.
• R: Adhere to Google’s R Style Guide.
• Utilize linters during development to ensure consistent styling. Adjust maximum line lengths based on team preferences, documented in a project-specific .editorconfig file, to maintain consistency across various editors.

Additional Resources

For more insights into the value of code reviews and how to enhance their effectiveness, refer to these articles:

• Why Code Reviews Matter (And Actually Save Time)
• The Importance of Code Reviews
• Code Review Checklist - To Perform Effective Code Reviews
• The Code Review Pyramid

Engaging in thorough and thoughtful code reviews strengthens the codebase and builds a collaborative culture that values quality and continuous improvement.

Contributors

Here is a list of the contributors who have helped improving The Art of Lazy Programming. Big.

• Charlie Cruz
• Shreyas Minocha

If you feel you’re missing from this list, feel free to add yourself in a PR.

License

All the content in this course, including the website source code, lecture notes, exercises, and lecture videos is licensed under Attribution-NonCommercial-ShareAlike 4.0 International CC BY-NC-SA 4.0.

This means that you are free to:

• Share — copy and redistribute the material in any medium or format
• Adapt — remix, transform, and build upon the material

Under the following terms:

• Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
• NonCommercial — You may not use the material for commercial purposes.
• ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.

This is a human-readable summary of (and not a substitute for) the license.

Contribution guidelines

You can submit corrections and suggestions to the course material by submitting issues and pull requests on our GitHub repo. Remember to add yourself to contributions.

Translation guidelines

You are free to translate the lecture notes and exercises as long as you follow the license terms. If your translation mirrors the course structure, please contact us so we can link your translated version from our page.