Complete Guide to Bash scp Command (Secure Copy)

Introduction to `scp`

The scp (Secure Copy) command is a network file transfer tool that uses SSH (Secure Shell) for secure data transfer between local and remote hosts. It provides encrypted authentication and data transfer, making it a reliable choice for moving files across networks securely.

Key Concepts

SSH-based: Uses the same authentication and security as SSH
Encryption: All data transferred is encrypted
Authentication: Supports password, key-based, and other SSH authentication methods
Recursive Copy: Can copy entire directory structures
Preserves Attributes: Can maintain file permissions and timestamps

1. Basic Syntax and Usage

Basic Syntax

# Basic syntax scp [options] source_file target_file # Local to remote scp file.txt user@remote_host:/path/to/destination/ # Remote to local scp user@remote_host:/path/to/file.txt /local/destination/ # Between two remote hosts scp user1@host1:/path/file.txt user2@host2:/path/destination/ # Examples # Copy local file to remote home directory scp document.pdf [email protected]:~/Documents/ # Copy remote file to current local directory scp [email protected]:/var/log/syslog . # Copy with different username scp [email protected]:~/data.txt .

Common Use Cases

# Copy single file scp backup.tar.gz admin@backup-server:/backups/ # Copy with different remote filename scp local.txt user@host:remote.txt # Copy multiple files scp file1.txt file2.txt file3.txt user@host:/destination/ # Copy with wildcards scp *.log user@host:/logs/ # Copy entire directory recursively scp -r documents/ user@host:/backup/ # Preserve file attributes (permissions, timestamps) scp -p important.conf user@host:/etc/

2. Command Options

-P (Port) Option

# Specify SSH port (note: uppercase P) scp -P 2222 file.txt user@host:/destination/ # Common alternative SSH port scp -P 2222 backup.tar.gz [email protected]:/backups/ # Multiple files with custom port scp -P 2222 *.log user@host:/var/log/ # Script with variable port PORT=2222 scp -P $PORT file.txt user@host:~ # Check port connectivity first check_port() { local host="$1" local port="${2:-22}" nc -zv "$host" "$port" 2>&1 | grep -q succeeded } if check_port server.com 2222; then scp -P 2222 file.txt [email protected]: else echo "Port 2222 not accessible" fi

-r (Recursive) Option

# Copy entire directory recursively scp -r projects/ user@host:/backup/ # Copy directory contents (not the directory itself) scp -r projects/* user@host:/backup/projects/ # Copy multiple directories scp -r dir1 dir2 dir3 user@host:/destination/ # Preserve symlinks when copying recursively scp -r -l projects/ user@host:/backup/ # Exclude certain files while copying copy_with_exclude() { local source="$1" local dest="$2" local exclude="${3:-.git}" tar --exclude="$exclude" -cf - "$source" | \ ssh user@host "tar -xf - -C $dest" } # Progress for recursive copy scp -rv big_directory/ user@host:/destination/

-p (Preserve) Option

# Preserve modification times, access times, and modes scp -p important.conf user@host:/etc/ # Preserve when copying backups scp -rp /var/backups/ user@backup-server:/backups/ # Verify preservation copy_with_attributes() { local src="$1" local dest="$2" # Get original attributes orig_perms=$(stat -c %a "$src") orig_mtime=$(stat -c %y "$src") # Copy with preservation scp -p "$src" "$dest" # Verify new_perms=$(stat -c %a "$dest") if [ "$orig_perms" = "$new_perms" ]; then echo "✓ Permissions preserved" else echo "✗ Permissions changed" fi }

-C (Compression) Option

# Enable compression (useful for slow connections) scp -C large_file.tar.gz user@host:/backup/ # Compress directory transfer scp -Cr documents/ user@host:/backup/ # Measure compression benefit test_compression() { local file="$1" local host="$2" echo "Without compression:" time scp "$file" "$host":~/test/ echo "With compression:" time scp -C "$file" "$host":~/test/ } # Adaptive compression based on file type smart_copy() { local file="$1" local dest="$2" # Don't compress already compressed files if [[ "$file" =~ \.(gz|bz2|zip|jpg|png)$ ]]; then scp "$file" "$dest" else scp -C "$file" "$dest" fi }

-l (Limit) Option

# Limit bandwidth (in Kbit/s) scp -l 1000 bigfile.iso user@host:/downloads/ # Limit to 1000 Kbit/s # Throttle transfer scp -l 500 -r videos/ user@host:/media/ # Dynamic rate limiting rate_limited_copy() { local file="$1" local dest="$2" local max_rate="${3:-1000}" # Default 1Mbit/s # Check current network usage current_load=$(ifstat 1 1 | tail -1 | awk '{print $1}') if [ "$(echo "$current_load > 50" | bc)" -eq 1 ]; then rate=$((max_rate / 2)) else rate=$max_rate fi scp -l "$rate" "$file" "$dest" }

-i (Identity) Option

# Use specific SSH private key scp -i ~/.ssh/id_rsa_custom file.txt user@host:/destination/ # Multiple key files scp -i ~/.ssh/key1 -i ~/.ssh/key2 file.txt user@host: # Key selection based on host copy_with_key() { local host="$1" local file="$2" local dest="$3" case "$host" in *.github.com) key="~/.ssh/github_rsa" ;; *.company.com) key="~/.ssh/company_rsa" ;; *) key="~/.ssh/id_rsa" ;; esac scp -i "$key" "$file" "user@$host:$dest" }

-v (Verbose) Option

# Verbose output for debugging scp -v file.txt user@host:/destination/ # Debug connection issues debug_scp() { local src="$1" local dest="$2" scp -vvv "$src" "$dest" 2>&1 | tee scp_debug.log echo "Debug log saved to scp_debug.log" } # Trace authentication process trace_auth() { scp -v file.txt user@host: 2>&1 | grep -E "Authentications|Authentication" } # Monitor connection phases monitor_phases() { scp -v file.txt user@host: 2>&1 | \ grep -E "Connecting|Connection established|Authenticated|Starting transfer" | \ while read line; do echo "$(date +%H:%M:%S) - $line" done }

3. Practical Examples

Backup Scripts

#!/bin/bash # Simple backup script backup_files() { local source_dir="$1" local remote_host="$2" local remote_dir="$3" local timestamp=$(date +%Y%m%d_%H%M%S) local backup_name="backup_${timestamp}.tar.gz" echo "Creating backup archive..." tar -czf "/tmp/$backup_name" -C "$source_dir" . echo "Copying to remote host..." scp "/tmp/$backup_name" "${remote_host}:${remote_dir}/" if [ $? -eq 0 ]; then echo "Backup successful: $backup_name" rm "/tmp/$backup_name" else echo "Backup failed" return 1 fi } # Incremental backup incremental_backup() { local source="$1" local remote="$2" local last_backup="/tmp/last_backup_time" # Get last backup time if [ -f "$last_backup" ]; then last_time=$(cat "$last_backup") else last_time="1970-01-01" fi # Find files modified since last backup find "$source" -type f -newermt "$last_time" -print0 | while IFS= read -r -d '' file; do rel_path="${file#$source/}" echo "Backing up: $rel_path" scp "$file" "${remote}:backup/$rel_path" done date +%Y-%m-%d > "$last_backup" } # Encrypted backup encrypted_backup() { local source="$1" local remote="$2" local passphrase="$3" tar -czf - "$source" | \ openssl enc -aes-256-cbc -salt -k "$passphrase" | \ ssh user@"$remote" "cat > backup_$(date +%Y%m%d).enc" echo "Encrypted backup complete" }

File Synchronization

#!/bin/bash # Sync directory with remote sync_directory() { local local_dir="$1" local remote_dir="$2" # First, copy new/modified files find "$local_dir" -type f -mtime -1 -exec scp {} "$remote_dir"/ \; # Then, handle deletions (requires list of remote files) ssh "${remote_dir%%:*}" "ls ${remote_dir#*:}" > /tmp/remote_files.txt while read file; do if [ ! -f "$local_dir/$file" ]; then echo "File $file exists only on remote" fi done < /tmp/remote_files.txt } # Bi-directional sync bidirectional_sync() { local dir1="$1" local dir2="$2" # Get latest files from each side echo "Syncing $dir1 -> $dir2" scp -r "$dir1"/* "$dir2"/ 2>/dev/null echo "Syncing $dir2 -> $dir1" scp -r "$dir2"/* "$dir1"/ 2>/dev/null } # Sync with checksum verification sync_with_checksum() { local src="$1" local dest="$2" # Calculate local checksums find "$src" -type f -exec md5sum {} \; > /tmp/local.md5 # Copy files scp -r "$src"/* "$dest"/ # Calculate remote checksums ssh "${dest%%:*}" "cd ${dest#*:} && md5sum *" > /tmp/remote.md5 # Compare diff /tmp/local.md5 /tmp/remote.md5 }

Deployment Scripts

#!/bin/bash # Deploy web application deploy_app() { local app_dir="$1" local server="$2" local deploy_dir="${3:-/var/www/html}" echo "Packaging application..." tar -czf /tmp/deploy.tar.gz -C "$app_dir" --exclude=.git --exclude=node_modules . echo "Deploying to $server..." scp /tmp/deploy.tar.gz "deploy@$server:/tmp/" ssh "deploy@$server" " cd $deploy_dir && sudo rm -rf * && sudo tar -xzf /tmp/deploy.tar.gz && sudo chown -R www-data:www-data . && sudo systemctl reload nginx " rm /tmp/deploy.tar.gz echo "Deployment complete" } # Rollback function rollback_deploy() { local server="$1" local backup_dir="${2:-/backups}" # List available backups ssh "deploy@$server" "ls -lt $backup_dir | head -20" echo "Enter backup date (YYYYMMDD):" read backup_date ssh "deploy@$server" " cd /var/www/html && sudo rm -rf * && sudo tar -xzf $backup_dir/backup_$backup_date.tar.gz && sudo systemctl reload nginx " }

4. Scripting with scp

Connection Testing

#!/bin/bash # Test SCP connectivity test_scp_connection() { local host="$1" local user="${2:-$USER}" local port="${3:-22}" local test_file="/tmp/scp_test_$$" # Create test file echo "test" > "$test_file" # Try upload if scp -P "$port" -q "$test_file" "$user@$host":/tmp/ 2>/dev/null; then echo "✓ Upload successful" # Try download if scp -P "$port" -q "$user@$host:/tmp/$(basename "$test_file")" /dev/null 2>/dev/null; then echo "✓ Download successful" ssh "$user@$host" "rm /tmp/$(basename "$test_file")" rm "$test_file" return 0 fi fi echo "✗ SCP connection failed" rm "$test_file" return 1 } # Check SCP prerequisites check_scp_prereqs() { local host="$1" # Check SSH connectivity if ! ssh -o ConnectTimeout=5 -q "$host" exit 2>/dev/null; then echo "SSH connection failed" return 1 fi # Check if SCP is available if ! ssh "$host" "which scp" >/dev/null 2>&1; then echo "SCP not available on remote host" return 1 fi # Check disk space space=$(ssh "$host" "df -h /tmp | tail -1 | awk '{print \$4}'") echo "Remote /tmp has $space free" return 0 }

Batch Processing

#!/bin/bash # Copy multiple files with progress batch_copy() { local source_dir="$1" local dest="$2" local pattern="${3:-*}" local log="/tmp/scp_batch_$(date +%s).log" total=$(ls "$source_dir"/$pattern 2>/dev/null | wc -l) count=0 for file in "$source_dir"/$pattern; do if [ -f "$file" ]; then count=$((count + 1)) echo "[$count/$total] Copying $(basename "$file")..." if scp -q "$file" "$dest/" >> "$log" 2>&1; then echo " ✓ Success" else echo " ✗ Failed" >> "$log" fi fi done echo "Batch copy complete. Log saved to $log" } # Parallel SCP transfers parallel_scp() { local files=("$@") local dest="${files[-1]}" unset 'files[-1]' for file in "${files[@]}"; do ( echo "Starting transfer of $(basename "$file")" scp "$file" "$dest/" echo "Completed $(basename "$file")" ) & done wait echo "All transfers complete" } # Retry logic scp_with_retry() { local src="$1" local dest="$2" local max_attempts="${3:-3}" local attempt=1 while [ $attempt -le $max_attempts ]; do echo "Attempt $attempt of $max_attempts" if scp "$src" "$dest"; then echo "Transfer successful" return 0 fi attempt=$((attempt + 1)) [ $attempt -le $max_attempts ] && sleep 5 done echo "Transfer failed after $max_attempts attempts" return 1 }

Monitoring and Logging

#!/bin/bash # Monitor SCP transfers monitor_scp() { local logfile="/var/log/scp_monitor.log" # Monitor SCP processes while true; do scp_pids=$(pgrep -f "scp .*") if [ -n "$scp_pids" ]; then for pid in $scp_pids; do # Get process info cmd=$(ps -p "$pid" -o cmd=) start=$(ps -p "$pid" -o lstart=) # Get network stats (requires lsof) if command -v lsof >/dev/null; then conn=$(lsof -p "$pid" 2>/dev/null | grep TCP | head -1) fi echo "$(date): Active SCP: $cmd" >> "$logfile" done fi sleep 10 done } # Log all SCP transfers log_scp() { local src="$1" local dest="$2" local log="/var/log/scp_transfers.log" { echo "=== SCP Transfer ===" echo "Timestamp: $(date)" echo "User: $(whoami)" echo "Source: $src" echo "Destination: $dest" echo "Size: $(du -h "$src" 2>/dev/null | cut -f1)" echo "===================" } >> "$log" scp "$src" "$dest" if [ $? -eq 0 ]; then echo "Status: SUCCESS" >> "$log" else echo "Status: FAILED" >> "$log" fi }

5. Advanced Techniques

Compression and Archiving

#!/bin/bash # Compress on the fly compress_and_send() { local dir="$1" local dest="$2" tar -czf - "$dir" | ssh user@"$dest" "cat > archive_$(date +%Y%m%d).tar.gz" } # Receive and decompress receive_and_extract() { local remote="$1" local remote_file="$2" local local_dir="$3" ssh user@"$remote" "cat $remote_file" | tar -xzf - -C "$local_dir" } # Split large files for transfer split_and_send() { local file="$1" local dest="$2" local chunk_size="${3:-100M}" local tmp_dir="/tmp/split_$$" mkdir -p "$tmp_dir" # Split file split -b "$chunk_size" "$file" "$tmp_dir/part_" # Send parts scp -r "$tmp_dir"/* "$dest/parts/" # Reassemble command echo "To reassemble: cat parts/* > $file" | ssh "$dest" "cat > reassemble.sh" rm -rf "$tmp_dir" } # Resume interrupted transfer resume_transfer() { local file="$1" local dest="$2" # Check if partial transfer exists if ssh "${dest%%:*}" "test -f ${dest#*:}.part"; then remote_size=$(ssh "${dest%%:*}" "stat -c%s ${dest#*:}.part") local_size=$(stat -c%s "$file") if [ "$local_size" -gt "$remote_size" ]; then echo "Resuming transfer..." tail -c +$((remote_size + 1)) "$file" | \ ssh "${dest%%:*}" "cat >> ${dest#*:}.part" # Rename when complete ssh "${dest%%:*}" "mv ${dest#*:}.part ${dest#*:}" fi else scp "$file" "$dest" fi }

Using Different Shells

#!/bin/bash # Remote command execution with scp scp_with_remote_cmd() { local file="$1" local remote="$2" local remote_cmd="$3" # Copy file scp "$file" "$remote:/tmp/" # Execute remote command ssh "$remote" "$remote_cmd /tmp/$(basename "$file")" } # Check file integrity after transfer verify_transfer() { local src="$1" local dest="$2" # Calculate local checksum local_sum=$(md5sum "$src" | cut -d' ' -f1) # Copy file scp "$src" "$dest" # Calculate remote checksum remote_sum=$(ssh "${dest%%:*}" "md5sum ${dest#*:}" | cut -d' ' -f1) if [ "$local_sum" = "$remote_sum" ]; then echo "✓ Transfer verified" return 0 else echo "✗ Checksum mismatch" return 1 fi } # Atomic file replacement atomic_replace() { local new_file="$1" local remote_target="$2" local temp_file="${remote_target}.new" # Copy to temp location scp "$new_file" "$remote_target_host:$temp_file" # Atomic rename ssh "$remote_target_host" "mv $temp_file $remote_target" }

6. Error Handling and Robustness

Comprehensive Error Handling

#!/bin/bash # Robust SCP function robust_scp() { local src="$1" local dest="$2" local max_attempts="${3:-3}" local timeout="${4:-30}" # Validate source if [ ! -e "$src" ]; then echo "Error: Source does not exist: $src" >&2 return 1 fi if [ ! -r "$src" ]; then echo "Error: Source not readable: $src" >&2 return 1 fi # Parse destination dest_host="${dest%%:*}" dest_path="${dest#*:}" if [ "$dest_host" = "$dest" ]; then # Local destination dest_dir=$(dirname "$dest_path") if [ ! -d "$dest_dir" ]; then mkdir -p "$dest_dir" fi else # Remote destination - check connectivity if ! ssh -o ConnectTimeout=5 -o BatchMode=yes "$dest_host" exit 2>/dev/null; then echo "Error: Cannot connect to $dest_host" >&2 return 1 fi # Check remote directory exists if ! ssh "$dest_host" "mkdir -p $(dirname "$dest_path")" 2>/dev/null; then echo "Error: Cannot create remote directory" >&2 return 1 fi fi # Attempt transfer with retry local attempt=1 while [ $attempt -le $max_attempts ]; do echo "Attempt $attempt of $max_attempts" if timeout "$timeout" scp -q "$src" "$dest" 2>/dev/null; then echo "Transfer successful" return 0 fi echo "Transfer failed, retrying in 5 seconds..." sleep 5 attempt=$((attempt + 1)) done echo "Transfer failed after $max_attempts attempts" >&2 return 1 } # Error classification classify_scp_error() { local exit_code=$1 local output="$2" case $exit_code in 0) echo "SUCCESS" ;; 1) if echo "$output" | grep -q "Permission denied"; then echo "AUTHENTICATION_FAILED" elif echo "$output" | grep -q "No such file"; then echo "FILE_NOT_FOUND" else echo "GENERAL_ERROR" fi ;; 2) echo "CONNECTION_FAILED" ;; 3) echo "PROTOCOL_ERROR" ;; *) echo "UNKNOWN_ERROR" ;; esac }

Recovery Strategies

#!/bin/bash # Resume failed transfer resume_failed() { local src="$1" local dest="$2" local state_file="/tmp/scp_resume_$$" # Check if partial transfer exists if ssh "${dest%%:*}" "test -f ${dest#*:}.part" 2>/dev/null; then remote_size=$(ssh "${dest%%:*}" "stat -c%s ${dest#*:}.part") local_size=$(stat -c%s "$src") if [ "$local_size" -gt "$remote_size" ]; then echo "Resuming from byte $remote_size" dd if="$src" bs=1024 skip=$((remote_size / 1024)) 2>/dev/null | \ ssh "${dest%%:*}" "dd of=${dest#*:}.part bs=1024 seek=$((remote_size / 1024)) 2>/dev/null" # Verify final size final_size=$(ssh "${dest%%:*}" "stat -c%s ${dest#*:}.part") if [ "$final_size" -eq "$local_size" ]; then ssh "${dest%%:*}" "mv ${dest#*:}.part ${dest#*:}" echo "Transfer completed" fi fi else # Start fresh transfer with part file scp "$src" "${dest}.part" && \ ssh "${dest%%:*}" "mv ${dest#*:}.part ${dest#*:}" fi } # Create recovery script create_recovery_script() { local files=("$@") local recovery_script="/tmp/recover_scp_$$.sh" { echo "#!/bin/bash" echo "# Recovery script for SCP transfers" echo "# Generated: $(date)" echo echo "FAILED=()" for file in "${files[@]}"; do echo "if [ ! -f \"$file\" ]; then" echo " echo \"Recovering $file...\"" echo " scp user@backup-server:backup/$(basename "$file") ." echo " [ \$? -eq 0 ] || FAILED+=(\"$file\")" echo "fi" echo done echo "if [ \${#FAILED[@]} -gt 0 ]; then" echo " echo \"Failed to recover: \${FAILED[@]}\"" echo "else" echo " echo \"All files recovered successfully\"" echo "fi" } > "$recovery_script" chmod +x "$recovery_script" echo "Recovery script created: $recovery_script" }

7. Performance Optimization

Bandwidth Optimization

#!/bin/bash # Optimize SCP for different networks optimize_scp() { local file="$1" local dest="$2" local network_type="${3:-auto}" # auto, lan, wifi, wan case "$network_type" in lan) # Fast LAN - no compression, large window scp -o IPQoS=throughput -o TCPKeepAlive=yes "$file" "$dest" ;; wifi) # WiFi - some compression, moderate window scp -C -o IPQoS=reliability "$file" "$dest" ;; wan) # WAN/WAN - maximum compression, small window scp -C -c aes128-ctr -o CompressionLevel=9 "$file" "$dest" ;; *) # Auto-detect # Check RTT to destination rtt=$(ping -c 1 "${dest%%:*}" 2>/dev/null | grep time= | cut -d= -f4) if [ -n "$rtt" ] && [ "$(echo "$rtt < 5" | bc)" -eq 1 ]; then optimize_scp "$file" "$dest" "lan" elif [ -n "$rtt" ] && [ "$(echo "$rtt < 50" | bc)" -eq 1 ]; then optimize_scp "$file" "$dest" "wifi" else optimize_scp "$file" "$dest" "wan" fi ;; esac } # Parallel chunked transfer parallel_chunk() { local file="$1" local dest="$2" local chunks="${3:-4}" local tmp_dir="/tmp/scp_parallel_$$" mkdir -p "$tmp_dir" # Get file size total=$(stat -c%s "$file") chunk_size=$((total / chunks)) # Split and transfer in parallel for ((i=0; i<chunks; i++)); do start=$((i * chunk_size)) if [ $i -eq $((chunks - 1)) ]; then end=$total else end=$(( (i + 1) * chunk_size )) fi ( dd if="$file" bs=1024 skip=$((start / 1024)) count=$(((end - start) / 1024)) 2>/dev/null | \ ssh "${dest%%:*}" "dd of=${dest#*:}.part$i bs=1024 seek=$((start / 1024)) 2>/dev/null" ) & done wait # Reassemble ssh "${dest%%:*}" "cat ${dest#*:}.part* > ${dest#*:} && rm ${dest#*:}.part*" rm -rf "$tmp_dir" }

Caching and Optimization

#!/bin/bash # Cache SCP connections setup_scp_cache() { local host="$1" # Setup persistent connection for multiple transfers ssh -o ControlMaster=auto -o ControlPath=/tmp/ssh-%r@%h:%p -o ControlPersist=10m "$host" exit # Use cached connection for SCP scp_with_cache() { scp -o ControlPath=/tmp/ssh-%r@%h:%p "$@" } } # Batch transfer with connection reuse batch_transfer() { local host="$1" shift local files=("$@") # Setup master connection ssh -fN -o ControlMaster=yes -o ControlPath=/tmp/ssh-%r@%h:%p "$host" # Transfer all files using same connection for file in "${files[@]}"; do scp -o ControlPath=/tmp/ssh-%r@%h:%p "$file" "$host:" done # Close master connection ssh -O exit -o ControlPath=/tmp/ssh-%r@%h:%p "$host" }

8. Security Considerations

Secure Practices

#!/bin/bash # Secure SCP wrapper secure_scp() { local src="$1" local dest="$2" # Verify host key ssh-keygen -F "${dest%%:*}" >/dev/null || { echo "Warning: Host not known" ssh-keyscan -H "${dest%%:*}" 2>/dev/null } # Use strong cipher scp -c aes256-ctr -o MACs=hmac-sha2-512 "$src" "$dest" } # Encrypt before transfer encrypt_before_scp() { local file="$1" local dest="$2" local password="$3" # Encrypt openssl enc -aes-256-cbc -salt -in "$file" -out "$file.enc" -k "$password" # Transfer scp "$file.enc" "$dest" # Cleanup rm "$file.enc" echo "To decrypt: openssl enc -d -aes-256-cbc -in $file.enc -out $file -k password" } # Sign and verify sign_and_send() { local file="$1" local dest="$2" local key="$HOME/.ssh/id_rsa" # Create signature openssl dgst -sha256 -sign "$key" -out "$file.sig" "$file" # Send both file and signature scp "$file" "$file.sig" "$dest" # Remote verification command echo "To verify: openssl dgst -sha256 -verify <(ssh-keygen -y -f $key.pub) -signature $file.sig $file" }

Access Control

#!/bin/bash # Restricted SCP for specific users restricted_scp() { local user="$1" local src="$2" local dest="$3" # Check user permissions if ! grep -q "^$user:" /etc/passwd; then echo "User $user does not exist" return 1 fi # Check allowed directories allowed_dirs=("/home/$user" "/backup/$user" "/tmp") dest_dir=$(dirname "$dest") allowed=0 for dir in "${allowed_dirs[@]}"; do if [[ "$dest_dir" == "$dir"* ]]; then allowed=1 break fi done if [ $allowed -eq 0 ]; then echo "Destination not allowed" return 1 fi scp "$src" "$dest" } # SCP with IP whitelist whitelist_scp() { local src="$1" local dest="$2" local client_ip="${SSH_CLIENT%% *}" # Check if IP is whitelisted if ! grep -q "^$client_ip$" /etc/scp_whitelist 2>/dev/null; then logger -t scp "Blocked SCP attempt from $client_ip" echo "Access denied" return 1 fi scp "$src" "$dest" }

9. Integration with Other Tools

With rsync for better sync

#!/bin/bash # Hybrid scp/rsync approach smart_sync() { local src="$1" local dest="$2" # Check if rsync is available on both ends if command -v rsync >/dev/null && ssh "${dest%%:*}" "command -v rsync" >/dev/null; then echo "Using rsync for efficient sync" rsync -avz -e ssh "$src" "$dest" else echo "Using scp (rsync not available)" scp -r "$src" "$dest" fi } # Scp with tar for better performance tar_scp() { local src="$1" local dest="$2" tar -cf - "$src" | ssh "${dest%%:*}" "tar -xf - -C ${dest#*:}" } # Scp with find for selective copy find_scp() { local src_dir="$1" local dest="$2" local pattern="$3" find "$src_dir" -name "$pattern" -type f -exec scp {} "$dest" \; }

With monitoring tools

#!/bin/bash # Monitor with pv scp_with_progress() { local src="$1" local dest="$2" # Get file size size=$(stat -c%s "$src") # Copy with progress bar pv -s "$size" "$src" | ssh "${dest%%:*}" "cat > ${dest#*:}" } # Monitor with custom progress scp_progress() { local src="$1" local dest="$2" scp "$src" "$dest" & pid=$! while kill -0 $pid 2>/dev/null; do # Get process size (simplified) if [ -f "$dest" ]; then transferred=$(stat -c%s "$dest" 2>/dev/null || echo 0) total=$(stat -c%s "$src") percent=$((transferred * 100 / total)) echo -ne "Progress: $percent% ($transferred/$total bytes)\r" fi sleep 1 done echo } # Log to systemd journal scp_journal() { local src="$1" local dest="$2" logger -t scp "Starting transfer: $src -> $dest" if scp "$src" "$dest"; then logger -t scp "Transfer complete: $src -> $dest" else logger -t scp "Transfer failed: $src -> $dest" fi }

10. Real-World Applications

Automated Backup System

#!/bin/bash # Comprehensive backup system backup_system() { local config="/etc/backup.conf" local log="/var/log/backup.log" # Load configuration source "$config" { echo "=== Backup Started: $(date) ===" for item in "${BACKUP_ITEMS[@]}"; do echo "Backing up: $item" # Create backup name backup_name="$(basename "$item")_$(date +%Y%m%d).tar.gz" # Compress tar -czf "/tmp/$backup_name" "$item" # Transfer if scp "/tmp/$backup_name" "${BACKUP_SERVER}:${BACKUP_DIR}/"; then echo "✓ $item backed up successfully" rm "/tmp/$backup_name" # Rotate old backups ssh "${BACKUP_SERVER}" "cd ${BACKUP_DIR} && ls -t $(basename "$item")_* | tail -n +${BACKUP_ROTATE} | xargs rm -f" else echo "✗ Failed to backup $item" fi done echo "=== Backup Complete: $(date) ===" } | tee -a "$log" } # Example config file cat > /etc/backup.conf << EOF BACKUP_SERVER="user@backup-server" BACKUP_DIR="/backups" BACKUP_ROTATE=7 BACKUP_ITEMS=( "/etc" "/home" "/var/www" "/var/log" ) EOF

Remote Deployment System

#!/bin/bash # Multi-server deployment deploy_to_servers() { local app_name="$1" local version="$2" local servers=("${@:3}") local deploy_log="/tmp/deploy_${app_name}_${version}.log" echo "Deploying $appName version $version to ${#servers[@]} servers" # Create deployment package package_name="${app_name}_${version}.tar.gz" tar -czf "/tmp/$package_name" -C "/builds/$appName" . # Deploy to each server for server in "${servers[@]}"; do echo -n "Deploying to $server... " { # Copy package scp "/tmp/$package_name" "$server:/tmp/" # Deploy on remote ssh "$server" " cd /opt/apps && sudo tar -xzf /tmp/$package_name && sudo chown -R app:app $app_name && sudo systemctl restart $app_name " } >> "$deploy_log" 2>&1 if [ $? -eq 0 ]; then echo "✓" else echo "✗" fi done rm "/tmp/$package_name" echo "Deployment log: $deploy_log" } # Rolling deployment rolling_deploy() { local servers=("$@") for server in "${servers[@]}"; do echo "Deploying to $server..." # Take out of load balancer ssh "$server" "sudo systemctl stop app" # Deploy scp app.tar.gz "$server:/tmp/" ssh "$server" " cd /opt/app && sudo tar -xzf /tmp/app.tar.gz && sudo systemctl start app " # Wait for health check sleep 10 # Verify if curl -f "http://$server:8080/health"; then echo "✓ $server deployed successfully" else echo "✗ $server deployment failed - rolling back" # Rollback logic here break fi done }

Log Collection System

#!/bin/bash # Collect logs from multiple servers collect_logs() { local date=$(date +%Y%m%d) local log_dir="/logs/collect_$date" mkdir -p "$log_dir" # Read server list while read server; do echo "Collecting logs from $server" # Collect syslog scp "$server:/var/log/syslog" "$log_dir/${server}_syslog" & # Collect application logs scp "$server:/var/log/app/*.log" "$log_dir/${server}_app/" & # Collect auth logs scp "$server:/var/log/auth.log" "$log_dir/${server}_auth.log" & # Wait for background jobs to complete wait echo "✓ Logs collected from $server" done < /etc/log_collector/servers.txt # Create archive tar -czf "/backups/logs_$date.tar.gz" "$log_dir" rm -rf "$log_dir" echo "Log collection complete: /backups/logs_$date.tar.gz" } # Real-time log aggregation aggregate_logs() { local servers=("$@") local logfile="/var/log/aggregated.log" while true; do for server in "${servers[@]}"; do # Get last 10 lines from each server ssh "$server" "tail -10 /var/log/app.log" | \ sed "s/^/[$server] /" >> "$logfile" done sleep 5 done }

11. Best Practices and Tips

Security Best Practices

#!/bin/bash # 1. Use key-based authentication instead of passwords ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 ssh-copy-id user@remote-server # 2. Use specific ciphers scp -c [email protected] file.txt user@host: # 3. Limit SCP in sshd_config # In /etc/ssh/sshd_config: # Subsystem sftp internal-sftp # Match User backup # ChrootDirectory /backup # ForceCommand internal-sftp # X11Forwarding no # AllowTcpForwarding no # 4. Use bastion hosts for complex networks scp -o ProxyJump=bastion-server user@internal-server:/file.txt . # 5. Log all transfers # In /etc/ssh/sshd_config: # Subsystem sftp /usr/lib/openssh/sftp-server -l INFO

Performance Tips

#!/bin/bash # 1. Use compression for text files scp -C large_text_file.log user@host: # 2. Don't compress already compressed files if [[ ! "$file" =~ \.(gz|bz2|zip|jpg|png)$ ]]; then scp -C "$file" user@host: else scp "$file" user@host: fi # 3. Use appropriate cipher for speed # Fastest: aes128-ctr scp -c aes128-ctr file.txt user@host: # 4. Increase window size for high latency scp -o IPQoS=throughput -o TCPKeepAlive=yes file.txt user@host: # 5. Use ControlMaster for multiple files ssh -fN -o ControlMaster=yes -o ControlPath=/tmp/ssh-%r@%h:%p user@host scp -o ControlPath=/tmp/ssh-%r@%h:%p file1.txt user@host: scp -o ControlPath=/tmp/ssh-%r@%h:%p file2.txt user@host: ssh -O exit -o ControlPath=/tmp/ssh-%r@%h:%p user@host

Scripting Best Practices

#!/bin/bash # 1. Always check return codes if ! scp file.txt user@host:/destination/; then echo "SCP failed" exit 1 fi # 2. Use timeout to prevent hanging timeout 60 scp largefile.txt user@host: || echo "SCP timed out" # 3. Quote variables to handle spaces scp "$file" "user@host:$dest" # 4. Use arrays for multiple files files=("file1.txt" "file with spaces.txt" "file3.txt") scp "${files[@]}" user@host: # 5. Create temporary files safely temp_file=$(mktemp) scp "user@host:file.txt" "$temp_file" # 6. Clean up temp files trap 'rm -f "$temp_file"' EXIT # 7. Use batch mode in scripts scp -o BatchMode=yes file.txt user@host: # 8. Set connection timeout scp -o ConnectTimeout=10 file.txt user@host:

Common Pitfalls and Solutions

#!/bin/bash # Pitfall 1: Host key verification # Solution: Add host key automatically ssh-keyscan -H hostname >> ~/.ssh/known_hosts # Pitfall 2: Permission denied # Solution: Check permissions chmod 600 ~/.ssh/id_rsa chmod 700 ~/.ssh # Pitfall 3: Path expansion issues # Solution: Use full paths or proper quoting scp "user@host:'/path/with spaces/file.txt'" . # Pitfall 4: Symlinks not preserved # Solution: Use tar to preserve links tar -chf - symlink_dir | ssh user@host "tar -xf - -C /dest" # Pitfall 5: Interrupted transfers # Solution: Use rsync or implement resume rsync -avP -e ssh file.txt user@host: # Pitfall 6: Firewall blocking # Solution: Test connectivity first nc -zv host 22 && scp file.txt user@host:

12. Command Summary and Cheat Sheet

Quick Reference

# Basic commands scp file.txt user@host: # Copy to remote home scp user@host:file.txt . # Copy from remote to local scp -r dir/ user@host:backup/ # Copy directory recursively scp -p file.txt user@host: # Preserve attributes scp -C file.txt user@host: # Enable compression scp -P 2222 file.txt user@host: # Use specific port # Advanced options scp -l 1000 file.txt user@host: # Limit bandwidth scp -c aes256-ctr file.txt user@host: # Use specific cipher scp -o ConnectTimeout=10 file.txt user@host: # Set timeout scp -o BatchMode=yes file.txt user@host: # Batch mode (no password) scp -v file.txt user@host: # Verbose output scp -q file.txt user@host: # Quiet mode # Multiple files scp file1.txt file2.txt file3.txt user@host: scp *.log user@host:/logs/ # With different usernames scp file.txt username@host:/path/ # Between remote hosts scp user1@host1:file.txt user2@host2:

Common Options

Option	Description	Example
`-P`	Port number	`scp -P 2222 file.txt host:`
`-p`	Preserve attributes	`scp -p file.txt host:`
`-r`	Recursive	`scp -r dir/ host:`
`-C`	Enable compression	`scp -C bigfile.tar host:`
`-l`	Limit bandwidth	`scp -l 1000 file.txt host:`
`-i`	Identity file	`scp -i ~/.ssh/key file.txt host:`
`-c`	Cipher specification	`scp -c aes256-ctr file.txt host:`
`-o`	SSH options	`scp -o ConnectTimeout=10 file.txt host:`
`-v`	Verbose	`scp -v file.txt host:`
`-q`	Quiet	`scp -q file.txt host:`

Exit Codes

# scp exit codes and meanings # 0 = Success # 1 = General error (file not found, permission denied) # 2 = Connection failed # 3 = Protocol error # 4 = No matching cipher/MAC scp file.txt user@host: case $? in 0) echo "Success" ;; 1) echo "File error or permission denied" ;; 2) echo "Connection failed" ;; 3) echo "Protocol error" ;; 4) echo "Cipher mismatch" ;; esac

Conclusion

The scp command is an essential tool for secure file transfer:

Key Takeaways

Security: All transfers are encrypted using SSH
Simplicity: Simple syntax similar to cp command
Flexibility: Supports single files, directories, and multiple files
Integration: Works seamlessly with SSH authentication
Scripting: Easy to incorporate into automation scripts

Best Practices Summary

Scenario	Recommendation
Single file	`scp file.txt user@host:`
Directory	`scp -r dir/ user@host:`
Large files	Use `-C` compression and `-l` rate limiting
Many small files	Use tar + ssh or rsync
Automated scripts	Use key-based auth and `-o BatchMode=yes`
Debugging	Use `-v` for verbose output
Security	Use strong ciphers and key-based auth
Reliability	Implement retry logic and error checking

When to Use Alternatives

rsync: Better for syncing directories, only transfers differences
sftp: Interactive file transfers, better for manual operations
ftp: Unencrypted, only for internal/trusted networks
curl: HTTP/HTTPS transfers, API interactions
nc: Simple unencrypted transfers, debugging

The scp command remains a reliable choice for secure file transfers, especially when simplicity and security are paramount.

Complete Bash Exercises with Solutions

Practice Bash scripting with solved exercises to improve your command-line and scripting skills.
Link: https://macronepal.com/complete-bash-exercises-with-solutions/

Complete Guide to Bash Crontab Command

Learn how to schedule and automate tasks efficiently using the Bash crontab command.
Link: https://macronepal.com/complete-guide-to-bash-crontab-command-schedule-tasks/

Complete Guide to Bash Arrays

Understand how to create, manage, and use arrays in Bash scripting.
Link: https://macronepal.com/complete-guide-to-bash-arrays/

Bash Quiz: Test Your Knowledge

Test your Bash scripting knowledge with quizzes and practical questions.
Link: https://macronepal.com/bash-quiz-test-your-knowledge/

Complete Guide to Bash Functions

Learn how to write reusable Bash functions for cleaner and efficient scripts.
Link: https://macronepal.com/complete-guide-to-bash-functions/

Complete Guide to Bash Loops

Master loops in Bash for repetitive tasks and automation.
Link: https://macronepal.com/complete-guide-to-bash-loops/

Complete Guide to Bash If-Else Statements

Learn decision-making in Bash using conditional statements.
Link: https://macronepal.com/complete-guide-to-bash-ifelse-statements/

Complete Guide to Bash Operators

Explore arithmetic, logical, and comparison operators in Bash.
Link: https://macronepal.com/complete-guide-to-bash-operators/

Complete Guide to Bash Data Types

Understand variables and different data types used in Bash scripting.
Link: https://macronepal.com/complete-guide-to-bash-data-types/

Complete Guide to Bash Variables

Learn how to declare, assign, and use variables in Bash scripts.
Link: https://macronepal.com/complete-guide-to-bash-variables/

Complete Guide to Bash Scripting

A complete introduction to writing Bash scripts for automation.
Link: https://macronepal.com/complete-guide-to-bash-scripting/

Complete Guide to Basic Bash Syntax

Learn the fundamental syntax needed to write Bash commands and scripts.
Link: https://macronepal.com/complete-guide-to-basic-bash-syntax/

Complete Guide to Bash Chgrp Command

Learn how to change group ownership of files and directories.
Link: https://macronepal.com/complete-guide-to-bash-chgrp-command-change-group-ownership-2/

Complete Guide to Bash Chgrp Command (Alternate)

Another detailed explanation of Bash group ownership management.
Link: https://macronepal.com/complete-guide-to-bash-chgrp-command-change-group-ownership/

Complete Guide to Bash File Permissions and Ownership

Understand Linux file permissions and ownership management.
Link: https://macronepal.com/complete-guide-to-bash-file-permissions-and-ownership/

Complete Guide to Bash Chmod Command

Learn how to modify file permissions using chmod.
Link: https://macronepal.com/complete-guide-to-bash-chmod-command-change-file-permissions/

Complete Guide to Bash Chown Command

Change file ownership using the Bash chown command.
Link: https://macronepal.com/complete-guide-to-bash-chown-command-change-file-ownership/

Complete Guide to Bash SCP Command

Securely transfer files between systems using SCP.
Link: https://macronepal.com/complete-guide-to-bash-scp-command-secure-copy/

Complete Guide to Bash SSH Command

Learn secure remote access and server management using SSH.
Link: https://macronepal.com/complete-guide-to-bash-ssh-command/

Bash Wget Command Complete Guide

Download files from the internet using the wget command.
Link: https://macronepal.com/bash-wget-command-complete-guide-to-non-interactive-network-downloader/

Introduction to scp