Translation(s): English - Français - Italiano

(!) ?Discussion

Shell Scripting Tricks

Note: Debian Wiki is not necessarily the best place to find help with shell scripts. If your scripting problem involves bash (or Bourne shell), you might try bash page instead. A bash FAQ is at http://mywiki.wooledge.org/BashFAQ and contains many more tricks than this page does.

How do I see how much disk space is left?

df 
df -h # for a human readable format in megs & gigs

How do I see where all my big files are? My file system is full.

cd /somewhere
du -sk * | sort -n
# Repeat as necessary

If you want to see big directories instead of files, you can do:

du -x /somewhere | sort -n | tail -10
# This shows the 10 largest directories under /somewhere

Another way:

find /somewhere -size +2000k -ls
# This shows all files over 2000 kilobytes under /somewhere

If you want to keep your .debs, you can filter them out of the list:

find / -size +2000k ! -name "*.deb" -ls
# This shows all files on your entire system over 2000 kilobytes that are not debian packages

I have a directory full of MP3s... how do I rename them all with underscores instead of spaces?

rename 's/ /_/g' *.mp3

The rename command is not a general Unix command but it's included with Perl which is of course installed by default in Debian.

How do I do that recursively?

That's a bit tricky. You can get a list of files to pass to rename using find, but if some of the directories are being renamed as well, that's not something rename can keep track of. You have to use -depth with find to make sure any files are renamed before the directories they're in.

cd /somewhere
find . -depth -name '* *' -type f -print0 | xargs -r0 rename 's/ /_/g'

How do I (recursively) rename all files from uppercase to lowercase?

Once again, this is hard if the directories themselves have uppercase letters in their names. Let's assume for the moment that they don't. Then:

cd /somewhere
find . -name '*[A-Z]*' -type f -print0 | xargs -0 rename 'y/A-Z/a-z/'

Unfortunately, whenever this question pops up, that's usually not enough -- the person asking usually has uppercase directory names too. So something like this might be required:

cd /somewhere
find . -type d -depth -name '*[A-Z]*' -print |
  while read dir; do dname="$(dirname $dir)"; bname="$(basename $dir)";
  newbname="$(echo $bname | tr [:upper:] [:lower:])"; mv "$dir" "$dname/$newbname"; done
# That renames the directories.  The -depth makes it go through the deepest ones first, so we don't rename
# A to a until we've already renamed A/B to A/b.
find . -name '*[A-Z]*' -type f -print0 | xargs -0 rename 'y/A-Z/a-z/'

Note that the more complex script above is imperfect: it will fail if any directories have newlines in their names, and it may also fail under some conditions if any files or directories have whitespace in their names. Use at your own risk. (Hint: if you want to test it before you commit to it, put an echo in front of the mv.)

In fact, this simpler solution might work better:

find /somewhere -depth -name '*[A-Z]*' -print0 | xargs -r0 rename 'y/A-Z/a-z/'

Here, too, it might be a good idea to put in an echo before the rename command to test before you actually run this.

How do I remove a file that begins with "-"?

unlink -foo

Three different ways:

rm -- -foo
rm ./-foo
using 'mc', and pressing F8 on the appropriate file

These two tricks work with nearly every command line tool, not only rm.

How do I monitor something in realtime?

If it's a log file, follow the tail of the file:

tail -f /var/log/messages

or

less +F /var/log/messages

If it's a general command, use watch:

watch -n 1 ls -l ~/some/file

This works well when monitoring a firewall with iptables / tc or like tasks.

How do I get only the filename in a full pathname?

Two ways:

basename /path/to/file
foo=/path/to/file ; echo ${foo##*/}

How do I test whether a directory has files in it?

if [ "$(ls -A somedir)" ]; then
    echo "It has files"
fi

Note: a single argument inside the brackets is equivalent to [ -n argument ]. There seems to be no clean way to do this using only shell builtins. (We had one earlier attempt which was close, but that failed if there was one file named * in the directory.)

Another way:

if [ "$(ls -A somedir | wc -l)" -gt 0 ]; then
    echo "found files"
fi

A shorter way: [ $(ls -A somedir) ] && echo "directory is not empty"

Note: The double ampersand works like a boolean "and", so the "if" is not required.

How do I start a process in background?

Just append an ampersand:

sleep 100 &

It prints a line that looks like:  [1] 6338 , indicating that this process is job number 1 and that its process ID is 6338.

To kill this job:

kill %1 

or

kill 6338 

Note: If you have started many processes in background and you want to list them, type jobs.

You can also switch a job running in the background into the foreground, then suspend this job, and then restart this job to the background. Let's start a process in the background:

sleep 100 &

To move it to the foreground (assuming that this is job number 1):

fg %1

Then, press Ctrl+Z. This suspends the process and send it to the background. Now, restart the suspended process while leaving it in the background:

bg %1

To kill this job:

kill %1

How do I work with arrays in bash?

Assigning to one:

arr=(one two three four)
myoggs=(*.ogg)

Iterating over one:

for f in "${myoggs[@]}"; do ...; done

Selecting a random element:

mysigs=($HOME/sigs/*)
cat ${mysigs[RANDOM % ${#mysigs[*]}]}

(In the previous example, note that the brackets for the array index also force a numeric evaluation context, in which RANDOM doesn't need a leading $ sign.)

You can't easily "delete" an element from the middle of an array, but you can hack around it by reindexing the array to skip over all the unset elements:

arr=(zero one two three four)
unset arr[1]
arr=("${arr[@]}")

This will also leave all the empty elements in place, deleting only the unset ones.

How do I use a variable within a variable (variable interpolation) in bash?

If you want to do something like ${$var}, use this format: ${!var}

So if you have:

FOO=one
BAR=FOO

then ${!BAR} will return "one"

This only works for referencing an existing variable, not for assignment. If you wish to assign to a dynamically generated variable name, you must use either eval:

IFACE=eth0
eval IP_${IFACE}=192.168.1.1

or, in many cases, just use an array:

N=17
ROW[N]="This is the 18th row (counting from 0)."

The eval trick can also be used for referencing:

IFACE=eth0
eval echo \$IP_${IFACE}

The eval makes a second pass over the code. The first pass substitutes $ for \$ and then substitutes the value of ${IFACE}, leaving "eval echo $IP_eth0". The second pass then substitutes the value of the ${IP_eth0} variable, assuming one exists.

How do I count the lines of files with certain extensions in a certain directory?

find $dir -name *.[$extensions] -exec cat \{\} \; | wc -l

finds the files in $dir with any of the extensions $extensions and reads the files using cat using 'wc -l' to count the lines of text.

You can so make

wc -l $dir/*.$extensions


CategorySystemAdministration | CategoryCommandLineInterface | CategoryRedundant: merge some parts with other shell/CLI pages (?ShellScripting)