Differences between revisions 9 and 10
Revision 9 as of 2009-08-25 09:44:33
Size: 7454
Editor: MichelBarret
Comment: Modify « wc -l » to « nl » and « xarg cat » to « -exec \{\} \; »
Revision 10 as of 2009-08-25 09:49:05
Size: 7475
Editor: MichelBarret
Deletions are marked like this. Additions are marked like this.
Line 203: Line 203:
find $dir -name *.[$extensions] -exec cat \{\} \; | nl
finds the files in $dir with any of the extensions $extensions and reads the files using cat using 'nl' to count the lines of text (see [[ShellCommands#N|ShellCommands]]).
find $dir -name *.[$extensions] -exec cat \{\} \; | wc -l
finds the files in $dir with any of the extensions $extensions and reads the files using cat using 'wc -l' to count the lines of text.

You can so
wc -l $dir/*

Translation(s): Français

(!) ?Discussion

Shell Scripting Tricks

Note: #debian is not necessarily the best place to find help with shell scripts. If your scripting problem involves bash (or Bourne shell), you might try #bash instead. The #bash FAQ is at http://wooledge.org/mywiki/BashFaq and contains many more tricks than this page does.

How do I see how much disk space is left?

df -h for a human readable format in megs & gigs

How do I see where all my big files are? My file system is full.

cd /somewhere
du -sk * | sort -n
# Repeat as necessary

If you want to see big directories instead of files, you can do:

du -x /somewhere | sort -n | tail -10
# This shows the 10 largest directories under /somewhere

Another way:

find /somewhere -size +2000k -ls
# This shows all files over 2000 kilobytes under /somewhere

If you want to keep your .debs, you can filter them out of the list:

find / -size +2000k ! -name "*.deb" -ls
# This shows all files on your entire system over 2000 kilobytes that are not debian packages

I have a directory full of MP3s... how do I rename them all with underscores instead of spaces?

rename 's/ /_/g' *.mp3

The rename command is not a general Unix command but it's included with Perl which is of course installed by default in Debian.

How do I do that recursively?

That's a bit tricky. You can get a list of files to pass to rename using find, but if some of the directories are being renamed as well, that's not something rename can keep track of. You have to use -depth with find to make sure any files are renamed before the directories they're in.

cd /somewhere
find . -depth -name '* *' -type f -print0 | xargs -r0 rename 's/ /_/g'

How do I (recursively) rename all files from uppercase to lowercase?

Once again, this is hard if the directories themselves have uppercase letters in their names. Let's assume for the moment that they don't. Then:

cd /somewhere
find . -name '*[A-Z]*' -type f -print0 | xargs -0 rename 'y/A-Z/a-z/'

Unfortunately, whenever this question pops up, that's usually not enough -- the person asking usually has uppercase directory names too. So something like this might be required:

cd /somewhere
find . -type d -depth -name '*[A-Z]*' -print |
  while read dir; do dname="$(dirname $dir)"; bname="$(basename $dir)";
  newbname="$(echo $bname | tr [:upper:] [:lower:])"; mv "$dir" "$dname/$newbname"; done
# That renames the directories.  The -depth makes it go through the deepest ones first, so we don't rename
# A to a until we've already renamed A/B to A/b.
find . -name '*[A-Z]*' -type f -print0 | xargs -0 rename 'y/A-Z/a-z/'

Note that the more complex script above is imperfect: it will fail if any directories have newlines in their names, and it may also fail under some conditions if any files or directories have whitespace in their names. Use at your own risk. (Hint: if you want to test it before you commit to it, put an echo in front of the mv.)

In fact, this simpler solution might work better:

find /somewhere -depth -name '*[A-Z]*' -print0 | xargs -r0 rename 'y/A-Z/a-z/'

Here, too, it might be a good idea to put in an echo before the rename command to test before you actually run this.

How do I remove a file that begins with "-"?

unlink -foo

Three different ways:

rm -- -foo
rm ./-foo
using 'mc', and pressing F8 on the appropriate file

These two tricks work with nearly every command line tool, not only rm.

How do I monitor something in realtime?

If it's a log file, follow the tail of the file:

tail -f /var/log/messages


less +F /var/log/messages

If it's a general command, use watch:

watch -n 1 ls -l ~/some/file

This works well when monitoring a firewall with iptables / tc or like tasks.

How do I get only the filename in a full pathname?

Two ways:

basename /path/to/file
foo=/path/to/file ; echo ${foo##*/}

How do I test whether a directory has files in it?

if [ "$(ls -A somedir)" ]; then
    echo "It has files"

Note: a single argument inside the brackets is equivalent to [ -n argument ]. There seems to be no clean way to do this using only shell builtins. (We had one earlier attempt which was close, but that failed if there was one file named * in the directory.)

Another way:

if [ "$(ls -A somedir | wc -l)" -gt 0 ]; then
    echo "found files"

A shorter way: [ $(ls -A somedir) ] && echo "directory is not empty"

Note: The double ampersand works like a boolean "and", so the "if" is not required.

How do I start a process in background?

Just append an ampersand:

cp bigfile bigfile2 &

or press Ctrl+Z while the command is running (this will stop it) and use for example

bg cp

to make it continue working and use

fg cp

to move it to the foreground (Ctrl+Z will stop again it and send it to the background) and use

kill cp

to kill it.

How do I work with arrays in bash?

Assigning to one:

arr=(one two three four)

Iterating over one:

for f in "${myoggs[@]}"; do ...; done

Selecting a random element:

cat ${mysigs[RANDOM % ${#mysigs[*]}]}

(In the previous example, note that the brackets for the array index also force a numeric evaluation context, in which RANDOM doesn't need a leading $ sign.)

You can't easily "delete" an element from the middle of an array, but you can hack around it by reindexing the array to skip over all the unset elements:

arr=(zero one two three four)
unset arr[1]

This will also leave all the empty elements in place, deleting only the unset ones.

How do I use a variable within a variable (variable interpolation) in bash?

If you want to do something like ${$var}, use this format: ${!var}

So if you have:


then ${!BAR} will return "one"

This only works for referencing an existing variable, not for assignment. If you wish to assign to a dynamically generated variable name, you must use either eval:

eval IP_${IFACE}=

or, in many cases, just use an array:

ROW[N]="This is the 18th row (counting from 0)."

The eval trick can also be used for referencing:

eval echo \$IP_${IFACE}

The eval makes a second pass over the code. The first pass substitutes $ for \$ and then substitutes the value of ${IFACE}, leaving "eval echo $IP_eth0". The second pass then substitutes the value of the ${IP_eth0} variable, assuming one exists.

How do I count the lines of files with certain extensions in a certain directory?

find $dir -name *.[$extensions] -exec cat \{\} \; | wc -l

finds the files in $dir with any of the extensions $extensions and reads the files using cat using 'wc -l' to count the lines of text.

You can so make

wc -l $dir/*.$extensions