Contents
-
Shell Scripting Tricks
- How do I see how much disk space is left?
- How do I see where all my big files are? My file system is full.
- I have a directory full of MP3s... how do I rename them all with underscores instead of spaces?
- How do I do that recursively?
- How do I (recursively) rename all files from uppercase to lowercase?
- How do I remove a file that begins with "-"?
- How do I monitor something in realtime?
- How do I get only the filename in a full pathname?
- How do I test whether a directory has files in it?
- How do I start a process in background?
- How do I work with arrays in bash?
- How do I use a variable within a variable (variable interpolation) in bash?
- How do I count the lines of files with certain extensions in a certain directory?
- How do I find longest file name in a directory?
Shell Scripting Tricks
Note: Debian Wiki is not necessarily the best place to find help with shell scripts. If your scripting problem involves bash (or Bourne shell), you might try bash page instead. A bash FAQ is at http://mywiki.wooledge.org/BashFAQ and contains many more tricks than this page does.
How do I see how much disk space is left?
df df -h # for a human readable format in megs & gigs
How do I see where all my big files are? My file system is full.
cd /somewhere du -sk * | sort -n # Repeat as necessary # or in human readable du -sch * | sort -h
If you want to see big directories instead of files, you can do:
du -x /somewhere | sort -n | tail -10 # This shows the 10 largest directories under /somewhere
Or visually with an interactive TUI, after installing ncdu:
ncdu /somewhere
Or an alternative, with duc:
duc index /somewhere duc ui /somewhere # interactive TUI duc gui /somewhere # interactive X11 sunburst duc graph /somewhere # generate PNG sunburst
Another way:
find /somewhere -size +2000k -ls # This shows all files over 2000 kilobytes under /somewhere
If you want to keep your .debs, you can filter them out of the list:
find / -size +2000k ! -name "*.deb" -ls # This shows all files on your entire system over 2000 kilobytes that are not debian packages
I have a directory full of MP3s... how do I rename them all with underscores instead of spaces?
rename 's/ /_/g' *.mp3 # if you want to copy them to a vfat mp3 player: detox -r
The rename command is not a general Unix command but it is available in the rename (it was included with Perl until Stretch, which was of course by default in Debian).
How do I do that recursively?
That's a bit tricky. You can get a list of files to pass to rename using find, but if some of the directories are being renamed as well, that's not something rename can keep track of. You have to use -depth with find to make sure any files are renamed before the directories they're in.
cd /somewhere find . -depth -name '* *' -type f -print0 | xargs -r0 rename 's/ /_/g'
How do I (recursively) rename all files from uppercase to lowercase?
Once again, this is hard if the directories themselves have uppercase letters in their names. Let's assume for the moment that they don't. Then:
cd /somewhere find . -name '*[A-Z]*' -type f -print0 | xargs -0 rename 'y/A-Z/a-z/'
Unfortunately, whenever this question pops up, that's usually not enough -- the person asking usually has uppercase directory names too. So something like this might be required:
cd /somewhere find . -type d -depth -name '*[A-Z]*' -print | while read dir; do dname="$(dirname $dir)"; bname="$(basename $dir)"; newbname="$(echo $bname | tr [:upper:] [:lower:])"; mv "$dir" "$dname/$newbname"; done # That renames the directories. The -depth makes it go through the deepest ones first, so we don't rename # A to a until we've already renamed A/B to A/b. find . -name '*[A-Z]*' -type f -print0 | xargs -0 rename 'y/A-Z/a-z/'
Note that the more complex script above is imperfect: it will fail if any directories have newlines in their names, and it may also fail under some conditions if any files or directories have whitespace in their names. Use at your own risk. (Hint: if you want to test it before you commit to it, put an echo in front of the mv.)
In fact, this simpler solution might work better:
find /somewhere -depth -name '*[A-Z]*' -print0 | xargs -r0 rename 'y/A-Z/a-z/'
Here, too, it might be a good idea to put in an echo before the rename command to test before you actually run this.
How do I remove a file that begins with "-"?
unlink -foo
Three different ways:
rm -- -foo rm ./-foo using 'mc', and pressing F8 on the appropriate file
These two tricks work with nearly every command line tool, not only rm.
And with the realy tough cases.
ls -i find . -xdev -inum $INODE -delete
How do I monitor something in realtime?
If it's a log file, follow the tail of the file:
tail -f /var/log/messages
or
less +F /var/log/messages
If it's a general command, use watch:
watch -n 1 ls -l ~/some/file
This works well when monitoring a firewall with iptables / tc or like tasks.
How do I get only the filename in a full pathname?
Two ways:
basename /path/to/file foo=/path/to/file ; echo ${foo##*/}
How do I test whether a directory has files in it?
if [ "$(ls -A somedir)" ]; then echo "It has files" fi
Note: a single argument inside the brackets is equivalent to [ -n argument ]. There seems to be no clean way to do this using only shell builtins. (We had one earlier attempt which was close, but that failed if there was one file named * in the directory.)
Another way:
if [ "$(ls -A somedir | wc -l)" -gt 0 ]; then echo "found files" fi
A shorter way: [ $(ls -A somedir) ] && echo "directory is not empty"
Note: The double ampersand works like a boolean "and", so the "if" is not required.
How do I start a process in background?
Just append an ampersand:
sleep 100 &
It prints a line that looks like: [1] 6338 , indicating that this process is job number 1 and that its process ID is 6338.
To kill this job:
kill %1
or
kill 6338
Note: If you have started many processes in background and you want to list them, type jobs.
You can also switch a job running in the background into the foreground, then suspend this job, and then restart this job to the background. Let's start a process in the background:
sleep 100 &
To move it to the foreground (assuming that this is job number 1):
fg %1
Then, press Ctrl+Z. This suspends the process and send it to the background. Now, restart the suspended process while leaving it in the background:
bg %1
To kill this job:
kill %1
How do I work with arrays in bash?
Assigning to one:
arr=(one two three four) myoggs=(*.ogg)
Iterating over one:
for f in "${myoggs[@]}"; do ...; done
Selecting a random element:
mysigs=($HOME/sigs/*) cat ${mysigs[RANDOM % ${#mysigs[*]}]}
(In the previous example, note that the brackets for the array index also force a numeric evaluation context, in which RANDOM doesn't need a leading $ sign.)
You can't easily "delete" an element from the middle of an array, but you can hack around it by reindexing the array to skip over all the unset elements:
arr=(zero one two three four) unset arr[1] arr=("${arr[@]}")
This will also leave all the empty elements in place, deleting only the unset ones.
How do I use a variable within a variable (variable interpolation) in bash?
If you want to do something like ${$var}, use this format: ${!var}
So if you have:
FOO=one BAR=FOO
then ${!BAR} will return "one"
This only works for referencing an existing variable, not for assignment. If you wish to assign to a dynamically generated variable name, you must use either eval:
IFACE=eth0 eval IP_${IFACE}=192.168.1.1
or, in many cases, just use an array:
N=17 ROW[N]="This is the 18th row (counting from 0)."
The eval trick can also be used for referencing:
IFACE=eth0 eval echo \$IP_${IFACE}
The eval makes a second pass over the code. The first pass substitutes $ for \$ and then substitutes the value of ${IFACE}, leaving "eval echo $IP_eth0". The second pass then substitutes the value of the ${IP_eth0} variable, assuming one exists.
How do I count the lines of files with certain extensions in a certain directory?
find $dir -name *.[$extensions] -exec cat \{\} \; | wc -l
finds the files in $dir with any of the extensions $extensions and reads the files using cat using 'wc -l' to count the lines of text.
You can so make
wc -l $dir/*.$extensions
How do I find longest file name in a directory?
find $dir -type f | awk -F/ 'BEGIN {l=0;d=""}length($NF)>l{l=length($NF);d=$NF;} END {print d}'
Change the type from f (regular file) to d (directory) to do the same but for the directories. To find the longest path, remove the option '-F' and delimiter '/' after awk.
CategorySystemAdministration | CategoryCommandLineInterface | CategoryRedundant: merge some parts with other shell/CLI pages (?ShellScripting)