Thread: [Linuxcommand-announce] [LinuxCommand.org: Tips, News And Rants] New Features In Bash Version 4.x -

As I mention in the introduction to The Linux Command Line, the command
line is a long lasting skill. It's quite possible that a script that
you wrote ten years ago still works perfectly well today. But even so,
every few years the GNU Project releases a new version of bash. While I
was writing the book, version 3.2 was the predominate version found in
Linux distributions. In February of 2009 however a new major version of
bash (4.0) appeared and it began to show up in distributions last fall.
Today the current version of bash is 4.1 and it, too ,is beginning to
show up in distributions such as Ubuntu 10.04 and (I presume, since I
haven't checked yet) Fedora 13.

So what's new in bash? A bunch of things, though most of them tend to
be rather small. In this series we will look at features that, I feel,
are of the most use to ordinary shell users starting with a couple of
the small ones.

Finding Your Version

How do you know if you are using the latest, greatest bash? By issuing
a command of course:

me@linuxbox: ~$ echo $BASH_VERSION
4.1.2(1)-release

bash maintains a shell variable called BASH_VERSION that always
contains the version number of the shell in use. The example above is
from my Ubuntu 10.04 test machine and it dutifully reveals that we are
running bash version 4.1.2.

Better help

The help command, which is used to display documentation for the
shell's builtin commands, got some much needed attention in the new
version of bash. The command has some new options and the help text
itself has been reformatted and improved. For example, here is the
result of the help cd command in bash 3.2:

bshotts@twin2:~$ help cd
cd: cd [-L|-P] [dir]
Change the current directory to DIR. The variable $HOME is the
default DIR. The variable CDPATH defines the search path for
the directory containing DIR. Alternative directory names in CDPATH
are separated by a colon (:). A null directory name is the same as
the current directory, i.e. `.'. If DIR begins with a slash (/),
then CDPATH is not used. If the directory is not found, and the
shell option `cdable_vars' is set, then try the word as a variable
name. If that variable has a value, then cd to the value of that
variable. The -P option says to use the physical directory structure
instead of following symbolic links; the -L option forces symbolic links
to be followed.

The same command in bash 4.1:

bshotts@twin7:~$ help cd
cd: cd [-L|-P] [dir]
Change the shell working directory.

Change the current directory to DIR. The default DIR is the value of the
HOME shell variable.

The variable CDPATH defines the search path for the directory containing
DIR. Alternative directory names in CDPATH are separated by a colon (:).
A null directory name is the same as the current directory. If DIR
begins
with a slash (/), then CDPATH is not used.

If the directory is not found, and the shell option `cdable_vars' is
set,
the word is assumed to be a variable name. If that variable has a value,
its value is used for DIR.

Options:
-L force symbolic links to be followed
-P use the physical directory structure without following symbolic
links

The default is to follow symbolic links, as if `-L' were specified.

Exit Status:
Returns 0 if the directory is changed; non-zero otherwise.

As you can see, the output is more "man page-like" than the previous
version, as well as better written. help also includes two new options,
-d which displays a short description of the command and the -m option
which displays the help text in full man page format.

New Redirections

It is now possible to combine both standard output and standard error
from a command and append it to a file using this form:

command &>> file

Likewise, it is possible to pipe the combined output streams using this:

command1 |& command2

where the standard output and standard error of command1 is piped into
the standard input of command2. This form may be used in place of the
traditional form:

command1 2>&1 | command2

Further Reading

Here are some change summaries for the 4.x version of bash:

- http://bash-hackers.org/wiki/doku.php/bash4
- http://tiswww.case.edu/php/chet/bash/NEWS




--
Posted By William Shotts to LinuxCommand.org: Tips, News And Rants at
4/06/2010 05:15:00 PM

In this installment, we are going to look at a couple of commands that
have been updated in bash 4.x

read Enhancements

The read command has gotten several small improvements. There is one
however that I thought was a real standout. You can now provide a
default value that will be accepted if the user presses the Enter key.
The new -i option is followed by a string containing the default text.
Note that for this option to work, the -e option (which enables the
readline library) must also be specified. Here is an example script:

#!/bin/bash

# read4: demo new read command feature

read -e -p "What is your user name? " -i $USER
echo "You answered: '$REPLY'"

When the script is executed, the user is prompted to answer, but a
default value is supplied which may be edited if desired:

bshotts@twin7:~$ read4
What is your user name? bshotts
You answered: 'bshotts'

case Improvements

The case compound command has been made more flexible. As you may
recall, case performs a multiple choice test on a string. In versions
of bash prior to 4.x, case allowed only one action to be performed on a
successful match. After a successful match, the command would
terminate. Here we see a script that tests a character:

#!/bin/bash

# case4-1: test a character

read -n 1 -p "Type a character > "
echo
case $REPLY in
[[:upper:]]) echo "'$REPLY' is upper case." ;;
[[:lower:]]) echo "'$REPLY' is lower case." ;;
[[:alpha:]]) echo "'$REPLY' is alphabetic." ;;
[[:digit:]]) echo "'$REPLY' is a digit." ;;
[[:graph:]]) echo "'$REPLY' is a visible character." ;;
[[:punct:]]) echo "'$REPLY' is a punctuation symbol." ;;
[[:space:]]) echo "'$REPLY' is a whitespace character." ;;
[[:xdigit:]]) echo "'$REPLY' is a hexadecimal digit." ;;
esac

Running this script produces this:

bshotts@twin7:~$ case4-1
Type a character > a
'a' is lower case.

The script works for the most part, but fails if a character matches
more than one of the POSIX characters classes. For example the
character "a" is both lower case and alphabetic, as well as a
hexadecimal digit. In bash prior to 4.x there was no way for case to
match more than one test. In bash 4.x however, we can do this:

#!/bin/bash

# case4-2: test a character

read -n 1 -p "Type a character > "
echo
case $REPLY in
[[:upper:]]) echo "'$REPLY' is upper case." ;;&
[[:lower:]]) echo "'$REPLY' is lower case." ;;&
[[:alpha:]]) echo "'$REPLY' is alphabetic." ;;&
[[:digit:]]) echo "'$REPLY' is a digit." ;;&
[[:graph:]]) echo "'$REPLY' is a visible character." ;;&
[[:punct:]]) echo "'$REPLY' is a punctuation symbol." ;;&
[[:space:]]) echo "'$REPLY' is a whitespace character." ;;&
[[:xdigit:]]) echo "'$REPLY' is a hexadecimal digit." ;;&
esac

Now when we run the script, we get this:

bshotts@twin7:~$ case4-2
Type a character > a
'a' is lower case.
'a' is alphabetic.
'a' is a visible character.
'a' is a hexadecimal digit.

The addition of the ";;&" syntax allows case to continue on to the next
test rather than simply terminating. There is also a ";&" syntax which
permits case to continue on to the next action regardless of the
outcome of the next test.

Further Reading

The bash man page:

- The Compound Commands subsection of the SHELL GRAMMAR section.
- The SHELL BUILTIN COMMANDS section.The Bash Hackers Wiki:

- http://bash-hackers.org/wiki/doku.php/commands/builtin/read
- http://bash-hackers.org/wiki/doku.php/syntax/ccmd/caseThe Bash
Reference Manual

- http://www.gnu.org/software/bash/manual/bashref.html#Bash-Builtins
-
http://www.gnu.org/software/bash/manual/bashref.html#Conditional-Constructs


--
Posted By William Shotts to LinuxCommand.org: Tips, News And Rants at
4/13/2010 05:12:00 PM

In this final installment of our series, we will look at perhaps the
most significant area of change in bash version 4.x: arrays.

Arrays In bash

Arrays were introduced in version 2 of bash about fifteen years ago.
Since then, they have been on the fringes of shell programming. I, in
fact, have never seen a shell script "in the wild" that used them. None
of the scripts on LinuxCommand.org, for example, use arrays.

Why is this? Arrays are widely used in other programming languages. I
see two reasons for this lack of popularity. First, arrays are not a
traditional shell feature. The Bourne shell, which bash was designed to
emulate and replace, offers no array support at all. Second, arrays in
bash are limited to a single dimension, an unusual limitation given
that virtually every other programming language supports
multi-dimensional arrays.

Some Background

I devote a chapter in my book to bash arrays but briefly, bash supports
single dimension array variables. Arrays behave like a column of
numbers in a spread sheet. A single array variable contains multiple
values called elements. Each element is accessed via an address called
an index or subscript. All versions of bash starting with version 2
support integer indexes. For example, to create a five element array in
bash called numbers containing the strings "zero" through "four", we
would do this:

bshotts@twin7:~$ numbers=(zero one two three four)

After creating the array, we can access individual elements by
specifying the array element's index:

bshotts@twin7:~$ echo ${numbers[2]}
two

The braces are required to prevent the shell from misinterpreting the
brackets as wildcard characters used by pathname expansion.

Arrays are not very useful on the command line but are very useful in
programming because they work well with loops. Here's an example using
the array we just created:

#!/bin/bash

# array-1: print the contents of an array

numbers=(zero one two three four)

for i in {0..4}; do
echo ${numbers[$i]}
done

When executed, the script prints each element in the numbers array:

bshotts@twin7:~$ array-1
zero
one
two
three
four

mapfile Command

bash version 4 added the mapfile command. This command copies a file
line-by-line into an array. It is basically a substitute for the
following code:

while read line
array[i]="$line"
i=$((i + 1))
done < file

with mapfile, you can use the following in place of the code above:

mapfile array < file

mapfile handles the case of a missing newline at the end of the file
and creates empty array elements when it encounters a blank line in the
file. It also supports ranges within the file and the array.

Associative Arrays

By far, the most significant new feature in bash 4.x is the addition of
associative arrays. Associative arrays use strings rather than integers
as array indexes. This capability allow interesting new approaches to
managing data. For example, could create an array called colors and use
color names as indexes:

colors["red"]="#ff0000"
colors["green"]="#00ff00"
colors["blue"]="#0000ff"

Associative array elements are accessed in much the same way as integer
indexed arrays:

echo ${colors["blue"]}

In the script that follows, we will look at several programming
techniques that can be employed in conjunction with associative arrays.
This script, called array-2, when given the name of a directory, prints
a lsting of the files in the directory along with the names of the the
file's owner and group owner. At the end of listing, the script prints
a tally of the number of files belonging to each owner and group. Here
we see the results (truncated for brevity) when the script is given the
directory /usr/bin:

bshotts@twin7:~$ array-2 /usr/bin
/usr/bin/2to3-2.6 root root
/usr/bin/2to3 root root
/usr/bin/a2p root root
/usr/bin/abrowser root root
/usr/bin/aconnect root root
/usr/bin/acpi_fakekey root root
/usr/bin/acpi_listen root root
/usr/bin/add-apt-repository root root
.
.
.
/usr/bin/zipgrep root root
/usr/bin/zipinfo root root
/usr/bin/zipnote root root
/usr/bin/zip root root
/usr/bin/zipsplit root root
/usr/bin/zjsdecode root root
/usr/bin/zsoelim root root

File owners:
daemon : 1 file(s)
root : 1394 file(s)

File group owners:
crontab : 1 file(s)
daemon : 1 file(s)
lpadmin : 1 file(s)
mail : 4 file(s)
mlocate : 1 file(s)
root : 1380 file(s)
shadow : 2 file(s)
ssh : 1 file(s)
tty : 2 file(s)
utmp : 2 file(s)

Here is a listing of the script:

1 #!/bin/bash
2
3 # array-2: Use arrays to tally file owners
4
5 declare -A files file_group file_owner groups owners
6
7 if [[ ! -d "$1" ]]; then
8 echo "Usage: array-2 dir" >&2
9 exit 1
10 fi
11
12 for i in "$1"/*; do
13 owner=$(stat -c %U "$i")
14 group=$(stat -c %G "$i")
15 files["$i"]="$i"
16 file_owner["$i"]=$owner
17 file_group["$i"]=$group
18 ((++owners[$owner]))
19 ((++groups[$group]))
20 done
21
22 # List the collected files
23 { for i in "${files[@]}"; do
24 printf "%-40s %-10s %-10s\n" \
25 "$i" ${file_owner["$i"]} ${file_group["$i"]}
26 done } | sort
27 echo
28
29 # List owners
30 echo "File owners:"
31 { for i in "${!owners[@]}"; do
32 printf "%-10s: %5d file(s)\n" "$i" ${owners["$i"]}
33 done } | sort
34 echo
35
36 # List groups
37 echo "File group owners:"
38 { for i in "${!groups[@]}"; do
39 printf "%-10s: %5d file(s)\n" "$i" ${groups["$i"]}
40 done } | sort

Line 5: Unlike integer indexed arrays, which are created by merely
referencing them, associative arrays must be created with the declare
command using the new -A option. In this script we create five arrays
as follows:

- files contains the names of the files in the directory, indexed by
file name
- file_group contains the group owner of each file, indexed by file name
- file_owner contains the owner of each file, indexed by file name
- groups contains the number of files belonging to the indexed group
- owners contains the number of files belonging to the indexed owner
Lines 7-10: Checks to see that a valid directory name was passed as a
positional parameter. If not, a usage message is displayed and the
script exits with an exit status of 1.

Lines 12-20: Loop through the files in the directory. Using the stat
command, lines 13 and 14 extract the names of the file owner and group
owner and assign the values to their respective arrays (lines 16, 17)
using the name of the file as the array index. Likewise the file name
itself is assigned to the files array (line 15).

Lines 18-19: The total number of files belonging to the file owner and
group owner are incremented by one.

Lines 22-27: The list of files is output. This is done using
the "${array[@]}" parameter expansion which expands into the entire
list of array element with each element treated as a separate word.
This allows for the possibility that a file name may contain embedded
spaces. Also note that the entire loop is enclosed in braces thus
forming a group command. This permits the entire output of the loop to
be piped into the sort command. This is necessary because the expansion
of the array elements is not sorted.

Lines 29-40: These two loops are similar to the file list loop except
that they use the "${!array[@]}" expansion which expands into the list
of array indexes rather than the list of array elements.

Further Reading
The Linux Command Line

- Chapter 36 (Arrays)
- Chapter 37 (Group commands)Other bash 4.x references:

- http://tldp.org/LDP/abs/html/bashver4.html
- http://mywiki.wooledge.org/BashFAQ/005A Wikipedia article on
associative arrays:

- http://en.wikipedia.org/wiki/Associative_arrayThe Complete HTML Color
Chart:

- http://www.immigration-usa.com/html_colors.html



--
Posted By William Shotts to LinuxCommand.org: Tips, News And Rants at
4/22/2010 04:00:00 PM