I just want to get all the files in a certain directory into a bash array (assuming that none of the files have a newline in the name):
So:
myarr=()
find . -maxdepth 1 -name "mysqldump*" | mapfile -t myarr; echo "${myarr[@]}"Empty result!
If I do the roundabout way of using a file, temporary or otherwise:
myarr=()
find . -maxdepth 1 -name "mysqldump*" > X
mapfile -t myarray < X
echo "${myarray[@]}"Result!
But why doesn't mapfile read properly from a pipe?
5 Answers
From man 1 bash:
Each command in a pipeline is executed as a separate process (i.e., in a subshell).
Such subshells inherit variables from the main shell but they are independent. This means mapfile in your original command operates on its own myarr. Then echo (being outside the pipe) prints empty myarr (which is the main shell's myarr).
This command works differently:
find . -maxdepth 1 -name "mysqldump*" | { mapfile -t myarr; echo "${myarr[@]}"; }In this case mapfile and echo operate on the same myarr (which is not the main shell's myarr).
To change the main shell's myarr you have to run mapfile in the main shell exactly. Example:
myarr=()
mapfile -t myarr < <(find . -maxdepth 1 -name "mysqldump*")
echo "${myarr[@]}"Bash runs the commands of a pipeline in a subshell environment, so any variable assignments etc. that take place within it aren't visible to the rest of the shell.
Dash (Debian's /bin/sh) as well as busybox's sh are similar, while zsh and ksh
run the last part in the main shell. In Bash, you can use shopt -s lastpipe to do the same, but it only works when job control is disabled, so not in interactive shells by default.
So:
$ bash -c 'x=a; echo b | read x; echo $x'
a
$ bash -c 'shopt -s lastpipe; x=a; echo b | read x; echo $x'
b(read and mapfile have the same issue.)
Alternatively (and as mentioned by Attie), use process substitution, which works like a generalized pipe, and is supported in Bash, ksh and zsh.
$ bash -c 'x=a; read x < <(echo b); echo $x'
bPOSIX leaves it unspecified if the parts of a pipeline run in subshells or not, so it can't really be said that any of the shells would be "wrong" in this.
2As Kamil has pointed out, each element in the pipeline is a separate process.
You can use the following process substitution to get find to run in a different process, with the mapfile invocation remaining in your current interpreter, allowing access to myarr afterwards:
myarr=()
mapfile -t myarr < <( find . -maxdepth 1 -name "mysqldump*" )
echo "${myarr[@]}"b < <( a ) will act similarly to a | b in terms of how the pipeline is wired - the difference is that b is executed "here".
If we use:
myarr=()
mapfile -t myarr < <( find . -maxdepth 1 -name "mysqldump*" )
echo "${myarr[@]}"${myarr[@]} is not lost but $? of <(find..) is lost
If we use:
find . -maxdepth 1 -name "mysqldump*" | mapfile -t myarr${myarr[@]} is lost but $? of find is not lost
if we use
shopt -s lastpipe
find . -maxdepth 1 -name "mysqldump*" | mapfile -t myarr${myarr[@]} is not lost, $? of find is not lost but you need to disable bash job control (set +m)
if we use something like this
line=$(find . -maxdepth 1 -name "mysqldump*"); rc=$?; mapfile -d '\n' myarr <<< "$line"${myarr[@]} is not lost, $? of find is not lost, bash job control (set -m) does not need to be disabled ... but you you can't use \0 null byte as delimiter (find ... -print0 | mapfile -d '' ...)
The only way I have found come from and need bash 4.4.x
exec {psfd}< <(find . -maxdepth 1 -name "mysqldump*" -print0); procsub_pid=$!
mapfile -d '\0' myarr <&$psfd
exec {psfd}<&-
wait "$procsub_pid"
echo $?
echo "${myarr[@]}"${myarr[@]} is not lost, $? of find is not lost, bash job control (set -m) does not need to be disabled and \0 null byte can be used as delimiter.
I don't know why, per se, but looking at shellcheck SC2206: Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a., I can see how to make your example work: use a herestring (<<<), like this:
# For bash mapfile -t array <<< "$var"
For your particular case, therefore, do this:
# Capture all `find` output into a regular "indexed" bash array by using
# `mapfile` and a herestring (`<<<`)
mapfile -t array <<< "$(find . -maxdepth 1 -name "mysqldump*")"
# Print all elements at once in a big blob (this also demonstrates how
# to pass all elements in the array as arguments to another function)
echo "${array[@]}"
# OR, print all elements nicely, one element per line
for element in "${array[@]}"; do echo " $element"
done
# OR, the same thing as just above but as a one-liner:
for element in "${array[@]}"; do echo " $element"; done
