Ggthjy

This is my rewrite of the program posted here, based on janos's comments. I also added the ability to read from stdin.

I've also included a test script. I wanted something lightweight I could easily re-run, but didn't want a dependency or full test framework like bats. Comments on the test script are also welcome.

New Program

#!/bin/bash



set -euo pipefail



die() {

  printf '%sn' "$1" >&2

  exit 1

}



show_help() {

>&2 echo "Usage: 



  $ $0 <filename>



The program will read from stdin if <filename> is not given.



Description:



  Fills in template files with ENV variables.  <filename> is assumed to be a

  template file whose variables are enclosed in double braces like:



  Some content with {{ MY_VAR1 }}

  or {{ MY_VAR1 }} and {{ MY_VAR2 }}



  where MY_VAR1 and MY_VAR2 and ENV variables.  Assuming that the ENV variables

  are set:



  $ export MY_VAR1=value_1

  $ export MY_VAR2=value_2



  then executing this script on a file with the above content will output:



  Some content with value_1

  or value_1 and value_2

"

}



if [[ $# -gt 0 && ( $1 = '-h' || $1 = '--help' ) ]]; then

  show_help

  exit 0

fi



# If given, ensure arg is a file:



if [[ $# -gt 0 && ! -f $1 ]]; then

  die "'$1' is not a file." 

fi



# If we're reading from stdin, save its contents to a temp file

# This is because we need to read it twice: first to extract 

# the required vars, then again to replace them.

if [[ $# -eq 0 ]]; then

  # Read stdin into a temp file

  tmpfile=$(mktemp)

  cat /dev/stdin > "$tmpfile"



  # Clean it up, saving to FDs to read from

  exec 3< "$tmpfile"

  exec 4< "$tmpfile"

  rm "$tmpfile"

else

  exec 3< "$1"

  exec 4< "$1"

fi



# Gather all the required template variables



vars=()

while IFS= read -r line; do

  vars+=( "$line" )

done < <( grep -Eo '{{ ([-_[:alnum:]]*) }}' <&3 | 

          grep -Eo '([-_[:alnum:]]*)' | 

          sort -u )



# Verify that all template variables exist



missing=()

for var in "${vars[@]}"; do

  if [[ -z ${!var+x} ]]; then

    missing+=( "$var" )

  fi

done



if [[ ${#missing[@]} -gt 0 ]]; then

  >&2 echo "The following required variables have not been exported:"

  for var in "${missing[@]}"; do

    >&2 echo "${var}"

  done

  exit 1

fi



# Dynamically construct the sed cmd to do the replacement



sed_cmd=

for var in "${vars[@]}"; do

  # sanitize the user's input (ie, the var's value) by prepending a backslash

  # to our sed delimiter (#) as well as to backslashes themselves, to prevent

  # the input from being interpreted by sed as a special character like a

  # backreference (1) or a tab (t), etc

  escaped_val=$(printf "%s" "${!var}" | sed -E 's|([#])|\1|g')

  sed_cmd+="s#\{\{ ${var} }}#${escaped_val}#g;"

done



sed -E "${sed_cmd}" <&4

Test Script

#!/bin/bash

# shellcheck disable=SC2030,SC2031

# disabling because we're modifying in a subshell by design



set -euo pipefail



# Do everything in a sub-process to keep parent process clean

(



new_test() {

  echo "$1"

  unset MY_VAR1

  unset MY_VAR2

}



  basic_tmpl='

hello there {{ MY_VAR1 }}

some other stuff



foo: {{ MY_VAR2 }}

line with both: {{ MY_VAR1 }} and {{ MY_VAR2 }}'



# Each test goes in a subshell too

(

  new_test "Should error when tmpl variables are missing"



  printf "%s" "$basic_tmpl" | ./fill_template 2>/dev/null 

    && echo 'FAIL' || echo 'PASS'

)



(

  new_test "Should succeed when tmpl variables are set"



  export MY_VAR1=val1

  export MY_VAR2=val2



  printf "%s" "$basic_tmpl" | ./fill_template >/dev/null 2>&1 

    && echo 'PASS' || echo 'FAIL'

)



(

  new_test "Basic template should produce expected output"



  export MY_VAR1=val1

  export MY_VAR2=val2



  result=$(printf '%s' "$basic_tmpl" | ./fill_template 2> /dev/null)

  expected='

hello there val1

some other stuff



foo: val2

line with both: val1 and val2'



  [[ "$result" = "$expected" ]] && echo 'PASS' || echo 'FAIL'

)



(

  new_test "Values with spaces/slashes/metachars still work"



  export MY_VAR1='/some/path/and_1_t+_'

  export MY_VAR2='blah _\_ baz'



  result=$(printf '%snn' "$basic_tmpl" | ./fill_template 2> /dev/null)

  expected='

hello there /some/path/and_1_t+_

some other stuff



foo: blah _\_ baz

line with both: /some/path/and_1_t+_ and blah _\_ baz'



  [[ "$result" = "$expected" ]] && echo 'PASS' || echo 'FAIL'

)



)

The grep matches a lot of strings that can't be valid bash identifiers ("alphanumeric characters and underscores, beginning with an alphabetic character or an underscore"). There are some magical exceptions like $* and $@ but I'm assuming you aren't trying to support those in your templates.

Reading inputs twice is not necessary; a single-pass approach would remove a lot of complexity from your program, and make it possible to process infinitely large inputs on STDIN. The only real drawback is that it will exit on the first undefined value, instead of producing a list.

And finally, everyone has their own definitions, but—to me—"pure bash" means "without the use of external programs." Making your program meet this bar not only makes it faster, it removes even more complexity, because you don't need to filter values through an external program that might misunderstand them. It also means your template values need not be exported. They only need to be visible to the current shell.

This version, implemented as a function, uses only bash builtins and passes your tests once ./fill_template is replaced by fill_template:

fill_template() {

  (( ${#@} )) || set -- /dev/stdin

  local file line eof original name value

  for file; do

    while true; do

      read -r line

      eof=$?

      while [[ $line =~ {{" "*([a-zA-Z_][_a-zA-Z0-9]*)" "*}} ]]; do

        original=$BASH_REMATCH

        name=${BASH_REMATCH[1]}

        value=${!name?"unset template variable: $name"}

        line=${line//$original/$value}

      done

      printf -- %s "$line"

      (( eof )) && break || printf "n"

    done <$file

  done

}