bash - Torque PBS passing environment variables that contain quotes -

i have python script. run this:

./make_graph data_directory "wonderful graph title" 

i have run script through scheduler. using -v pass arguments python script through qsub.

qsub make_graph.pbs -v args="data_directory \"wonderful graph title\"" 

i have tried many combinations of ', ", \" escaping , can't right. quoting around 'wonderful graph title' either lost or mangled.

here excerpt pbs script

if [ -z "${args+xxx}" ];         echo "no args specified!"         exit 1 fi  cmd="/path/make_graph $args" echo "cmd: $cmd"  echo "job started on `hostname` @ `date`" ${cmd} 

what proper way pass string parameter contains spaces through qsub environment variable? there better way this? maybe more general bash problem.

update: answer based on sge qsub rather torque qsub, cli different. in particular, torque qub doesn't seem support direct argument passing, second approach doesn't work.

---

this problem of proper quoting , has little grid engine submission itself. if want fix current script, should use eval "${cmd}" rather ${cmd}. here's detailed analysis of happens when ${cmd} alone (in analysis assume there's nothing funny in path):

1. your qsub command line processed , quotes removed, args environment variable passed data_directory "wonderful graph title".

2. you did cmd="/path/make_graph $args", value of cmd /path/make_graph data_directory "wonderful graph title" (i'm presenting string literal without quoting, is, value literally contains quote characters).

3. you did ${cmd}. bash performs parameter expansion on this, amounts to:

   1. expanding ${cmd} value /path/make_graph data_directory "wonderful graph title";
   2. since ${cmd} not quoted, perform word splitting, in end command line has 5 words: /path/make_graph, data_directory, "wonderful, graph, title". last 4 treated arguments make_graph, not want.

on other hand, if use eval "${cmd}", if typed /path/make_graph data_directory "wonderful graph title" interactive shell, desired behavior.

you should read more eval, parameter expansion, etc. in bash reference manual.

the corrected script:

#!/usr/bin/env bash [[ -z ${args+xxx} ]] && { echo "no args specified!" >&2; exit 1; }  cmd="/path/make_graph ${args}" echo "cmd: ${cmd}"  echo "job started on $(hostname) @ $(date)" # backticks deprecated eval "${cmd}" 

by way, test this, don't need submit grid engine; do

args="data_directory \"wonderful graph title\"" bash make_graph.pbs 

---

okay, pointed out what's wrong , patched it. "proper way" pass arguments grid engine jobs? no, don't think so. arguments arguments, , should not confused environment variables. qsub allows pass arguments directly (qsub synopsis: qsub [ options ] [ command | -- [ command_args ]]), why encode them in env var , end worrying quoting?

here's better way write submission script:

#!/usr/bin/env bash [[ $# == 0 ]] && { echo "no args specified!" >&2; exit 1; }  cmd="/path/make_graph $@" echo "cmd: ${cmd}"  echo "job started on $(hostname) @ $(date)" # backticks deprecated /path/make_graph "$@" 

here "$@" equivalent "$1" "$2" ... — faithfully passing arguments (see relevant section in bash reference manual).

one thing unfortunate this, though, although command executed correct, 1 printed may not quoted. instance, if do

qsub make_graph.pbs data_directory "wonderful graph title" 

then gets executed make_graph.pbs data_directory "wonderful graph title", printed cmd make_graph.pbs data_directory wonderful graph title. , there's no easy way fix this, far know, since quotes removed arguments no matter how word splitting done. if command printed important you, there 2 solutions:

1. use dedicated "shell escaper" (pretty easy write 1 yourself) quote arguments before printing;

2. use scripting language shell quoting readily available, e.g., python (shlex.quote) or ruby (shellwords.shellescape).