1433 lines
48 KiB
Groff
1433 lines
48 KiB
Groff
.\" OpenPBS (Portable Batch System) v2.3 Software License
|
|
.\"
|
|
.\" Copyright (c) 1999-2000 Veridian Information Solutions, Inc.
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" ---------------------------------------------------------------------------
|
|
.\" For a license to use or redistribute the OpenPBS software under conditions
|
|
.\" other than those described below, or to purchase support for this software,
|
|
.\" please contact Veridian Systems, PBS Products Department ("Licensor") at:
|
|
.\"
|
|
.\" www.OpenPBS.org +1 650 967-4675 sales@OpenPBS.org
|
|
.\" 877 902-4PBS (US toll-free)
|
|
.\" ---------------------------------------------------------------------------
|
|
.\"
|
|
.\" This license covers use of the OpenPBS v2.3 software (the "Software") at
|
|
.\" your site or location, and, for certain users, redistribution of the
|
|
.\" Software to other sites and locations. Use and redistribution of
|
|
.\" OpenPBS v2.3 in source and binary forms, with or without modification,
|
|
.\" are permitted provided that all of the following conditions are met.
|
|
.\" After December 31, 2001, only conditions 3-6 must be met:
|
|
.\"
|
|
.\" 1. Commercial and/or non-commercial use of the Software is permitted
|
|
.\" provided a current software registration is on file at www.OpenPBS.org.
|
|
.\" If use of this software contributes to a publication, product, or service
|
|
.\" proper attribution must be given; see www.OpenPBS.org/credit.html
|
|
.\"
|
|
.\" 2. Redistribution in any form is only permitted for non-commercial,
|
|
.\" non-profit purposes. There can be no charge for the Software or any
|
|
.\" software incorporating the Software. Further, there can be no
|
|
.\" expectation of revenue generated as a consequence of redistributing
|
|
.\" the Software.
|
|
.\"
|
|
.\" 3. Any Redistribution of source code must retain the above copyright notice
|
|
.\" and the acknowledgment contained in paragraph 6, this list of conditions
|
|
.\" and the disclaimer contained in paragraph 7.
|
|
.\"
|
|
.\" 4. Any Redistribution in binary form must reproduce the above copyright
|
|
.\" notice and the acknowledgment contained in paragraph 6, this list of
|
|
.\" conditions and the disclaimer contained in paragraph 7 in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\"
|
|
.\" 5. Redistributions in any form must be accompanied by information on how to
|
|
.\" obtain complete source code for the OpenPBS software and any
|
|
.\" modifications and/or additions to the OpenPBS software. The source code
|
|
.\" must either be included in the distribution or be available for no more
|
|
.\" than the cost of distribution plus a nominal fee, and all modifications
|
|
.\" and additions to the Software must be freely redistributable by any party
|
|
.\" (including Licensor) without restriction.
|
|
.\"
|
|
.\" 6. All advertising materials mentioning features or use of the Software must
|
|
.\" display the following acknowledgment:
|
|
.\"
|
|
.\" "This product includes software developed by NASA Ames Research Center,
|
|
.\" Lawrence Livermore National Laboratory, and Veridian Information
|
|
.\" Solutions, Inc.
|
|
.\" Visit www.OpenPBS.org for OpenPBS software support,
|
|
.\" products, and information."
|
|
.\"
|
|
.\" 7. DISCLAIMER OF WARRANTY
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. ANY EXPRESS
|
|
.\" OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
|
|
.\" OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT
|
|
.\" ARE EXPRESSLY DISCLAIMED.
|
|
.\"
|
|
.\" IN NO EVENT SHALL VERIDIAN CORPORATION, ITS AFFILIATED COMPANIES, OR THE
|
|
.\" U.S. GOVERNMENT OR ANY OF ITS AGENCIES BE LIABLE FOR ANY DIRECT OR INDIRECT,
|
|
.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
|
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
|
|
.\" OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
|
|
.\" LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
|
|
.\" NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
|
|
.\" EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
.\"
|
|
.\" This license will be governed by the laws of the Commonwealth of Virginia,
|
|
.\" without reference to its choice of law rules.
|
|
.if \n(Pb .ig Iq
|
|
.TH qsub 1B "" Local PBS
|
|
.so ../ers/ers.macros
|
|
.Iq
|
|
.SH NAME
|
|
qsub \- submit pbs job
|
|
.SH SYNOPSIS
|
|
qsub [\-a date_time] [\-A account_string] [\-b secs] [\-c checkpoint_options]
|
|
[\-C directive_prefix]
|
|
[\-d path] [\-D path] [\-e path] [\-f] [\-F] [\-h] [\-i idle_slot_limit] [\-I] [\-j join] [\-k keep] [\-K kill_delay]
|
|
[\-l resource_list] [\-L NUMA_resource_list] [\-m mail_options] [\-M user_list] [\-n node_exclusive] [\-N name]
|
|
[\-o path] [\-p priority] [\-P proxy_username[:group]] [\-q destination] [\-r c]
|
|
[\-S path_list] [\-t array_request] [\-T prologue/epilogue script_name]
|
|
[\-u user_list] [\-v variable_list] [\-V] [\-w path]
|
|
[\-W additional_attributes] [\-x] [\-X] [\-z] [script]
|
|
.SH DESCRIPTION
|
|
To create a job is to submit an executable script to a batch server.
|
|
The batch server will be the default server unless the
|
|
.Ar \-q
|
|
option is specified. See discussion of PBS_DEFAULT under Environment Variables
|
|
below.
|
|
Typically, the script is a shell script which will be executed
|
|
by a command shell such as sh or csh.
|
|
.LP
|
|
Options on the
|
|
.B qsub
|
|
command allow the specification of attributes which affect the behavior
|
|
of the job.
|
|
.if !\n(Pb .ig Ig
|
|
The job is created by sending a
|
|
.I "Queue Job"
|
|
batch request to the batch server.
|
|
.Ig
|
|
.LP
|
|
The qsub
|
|
command will pass certain environment variables in the
|
|
.At Variable_List
|
|
attribute of the job.
|
|
These variables will be available to the job.
|
|
The value for the following variables will be taken from the environment
|
|
of the qsub
|
|
command: \fBHOME\fP, \fBLANG\fP, \fBLOGNAME\fP, \fBPATH\fP, \fBMAIL\fP,
|
|
\fBSHELL\fP,
|
|
and \fBTZ\fP. These values will be assigned to a new name which is the
|
|
current name prefixed with the string "PBS_O_".
|
|
For example, the job will have access to an environment variable named
|
|
.B PBS_O_HOME
|
|
which have the value of the variable
|
|
.B HOME
|
|
in the qsub command environment.
|
|
.LP
|
|
In addition to the above, the following environment variables will be available
|
|
to the batch job.
|
|
.if !\n(Pb .ig Ig
|
|
(The values of the following environment variables are established by qsub.)
|
|
.Ig
|
|
.IP \fBPBS_O_HOST\fP
|
|
the name of the host upon which the qsub command is running.
|
|
.IP \fBPBS_SERVER\fP
|
|
the hostname of the pbs_server which qsub submits the job to.
|
|
.IP \fBPBS_O_QUEUE\fP
|
|
the name of the original queue to which the job was submitted.
|
|
.if !\n(Pb .ig Ig
|
|
(It is established by the server which creates the job, not qsub.)
|
|
.Ig
|
|
.IP \fBPBS_O_WORKDIR\fP
|
|
the absolute path of the current working directory of the qsub command.
|
|
.if !\n(Pb .ig Ig
|
|
.LP
|
|
The following are established by the server executing the job,
|
|
not the qsub command.
|
|
.Ig
|
|
.IP \fBPBS_ARRAYID\fP
|
|
each member of a job array is assigned a unique identifier (see \-t)
|
|
.IP \fBPBS_ENVIRONMENT\fP
|
|
set to
|
|
.Ty PBS_BATCH
|
|
to indicate the job is a batch job, or to
|
|
.Ty PBS_INTERACTIVE
|
|
to indicate the job is a PBS interactive job, see \-I option.
|
|
.IP \fBPBS_JOBID\fP
|
|
the job identifier assigned to the job by the batch system.
|
|
.IP \fBPBS_JOBNAME\fP
|
|
the job name supplied by the user.
|
|
.IP \fBPBS_NODEFILE\fP
|
|
the name of the file contain the list of nodes assigned to the job
|
|
(for parallel and cluster systems).
|
|
.IP \fBPBS_QUEUE\fP
|
|
the name of the queue from which the job is executed.
|
|
.SH OPTIONS
|
|
.IP "\-a date_time" 8
|
|
Declares the time after which the job is eligible for execution.
|
|
.RS
|
|
.LP
|
|
The
|
|
.Ar date_time
|
|
argument is in the form:
|
|
.Ty "[[[[CC]YY]MM]DD]hhmm[.SS]"
|
|
.LP
|
|
Where CC is the first two digits of the year (the century),
|
|
YY is the second two digits of the year, MM is the two digits for the month,
|
|
DD is the day of the month, hh is the hour, mm is the minute,
|
|
and the optional SS is the seconds.
|
|
.LP
|
|
If the month,
|
|
.Ty MM ,
|
|
is not specified, it will default to the current month if the specified day
|
|
.Ty DD ,
|
|
is in the future. Otherwise, the month will be set to next month.
|
|
Likewise, if the day,
|
|
.Ty DD ,
|
|
is not specified, it will default to today if the time
|
|
.Ty hhmm
|
|
is in the future. Otherwise, the day will be set to tomorrow.
|
|
For example, if you submit a job at 11:15am with a time of
|
|
.Ty "\-a 1110" ,
|
|
the job will be eligible to run at 11:10am tomorrow.
|
|
.if !\n(Pb .ig Ig
|
|
See the date_time operand for the touch(1) command defined by POSIX.2.
|
|
.LP
|
|
The
|
|
.At Execution_Time
|
|
job attribute will be set to the number of seconds since Epoch which
|
|
is equivalent to the Universal time expressed by the local time in the
|
|
.Ar date_time
|
|
argument.
|
|
If the
|
|
.Ar \-a
|
|
option is not specified, the
|
|
.At Execution_Time
|
|
attribute is unset which represents a time zero or no delay.
|
|
.Ig
|
|
.RE
|
|
.IP "\-A account_string" 8
|
|
Defines the account string associated with the job.
|
|
The
|
|
.Ar account_string
|
|
is an undefined string of characters and is interpreted by the server
|
|
which executes the job. See section 2.7.1 of the PBS ERS.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Account_Name
|
|
attribute is set to the account string.
|
|
If account_string is unset, it is not passed with the job to the job executor.
|
|
.Ig
|
|
.IP "\-b seconds"
|
|
Defines the maximum number of seconds qsub will block attempting to contact
|
|
pbs_server. If pbs_server is down, or for a variety of communication failures,
|
|
qsub will continually retry connecting to pbs_server for job submission. This
|
|
value overrides the
|
|
.B CLIENTRETRY
|
|
parameter in
|
|
.B torque.cfg.
|
|
This is a non-portable TORQUE extension. Portability-minded users can use the
|
|
.B PBS_CLIENTRETRY
|
|
environmental variable. A negative value is
|
|
interpreted as infinity. The default is 0.
|
|
.IP "\-c checkpoint_options"
|
|
Defines the options that will apply to the job. If the job
|
|
executes upon a host which does not support checkpoint, these options will
|
|
be ignored.
|
|
.IP
|
|
Valid checkpoint options are:
|
|
.RS
|
|
.IP none 3
|
|
No checkpointing is to be performed.
|
|
.if !\n(Pb .ig Ig
|
|
The job's
|
|
.At Checkpoint
|
|
attribute is set to the string
|
|
.Ty """none""" .
|
|
.Ig
|
|
.IP enabled 3
|
|
Specify that checkpointing is allowed but must be explicitly invoked by either the
|
|
.B
|
|
qhold
|
|
or
|
|
.B
|
|
qchkpt
|
|
commands.
|
|
.if !\n(Pb .ig Ig
|
|
The job's
|
|
.At Checkpoint
|
|
attribute is set to the string
|
|
.Ty """enabled""" .
|
|
.Ig
|
|
.IP shutdown 3
|
|
Specify that checkpointing is to be done on a job at pbs_mom shutdown.
|
|
.if !\n(Pb .ig Ig
|
|
The job's
|
|
.At Checkpoint
|
|
attribute is set to the string
|
|
.Ty """shutdown""" .
|
|
.Ig
|
|
.IP periodic 3
|
|
Specify that periodic checkpointing is enabled. The default interval is 10 minutes and can be changed by the $checkpoint_interval option in the mom config file or by specifying an interval when the job is submitted
|
|
.if !\n(Pb .ig Ig
|
|
The job's
|
|
.At Checkpoint
|
|
attribute is set to the string
|
|
.Ty """periodic""" .
|
|
.Ig
|
|
.IP interval=minutes 3
|
|
Checkpointing is to be performed at an interval of
|
|
.Ar minutes ,
|
|
which is the integer number of minutes of wall time used by the job.
|
|
This value must be greater than zero.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Checkpoint
|
|
attribute is set to the string specified by
|
|
.Ar """interval=minutes""" .
|
|
.Ig
|
|
.IP depth=number 3
|
|
Specify a number (depth) of checkpoint images to be kept in the checkpoint directory.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Checkpoint
|
|
attribute is set to the string specified by
|
|
.Ar """depth=number""" .
|
|
.Ig
|
|
.IP dir=path 3
|
|
Specify a checkpoint directory (default is /var/spool/torque/checkpoint).
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Checkpoint
|
|
attribute is set to the string specified by
|
|
.Ar """dir=path""" .
|
|
.Ig
|
|
.RE
|
|
.IP "\-C directive_prefix" 8
|
|
Defines the prefix that declares a directive to the qsub command within the
|
|
script file. See the paragraph on script directives in the
|
|
Extended Description section.
|
|
.IP
|
|
If the
|
|
.Ar \-C
|
|
option is presented with a
|
|
.Ar directive_prefix
|
|
argument that is the null string, qsub
|
|
will not scan the script file for directives.
|
|
.if !\n(Pb .ig Ig
|
|
The directive prefix is not a job attribute. It is used solely within the
|
|
qsub command.
|
|
.Ig
|
|
.IP "\-d path" 8
|
|
Defines the working directory path to be used for the job. If the
|
|
.Ar \-d
|
|
option is not specified, the default working directory is the home directory.
|
|
This option sets the environment variable PBS_O_INITDIR.
|
|
.Ig
|
|
.IP "\-D path" 8
|
|
Defines the root directory to be used for the job.
|
|
This option sets the environment variable PBS_O_ROOTDIR.
|
|
.Ig
|
|
.IP "\-e path" 8
|
|
Defines the path to be used for the standard error stream of the batch job.
|
|
The
|
|
.Ar path
|
|
argument is of the form:
|
|
.br
|
|
.Ty "\ \ \ \ [hostname:][path_name]"
|
|
.br
|
|
where
|
|
.Ty hostname
|
|
is the name of a host to which the file will be returned and
|
|
.Ty path_name
|
|
is the path
|
|
name on that host in the syntax recognized by POSIX.
|
|
The argument will be interpreted as follows:
|
|
.RS
|
|
.IP \fBpath_name\fP
|
|
Where path_name is not an absolute path name, then the qsub command will
|
|
expand the path name relative to the current working directory of the command.
|
|
The command will supply the name of the host upon which it is executing for the
|
|
.Ar hostname
|
|
component.
|
|
.IP \fBhostname:path_name\fP
|
|
Where path_name is not an absolute path name, then the qsub command will not
|
|
expand the path name relative to the current working directory of the command.
|
|
On delivery of the standard error, the path name will be expanded relative
|
|
to the user's home directory on the \fBhostname\fP system.
|
|
.IP \fBpath_name\fP
|
|
Where path_name specifies an absolute path name, then the qsub will supply
|
|
the name of the host on which it is executing for the
|
|
.Ar hostname
|
|
.IP \fBhostname:path_name\fP
|
|
Where path_name specifies an absolute path name, the path will be used as
|
|
specified.
|
|
.Ar hostname .
|
|
.IP \fBhostname:\fP
|
|
Where hostname specifies the name of the host that the file should be returned
|
|
to. The path will be the default file name.
|
|
.RE
|
|
.IP
|
|
If the
|
|
.Ar \-e
|
|
option is not specified or the \fBpath_name\fP is not specified or is specified
|
|
and is a directory, the default file name for the standard error stream
|
|
will be used. The default name has the following form:
|
|
.br
|
|
\ \ \ \ \fBjob_name.esequence_number\fP
|
|
.br
|
|
where \fBjob_name\fP is the name of the job, see
|
|
.Ar \-N
|
|
option, and \fBsequence_number\fP is the job number assigned when the
|
|
job is submitted.
|
|
.if !\n(Pb .ig Ig
|
|
This option sets the job attribute
|
|
.At Error_Path .
|
|
.Ig
|
|
.IP "\-f" 8
|
|
Specifies that the job is fault tolerant. The
|
|
.At fault_tolerant
|
|
attribute will be set to true, which indicates that the job can
|
|
survive the loss of a mom other than the "mother superior" mom
|
|
(the first node in the exec hosts )
|
|
.Ig
|
|
.IP "\-F" 8
|
|
Specifies the arguments that will be passed to the job script when the script is launched.
|
|
.br
|
|
The accepted syntax is:
|
|
.br
|
|
qsub -F "myarg1 myarg2 myarg3=myarg3value" myscript2.sh
|
|
.Ig
|
|
.IP "\-h" 8
|
|
Specifies that a user hold be applied to the job at submission time.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Hold_Types
|
|
attribute will be set to USER, "u".
|
|
If \-h is not specified, then
|
|
.At Hold_Types
|
|
is set to NONE, "n".
|
|
.Ig
|
|
.IP "\-i idle_slot_lmit" 8
|
|
Sets an idle slot limit for the job array being submitted. If this parameter is set for a non-array job, it will be rejected. Additionally, if the user requests an idle slot limit that exceeds the the server parameter's default, the job will be rejected. See also the idle_slot_limit server parameter.
|
|
.br
|
|
$ qsub -t 0-99 -i 10 script.sh
|
|
.br
|
|
The submitted array will only instantiate 10 idle jobs; instead of all 100 jobs at submission time.
|
|
.Ig
|
|
.IP "\-I" 8
|
|
Declares that the job is to be run "interactively". The job will be queued
|
|
and scheduled as any PBS batch job, but when executed, the standard input,
|
|
output, and error streams of the job are connected through qsub to the
|
|
terminal session in which qsub is running. Interactive jobs are forced
|
|
to not rerunable.
|
|
See the "Extended Description" paragraph for addition information of
|
|
interactive jobs.
|
|
.if !\n(Pb .ig Ig
|
|
.SM
|
|
The \-I option is a violation of the POSIX 1003.2d standard. Option key
|
|
letters not defined by the standard, such as I, are reserved for future
|
|
revisions of the standard. PBS can be built with the symbol
|
|
PBS_NO_POSIX_VIOLATION defined, in which case the \-I option is removed.
|
|
The interactive attribute may still be specified via the \-W option.
|
|
.NL
|
|
.Ig
|
|
.IP "\-j join" 8
|
|
Declares if the standard error stream of the job will be merged with the
|
|
standard output stream of the job.
|
|
.IP
|
|
An option argument value of
|
|
.Ty oe
|
|
directs that the two streams will be merged, intermixed, as standard output.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Join_Path
|
|
job attribute is set to "oe".
|
|
.Ig
|
|
An option argument value of
|
|
.Ty eo
|
|
directs that the two streams will be merged, intermixed, as standard error.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Join_Path
|
|
job attribute is set to "eo".
|
|
.Ig
|
|
.IP
|
|
If the
|
|
.Ar join
|
|
argument is
|
|
.Ty n
|
|
or the option is not specified,
|
|
the two streams will be two separate files.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Join_Path
|
|
job attribute is set to "n".
|
|
.Ig
|
|
.IP "\-k keep" 8
|
|
Defines which (if either) of standard output or standard error
|
|
will be retained on the execution host.
|
|
If set for a stream, this option overrides the path name for that stream.
|
|
If not set, neither stream is retained on the execution host.
|
|
.IP
|
|
The argument is either the single letter "e" or "o",
|
|
or the letters "e" and "o" combined in either order.
|
|
Or the argument is the letter "n".
|
|
.if !\n(Pb .ig Ig
|
|
Repetition of characters is permitted, but "n" may not appear in the same
|
|
option argument with the other two characters.
|
|
The attribute
|
|
.At Keep_Files
|
|
is set to the argument.
|
|
.Ig
|
|
.RS
|
|
.IP e 3
|
|
The standard error stream is to retained on the execution host.
|
|
The stream will be placed in the home directory of the user under whose
|
|
user id the job executed. The file name will be the default file name
|
|
given by:
|
|
\fBjob_name.esequence\fP
|
|
where \fBjob_name\fP is the name specified for the job, and \fBsequence\fP is
|
|
the sequence number component of the job identifier.
|
|
.if !\n(Pb .ig Ig
|
|
The attribute is set to KEEP_ERROR.
|
|
.Ig
|
|
.IP o 3
|
|
The standard output stream is to retained on the execution host.
|
|
The stream will be placed in the home directory of the user under whose
|
|
user id the job executed. The file name will be the default file name
|
|
given by: \fBjob_name.osequence\fP
|
|
where \fBjob_name\fP is the name specified for the job, and \fBsequence\fP is
|
|
the sequence number component of the job identifier.
|
|
.if !\n(Pb .ig Ig
|
|
The attribute is set to KEEP_OUTPUT.
|
|
.Ig
|
|
.IP eo 3
|
|
Both the standard output and standard error streams will be retained.
|
|
.if !\n(Pb .ig Ig
|
|
The attribute is set to "KEEP_OUTPUT\ |\ KEEP_ERROR".
|
|
.Ig
|
|
.IP oe 3
|
|
Both the standard output and standard error streams will be retained.
|
|
.if !\n(Pb .ig Ig
|
|
The attribute is set to "KEEP_OUTPUT\ |\ KEEP_ERROR".
|
|
.Ig
|
|
.IP n 3
|
|
Neither stream is retained.
|
|
.RE
|
|
.IP "\-K kill_delay" 8
|
|
Specifies a job specific kill delay in seconds. When specified, this job will
|
|
be sent a SIGTERM, followed by a SIGKILL the specified number of seconds later.
|
|
.RE
|
|
.IP "\-l resource_list" 8
|
|
Defines the resources that are required by the job and establishes a limit
|
|
to the amount of resource that can be consumed.
|
|
If not set for a generally available resource, such as CPU time, the limit
|
|
is infinite.
|
|
The
|
|
.Ar resource_list
|
|
argument is of the form:
|
|
.br
|
|
.Ty "\ \ \ \ resource_name[=[value]][,resource_name[=[value]],...]
|
|
.if !\n(Pb .ig Ig
|
|
.IP
|
|
For each resource listed in the
|
|
.Ar resource_list ,
|
|
one entry will be added to the
|
|
.At Resource_List
|
|
attribute of the job. The entry contains the resource name and its
|
|
requested value. No white space is allowed in the value.
|
|
Other than syntax, qsub performs no resource or value
|
|
checking. The checking is performed by the execution server.
|
|
.Ig
|
|
.RE
|
|
.IP "\-L NUMA_resource_list" 8
|
|
Defines the NUMA-aware resource requests for NUMA hardware. This option will work with non-NUMA hardware.
|
|
Syntax for
|
|
.Ar NUMA_resource_list
|
|
is:
|
|
.br
|
|
.Ty "\ \ \ \ tasks=#[:lprocs=#|all]
|
|
.br
|
|
.Ty "\ \ \ \ [:{usecores|usethreads|allowthreads}]
|
|
.br
|
|
.Ty "\ \ \ \ [:place={socket|numanode|core|thread}[=#]{node}][:memory=#][:swap=#][:maxtpn=#][:gpus=#[:<mode>]][:mics=#]
|
|
.br
|
|
.Ty "\ \ \ \ [:gres=<gres>][:feature=<feature>]
|
|
.br
|
|
.Ty "\ \ \ \ [[:{cpt|cgroup_per_task}]|[:{cph|cgroup_per_host}]]
|
|
.Ig
|
|
.RE
|
|
.IP "\-m mail_options " 8
|
|
Defines the set of conditions under which the execution server will
|
|
send a mail message about the job. The mail_options
|
|
argument is a string which consists of either the single character "\fBn\fP" or "\fBp\fP",
|
|
or one or more of the characters "\fBa\fP", "\fBb\fP", "\fBe\fP", and "\fBf\fP".
|
|
|
|
If the character "\fBn\fP" is specified, no normal mail is sent. Mail for job cancels and other events outside of normal job processing are still sent.
|
|
If the character "\fBp\fP" is specified, mail will never be sent for the job.
|
|
|
|
For the characters "\fBa\fP", "\fBb\fP", "\fBe\fP", and "\fBf\fP":
|
|
.RS
|
|
.IP a 3
|
|
Mail is sent when the job is aborted by the batch system.
|
|
.Ig
|
|
.IP b 3
|
|
Mail is sent when the job begins execution.
|
|
.Ig
|
|
.IP e 3
|
|
Mail is sent when the job terminates.
|
|
.Ig
|
|
.IP f 3
|
|
Mail is sent when the job terminates with a non-zero exit code.
|
|
.RE
|
|
.IP
|
|
If the
|
|
.Ar \-m
|
|
option is not specified, mail will be sent if the job is aborted.
|
|
|
|
.RE
|
|
.IP "\-M user_list" 8
|
|
Declares the list of users to whom mail is sent by the execution server
|
|
when it sends mail about the job.
|
|
.IP
|
|
The
|
|
.Ar user_list
|
|
argument is of the form:
|
|
.br
|
|
.Ty "\ \ \ \ user[@host][,user[@host],...]"
|
|
.br
|
|
If unset, the list defaults to the submitting user at the qsub host, i.e. the
|
|
job owner.
|
|
.if !\n(Pb .ig Ig
|
|
.IP
|
|
The
|
|
.At Mail_Users
|
|
attribute is set to the argument.
|
|
.Ig
|
|
.IP "\-n node_exclusive" 8
|
|
|
|
Allows a user to specify an exclusive-node access/allocation request for the job. This will set node_exclusive = True in the output of qstat -f <job ID>. By default, this only applies for cpusets, and only for compatible schedulers.
|
|
|
|
For systems that use Moab and have cgroups enabled, the recommended manner for assigning all cores is to use NUMA syntax: "-L tasks=<count>:lprocs=all:place=node".
|
|
|
|
With cgroups, the ("-l") syntax (lowercase L) will, by default, restrict to the number of cores requested, or to the resources_default.procs value (i.e., 1 core, typically). In order to override this behavior and have Moab assign all the cores on a node while using "-l...singlejob" and/or "-n" (in other words, without "-L ...lprocs=all..."), you must also set RMCFG[<torque>] FLAGS=MigrateAllJobAttributes in moab.cfg.
|
|
|
|
|
|
.IP "\-N name " 8
|
|
Declares a name for the job.
|
|
The name specified may be up to and including 15 characters in length.
|
|
It must consist of printable, non white space characters with the first
|
|
character alphabetic.
|
|
.if !\n(Pb .ig Ig
|
|
[The POSIX 1003.2d Standard calls for only alphanumeric characters, but then
|
|
calls for the use of the script file base name as the job name if a name is
|
|
not specified. The file name may contain other than alphanumeric characters.
|
|
Therefore I \*Qinterpret\*U the standard as allowing printable characters.]
|
|
Names taken from the script name may have a non-alphabetic character first.
|
|
If the script basename is greater than 15 characters, it will be truncated
|
|
to 15.
|
|
.Ig
|
|
.IP
|
|
If the
|
|
.Ar \-N
|
|
option is not specified, the job name will be the base name of the job
|
|
script file specified on the command line. If no script file name was
|
|
specified and the script was read from the standard input, then the job
|
|
name will be set to
|
|
.Ty STDIN .
|
|
.if !\n(Pb .ig Ig
|
|
.IP
|
|
The
|
|
.At Job_Name
|
|
attribute is set to the name.
|
|
.Ig
|
|
.IP "\-o path" 8
|
|
Defines the path to be used for the standard output stream of the batch job.
|
|
The
|
|
.Ar path
|
|
argument is of the form:
|
|
.br
|
|
.Ty "\ \ \ \ [hostname:][path_name]"
|
|
.br
|
|
where
|
|
.Ty hostname
|
|
is the name of a host to which the file will be returned and
|
|
.Ty path_name
|
|
is the path
|
|
name on that host in the syntax recognized by POSIX.
|
|
The argument will be interpreted as follows:
|
|
.RS
|
|
.IP \fBpath_name\fP
|
|
Where path_name is not an absolute path name, then the qsub command will
|
|
expand the path name relative to the current working directory of the command.
|
|
The command will supply the name of the host upon which it is executing for the
|
|
.Ar hostname
|
|
component.
|
|
.IP \fBhostname:path_name\fP
|
|
Where path_name is not an absolute path name, then the qsub command will not
|
|
expand the path name relative to the current working directory of the command.
|
|
On delivery of the standard output, the path name will be expanded relative
|
|
to the user's home directory on the \fBhostname\fP system.
|
|
.IP \fBpath_name\fP
|
|
Where path_name specifies an absolute path name, then the qsub will supply
|
|
the name of the host on which it is executing for the
|
|
.Ar hostname
|
|
.IP \fBhostname:path_name\fP
|
|
Where path_name specifies an absolute path name, the path will be used as
|
|
specified.
|
|
.Ar hostname .
|
|
.IP \fBhostname:\fP
|
|
Where hostname specifies the name of the host that the file should be returned
|
|
to. The path will be the default file name.
|
|
.RE
|
|
.IP
|
|
If the
|
|
.Ar \-o
|
|
option is not specified or the \fBpath_name\fP is not specified or is specified
|
|
and is a directory, the default file name for the standard output stream
|
|
will be used. The default name has the following form:
|
|
.br
|
|
\ \ \ \ \fBjob_name.osequence_number\fP
|
|
.br
|
|
where \fBjob_name\fP is the name of the job, see
|
|
.Ar \-N
|
|
option, and \fBsequence_number\fP is the job number assigned when the
|
|
job is submitted.
|
|
.if !\n(Pb .ig Ig
|
|
This option sets the job attribute
|
|
.At Output_Path .
|
|
.Ig
|
|
.IP "\-p priority" 8
|
|
Defines the priority of the job. The
|
|
.Ar priority
|
|
argument must be a integer between \-1024 and +1023 inclusive.
|
|
The default is no priority which is equivalent to a priority of zero.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Priority
|
|
job attribute is set to this signed integer value.
|
|
.Ig
|
|
.IP "\-P proxy_user[:group]" 8
|
|
Proxy user for whom the job should be submitted.
|
|
This option is only available for the super user.
|
|
.Ig
|
|
.IP "\-q destination" 8
|
|
Defines the destination of the job. The
|
|
.Ar destination
|
|
names a queue, a server, or a queue at a server.
|
|
.IP
|
|
The qsub command will submit the script to the server defined by the
|
|
.Ar destination
|
|
argument.
|
|
.if !\n(Pb .ig Ig
|
|
The server named by the destination is the one to which qsub sends the
|
|
.I "Queue Job"
|
|
batch request.
|
|
.Ig
|
|
If the destination is a
|
|
.I "routing queue,"
|
|
the job may be routed by the server to a new destination.
|
|
.IP
|
|
If the
|
|
.Ar \-q
|
|
option is not specified, the qsub
|
|
command will submit the script to the default server.
|
|
See PBS_DEFAULT under the Environment Variables section on this man page
|
|
and the PBS ERS section 2.7.4, "Default Server".
|
|
.IP
|
|
If the
|
|
.Ar \-q
|
|
option is specified, it is in one of the following three forms:
|
|
.br
|
|
.Ty "\ \ \ \ queue"
|
|
.br
|
|
.Ty "\ \ \ \ @server"
|
|
.br
|
|
.Ty "\ \ \ \ queue@server"
|
|
.IP
|
|
If the
|
|
.Ar destination
|
|
argument names a queue and does not name a server, the job will be submitted
|
|
to the named queue at the default server.
|
|
.IP
|
|
If the
|
|
.Ar destination
|
|
argument names a server and does not name a queue, the job will be submitted
|
|
to the default queue at the named server.
|
|
.IP
|
|
If the
|
|
.Ar destination
|
|
argument names both a queue and a server, the job will be submitted to
|
|
the named queue at the named server.
|
|
.IP "\-r y|n" 8
|
|
Declares whether the job is rerunable.
|
|
See the
|
|
.B qrerun
|
|
command.
|
|
The option argument
|
|
is a single character, either
|
|
.Ty y
|
|
or
|
|
.Ty n .
|
|
.if !\n(Pb .ig Ig
|
|
Also see
|
|
.I rerunable
|
|
in the glossary. Interactive jobs are forced to not rerunable.
|
|
.Ig
|
|
.IP
|
|
If the argument is "\fBy\fP", the job is rerunable.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Rerunable
|
|
attribute is set to the character 'y'.
|
|
.Ig
|
|
If the argument is "\fBn\fP", the job is not rerunable.
|
|
The default value is 'y', rerunable.
|
|
.IP "\-S path_list" 8
|
|
Declares the shell that interprets the job script.
|
|
.IP
|
|
The option argument
|
|
.Ar path_list
|
|
is in the form:
|
|
.br
|
|
.Ty "\ \ \ \ path[@host][,path[@host],...]"
|
|
.br
|
|
Only one path may be specified for any host named. Only one path may be
|
|
specified without the corresponding host name. The path selected will be
|
|
the one with the host name that matched the name of the execution host.
|
|
If no matching host is found, then the path specified without
|
|
a host will be selected, if present.
|
|
.IP
|
|
If the
|
|
.Ar \-S
|
|
option is not specified, the option argument is the null string, or
|
|
no entry from the
|
|
.Ar path_list
|
|
is selected, the execution will use the user's login shell
|
|
on the execution host.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Shell_Path_List
|
|
attribute is set to the
|
|
.Ar path_list
|
|
argument if present, otherwise it is set to the null string.
|
|
.Ig
|
|
.IP "\-t array_request" 8
|
|
Specifies the task ids of a job array. Single task arrays are allowed.
|
|
.IP
|
|
The
|
|
.Ar array_request
|
|
argument is an integer id or a range of integers. Multiple ids
|
|
or id ranges can be combined in a comma delimted list. Examples :
|
|
\-t 1-100 or \-t 1,10,50-100
|
|
.IP
|
|
An optional slot limit can be specified to limit the amount of jobs that can run
|
|
concurrently in the job array. The default value is unlimited. The slot limit
|
|
must be the last thing specified in the array_request and is delimited from the
|
|
array by a percent sign (%).
|
|
.IP
|
|
qsub script.sh -t 0-299%5
|
|
.IP
|
|
This sets the slot limit to 5. Only 5 jobs from this array can run at the same
|
|
time.
|
|
.IP
|
|
Note: You can use qalter to modify slot limits on an array. The server parameter
|
|
max_slot_limit can be used to set a global slot limit policy.
|
|
.IP "\-T script_name" 8
|
|
Allows for per job prologue and epilogue scripts. The full script name will be
|
|
prologue.[name] or epilogue.[name]. For the job submission, only request
|
|
the name of the prologue or epilogue script.
|
|
.IP
|
|
Example:
|
|
.Ty "qsub -T prescript"
|
|
.br
|
|
Specifies to use the script prologue.prescript
|
|
.IP "\-u user_list" 8
|
|
Defines the user name under which the job is to run on the execution system.
|
|
.IP
|
|
The
|
|
.Ar user_list
|
|
argument is of the form:
|
|
.br
|
|
.Ty "\ \ \ \ user[@host][,user[@host],...]"
|
|
.br
|
|
Only one user name may be given per specified host.
|
|
Only one of the
|
|
.Ty user
|
|
specifications may be supplied without the corresponding
|
|
.Ty host
|
|
specification. That user name will used for execution on any host not
|
|
named in the argument list.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At User_List
|
|
attribute is set to the value of
|
|
.Ar user_list .
|
|
.Ig
|
|
If unset, the user list defaults to the user who is running qsub.
|
|
.IP "\-v variable_list"
|
|
Expands the list of environment variables that are exported to the job.
|
|
.IP
|
|
In addition to the variables described in the "Description" section above,
|
|
.Ar variable_list
|
|
names environment variables from the qsub
|
|
command environment which are made available to the job when it executes.
|
|
The
|
|
.Ar variable_list
|
|
is a comma separated list of strings of the form
|
|
.Ty variable
|
|
or
|
|
.Ty variable=value .
|
|
These variables and their values are passed to the job.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Variable_List
|
|
attribute is appended with the variables in
|
|
.Ar user_list
|
|
and their values.
|
|
.Ig
|
|
.IP "\-V" 8
|
|
Declares that all environment variables in the qsub
|
|
command's environment are to be exported to the batch job.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At Variable_List
|
|
attribute is appended with the variables in the qsub
|
|
command's environment and their values.
|
|
.Ig
|
|
.IP "\-w path" 8
|
|
Defines the working directory path to be used for the job. If the
|
|
.Ar \-w
|
|
option is not specified, the default working directory is the current directory.
|
|
This option sets the environment variable PBS_O_WORKDIR.
|
|
.Ig
|
|
.IP "\-W additional_attributes" 8
|
|
The \-W option allows for the specification of additional job attributes.
|
|
.if !\n(Pb .ig Ig
|
|
.SM
|
|
POSIX.2 reserves all undefined option letters for future versions of the
|
|
standard. The single letter 'W' is allowed for extensions. PBS makes use
|
|
of the \-W to specify attributes which are extensions to POSIX 1003.2d.
|
|
.NL
|
|
.Ig
|
|
The general syntax of the \-W is in the form:
|
|
.br
|
|
.Ty "\ \ \ \ \-W attr_name=attr_value[,attr_name=attr_value...]"
|
|
.br
|
|
Note if white space occurs anywhere within the option argument string or the
|
|
equal sign, "=", occurs within an
|
|
.Ar attribute_value
|
|
string, then the string must be enclosed with either single or double quote
|
|
marks.
|
|
.IP
|
|
PBS currently supports the following attributes within the \-W option.
|
|
.IP
|
|
.Ty "depend=dependency_list"
|
|
.br
|
|
Defines the dependency between this and other jobs. The
|
|
.Ar dependency_list
|
|
is in the form:
|
|
.br
|
|
.Ty "type[:argument[:argument...][,type:argument...]" .
|
|
.br
|
|
The
|
|
.I argument
|
|
is either a numeric count or a PBS job id according to
|
|
.I type .
|
|
If argument is a count, it must
|
|
be greater than 0. If it is a job id and not fully specified
|
|
in the form
|
|
.Ty seq_number.server.name ,
|
|
it will be expanded according to the default server rules which apply to
|
|
job IDs on most commands.
|
|
If
|
|
.I argument
|
|
is null (the preceding colon need not be specified), the
|
|
dependency of the corresponding type is cleared (unset).
|
|
.RS 12
|
|
.IP "\fBsynccount:count\fP" 4
|
|
This job is the first in a set of jobs to be executed at the same time.
|
|
.I Count
|
|
is the number of additional jobs in the set.
|
|
.IP "\fBsyncwith:jobid\fP" 4
|
|
This job is an additional member of a set of jobs to be executed at the
|
|
same time.
|
|
In the above and following dependency types,
|
|
.I jobid
|
|
is the job identifier of the first job in the set.
|
|
.IP "\fBafter:jobid[:jobid...]\fP" 4
|
|
This job may be scheduled for execution at any point after jobs
|
|
.I jobid
|
|
have started execution.
|
|
.IP "\fBafterok:jobid[:jobid...]\fP" 4
|
|
This job may be scheduled for execution only after jobs
|
|
.I jobid
|
|
have terminated with no errors.
|
|
See the csh warning under "Extended Description".
|
|
.IP "\fBafternotok:jobid[:jobid...]\fP" 4
|
|
This job may be scheduled for execution only after jobs
|
|
.I jobid
|
|
have terminated with errors.
|
|
See the csh warning under "Extended Description".
|
|
.IP "\fBafterany:jobid[:jobid...]\fP" 4
|
|
This job may be scheduled for execution after jobs
|
|
.I jobid
|
|
have terminated, with or without errors.
|
|
.IP "\fBon:count\fP" 4
|
|
This job may be scheduled for execution after \fBcount\fP dependencies on
|
|
other jobs have been satisfied. This form is used in conjunction
|
|
with one of the \fBbefore\fP forms, see below.
|
|
.IP \fBbefore:jobid[:jobid...]\fP 4
|
|
When this job has begun execution, then jobs \fBjobid...\fP may begin.
|
|
.IP \fBbeforeok:jobid[:jobid...]\fP 4
|
|
If this job terminates execution without errors, then jobs
|
|
\fBjobid...\fP may begin.
|
|
See the csh warning under "Extended Description".
|
|
.IP \fBbeforenotok:jobid[:jobid...]\fP 4
|
|
If this job terminates execution with errors, then jobs
|
|
\fBjobid...\fP may begin.
|
|
See the csh warning under "Extended Description".
|
|
.IP \fBbeforeany:jobid[:jobid...]\fP 4
|
|
When this job terminates execution, jobs \fBjobid...\fP may begin.
|
|
.IP
|
|
If any of the \fBbefore\fP forms are used, the jobs referenced by \fBjobid\fP
|
|
must have been submitted with a dependency type of \fBon\fP.
|
|
.IP "\fBArray Dependencies\fP" 4
|
|
It is now possible to have a job depend on an array. These dependencies are in the form depend=arraydep:arrayid[num]. If [num] is not present, then the dependencies applies to the entire array. If [num] is present, then num means the number of jobs that must meet the condition for the dependency to be satisfied.
|
|
.IP "\fBafterstartarray:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution only after jobs in
|
|
.I arrayid
|
|
have started execution.
|
|
.IP "\fBafterokarray:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution only after jobs in
|
|
.I arrayid
|
|
have terminated with no errors.
|
|
.IP "\fBafternotok:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution only after jobs in
|
|
.I arrayid
|
|
have terminated with errors.
|
|
.IP "\fBafteranyarray:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution after jobs in
|
|
.I array id
|
|
have terminated, with or without errors.
|
|
.IP "\fBbeforestartarray:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution only before jobs in
|
|
.I arrayid
|
|
have started execution.
|
|
.IP "\fBbeforeokarray:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution only before jobs in
|
|
.I arrayid
|
|
have terminated with no errors.
|
|
.IP "\fBbeforenotok:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution only before jobs in
|
|
.I arrayid
|
|
have terminated with errors.
|
|
.IP "\fBbeforeanyarray:arrayid[count]\fP" 4
|
|
This job may be scheduled for execution before jobs in
|
|
.I array id
|
|
have terminated, with or without errors.
|
|
.if !\n(Pb .ig Ig
|
|
.IP
|
|
The
|
|
.At depend
|
|
attribute is set to the value of the
|
|
.Ar dependency
|
|
option argument.
|
|
.Ig
|
|
.IP
|
|
If any of the \fBbefore\fP forms are used, the jobs referenced by \fBjobid\fP
|
|
must have the same owner as the job being submitted. Otherwise, the dependency
|
|
is ignored.
|
|
.LP
|
|
Error processing of the existence, state, or condition of the job on which the
|
|
newly submitted job is a deferred service, i.e. the check is performed after
|
|
the job is queued. If an error is detected, the new job will be deleted by
|
|
the server. Mail will be sent to the job submitter stating the error.
|
|
.LP
|
|
Dependency examples:
|
|
.br
|
|
.Ty "qsub \-W depend=afterok:123.big.iron.com /tmp/script"
|
|
.br
|
|
.Ty "qsub \-W depend=before:234.hunk1.com:235.hunk1.com /tmp/script"
|
|
.br
|
|
.Ty "qsub \-W depend=afterokarray:21.tom.com[] /tmp/script"
|
|
.br
|
|
.Ty "qsub \-W depend=beforenotokarray:22.tom.com[][5] /tmp/script"
|
|
.RE
|
|
.IP
|
|
.Ty group_list=g_list
|
|
.br
|
|
Defines the group name under which the job is to run on the execution system.
|
|
The
|
|
.Ar g_list
|
|
argument is of the form:
|
|
.br
|
|
.Ty "group[@host][,group[@host],...]"
|
|
.br
|
|
Only one group name may be given per specified host.
|
|
Only one of the
|
|
.Ty group
|
|
specifications may be supplied without the corresponding
|
|
.Ty host
|
|
specification. That group name will used for execution on any host not
|
|
named in the argument list.
|
|
.if !\n(Pb .ig Ig
|
|
The
|
|
.At group_list
|
|
attribute is set to the value of
|
|
.Ar g_list .
|
|
.Ig
|
|
If not set, the
|
|
.At group_list
|
|
defaults to the primary group of the user under which the job will be run.
|
|
.IP
|
|
.Ty "interactive=true"
|
|
.br
|
|
If the interactive attribute is specified, the job is an interactive job.
|
|
The \-I option is a alternative method of specifying this attribute.
|
|
.IP
|
|
.Ty "stagein=file_list"
|
|
.br
|
|
.Ty "stageout=file_list"
|
|
.br
|
|
Specifies
|
|
.if !\n(Pb .ig Ig
|
|
the
|
|
.At stagein
|
|
or
|
|
.At stageout
|
|
attribute, listing
|
|
.Ig
|
|
which files are staged (copied) in before job start or staged out
|
|
after the job completes execution.
|
|
On completion of the job, all staged-in and staged-out files
|
|
are removed from the execution system. The
|
|
.Ar file_list
|
|
is in the form
|
|
.br
|
|
.Ty "local_file@hostname:remote_file[,...]"
|
|
.br
|
|
regardless of the direction of the copy.
|
|
The name
|
|
.Ty local_file
|
|
is the name of the file on the system where the job executed.
|
|
It may be an absolute path
|
|
or relative to the home directory of the user.
|
|
The name
|
|
.Ty remote_file
|
|
is the destination name on the host specified by
|
|
.Ty hostname .
|
|
The name may be absolute or relative to the user's home directory on the
|
|
destination host.
|
|
The use of wildcards in the file name is not recommended.
|
|
.if !\n(Pb .ig Ig
|
|
Since rcp (or scp) is run via rsh, it will pick up matching names from the
|
|
remote system. However, pbs_mom will does not expand the wildcards and will
|
|
fail to delete the staged files on job termination.
|
|
.Ig
|
|
The file names map to a remote copy program (rcp) call
|
|
on the execution system in the follow manner:
|
|
.br
|
|
For stagein: rcp hostname:remote_file local_file
|
|
.br
|
|
For stageout: rcp local_file hostname:remote_file
|
|
.br
|
|
Data staging examples:
|
|
.br
|
|
.Ty "\-W stagein=/tmp/input.txt@headnode:/home/user/input.txt"
|
|
.br
|
|
.Ty "\-W stageout=/tmp/output.txt@headnode:/home/user/output.txt"
|
|
.br
|
|
If TORQUE has been compiled with wordexp support, then variables can
|
|
be used in the specified paths. Currently only $PBS_JOBID, $HOME, and
|
|
$TMPDIR are supported for stagein.
|
|
.RE
|
|
.IP
|
|
.Ty umask=XXX
|
|
.br
|
|
Sets umask used to create stdout and stderr spool files in pbs_mom
|
|
spool directory. Values starting with 0 are treated as octal values,
|
|
otherwise the value is treated as a decimal umask value.
|
|
.br
|
|
.IP "\-x"
|
|
When running an interactive job, the \-x flag makes it so that the script won't be parsed for PBS directives, but instead will be a command that is launched once the interactive job has started. The job will terminate at the completion of this command.
|
|
.IP "\-X" 8
|
|
Enables X11 forwarding. The DISPLAY environment variable must be set.
|
|
.IP "\-z" 8
|
|
Directs that the qsub
|
|
command is not to write the job identifier assigned to the job to
|
|
the command's standard output.
|
|
.in 0
|
|
.LP
|
|
.SH OPERANDS
|
|
The qsub command accepts a
|
|
.Ar script
|
|
operand that is the path to the script of the job.
|
|
If the path is relative, it will be expanded relative to the working
|
|
directory of the qsub command.
|
|
.LP
|
|
If the
|
|
.Ar script
|
|
operand is not provided or the operand is the single character "\-", the
|
|
qsub command reads the script from standard input.
|
|
.if \n(Pb .ig Ig
|
|
When the script is being read from Standard Input, qsub will copy the
|
|
file to a temporary file. This temporary file is passed to the library
|
|
interface routine pbs_submit. The temporary file is removed by qsub
|
|
after pbs_submit returns or upon the receipt of a signal which would cause
|
|
qsub to terminate.
|
|
.Ig
|
|
.SH STANDARD INPUT
|
|
The qsub command reads the script for the job from standard input if the
|
|
.Ar script
|
|
operand is missing or is the single character "\-".
|
|
.SH INPUT FILES
|
|
The
|
|
.Ar script
|
|
file is read by the qsub command.
|
|
Qsub acts upon any directives found in the script.
|
|
.LP
|
|
When the job is created, a copy of the script file is made and that copy
|
|
cannot be modified.
|
|
.SH STANDARD OUTPUT
|
|
Unless the
|
|
.Ar \-z
|
|
option is set, the job identifier assigned to the job will be written to
|
|
standard output if the job is successfully created.
|
|
.SH STANDARD ERROR
|
|
The
|
|
qsub
|
|
command will write a diagnostic message to standard error for
|
|
each error occurrence.
|
|
.SH ENVIRONMENT VARIABLES
|
|
The values of some or all of the variables in the qsub
|
|
command's environment are exported with the job, see the \-v and \-V options.
|
|
.LP
|
|
The environment variable
|
|
.B PBS_DEFAULT
|
|
defines the name of the default server. Typically, it corresponds to the
|
|
system name of the host on which the server is running.
|
|
If PBS_DEFAULT is not set, the default is defined by an administrator
|
|
established file.
|
|
.LP
|
|
The environment variable
|
|
.B PBS_DPREFIX
|
|
determines the prefix string which identifies directives in the script.
|
|
.LP
|
|
The environment variable
|
|
.B PBS_CLIENTRETRY
|
|
defines the maximum number of seconds qsub will block. See the \-b option
|
|
above. Despite the name, currently qsub is the only client that supports
|
|
this option.
|
|
.SH TORQUE.CFG
|
|
The torque.cfg file, located in PBS_SERVER_HOME (/var/spool/torque by default)
|
|
controls the behavior of the qsub command. This file contains a list of parameters
|
|
and values separated by whitespace
|
|
.LP
|
|
.B QSUBSLEEP
|
|
takes an integer operand which specifies time to sleep when running qsub command.
|
|
Used to prevent users from overwhelming the scheduler.
|
|
.LP
|
|
.B SUBMITFILTER
|
|
specifies the path to the submit filter used to pre-process job submission. The
|
|
default path is $(libexecdir)/qsub_filter, which falls back to
|
|
/usr/local/sbin/torque_submitfilter for backwards compatibility. This torque.cfg
|
|
parameter overrides this default.
|
|
.LP
|
|
.B SERVERHOST
|
|
specifies the value for the PBS_SERVER environment variable
|
|
.LP
|
|
.B QSUBHOST
|
|
specifies the hostname for the jobs QSUB_O_HOST variable
|
|
.LP
|
|
.B QSUBSENDUID
|
|
specifies a uid to use for the jobs PBS_O_UID variable
|
|
.LP
|
|
.B XAUTHPATH
|
|
specifies the path to xauth
|
|
.LP
|
|
.B CLIENTRETRY
|
|
specifies the integer seconds between retry attempts to communicate with pbs_server
|
|
.LP
|
|
.B VALIDATEGROUP
|
|
set this parameter to force qsub to verify the submitter's group id
|
|
.LP
|
|
.B DEFAULTCKPT
|
|
specifies the default value for the jobs checkpoint attribute. The user overrides
|
|
this with the \-c qsub option.
|
|
.LP
|
|
.B VALIDATEPATH
|
|
set this parameter to force qsub to validate local existence of a "\-d" working directory
|
|
.LP
|
|
.B RERUNNABLEBYDEFAULT
|
|
this parameter specifies if a job is rerunnable by default. The default is true, setting
|
|
this to false causes the rerunnable attribute value to be false unless the users specifies
|
|
otherwise with the \-r option
|
|
.LP
|
|
.B FAULT_TOLERANT_BY_DEFAULT
|
|
this parameter specifies if a job is fault tolerant by default. The default value for
|
|
the fault_tolerant job attribute is false, setting this parameter to true causes the
|
|
default value of the attribute to be true. The user can specify their preference with
|
|
the \-f qsub option.
|
|
.LP
|
|
For example:
|
|
.RS
|
|
.Cs
|
|
.Ty "QSUBSLEEP 2"
|
|
.Ty "RERUNNABLEBYDEFAULT false"
|
|
.Ce
|
|
.RE
|
|
.LP
|
|
.SH EXTENDED DESCRIPTION
|
|
.LP
|
|
Script Processing:
|
|
.LP
|
|
A job script may consist of PBS directives, comments and executable statements.
|
|
A PBS directive provides a way of specifying job attributes in addition to
|
|
the command line options.
|
|
For example:
|
|
.RS
|
|
.Cs
|
|
.Ty ":"
|
|
.Ty "#PBS \-N Job_name"
|
|
.Ty "#PBS \-l walltime=10:30,mem=320kb"
|
|
.Ty "#PBS \-m be"
|
|
.Ty "#"
|
|
.Ty "step1 arg1 arg2"
|
|
.Ty "step2 arg3 arg4"
|
|
.Ce
|
|
.RE
|
|
.LP
|
|
The qsub
|
|
command scans the lines of the script file for directives.
|
|
An initial line in the script that begins with the characters "#!" or the
|
|
character ":" will be ignored and scanning will start with the next line.
|
|
Scanning will continue until the first executable line, that is a line that
|
|
is not blank, not a directive line, nor a line whose first non white space
|
|
character is "#".
|
|
If directives occur on subsequent lines, they will be ignored.
|
|
.LP
|
|
A line in the script file will be processed as a directive to qsub
|
|
if and only if the string of characters starting with the first non white
|
|
space character on the line and of the same length as the directive prefix
|
|
matches the directive prefix.
|
|
.LP
|
|
The remainder of the directive line consists of the options to qsub
|
|
in the same syntax as they appear on the command line.
|
|
The option character is to be preceded with the "\-" character.
|
|
.LP
|
|
If an option is present in both a directive and on the command line, that
|
|
option and its argument, if any, will be ignored in the directive. The
|
|
command line takes precedence.
|
|
.LP
|
|
If an option is present in a directive and not on the command line, that
|
|
option and its argument, if any, will be processed as if it had occurred
|
|
on the command line.
|
|
.LP
|
|
The directive prefix string will be determined in order of preference
|
|
from:
|
|
.RS 4
|
|
.sp
|
|
The value of the
|
|
.Ar \-C
|
|
option argument if the option is specified on the command line.
|
|
.sp
|
|
The value of the environment variable
|
|
.B PBS_DPREFIX
|
|
if it is defined.
|
|
.sp
|
|
The four character string
|
|
.Ty #PBS .
|
|
.sp
|
|
.RE
|
|
If the
|
|
.Ar \-C
|
|
option is found in a directive in the script file, it will be ignored.
|
|
.LP
|
|
User Authorization:
|
|
.LP
|
|
When the user submits a job from a system other than the one on which the
|
|
PBS Server is running, the name under which the job is to be executed is
|
|
selected according to the rules listed under the \-u option.
|
|
The user submitting the job must be authorized to run the job under the
|
|
execution user name.
|
|
This authorization is provided if
|
|
.RS
|
|
.IP (1) 5
|
|
The host on which qsub is run is trusted by the execution host (see
|
|
/etc/hosts.equiv),
|
|
.IP (2)
|
|
The execution user has an .rhosts file naming the submitting user on the
|
|
submitting host.
|
|
.RE
|
|
.LP
|
|
C-Shell .logout File:
|
|
.LP
|
|
The following warning applies for users of the c-shell, csh.
|
|
If the job is executed under the csh and a
|
|
.I .logout
|
|
file exists in the home directory in which the job executes,
|
|
the exit status of the job is that of the .logout script, not the job script.
|
|
This may impact any inter-job dependencies.
|
|
To preserve the job exit status, either remove the .logout file or place the
|
|
following line as the first line in the .logout file
|
|
.br
|
|
.Ty "\ \ \ set EXITVAL = $status"
|
|
.br
|
|
and the following line as the last executable line in .logout
|
|
.br
|
|
.Ty "\ \ \ exit $EXITVAL"
|
|
.LP
|
|
Interactive Jobs:
|
|
.LP
|
|
If the
|
|
.Ar \-I
|
|
option is specified on the command line or in a script directive, or if
|
|
the "interactive" job attribute declared true via the \-W option,
|
|
.Ty "\-W interactive=true" ,
|
|
either on the command line or in a script directive, the job is an interactive
|
|
job. The script will be processed for directives, but will not be included
|
|
with the job. When the job begins execution, all input to the job is from
|
|
the terminal session in which qsub is running.
|
|
.LP
|
|
When an interactive job is submitted, the qsub command will not
|
|
terminate when the job is submitted. Qsub will remain running until the
|
|
job terminates, is aborted, or the user interrupts qsub with an SIGINT
|
|
(the control-C key). If qsub is interrupted prior to job start,
|
|
it will query if the user wishes to exit.
|
|
If the user response "yes", qsub exits and the job is aborted.
|
|
.LP
|
|
Once the interactive job has started execution, input to and output from the
|
|
job pass through qsub. Keyboard generated interrupts are passed to the job.
|
|
Lines entered that begin with the tilde ('~') character and contain special
|
|
sequences are escaped by qsub. The recognized escape sequences are:
|
|
.RS
|
|
.IP "~."
|
|
Qsub terminates execution. The batch job is also terminated.
|
|
.IP "~susp"
|
|
Suspend the qsub program if running under the C shell. "susp" is the suspend
|
|
character, usually CNTL-Z.
|
|
.IP "~asusp"
|
|
Suspend the input half of qsub (terminal to job), but allow output to continue
|
|
to be displayed. Only works under the C shell. "asusp" is the auxiliary
|
|
suspend character, usually CNTL-Y.
|
|
.RE
|
|
.LP
|
|
.SH EXIT STATUS
|
|
Upon successful processing, the qsub exit status will be a value of zero.
|
|
.LP
|
|
If the qsub command fails, the
|
|
command exits with a value greater than zero.
|
|
.SH SEE ALSO
|
|
qalter(1B), qdel(1B), qhold(1B), qmove(1B), qmsg(1B), qrerun(1B),
|
|
qrls(1B), qselect(1B), qsig(1B), qstat(1B), pbs_connect(3B),
|
|
pbs_job_attributes(7B),
|
|
pbs_queue_attributes(7B),
|
|
pbs_resources_irix5(7B), pbs_resources_sp2(7B), pbs_resources_sunos4(7B),
|
|
pbs_resources_unicos8(7B), pbs_server_attributes(7B), and
|
|
pbs_server(8B)
|
|
\" turn off any extra indent left by the Sh macro
|
|
.RE
|