136 lines
4.2 KiB
Plaintext
136 lines
4.2 KiB
Plaintext
PBS Tools
|
|
|
|
Website: http://www.osc.edu/~troy/pbstools/
|
|
Git repo: https://github.com/tabaer/pbstools
|
|
|
|
PBS Tools is a collection of utilities that have been developed at
|
|
OSC, NICS, and elsewhere to aid in the administration and management
|
|
of PBS variants (including OpenPBS, PBS Pro, and TORQUE). They have
|
|
been developed primarily on Linux clusters, but most of them should be
|
|
applicable to virtually any system running PBS.
|
|
|
|
|
|
User tools:
|
|
-----------
|
|
|
|
bin/ja -- Job accounting within a PBS job; similar to "ja" in NQE.
|
|
|
|
bin/pbsdcp -- Distributed copy command within a PBS job. This can
|
|
optionally include an MPI-based scatter functionality that should
|
|
scale better than the default rcp-based approach.
|
|
|
|
bin/qexec -- Workalike for SGE qlogin/qrsh/qsh
|
|
|
|
bin/parallel-command-processor -- An MPI program for running an array
|
|
of serial jobs in the context of a single parallel job (a.k.a. the poor
|
|
man's job arrays).
|
|
|
|
bin/supermover -- A highly configurable data movement tool that can
|
|
wrap scp, globus-url-copy, hsi, and a host of other data movement
|
|
utilities.
|
|
|
|
bin/dmsub -- Submits data movement jobs using a variety of batch and
|
|
data movement back-ends; understands the transfer description formats
|
|
of RFT, DMover, and Stork. May rely on supermover, depending on how
|
|
it's configured.
|
|
|
|
bin/dagsub -- A workalike for condor_submit_dag. This allows the
|
|
submission of large, complex sets of dependent jobs using a relatively
|
|
simple syntax. Relies on dmsub for emulating Stork data movement jobs.
|
|
|
|
|
|
Admin tools:
|
|
------------
|
|
|
|
bin/js -- Command line lookup of job scripts in the accounting
|
|
database.
|
|
|
|
bin/jobinfo -- Command line lookup of usage data in the accounting
|
|
database.
|
|
|
|
sbin/reaver -- Find (and optionally clean up) processes on a host
|
|
which have not been allocated jobs on that host; requires root
|
|
privilege.
|
|
|
|
|
|
Reporting tools:
|
|
----------------
|
|
|
|
sbin/jobstats -- Generate an HTML report summarizing PBS job
|
|
statistics over a period of months. Relies on gnuplot; requires root
|
|
privilege.
|
|
|
|
sbin/find-outlyers -- Extract statistical outlyers from the
|
|
jobdist.dat file generated by jobstats.
|
|
|
|
|
|
Database/web tools:
|
|
-------------------
|
|
Note: these all assume a MySQL DB back end running on the same server
|
|
as pbs_server and PHP-enabled httpd.
|
|
|
|
etc/create-tables.sql -- Create "pbsacct" database, table, and two
|
|
user accounts (pbsacct and webapp) used by the other DB tools. To
|
|
initialize, run "mysql -u root -p passwd <create-tables.sql".
|
|
|
|
sbin/job-db-update -- Parses PBS job accounting records and inserts
|
|
them into DB.
|
|
|
|
sbin/jobscript-to-db -- Inserts one or more job scripts into the job DB.
|
|
|
|
sbin/spool-jobscripts -- Spools newly submitted jobscripts to /tmp and
|
|
then invokes jobscript-to-db on them.
|
|
|
|
web/jobinfo.php -- Presents information on a single job based from the
|
|
job DB.
|
|
|
|
web/jobs-by-user.php -- Presents information on all jobs by a given
|
|
user (over an optional date range) from the job DB.
|
|
|
|
web/jobs-by-group.php -- Presents information on all jobs by a given
|
|
group (over an optional date range) from the job DB.
|
|
|
|
web/jobs-by-node.php -- Presents information on all jobs on a given
|
|
node (over an optional date range) from the job DB.
|
|
|
|
web/jobstats*.php -- Presents statistics on all jobs (over an optional
|
|
date range) from the job DB.
|
|
|
|
web/software-usage*.php -- Presents information on software
|
|
usage on all jobs (over an optional date range) from the job DB.
|
|
|
|
web/active-users.php -- Presents a list of the most active users over
|
|
a date range.
|
|
|
|
web/active-groups.php -- Presents a list of the most active groups over
|
|
a date range.
|
|
|
|
web/problem-jobs.php -- Searches for potentially problematic jobs over
|
|
a date range. (Note that what constitutes "problematic" is extremely
|
|
site specific.)
|
|
|
|
web/usage-summary.php -- Presents a usage summary over a date range.
|
|
|
|
web/site-specific.php -- Site specific settings.
|
|
|
|
web/sql-term.php -- "Terminal" for running arbitrary SQL queries on
|
|
job DB.
|
|
|
|
examples/db-example -- An example of putting job data from multiple PBS
|
|
instances into a single job DB.
|
|
|
|
|
|
Suggestions:
|
|
------------
|
|
|
|
Comments and suggestions on this software are welcomed; please send
|
|
them to Troy Baer <troy@osc.edu>.
|
|
|
|
|
|
Copyright:
|
|
----------
|
|
PBS Tools are copyright 2005-2016 Supercomputer Center and/or
|
|
copyright 2008-2015 University of Tennessee, and are licensed under
|
|
the terms of the GNU GPL v2.
|
|
|