1

2021-09-29 21:06:16 +08:00
parent b200bb2f20
commit 2bb9621e30
2960 changed files with 642536 additions and 1 deletions
@@ -0,0 +1,135 @@
+PBS Tools
+
+Website:  http://www.osc.edu/~troy/pbstools/
+Git repo:  https://github.com/tabaer/pbstools
+
+PBS Tools is a collection of utilities that have been developed at
+OSC, NICS, and elsewhere to aid in the administration and management
+of PBS variants (including OpenPBS, PBS Pro, and TORQUE).  They have
+been developed primarily on Linux clusters, but most of them should be
+applicable to virtually any system running PBS.
+
+
+User tools:
+-----------
+
+bin/ja -- Job accounting within a PBS job; similar to "ja" in NQE.
+
+bin/pbsdcp -- Distributed copy command within a PBS job.  This can
+optionally include an MPI-based scatter functionality that should
+scale better than the default rcp-based approach.
+
+bin/qexec -- Workalike for SGE qlogin/qrsh/qsh
+
+bin/parallel-command-processor -- An MPI program for running an array
+of serial jobs in the context of a single parallel job (a.k.a. the poor
+man's job arrays).
+
+bin/supermover -- A highly configurable data movement tool that can
+wrap scp, globus-url-copy, hsi, and a host of other data movement
+utilities.
+
+bin/dmsub -- Submits data movement jobs using a variety of batch and
+data movement back-ends; understands the transfer description formats
+of RFT, DMover, and Stork.  May rely on supermover, depending on how
+it's configured.
+
+bin/dagsub -- A workalike for condor_submit_dag.  This allows the
+submission of large, complex sets of dependent jobs using a relatively
+simple syntax.  Relies on dmsub for emulating Stork data movement jobs.
+
+
+Admin tools:
+------------
+
+bin/js -- Command line lookup of job scripts in the accounting
+database.
+
+bin/jobinfo -- Command line lookup of usage data in the accounting
+database.
+
+sbin/reaver -- Find (and optionally clean up) processes on a host
+which have not been allocated jobs on that host; requires root
+privilege.
+
+
+Reporting tools:
+----------------
+
+sbin/jobstats -- Generate an HTML report summarizing PBS job
+statistics over a period of months.  Relies on gnuplot; requires root
+privilege.
+
+sbin/find-outlyers -- Extract statistical outlyers from the
+jobdist.dat file generated by jobstats.
+
+
+Database/web tools:
+-------------------
+Note:  these all assume a MySQL DB back end running on the same server
+as pbs_server and PHP-enabled httpd.
+
+etc/create-tables.sql -- Create "pbsacct" database, table, and two
+user accounts (pbsacct and webapp) used by the other DB tools.  To
+initialize, run "mysql -u root -p passwd <create-tables.sql".
+
+sbin/job-db-update -- Parses PBS job accounting records and inserts
+them into DB.
+
+sbin/jobscript-to-db -- Inserts one or more job scripts into the job DB.
+
+sbin/spool-jobscripts -- Spools newly submitted jobscripts to /tmp and
+then invokes jobscript-to-db on them.
+
+web/jobinfo.php -- Presents information on a single job based from the
+job DB.
+
+web/jobs-by-user.php -- Presents information on all jobs by a given
+user (over an optional date range) from the job DB.
+
+web/jobs-by-group.php -- Presents information on all jobs by a given
+group (over an optional date range) from the job DB.
+
+web/jobs-by-node.php -- Presents information on all jobs on a given
+node (over an optional date range) from the job DB.
+
+web/jobstats*.php -- Presents statistics on all jobs (over an optional
+date range) from the job DB.
+
+web/software-usage*.php -- Presents information on software
+usage on all jobs (over an optional date range) from the job DB.
+
+web/active-users.php -- Presents a list of the most active users over
+a date range.
+
+web/active-groups.php -- Presents a list of the most active groups over
+a date range.
+
+web/problem-jobs.php -- Searches for potentially problematic jobs over
+a date range.  (Note that what constitutes "problematic" is extremely
+site specific.)
+
+web/usage-summary.php -- Presents a usage summary over a date range.
+
+web/site-specific.php -- Site specific settings.
+
+web/sql-term.php -- "Terminal" for running arbitrary SQL queries on
+job DB.
+
+examples/db-example -- An example of putting job data from multiple PBS
+instances into a single job DB.
+
+
+Suggestions:
+------------
+
+Comments and suggestions on this software are welcomed; please send
+them to Troy Baer <troy@osc.edu>.
+
+
+Copyright:
+----------
+PBS Tools are copyright 2005-2016 Supercomputer Center and/or
+copyright 2008-2015 University of Tennessee, and are licensed under
+the terms of the GNU GPL v2.
+