Monitor Panel
The Monitor panel can be used to manage and monitor the progress of any Schrödinger computational job.
To open this panel, choose Window → Job Monitor, or click the Jobs button on the toolbar, then click Monitor.
- Using
- Features
- Additional Resources
Using the Monitor Panel
The Monitor panel is used for showing information on jobs, monitoring the progress of jobs, and managing jobs. The information is obtained from your jobs database. You can use the Monitor panel to monitor all your jobs, regardless of whether they were started from Maestro or not.
Only one job can be monitored at any given time. To monitor a job, double-click it in the table in the Jobs tab. The table row is colored blue, and the File tab is placed on top, so you can see the log file. The display is updated as the log file changes.
Monitoring a job is necessary for the job to be incorporated under the following circumstances:
- If Do not incorporate was selected in the Job Settings Dialog Box.
- If you choose not to incorporate a job when prompted by a banner.
Most jobs write information to a log file as the job progresses. This file is copied back periodically to the job submission directory, and is available to the Monitor panel. Some jobs copy back other files periodically, and these files can also be monitored.
When a job from the current project that is being monitored finishes, the results are automatically incorporated into the project using the settings that were active when the job was started. Once incorporation is complete the newly incorporated entries are selected in the project and the first of these is included in the Workspace. The job can then be removed from the list of jobs with the Delete button. However, if the job was submitted to a host that supports detached jobs (i.e. the launch host can go offline while the job runs), the job is not incorporated, but a download icon is displayed in the Status column.
Clicking the icon downloads and incorporates the job.
Incorporation can be undone. Undoing incorporation removes the new entries from the project and restores the Workspace to the state it was in just before monitoring mode was entered.
Most Maestro operations stop the monitoring of a job. You can resume or begin monitoring a job at any time by selecting it from the list of jobs in the Monitor panel.
Panels that do not have a Job toolbar, such as the Prime – Structure Prediction and Phase – Develop Pharmacophore Model panels, have a button that indicates whether a job is running. The icon rotates when the job is running, and stops rotating when the job has finished. For these panels, the Monitor panel can be opened by clicking the button.
Monitor Panel Features
The Monitor panel has a jobs table and two tabs. Below the table is a set of action buttons that can be applied to jobs.
Jobs table
The jobs table lists jobs started by the current user, using a tree format. The job information is taken from the job database for the user. This information is updated periodically.
To monitor a job, select the table row and click Monitor, or double-click the table row. The table row is highlighted in blue,and the File tab is placed on top with the log file displayed.
You can select multiple rows in the table with the usual shift-click and control-click actions, or you can select all rows by using the shortcut menu. If you select multiple rows, the Details and File tabs and the Monitor button are unavailable. You can apply an action to the selected jobs with the other action buttons. The actions are also available from the shortcut menu. You can also show and hide subjobs, using the "turner" in the parent job row.
You can sort the table rows by clicking in the heading of the column whose values you want to sort by. Subjobs are sorted in chronological order by default. Subjobs are kept under the parent job during sorting, so the top-level jobs are sorted by the column values, then for each job, the subjobs are sorted. You can also show and hide subjobs by clicking the "turner" for the parent.
The table cells have tooltips, which are displayed when you pause the pointer over the cell. The tooltips display the full content of the cell, except for the Status column, where an informative message describing the meaning of the status is displayed, and the Host column, where the host entry name from the hosts file is displayed.
| Job ID | ID by which the job is identified to Job Control. Jobs that have subjobs are displayed with a + or − symbol to the left of the ID. The + symbol indicates that the list is hidden; the − symbol indicates that the list is displayed. To display or undisplay the list, click the symbol. Subjob IDs are indented relative to the parent job ID. | |
| Name | Name of the job. The job name can be specified on the Job toolbar or in the Job Settings dialog box when you start a job from Maestro, or on the command line. Subjob job names are indented relative to the parent job name. | |
| Status | Job status. The status for subjobs is indented relative to the parent job status. See Job Status Descriptors below for information on the content of this column. | |
| Errs | Number of errors logged for this job. These errors are not necessarily fatal errors. Most of them are for tasks initiated by Job Control. You can view details of the errors by selecting the job record in the Details tab and viewing it in the File tab. | |
| Start Time | Time at which the job was started | |
| Host | Host to which the job was submitted or on which the job is running. This is the actual host name, such as the name of the machine or cluster node, not the name of the host entry in the hosts file. The host entry name is available in the tooltip for this cell and in the job summary (shown in the Details tab). | |
| Application | Name of the application being run (e.g. MacroModel). | |
| Project | Name of the project to which the job belongs. |
Show option menu
This menu allows you to choose the class of jobs that is displayed in the table. There are three options:
- Jobs from this project only
- Active jobs only
- All jobs
Action Buttons
These buttons perform actions on the table or table rows. The buttons are spread over two rows in the panel.
- Refresh
-
Update the status of all running jobs.
- Clean Up
-
Clean up the job database. This action removes records for the incorporated jobs and completed jobs that cannot be incorporated. The command that is executed is
$SCHRODINGER/jobcontrol -delete all. - Monitor
-
Monitors the currently selected job. Switches to the File tab and by default displays the log file for the job. If the job includes structural monitoring the generated structures will be displayed in the Workspace.
- Pause
-
Suspend the selected jobs. The jobs are not stopped, but they will take no CPU time and make no progress on the calculation.
- Resume
-
Resume progress on the selected jobs, after they have been paused.
- Stop
-
Stop the selected jobs at the next convenient stopping point. This point is defined by the application and allows a clean shutdown of the job, with return of the results up to the stopping point. This action is only available for some applications (MacroModel, Jaguar, Desmond). Some time may elapse before the job finally stops.
- Kill
-
Force termination of the selected jobs as soon as possible. Subjobs are killed by the parent job to ensure a clean job termination. Note that it may not be possible to successfully incorporate the output from a killed job into the project.
- Delete
-
Delete the selected job from the job database and delete the associated files. If you delete a job that has subjobs, the subjobs are also deleted. When you click Delete, a dialog box is displayed with two options for cleaning up the job:
- Remove from job database and remove monitoring files—removes the job from the database and deletes temporary files that were created for monitoring purposes only.
- Remove all files associated with this job—deletes all the input and output files associated with this job as well as removing the job from the database.
- Postmortem
-
Create a postmortem report on each of the selected jobs. Opens the Create Postmortem Archive Dialog Box in which you can choose options for information to go into the report (archive), and create reports for the jobs.
Details Tab
This tab shows details of the selected job. The job information is repeated in the Job Summary text area. The Files table lists all files associated with the job except for the structure files. When you select a file in the table, the file is displayed in the File tab. If you double-click a file, the File tab is displayed with the selected file. By default, the log file is selected. Gray rows in the table mark files that are not available for viewing.
The job record for the job is the last file in the list. This file shows details of the progress of the job and is useful for diagnosing errors.
File Tab
The File tab displays the file that is selected in the Details tab. By default, this file is the log file for the job. The display is updated at regular intervals. When new text is added, the display scrolls to show any new text if the display is already at the end of the file. If the display is not at the end of the file when text is added, the display remains at the same location in the file, and you must scroll down to the end of the file to see the new text.
Job Status Descriptors
The status of a job can be one of the following descriptors:
| launched | Job submitted and a JobId assigned |
| submitted | Job waiting to start on a batch queue |
| started | Environment for job is being set up |
| running | Job running |
| paused | Job temporarily suspended |
| exited | Job terminated and being cleaned up |
| completed | Job terminated and cleaned up. Followed by the exit status (see below) |
| incorporated | job output incorporated into Maestro project. Followed by the exit status (see below) |
| stranded | job could not be reached by job control |
| unreachable | machine running the job could not be reached by job control |
When the job status is completed or incorporated, it is followed by a colon and the exit status. The possible exit status values are:
| finished | Job finished successfully |
| fizzled | Job failed before the program could be run |
| stopped | Job stopped gracefully at user's request |
| killed | Job was killed by a kill request |
| died | Job failed during program execution |
If the job was submitted from a host such as a laptop to a host that supports running detached jobs, the status is shown as
last-known-status : offline
if the job submission host cannot contact the execution host. When the job finishes, a download icon is shown in the Status column.
Clicking the icon downloads and incorporates the job.