Restarting multisim Jobs

One of the advantages of using Multisim is the flexibility it provides when restarting jobs.

In most cases, the multisim job can be restarted with a command similar to the following:

$SCHRODINGER/utilities/multisim -RESTART myjob-multisim_checkpoint -d myjob_stage-out.tgz -JOBNAME myjob

where you specify the checkpoint file with the -RESTART option and the -out.tgz file of the last completed stage with the -d option. Note that myjob must match the JOBNAME of the original job you are restarting, and that the job will be restarted on the same compute hosts as before.

The -d option is usually necessary because the subsequent stages need access to the data produced by the last successfully completed stage. For example, if stage 8 is partially done (some subjobs finished, some did not), then you would also have to include myjob_7-out.tgz and myjob_8-out.tgz, and the command to use would be:

$SCHRODINGER/utilities/multisim -RESTART myjob-multisim_checkpoint -d myjob_8-out.tgz -d myjob_7-out.tgz -JOBNAME myjob

In certain uncommon cases, data is required from a stage earlier than the last completed one. Any related errors can be addressed by also including the relevant prior stage(s).

Creating Stage tgz Files

If the partially completed stage files are written to the job directory, it has to be tarred up to make myjob_8-out.tgz before running the above multisim command. To do this, you can use the command:

tar -czvf myjob_8-out.tgz myjob.cpt myjob.ene myjob-in.cms myjob.log myjob-multisim_checkpoint_8 myjob-multisim_checkpoint myjob-out.cfg myjob-out.cms myjob_trj

This will tar up the last stage that was running so it can be used in the above restart command.

Modifying a Restarted Job

You can modify the characteristics of some stages when you restart a job, by using the -set option. For example, if you use the command:

$SCHRODINGER/utilities/multisim -RESTART myjob-multisim_checkpoint -d myjob_7-out.tgz -JOBNAME myjob -set "stage[8].time = 1200.0"

then stage 8 is modified so that its subjobs run with the time set to 1200.0. However, if you also provide the stage 8 .tgz file and some subjobs from stage 8 have finished, only the unfinished jobs will run with the new settings.

You can restart a job with a different .msj file. For instance, the command:

$SCHRODINGER/utilities/multisim -RESTART myjob-multisim_checkpoint -d myjob_7-out.tgz -JOBNAME myjob -m newworkflow.msj

runs the uncompleted stages according to newworkflow.msj. This new .msj file must contain the same completed stages as the original job. Any of the remaining stages can be modified or deleted, and new stages may be inserted. You can also restart a job with a different .cfg file by specifying the new .cfg file with the -c option.

Restarting from a Specific Stage

To restart a job from a completed stage, you can specify the stage number right after the checkpoint file name. For instance, the command:

$SCHRODINGER/utilities/multisim -RESTART myjob-multisim_checkpoint:8 -d myjob_7-out.tgz -JOBNAME myjob

reruns the rest of the workflow starting from stage 8. The archive specified with -d must be the one for the prior stage (which is stage 7 in this example).

In most cases, multisim automatically detects and uses additional input files needed for a job when it is restarted. If the job fails because an existing input file was not detected when restarting a job you can specify that file by using the -ADD_FILE option.