Configure the jobserverd Service

1. The jobserverd process needs to be able to find your queue executables, so they must be in the PATH environment variable.

   Examples

   • Slurm: /etc/profile.d/slurm.sh

     export PATH=$PATH:/opt/slurm/bin/:/opt/slurm/sbin/

   • SGE and variants: /etc/profile.d/sge.sh

     export PATH=$PATH:/opt/sge/bin/:/opt/sge/bin/lx-amd64/

   • LSF: /etc/profile.d/lsf.sh

     export PATH=$PATH:/usr/share/lsf/10.1/linux2.6-glibc2.3-x86_64/etc:/usr/share/lsf/10.1/linux2.6-glibc2.3-x86_64/bin

   • PBSPro: /etc/profile/d/pbs.sh

     export PATH=$PATH:/opt/pbs/bin

2. Test manual startup

   To ensure Job Server was set up correctly, start up the jobserverd process manually in debug mode:

   On the Job Server host, change to the Job Server user

   sudo su - jobserver

   Then:

   /bin/bash -c "source /etc/profile.d/<profile.sh> && <jobserver_dir>/bin/jobserverd --debug --dir <jobserver_dir>"

   If jobserverd does not start correctly, you can look at the log file to troubleshoot the issue. The jobserverd process writes its own log files into the file:

   <jobserver_dir>/logs/jobserverd.log

   To display the output in a more readable form, you can use the "jq" program, which is available in all standard Linux software repositories. For example:

   cat <jobserver_dir>/logs/jobserverd.log | jq

   Errors related to startup may also be in <jobserver_dir>/logs/startup.log.

   If jobserverd does start correctly, it returns something similar to:

   Using version 62122 revision=33cb14d 
   Pid: 5779 
   File server port: 8029 
   Job server port: 8030 
   Hostname: localhost				

   Exit the process (ctrl + c).

3. Create a systemd service file for the Job Server process
   /etc/systemd/system/jobserverd.service using the following as template:

   [Unit]
   Description=Schrodinger Job Server
   # comment out the next line if your systemd version is older than version 230
   StartLimitIntervalSec=0
   
   [Service]
   ExecStart=/bin/bash -c "source /etc/profile.d/<profile.sh> \
            && <jobserver_dir>/bin/jobserverd --dir <jobserver_dir>"
            
   User=jobserver
   Group=jobserver
   Restart=on-failure
   LimitNOFILE=1048576
   RestartSec=15
   
   [Install]
   WantedBy=multi-user.target

   For more information about systemd service files, enter man systemd.service on the command line, or read the man page online: systemd.service.

   For more information about StartLimitIntervalSec, enter man systemd.unit or read the man page online: systemd.unit. You can check the version of systemd with the command
   systemctl --version

4. Start the jobserverd systemd service and enable the service:

   sudo systemctl daemon-reload
   sudo systemctl enable jobserverd
   sudo systemctl start jobserverd

5. Confirm that the jobserverd service is running:

   systemctl status jobserverd --full

OPTIONAL: Confirm that the jobserverd restarts when the host is rebooted
We recommend confirming that the jobserverd process restarts when the host is rebooted. Running jobs won’t be affected as long as the jobserverd process returns in less than two hours.

shutdown -r now

If the jobserverd process does not start, use the following command to query the operating system’s journal for errors related to the "jobserver" user account:

journalctl -xe -u jobserverd

This will query the operating system’s journal for errors related to the "jobserver" user account.