1. Double-click the Materials Science icon
   • (No icon? See Starting Maestro)

Figure 2-1. Change Working Directory option.

2. Go to File > Change Working Directory
3. Find your directory, and click Choose
4. Pre-generated files are included for running jobs or examining output. Download the zip file here: schrodinger.com/sites/default/files/s3/release/current/Tutorials/zip/sweetness_ml.zip
5. After downloading the zip file, unzip the contents in your Working Directory for ease of access throughout the tutorial

Figure 2-2. Save Project panel.

6. Go to File > Save Project As
7. Change the File name to sweetness_ml_tutorial, click Save
   • The project is now named sweetness_ml_tutorial.prj

Figure 2-3. Import the starting structures.

In this tutorial, we will use a data set of 1918 small molecules, defined by SMILES strings and marked as either sweet (1) or not sweet (0). The data is from BitterSweet: Building machine learning models for predicting the bitter and sweet taste of molecules.

Let’s import these structures now:

8. Go to File > Import Structures
9. Change Files of type to Smiles (*.smi*.csv…)
10. Navigate to where you downloaded the provided tutorial files (presumably in your working directorythe location where files are saved) and choose Section_02 > train.csv from the provided tutorial files
11. Click Open
    • The Import SMILES panel pops up

Figure 2-4. Import SMILES settings.

12. For SMILES Column: choose SMILES
    • The SMILES column in the CSV contains the SMILES patterns
13. For ENTRY TITLE Column: choose Title
14. Ensure Discard any additional properties is unchecked
15. Click OK
    • The import will take a few moments

Figure 2-5. Viewing the Sweet tag in the Project Table.

Each entry has a ‘Sweet’ value associated with it (0 or 1). These can be visualized in the Project Tabledisplays the contents of a project and is also an interface for performing operations on selected entries, viewing properties, and organizing structures and data (). Use the Property Tree () to add the Sweet property (under All > [Unclassified] > Sweet). You can also add the ‘Sweet’ property to your entry list via the Change table settings ().

Figure 3-1. Choosing task and options.

1. Ensure that all 1918 entries are selected(1) the atoms are chosen in the Workspace. These atoms are referred to as "the selection" or "the atom selection". Workspace operations are performed on the selected atoms. (2) The entry is chosen in the Entry List (and Project Table) and the row for the entry is highlighted. Project operations are performed on all selected entries from the entry list by selecting the entry group
2. Go to Tasks > Materials > Informatics > DeepAutoQSAR
   • The DeepAutoQSAR panel opens
3. Ensure that Build model is checked
4. For Model type, choose Classification
   • Because the data is categorical (0, 1), we use a classification model
   • For categorical inputs, the data should always be (0, 1, 2, etc. as needed)
5. Ensure that for Use structures from, Project Table (1918 selected entries) is chosen

Figure 3-2. Defining the training.

6. Change the Prediction property to Sweet via the dropdown menu
7. Set 90% for the Random split
   • This is the percentage of data to set aside between train and test sets, where 90% of the data is used to train the model and 10% of the data is used to test the model
   • This is a relatively large data set. The 90:10 split ensures that there is significantly more data in the training set than the test set, but still enough data in the test set to assess model performance.
8. Set the Training time to 12 hours

 

Note: Training for the current model concludes after the specified time limit is reached, but no subsequent models are trained. Note that the total elapsed time may significantly exceed this limit if a particular model requires an extended training duration. A minimum of two replicates is trained for every model type. For datasets with ~2000 inputs like this one, a 12 hour training time can help ensure the best models are determined

Figure 3-3. Naming and running the job.

9. Change the Job name to BuildTask_Sweetness
10. Adjust the job settings () as needed
    • This job requires a Linux host
11. This job takes roughly 5 hours to complete. If you would like to run the job yourself, click Run. Otherwise, provided data is available for proceeding in Section 4.

Note: If you do not have remote submission to Linux from MS Maestro set up, an alternative is to use the Write option in the job settings dropdown to generate a directory containing the submission scripts, which could then be transferred and run on a Linux host.

12. Close the DeepAutoQSAR panel

Figure 4-1. Loading the trained model.

When the job is complete, note that no new entry group is added to the entry lista simplified view of the Project Table that allows you to perform basic operations such as selection and inclusion. The output can be analyzed and used for predictions back in the DeepAutoQSAR panel:

1. Return to Tasks > Materials > Informatics > DeepAutoQSAR
   • The DeepAutoQSAR panel opens
2. For Choose task, switch to Make Predictions
3. To choose the Model file click Browse, navigate to the Section_04 > BuildTask_sweetness > BuildTask_sweetness_model.qzip file, select it, and click Open
   • The panel will parse the .qzip file and the Model Summary section will be populated

Figure 4-2. Viewing the Model Summary.

Begin by analyzing the Model Summary output. The data presented is a summary of the statistics of the model.

4. Click View Full Report

Figure 4-3. Viewing the Report tab.

The Report tab includes a raw copy of the JSON output of DeepAutoQSAR. This report provides detailed information about the top-performing model ensemble throughout the training process, including its evaluation metrics, the model architecture used (e.g., deep neural networks, random forests, etc.), and relevant model hyperparameters.

5. Click on the Plot tab

Figure 4-4. Viewing the ROC curve.

For categorical models such as this, the Plot tab shows an ROC plot. An ideal classifier would have a ROC plot shifted upper left with a True Positive Rate of 1, False Positive Rate of 0, and Area Under the Curve of 1.

6. Close the DeepAutoQSAR Report Viewer

Note: For regression models, a scatter plot and regression line are shown.

Figure 4-5. Imported structures in the entry list.

Now, we will use the constructed model to make predictions on an unseen data set of organic molecules that were not in the training data. These complexes have known sweetness values from BitterSweet, which we can use to assess the quality of the model for making predictions outside the training set.

7. Close the DeepAutoQSAR panel (or simply minimize it or adjust to the side of your window – we will return to it in a moment)
8. Go to File > Import Structures
9. Navigate to where you downloaded the provided tutorial files (presumably in your working directorythe location where files are saved), choose Section_04 > test.csv and click Open. If you need a refresher on how to import SMILES strings, revisit Section 2.
   • 214 new entries are added to the entry lista simplified view of the Project Table that allows you to perform basic operations such as selection and inclusion.
10. Select(1) the atoms are chosen in the Workspace. These atoms are referred to as "the selection" or "the atom selection". Workspace operations are performed on the selected atoms. (2) The entry is chosen in the Entry List (and Project Table) and the row for the entry is highlighted. Project operations are performed on all selected entries the entire test (214) group from the entry lista simplified view of the Project Table that allows you to perform basic operations such as selection and inclusion
    • Recall that selecting means to highlight the group in the entry list

Figure 4-6. Settings for the prediction job in the DeepAutoQSAR panel.

11. Return to the DeepAutoQSAR panel
12. Ensure that the panel reflects the progress from the above steps: Make Predictions is selected, the .qzip file is loaded and the Model Summary is shown
13. In the Make Predictions section of the panel, ensure that Project Table (214 selected entries) is chosen for Use structures from
14. For Output property name, input Predicted_Sweetness
    • This will be the name of the predicted property in the project table

Figure 4-7. Running the prediction job.

15. Change the Job name to PredictTask_Sweetness
16. Adjust the job settings () as needed
    • This job requires a Linux host
17. This job takes roughly 5 minutes to complete. If you would like to run the job yourself, click Run. Otherwise, provided data is available for import from the provided tutorial files: Section_04 > PredictTask_Sweetness > PredictTask_Sweetness_output.maegz
18. Close the DeepAutoQSAR panel

Figure 4-8. Viewing the output in the Project Table.

When the job is complete or after importing, a new entry group is added to the entry lista simplified view of the Project Table that allows you to perform basic operations such as selection and inclusion titled PredictTask_Sweetness_output (214) containing the same 214 entries, but now with predicted sweetness properties. The data can be analyzed in the Project Tabledisplays the contents of a project and is also an interface for performing operations on selected entries, viewing properties, and organizing structures and data

19. Open the Project Table ()
20. Use the Property Tree () to include the Predicted Sweetness score and uncertainty properties (Check the properties of interest under All > Maestro)

 

We can see predicted scores for the various molecules as well as uncertainty values. Note that the output values are probabilities. We can apply a simple cutoff of <0.5 as 0 and >0.5 as 1 for making sweetness predictions with our categorical model.

To compare these values to the known values we can export data to a spreadsheet (Project Table > Data > Export > Spreadsheet) and analyze.