QSAR

13       Genetic Function Approximation

The genetic function approximation (GFA) algorithm (Rogers and Hopfinger) offers a new approach to the problem of building quantitative structure-activity relationship (QSAR) and quantitative structure-property relationship (QSPR) models. Replacing regression analysis with the GFA algorithm allows the construction of models competitive with or superior to those produced by standard techniques and makes available additional information not provided by other techniques. Unlike most other analysis algorithms, GFA provides you with multiple models, where the populations of the models are created by evolving random initial models using a genetic algorithm. GFA can build models using not only linear polynomials but also higher-order polynomials, splines, and other nonlinear functions.

This chapter describes

• Overview of genetic function approximation on page 234
• Using genetic partial least squares on page 235
• Starting a genetic analysis on page 235
• Performing a genetic analysis on page 236
• Continuing the evolution of the current population on page 239
• Randomizing the current population on page 240
• Selecting equation term types on page 241
• Specifying mutation probabilities on page 243
• Specifying other genetic analysis preferences on page 244
• You can change the default using the Statistical Method popup. If you choose PLS, you also can set the number of components and the type of data scaling to perform. on page 247
• Running a G/PLS calculation on page 247
• Setting G/PLS preferences on page 247

Genetic algorithms are derived from an analogy with the spread of mutations in a population. In this analogy, "individuals" are represented as a one-dimensional string of bits. An initial population of individuals is created, usually with random initial bits. A fitness function is used to estimate the "quality" of an individual, so that the "best" individuals receive the best fitness scores. Individuals with the best scores are more likely to propagate their genetic material to offspring through crossover, in which pieces of genetic material are taken from each parent and recombined to create the child. After many such mating steps, the average fitness of the individuals in the population shoule increase, as good combinations of genes are discovered and spread through the population. Genetic algorithms are especially good at searching problem spaces having a large number of dimensions, since they conduct a very efficient, directed sampling of the large space of possibilities.

Friedman's MARS algorithm is a statistical technique for modeling data. It provides an error measure, called the lack-of-fit (LOF) score, that automatically penalizes models with too many features. It also inspired the use of splines as a powerful tool for nonlinear modeling.

The GFA algorithm uses a genetic algorithm to perform a search over the space of possible QSAR/QSPR models using the LOF score to estimate the fitness of each model. Such evolution of a population of randomly constructed models leads to the discovery of highly predictive QSARs/QSPRs.

The GFA algorithm approach has a number of important advantages over other techniques:

• It builds multiple models rather than a single model.
• It automatically selects which features are to be used in the models.
• It is better at discovering combinations of features that take advantage of correlations between multiple features.
• It incorporates Friedman's LOF error measure, which estimates the most appropriate number of features, resists overfitting, and allows control over the smoothness of fit.
• It can use a larger variety of equation term types in construction of its models (for example, splines, step functions, or high-order polynomials).
• It provides, through study of the evolving models, additional information not available from standard regression analysis (such as the preferred model length and useful partitions of the dataset).

The G/PLS algorithm uses GFA to select appropriate basis functions to be used in a model of the data and pls regression to weight the basis functions' relative contributions in the final model. Application of G/PLS allows the construction of larger QSAR equations while avoiding overfitting and eliminating most variables.

G/PLS is run in the same way as GFA. To set up a G/PLS calculation, you must specify the G/PLS method and certain other settings in the Configure GFA control panel. For more information, see Setting genetic analysis preferences on page 240.

Starting a genetic analysis

Before you begin

• You should be familiar with basic QSAR+ concepts and procedures.
• You should have added molecular structures, biological activity information, and descriptor values to a study table. Additionally, you should have selected the dependent and independent variables for your analysis.

To use GFA as the statistical method for a QSAR analysis, choose GFA from the Method popup at the top of the study table, then click the Run button on the QSAR tool bar. Configuring GFA options is described in Genetic function approximation on page 208.

Performing a genetic analysis

Performing a genetic analysis using the built-in settings consists of five basic steps.

1. Starting the analysis

• Add molecular structures, biological activity information, and the appropriate descriptor values to the study table.
• Select the appropriate independent variables (that is, the variables that can be used in building the models) and one dependent variable (that is, the variable you want to predict).

2. Building the initial population

You can change the size of the initial equation population, as well as change both the number and type of terms to be used in each of these initial equations. For more information, see Setting genetic analysis preferences.

3. Evolving the population

You can increase or decrease the number of generations that the initial equation population is evolved. For more information, see Working with the current equation population on page 238. You can also specify a variety of equation mutations that can occur, as well as determine the probability that each type of mutation is attempted in a generation. For more information, see Setting genetic analysis preferenceson page 240.

By changing the smoothing parameter d, you can control the bias in the scoring factor between equations with different numbers of terms. For more information, see Setting genetic analysis preferences on page 240.

4. Reviewing the evolved equations

Genetic analysis produces a graph the shows the frequency that each term (that is, descriptor) is used in all equations in the final population.

5. Using the equations

• Choose your preferred QSAR model based on score, appropriateness of features, feature combinations, and so on. You can then save this preferred model for further statistical analysis. For information about saving QSAR equations, see Saving QSAR equations on page 257.
• Study the entire equation population looking for information on feature use, patterns, regions in which different equations predict well, and so on. These patterns may suggest ideas for new compounds.
• Continue evolving the same population. For more information about continuing the evolution of a current population, see Working with the current equation population on page 238.
• Create a new, random population and evolve that population using the default GFA settings. For more information about randomizing the current population, see Working with the current equation population on page 238.
• Change the default GFA settings and continue the evolution or restart the analysis with a new population. For more information about changing the default values that control the processing performed by GFA, see Setting genetic analysis preferences on page 240.

If you repeat this step, QSAR+ discards the previous equation population, randomly generates a new equation population, and then evolves that new population.

However, if you want to continue the evolution from the point at which the previous evolution left off or to refresh the current equation population, you want to work with the current equation population.

This section describes the following activities related to working with the current equation population:

Continuing the evolution of the current population (page 239)

Randomizing the current population (page 240)

Before you begin

• You should have a study table containing molecular structures, biological activity information, and the appropriate descriptor values.
• You should have selected the appropriate dependent (Y) and independent (X) variables.

To work with the current equation population

Select Preferences/Statistical Methods... on the study table menu bar and be sure GFA is selected in the Statistical Method popup.

You use this control panel both to continue evolving and to randomize the current population.

Continuing the evolution of the current population

To continue the evolution

1.   In the Generations entry box on the Statistical Method Preferences control panel, enter the number of generations for which you want to evolve the current population.

This value is now also used whenever you choose GFA from the Method popup on the study table and click the Run icon on the study table tool bar.

2.   Click the Run icon. When you do so, the following occurs:

The current population of equations is evolved for the specified number of generations. This evolution occurs according to the default values used in GFA or according to the preferences that you specify. For detailed information about preferences, see Setting genetic analysis preferences on page 240.

The evolved equation population is displayed in the equation viewer. You can use the equation viewer to examine, sort, and graph various equations. For detailed information about the equation viewer, see Chapter 14, Using the Equation Viewer.

Setting genetic analysis preferences

This section explains how you can exercise control over a genetic analysis:

Selecting equation term types (page 241)

Specifying mutation probabilities (page 243)

Specifying other genetic analysis preferences (page 244)

You change the default values for genetic analysis by using the Configure GFA control panel. Select Configure on the GENETIC ANALYSIS (GFA) card or select Configure GFA... from the Statistical Method Preferences control panel (be sure the Statistical Method is set to GFA). The Configure GFA control panel allows you to specify the type of terms to be used in equations, mutation probabilities, and other preferences for running the algorithm.

You use this control panel to perform the activities described in the remainder of this section.

Selecting equation term types

You can choose from among five different types of equation terms, as follows:

• Linear -- Linear polynomial terms. This is the default because this type of term is used for 90 % of all modeling. A linear polynomial term type simply means that each feature in the dataset is used exactly as in a linear regression. This type of term is sometimes used in conjunction with linear spline terms or quadratic polynomial terms (below).
• Spline -- Linear spline terms. These terms can be useful if the number of features is small enough so that further reduction in the number of features is not needed. You can then use spline terms to explore partitions of the samples in order to find features that are predictive only over a limited range of values.

Splines are not always useful. If the variables selected are truly linear in their effect on biological activity, splines do not reveal any more-predictive models and may confuse the model building with chance correlations.

The spline terms used in GFA are truncated power splines and are denoted by angle brackets. For example, <f(x) - a> equals zero if the value of (f(x) - a) is negative; otherwise, it equals (f(x) - a). For example, <LogP - 5.5> is zero when LogP < 5.5; otherwise, it is equal to (LogP - 5.5), as shown in the following graph of the truncated power spline <LogP - 5.5>:

The constant a is called the knot of the spline. When a spline term is created, the knot is set using the value of the given feature from a random data sample.

A spline term partitions the data samples into two classes, depending on the value of its feature. The value of the spline is zero for one of the classes and nonzero for the other class. When a spline term is used in a model, the contribution of members of the first class can be adjusted independent of the members of the second class. Thus, regression with splines allows the incorporation of features that do not have a linear effect over their entire range.

Splines are interpreted as performing either range identification or outlier removal:

Range identification -- If there are many members in the nonzero partition, the spline identifies a range of effect. For example, the interpretation of the term <LogP - 5.5> in a model is that only high values of LogP affect the response.

Outlier removal -- If there are only a few members of the nonzero set, the spline identifies outliers. Regression can use the spline term to fit these members independent of the other terms of the model by, in effect, making them special cases based on the extreme value of a feature.

• Quadratic (x)2 -- Quadratic polynomial terms. Quadratic polynomials are simply second-degree polynomial terms. This term type allows pure quadratic terms to be added. Unlike the offset quadratic term, this term contains no constant, so it requires no special treatment by the LOF error measure.
• Offset Quad (x-a)2 -- This allows offset quadratic terms to be added to the equation. Because these terms contain an extra constant, they are rated by the lack-of-fit (LOF) error measure as equal in cost to two linear terms. Based on your intuition about the dataset with which you are working, you may want to use quadratic terms alone, or in combination with linear polynomial terms (above).
• Quad Spline -- Quadratic polynomial spline terms. These are simply linear spline terms taken to the power of two. While not commonly used, you may occasionally find a reason to suspect that the variables in a dataset may best be modeled by an equation that is quadratic over part of its range and ineffectual over the rest of its range.

Click the appropriate icon.

The icon is highlighted to indicate that equation terms of that type can be used to construct equations.

To deselect an equation term type

Click a highlighted icon to indicate that terms of that type cannot be used to construct equations.

Specifying mutation probabilities

Probability refers to the percentage of the time after a child equation is created that a mutation is attempted. If the attempted mutation lowers the fitness score of the child equation, that mutation is not kept. Instead, the original child equation is allowed to proceed. This makes high mutation-probability values relatively safe because, at worst, equation-mutation operations can cause no harm.

You can choose from among the following equation-mutation operations:

• Add New Term -- This mutation helps generate diversity in the equation population by randomly adding a newly generated term to a child equation. Only those term types that you select (as described above) are added to child equations.
• Shift Spline Knot -- This mutation tries to improve the score of a child equation that contains a spline term by shifting its knot. This helps to optimize the partitioning that is done by splines. For more information about spline terms, see Selecting equation term types on page 241.
• Reduce Equation -- This mutation tries eliminating each term of a child equation, finally eliminating the term that contributes least to the model. This mutation operation creates pressure for small, compact models.
• Extend Equation -- This mutation extends a child equation by adding a linear polynomial term that contains a variable of the data. Each feature is tried in turn, and the feature that contributes most is kept.

Use the slider or the entry box after the name of each equation-mutation operation to specify the probability that the specified mutation is attempted in a generation. The probability value should be an integer between 0 and 100.

Equation-mutation operations with a probability value of 0 are not performed on a child equation.

Specifying other genetic analysis preferences

• The size of the equation population.
• The smoothing parameter d, which controls the scoring bias between equations of different sizes (as measured by the lack-of-fit (LOF) score).
• The number of terms in each randomly constructed equation.
• The length of an equation.

• When you select GFA as your statistical method and request generation of a QSAR equation by clicking the Run icon on the QSAR tool bar.
• When you randomize the current equation population (that is, when you click Randomize on the Statistical Method Preferences control panel when the Statistical Method popup is set to GFA). For more information about randomizing the equation population, see Working with the current equation population on page 238.

To establish the population size

Enter the appropriate number of equations in the Population Size entry box. This value takes effect the next time an equation population is built.

Setting the smoothing parameter d

• Increase the value (for example, from 1.0 to 2.0) to create a bias toward smoother (that is, smaller) models. Equations with a greater number of terms have a larger penalty and, therefore, must predict significantly better in order to score well and survive in the population.
• Decrease the value (for example, from 1.0 to 0.5) to lessen the bias toward smaller models. The GFA algorithm can explore larger models.

Enter the appropriate value in the Smoothness (d) entry box.

Setting the number of equation terms

• Only those types of equation terms that you select are used to construct the equations. For more information about selecting these terms, see Selecting equation term types on page 241.
• As evolution proceeds, the actual equations that survive in the population may contain a greater or less number of terms, depending on the quality of the models found in the dataset.

You can do either of the following:

• Enter the number of terms in the Randomized Equation Length entry box.
• Use the up- or down-arrows to change the value displayed in the Randomized Equation Length entry box.

To specify equations of fixed length,

Select Fixed Length Equations from the popup. An entry box is displayed that is used to specify the length of the equation.

Setting the regression method

For the GFA method, the default is least squares. For G/PLS, the default is pls, with four components and no data scaling.

You can change the default using the Statistical Method popup. If you choose PLS, you also can set the number of components and the type of data scaling to perform.

Using genetic partial least squares

The G/PLS algorithm uses GFA to select appropriate basis functions to be used in a model of the data and pls regression as the fitting technique to weigh the basis functions' relative contributions in the final model. Application of G/PLS allows the construction of larger QSAR equations while avoiding overfitting and eliminating most variables.

This section describes

Running a G/PLS calculation (below)

Setting G/PLS preferences (page 248)

Before you begin

You should have added molecular structures, biological activity information, and descriptor values to a study table. Additionally, you should have selected the dependent and independent variables for your analysis.

Setting up and running a G/PLS calculation

1.   Select G/PLS from the Statistical Method popup on the Statistical Method control pane or select G/PLS from the Method popup at the top of the study table.

When you select G/PLS, the Statistical Method Preferences control panel's appearance changes.

2.   If you want to adjust the default G/PLS parameters, click Configure GFA.... The Configure GFA control panel appears. For detailed information on preferences, see Setting genetic analysis preferences on page 240.

3.   After selecting G/PLS as your method, select Configure on the GENETIC ANALYSIS (GFA) card or Configure GFA... in the Statistical Method Preferences control panel. The Configure GFA control panel appears in G/PLS mode.

4.   Select the appropriate preferences for your work and the G/ PLS algorithm. For information on setting preferences, see the next section.

5.   When you have completed setting preferences, begin a G/PLS calculation by clicking Run on the study table toolbar.

1.   Select a value for Randomized Equation Length by entering a number in the entry box. Values between 5 and 15 are typical for a G/PLS run.

2.   Select Fix Equation Length from the popup, then enter a value for the number of components in the equation. This number must be the same as the number you entered in the Randomized Equation Length box.

3.   Choose PLS from the popup at the bottom left of the Genetic Preferences section of the control panel.

4.   Specify the number of components (latent variables) in the components entry box. The more components selected, the more detail is represented in the QSAR model.

5.   Use the popup in the bottom right of the control panel to select the scaling method to apply to variable variances for the PLS calculation:

Scaled -- Normalize all variables to a variance of 1.0.

Unit Scaling -- Scale all variables of the same unit as a group, then equalize the average variance of each group with the average variance of all other groups. This is the most useful method for analysis of data having meaningful differences of variance and between variables of the same type, such as field or probe data.

No Scaling -- Leave data as originally calculated.

Scaling is important because pls tries to preserve the difference in variance between variables. In most QSAR datasets, the differences are due primarily to the choice of unit and are not meaningful. Therefore, some scaling generally should be applied.