Investigating the influence of data splitting on the predictive ability of QSAR/QSPR models

The study was aimed at investigating how the method of splitting data into a training set and a test set influences the external predictivity of quantitative structure-activity and/or structure-property relationships (QSAR/QSPR) models. Six models of good quality were collected from the literature and then redeveloped and validated on the basis of five alternative splitting algorithms, namely: (i) a commonly used algorithm ('Z:1'), in which every zth (e.g. third) from the compounds sorted ascending (according to the response values, y) is selected into the test set; (ii-iv) three variations of the Kennard-Stone algorithm; and (v) the duplex algorithm. The external validation statistics reported for each model served as a basis for the final comparison. We demonstrated that the splitting techniques utilizing the values of molecular descriptors alone (X) or in combination with the model response (y) always lead to the development of the models yielding better external predictivity in comparison with the models designed with methodologies based on the y-values only. Moreover, we showed that the external validation coefficient (Q2EXT) is more sensitive to the splitting technique than the root mean square error of prediction (RMSEP). This difference becomes especially important when the test set is relatively small (between 5-10 compounds). In the case of the models trained/validated with a small number of compounds, it is strongly recommended that both statistics (Q2EXT and RMSEP) are taken into account for the external predictivity evaluation.

PUZYN Tomasz;  MOSTRAG-SZLICHTYNG A.;  GAJEWICZ Agnieszka;  SKRZYNSKI Michal;  WORTH Andrew; 

2011-11-03

SPRINGER/PLENUM PUBLISHERS

JRC63016

1040-0400,   

http://www.springerlink.com/content/t5847116x5672609/,    https://publications.jrc.ec.europa.eu/repository/handle/JRC63016,   

10.1007/s11224-011-9757-4,