Model validation is the most important part of building a supervised model.For building a model with good generalization performance one must have a sensible data splitting strategy,and this is crucial for model valid...Model validation is the most important part of building a supervised model.For building a model with good generalization performance one must have a sensible data splitting strategy,and this is crucial for model validation.In this study,we con-ducted a comparative study on various reported data splitting methods.The MixSim model was employed to generate nine simulated datasets with different probabilities of mis-classification and variable sample sizes.Then partial least squares for discriminant analysis and support vector machines for classification were applied to these datasets.Data splitting methods tested included variants of cross-validation,bootstrapping,bootstrapped Latin partition,Kennard-Stone algorithm(K-S)and sample set partitioning based on joint X-Y distances algorithm(SPXY).These methods were employed to split the data into training and validation sets.The estimated generalization performances from the validation sets were then compared with the ones obtained from the blind test sets which were generated from the same distribution but were unseen by the train-ing/validation procedure used in model construction.The results showed that the size of the data is the deciding factor for the qualities of the generalization performance estimated from the validation set.We found that there was a significant gap between the performance estimated from the validation set and the one from the test set for the all the data splitting methods employed on small datasets.Such disparity decreased when more samples were available for training/validation,and this is because the models were then moving towards approximations of the central limit theory for the simulated datasets used.We also found that having too many or too few samples in the training set had a negative effect on the estimated model performance,suggesting that it is necessary to have a good balance between the sizes of training set and validation set to have a reliable estimation of model performance.We also found that systematic sampling method such a展开更多
China implemented the public hospital reform in 2012. This study utilized bootstrapping data envelopment analysis（DEA） to evaluate the technical efficiency（TE） and productivity of county public hospitals in Easter...China implemented the public hospital reform in 2012. This study utilized bootstrapping data envelopment analysis（DEA） to evaluate the technical efficiency（TE） and productivity of county public hospitals in Eastern, Central, and Western China after the 2012 public hospital reform. Data from 127 county public hospitals（39, 45, and 43 in Eastern, Central, and Western China, respectively） were collected during 2012–2015. Changes of TE and productivity over time were estimated by bootstrapping DEA and bootstrapping Malmquist. The disparities in TE and productivity among public hospitals in the three regions of China were compared by Kruskal–Wallis H test and Mann–Whitney U test. The average bias-corrected TE values for the four-year period were 0.6442, 0.5785, 0.6099, and 0.6094 in Eastern, Central, and Western China, and the entire country respectively, with average non-technical efficiency, low pure technical efficiency（PTE）, and high scale efficiency found. Productivity increased by 8.12%, 0.25%, 12.11%, and 11.58% in China and its three regions during 2012–2015, and such increase in productivity resulted from progressive technological changes by 16.42%, 6.32%, 21.08%, and 21.42%, respectively. The TE and PTE of the county hospitals significantly differed among the three regions of China. Eastern and Western China showed significantly higher TE and PTE than Central China. More than 60% of county public hospitals in China and its three areas operated at decreasing return scales. There was a considerable space for TE improvement in county hospitals in China and its three regions. During 2012–2015, the hospitals experienced progressive productivity; however, the PTE changed adversely. Moreover, Central China continuously achieved a significantly lower efficiency score than Eastern and Western China. Decision makers and administrators in China should identify the causes of the observed inefficiencies and take appropriate measures to increase the efficiency of county public hospitals in the th展开更多
Some studies have suggested that early surgical treatment can effectively improve the prognosis of cervical spinal cord injury without radiological abnormality, but no research has focused on the development of a prog...Some studies have suggested that early surgical treatment can effectively improve the prognosis of cervical spinal cord injury without radiological abnormality, but no research has focused on the development of a prognostic model of cervical spinal cord injury without radiological abnormality. This retrospective analysis included 43 patients with cervical spinal cord injury without radiological abnormality. Seven potential factors were assessed: age, sex, external force strength causing damage, duration of disease, degree of cervical spinal stenosis, Japanese Orthopaedic Association score, and physiological cervical curvature. A model was established using multiple binary logistic regression analysis. The model was evaluated by concordant profiling and the area under the receiver operating characteristic curve. Bootstrapping was used for internal validation. The prognostic model was as follows: logit(P) =-25.4545 + 21.2576 VALUE + 1.2160SCORE-3.4224 TIME, where VALUE refers to the Pavlov ratio indicating the extent of cervical spinal stenosis, SCORE refers to the Japanese Orthopaedic Association score(0–17) after the operation, and TIME refers to the disease duration(from injury to operation). The area under the receiver operating characteristic curve for all patients was 0.8941(95% confidence interval, 0.7930–0.9952). Three factors assessed in the predictive model were associated with patient outcomes: a great extent of cervical stenosis, a poor preoperative neurological status, and a long disease duration. These three factors could worsen patient outcomes. Moreover, the disease prognosis was considered good when logit(P) ≥-2.5105. Overall, the model displayed a certain clinical value. This study was approved by the Biomedical Ethics Committee of the Second Affiliated Hospital of Xi'an Jiaotong University, China(approval number: 2018063) on May 8, 2018.展开更多
基金YX and RG thank Wellcome Trust for funding MetaboFlow(Grant 202952/Z/16/Z).
文摘Model validation is the most important part of building a supervised model.For building a model with good generalization performance one must have a sensible data splitting strategy,and this is crucial for model validation.In this study,we con-ducted a comparative study on various reported data splitting methods.The MixSim model was employed to generate nine simulated datasets with different probabilities of mis-classification and variable sample sizes.Then partial least squares for discriminant analysis and support vector machines for classification were applied to these datasets.Data splitting methods tested included variants of cross-validation,bootstrapping,bootstrapped Latin partition,Kennard-Stone algorithm(K-S)and sample set partitioning based on joint X-Y distances algorithm(SPXY).These methods were employed to split the data into training and validation sets.The estimated generalization performances from the validation sets were then compared with the ones obtained from the blind test sets which were generated from the same distribution but were unseen by the train-ing/validation procedure used in model construction.The results showed that the size of the data is the deciding factor for the qualities of the generalization performance estimated from the validation set.We found that there was a significant gap between the performance estimated from the validation set and the one from the test set for the all the data splitting methods employed on small datasets.Such disparity decreased when more samples were available for training/validation,and this is because the models were then moving towards approximations of the central limit theory for the simulated datasets used.We also found that having too many or too few samples in the training set had a negative effect on the estimated model performance,suggesting that it is necessary to have a good balance between the sizes of training set and validation set to have a reliable estimation of model performance.We also found that systematic sampling method such a
基金supported by the National Natural Science Foundation of China(No.71473099)
文摘China implemented the public hospital reform in 2012. This study utilized bootstrapping data envelopment analysis（DEA） to evaluate the technical efficiency（TE） and productivity of county public hospitals in Eastern, Central, and Western China after the 2012 public hospital reform. Data from 127 county public hospitals（39, 45, and 43 in Eastern, Central, and Western China, respectively） were collected during 2012–2015. Changes of TE and productivity over time were estimated by bootstrapping DEA and bootstrapping Malmquist. The disparities in TE and productivity among public hospitals in the three regions of China were compared by Kruskal–Wallis H test and Mann–Whitney U test. The average bias-corrected TE values for the four-year period were 0.6442, 0.5785, 0.6099, and 0.6094 in Eastern, Central, and Western China, and the entire country respectively, with average non-technical efficiency, low pure technical efficiency（PTE）, and high scale efficiency found. Productivity increased by 8.12%, 0.25%, 12.11%, and 11.58% in China and its three regions during 2012–2015, and such increase in productivity resulted from progressive technological changes by 16.42%, 6.32%, 21.08%, and 21.42%, respectively. The TE and PTE of the county hospitals significantly differed among the three regions of China. Eastern and Western China showed significantly higher TE and PTE than Central China. More than 60% of county public hospitals in China and its three areas operated at decreasing return scales. There was a considerable space for TE improvement in county hospitals in China and its three regions. During 2012–2015, the hospitals experienced progressive productivity; however, the PTE changed adversely. Moreover, Central China continuously achieved a significantly lower efficiency score than Eastern and Western China. Decision makers and administrators in China should identify the causes of the observed inefficiencies and take appropriate measures to increase the efficiency of county public hospitals in the th
基金supported by the National Natural Science Foundation of China,No.30672136(to HPL)
文摘Some studies have suggested that early surgical treatment can effectively improve the prognosis of cervical spinal cord injury without radiological abnormality, but no research has focused on the development of a prognostic model of cervical spinal cord injury without radiological abnormality. This retrospective analysis included 43 patients with cervical spinal cord injury without radiological abnormality. Seven potential factors were assessed: age, sex, external force strength causing damage, duration of disease, degree of cervical spinal stenosis, Japanese Orthopaedic Association score, and physiological cervical curvature. A model was established using multiple binary logistic regression analysis. The model was evaluated by concordant profiling and the area under the receiver operating characteristic curve. Bootstrapping was used for internal validation. The prognostic model was as follows: logit(P) =-25.4545 + 21.2576 VALUE + 1.2160SCORE-3.4224 TIME, where VALUE refers to the Pavlov ratio indicating the extent of cervical spinal stenosis, SCORE refers to the Japanese Orthopaedic Association score(0–17) after the operation, and TIME refers to the disease duration(from injury to operation). The area under the receiver operating characteristic curve for all patients was 0.8941(95% confidence interval, 0.7930–0.9952). Three factors assessed in the predictive model were associated with patient outcomes: a great extent of cervical stenosis, a poor preoperative neurological status, and a long disease duration. These three factors could worsen patient outcomes. Moreover, the disease prognosis was considered good when logit(P) ≥-2.5105. Overall, the model displayed a certain clinical value. This study was approved by the Biomedical Ethics Committee of the Second Affiliated Hospital of Xi'an Jiaotong University, China(approval number: 2018063) on May 8, 2018.
