Objectives: To develop and validate an algorithm to predict occult nodal metastasis in clinically node negative oral cavity squamous cell carcinoma (OCSCC) using machine learning. To compare algorithm performance to a model based on tumor depth of invasion (DOI). Materials and methods: Patients who underwent primary tumor extirpation and elective neck dissection from 2007 to 2013 for clinical T1-2N0 OCSCC were identified from the National Cancer Database (NCDB). Multiple machine learning algorithms were developed to predict pathologic nodal metastasis using clinicopathologic data from 782 patients. The algorithm was internally validated using test data from 654 patients in NCDB and was then externally validated using data from 71 patients treated at a single academic institution. Performance was measured using area under the receiver operating characteristic (ROC) curve (AUC). Machine learning and DOI model performance were compared using Delong’s test for two correlated ROC curves. Results: The best classification performance was achieved with a decision forest algorithm (AUC=0.840). When applied to the single-institution data, the predictive performance of machine learning exceeded that of the DOI model (AUC=0.657, p=0.007). Compared to the DOI model, machine learning reduced the number of neck dissections recommended while simultaneously improving sensitivity and specificity. Conclusion: Machine learning improves prediction of pathologic nodal metastasis in patients with clinical T1- 2N0 OCSCC compared to methods based on DOI. Improved predictive algorithms are needed to ensure that patients with occult nodal disease are adequately treated while avoiding the cost and morbidity of neck dissection in patients without pathologic nodal disease.
Keywords: Oral cancer | Squamous cell carcinoma | Machine learning | Artificial intelligence

Recent advances in machine learning and artificial intelligence offer promising applications to radiology quality improvement initiatives as they relate to the radiology value network. Coordination within the interlocking web of systems, events, and stakeholders in the radiology value network may be mitigated though standardization, automation, and a focus on workflow efficiency. In this article the authors present applications of these various strategies via use cases for quality improvement projects at different points in the radiology value network. In addition, the authors discuss opportunities for machine-learning applications in data aggregation as opposed to traditional applications in data extraction.
Key Words: Machine learning | artificial intelligence | radiology quality improvement | radiology value network | data aggregation

Background: We explored whether the use of deep learning to model combinations of symptom-physical signs and objective tests, such as lung function tests and the bronchial challenge test, would improve model performance in predicting the initial diagnosis of adult asthma when compared to the conventional machine learning diagnostic method. Methods: The data were obtained from the clinical records on prospective study of 566 adult outpatients who visited Kindai University Hospital for the first time with complaints of non-specific respiratory symptoms. Asthma was comprehensively diagnosed by specialists based on symptom-physical signs and objective tests. Model performance metrics were compared to logistic analysis, support vector machine (SVM) learning, and the deep neural network (DNN) model. Results: For the diagnosis of adult asthma based on symptom-physical signs alone, the accuracy of the DNN model was 0.68, whereas that for the SVM was 0.60 and for the logistic analysis was 0.65. When adult asthma was diagnosed based on symptom-physical signs, biochemical findings, lung function tests, and the bronchial challenge test, the accuracy of the DNN model increased to 0.98 and was significantly higher than the 0.82 accuracy of the SVM and the 0.94 accuracy of the logistic analysis. Conclusions: DNN is able to better facilitate diagnosing adult asthma, compared with classical machine learnings, such as logistic analysis and SVM. The deep learning models based on symptom-physical signs and objective tests appear to improve the performance for diagnosing adult asthma
Keywords: Artificial intelligence | Asthma | Deep learning | Diagnosis | Support vector machine

OBJECTIVE: Virtual reality simulators track all movements and forces of simulated instruments, generating enormous datasets which can be further analyzed with machine learning algorithms. These advancements may increase the understanding, assessment and training of psychomotor performance. Consequently, the application of machine learning techniques to evaluate performance on virtual reality simulators has led to an increase in the volume and complexity of publications which bridge the fields of computer science, medicine, and education. Although all disciplines stand to gain from research in this field, important differences in reporting exist, limiting interdisciplinary communication and knowledge transfer. Thus, our objective was to develop a checklist to provide a general framework when reporting or analyzing studies involving virtual reality surgical simulation and machine learning algorithms. By including a total score as well as clear subsections of the checklist, authors and reviewers can both easily assess the overall quality and specific deficiencies of a manuscript. DESIGN: The Machine Learning to Assess Surgical Expertise (MLASE) checklist was developed to help computer science, medicine, and education researchers ensure quality when producing and reviewing virtual reality manuscripts involving machine learning to assess surgical expertise. SETTING: This study was carried out at the McGill Neurosurgical Simulation and Artificial Intelligence Learning Centre. PARTICIPANTS: The authors applied the checklist to 12 articles using machine learning to assess surgical expertise in virtual reality simulation, obtained through a systematic literature review. RESULTS: Important differences in reporting were found between medical and computer science journals. The medical journals proved stronger in discussion quality and weaker in areas related to study design. The opposite trends were observed in computer science journals. CONCLUSIONS: This checklist will aid in narrowing the knowledge divide between computer science, medicine, and education: helping facilitate the burgeoning field of machine learning assisted surgical education. ( J Surg Ed 000:1
10.
2019 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
KEY WORDS: simulation| surgery | education | artificial intelligence | assessment | machine learning

AI systems are usually evaluated on a range of problem instances and compared to other AI systems that use different strategies. These instances are rarely independent. Machine learning, and supervised learning in particular, is a very good example of this. Given a machine learning model, its behaviour for a single instance cannot be understood in isolation but rather in relation to the rest of the data distribution or dataset. In a dual way, the results of one machine learning model for an instance can be analysed in comparison to other models. While this analysis is relativeto a population or distribution of models, it can give much more insight than an isolated analysis. Item response theory (IRT) combines this duality between itemsand respondentsto extract latent variables of the items (such as discrimination or difficulty) and the respondents (such as ability). IRT can be adapted to the analysis of machine learning experiments (and by extension to any other artificial intelligence experiments). In this paper, we see that IRT suits classification tasks perfectly, where instances correspond to items and classifiers correspond to respondents. We perform a series of experiments with a range of datasets and classification methods to fully understand what the IRT parameters such as discrimination, difficulty and guessing mean for classification instances (and their relation to instance hardness measures) and how the estimated classifier ability can be used to compare classifier performance in a different way through classifier characteristic curves.
Keywords: Artificial intelligence evaluation | Item response theory | Machine learning | Instance hardness | Classifier metrics