Objectives: Endotracheal intubation (ETI) is an important skill for all emergency providers; our ability to train and assess our learners is integral to providing optimal patient care. The primary aim of this study was to assess the inter-rater reliability (IRR) and discriminant validity of a novel field ETI assessment tool using a checklist-derived performance score (PS) and critical failure (CF) rate. Methods: Forty-three participants (18 paramedic students, 11 paramedics, and 14 emergency physicians [EPs]) performed ETI during a simulated trauma scenario on a pseudo-ventilated cadaver. Each participant was assessed by two experienced raters. IRR was calculated using the intraclass correlation coefficient. Regarding discriminant validity, a Kruskal-Wallis test was used to analyze PSs and a χ 2 test was used for CFs. Mean global rating scale (GRS) scores were compared using an analysis of variance. Results: The ETI assessment tool had excellent IRR, with an intraclass correlation coefficient of 0.94. There was a significant difference in PSs, CFs, and GRSs (p < 0.05) between cohorts. Conclusion: The novel field ETI assessment tool has excellent reliability among trained raters and discriminates between experienced ETI providers (EPs) and less experienced ETI performers using PSs, CFs, and GRSs on a fresh cadaveric model.