Use of natural language processing method to identify regional anesthesia from clinical notes

Introduction: Accurate data capture is integral for research and quality improvement efforts. Unfortunately, limited guidance for defining and documenting regional anesthesia has resulted in wide variation in documentation practices, even within individual hospitals, which can lead to missing and inaccurate data. This cross-sectional study sought to evaluate the performance of a natural language processing (NLP)-based algorithm developed to identify regional anesthesia within unstructured clinical notes.

Methods: We obtained postoperative clinical notes for all patients undergoing elective non-cardiac surgery with general anesthesia at one of six Veterans Health Administration hospitals in California between January 1, 2017, and December 31, 2022. After developing and executing our algorithm, we compared our results to a frequently used referent, the Corporate Data Warehouse structured data, to assess the completeness and accuracy of the currently available data. Measures of agreement included sensitivity, positive predictive value, false negative rate, and accuracy.

Results: We identified 27,713 procedures, of which 9310 (33.6%) received regional anesthesia. 96.6% of all referent regional anesthesia cases were identified in the clinic notes with a very low false negative rate and good accuracy (false negative rate=0.8%, accuracy=82.5%). Surprisingly, the clinic notes documented more than two times the number of regional anesthesia cases that were documented in the referent (algorithm n=9154 vs referent n=4606).

Discussion: While our algorithm identified nearly all regional anesthesia cases from the referent, it also identified more than two times as many regional anesthesia cases as the referent, raising concerns about the accuracy and completeness of regional anesthesia documentation in administrative and clinical databases. We found that NLP was a promising alternative for identifying clinical information when existing databases lack complete documentation.