%0 Journal Article
%T Inter-rater reliability of ACS-NSQIP colorectal procedure coding in Canada.
%A Xiong Y
%A Spence RT
%A Hirsch G
%A Walsh MJ
%A Neumann K
%J Am J Surg
%V 0
%N 0
%D 2024 May 31
%M 38944624
%F 3.125
%R 10.1016/j.amjsurg.2024.115787
%X <B>BACKGROUND: </B>The American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) uses Current Procedural Terminology (CPT) codes for risk-adjusted calculations. This study evaluates the inter-rater reliability of coding colorectal resections across Canada by ACS-NSQIP surgical clinical nurse reviewers (SCNR) and its impact on risk predictions.<BR><B>METHODS: </B>SCNRs in Canada were asked to code simulated operative reports. Percent agreement and free-marginal kappa correlation were calculated. The ACS-NSQIP risk calculator was utilized to illustrate its impact on risk prediction.<BR><B>RESULTS: </B>Responses from 44 of 150 (29.3 ​%) SCNRs revealed 3 to 6 different codes chosen per case, with agreement ranging from 6.7 ​% to 62.3 ​%. Free-marginal kappa correlation ranged from moderate agreement (0.53) to high disagreement (-0.17). ACS-NSQIP risk calculator predicted large absolute differences in risk for serious complications (0.2 ​%-13.7 ​%) and mortality (0.2 ​%-6.3 ​%).<BR><B>CONCLUSIONS: </B>This study demonstrated low inter-rater reliability in coding ACS-NSQIP colorectal procedures in Canada among SCNRs, impacting risk predictions.