Toward Reliable Automatic Proxies for Code Comprehensibility

Improving code comprehensibility requires measuring it reliably, yet no reliable automatic proxy for comprehensibility exists. Developing one requires a validated human-study baseline to test against. However, the human-study proxies used in the literature (e.g., Likert-scale ratings or response time on output-prediction tasks) have not been systematically validated either, and the construct itself lacks a standard definition. We address this foundational gap first: we introduce a conceptualization of comprehensibility based on expert agreement, construct a ground truth via the Delphi protocol, and evaluate commonly used human-study proxies against it. We find that proxies based on input--output reasoning and response time are particularly reliable, while syntax-based proxies are especially unreliable. With this validated foundation, we propose a controlled human study to test whether code verifiability---the ease with which static verification tools can analyze code---can serve as a reliable automatic proxy for comprehensibility. If verifiability alone explains comprehension difficulty well, it can be used as a standalone automatic proxy; if it only partially explains it, it becomes one signal among several in a combined automatic proxy. Beyond this study, the research agenda includes reassessing prior work in light of proxy reliability, evaluating LLM-based agents as substitutes for human participants, and developing tools that leverage reliable automatic proxies to improve code comprehensibility.