91ºÚÁÏÍø

Software maintenance makes up a large share of the total cost of software over its lifetime, and a big part of that cost comes from understanding existing code. One way to make code easier to understand is through documentation, especially comments that summarize what the code does or explain why it does it. In this work, we introduce ComCat, a system that uses large language models (LLMs, which are AI models trained on very large amounts of text) together with expert guidance to automatically generate useful comments for source code. ComCat is designed to choose the most relevant and informative comment for a specific piece of code. For C/C++ files, the system works in three steps: it first finds places where comments would be most helpful, then decides what kind of comment is needed, and finally writes the comment. In a study with human participants, ComCat’s comments improved code understanding on three software engineering tasks by up to 13% for most participants. The generated comments were also judged to be at least as accurate and readable as human-written comments, and they were preferred over standard ChatGPT-generated comments for up to 92% of code snippets. We also released a dataset containing code snippets, human-written comments, and human-labeled comment categories. Overall, ComCat shows that LLMs can be used to meaningfully improve how well people understand code.

Fig. 1.

ComCatÌýpipeline and study procedure. We use three instances of HSR to informÌýComCat’s design (1) and evaluate developer performance (2) and preference (3) with our tool.ÌýComCatÌýtakes C/C++ code as input, using a Code Parser to identify code Snippets to be commented. These Snippets are classified, and the class of each Snippet is used in combination with our Template Catalog to create a prompt for each Snippet. These prompt ChatGPT, which outputs the commented code. This pipeline is informed by developer expertise, but it is fully automated and requires no human intervention.