|Copilot Research Asks Who Will Clean Up The Mess
|Written by Sue Gee
|Wednesday, 31 January 2024
A whitepaper from GitClear sets out to investigate the quality and maintainability of AI-assisted code compared to what would have been written by a human. It highlights a worrying increase in "code churn" due to reliance on Copilot.
We've previously reported on devs' enthusiasm for ChatGPT, Copilot and other forms of AI assistance and have reported on claims for improved productivity. Now some research has been published that raises doubts. It comes from GitClear and the findings are reported in a whitepaper:
Backing up the reference in the subtitle to the negative impact Copilot is having on code quality, the report shared a tweet from Adam Tornhill, the author of Your Code as a Crime Scene:
making the comment:
GitHub claims that code is written "55% faster" with Copilot. But what about code that shouldn't be written in the first place? The problem here is that code spends 10x more time being read than being written, according to Robert Martin, author of Clean Code: A Handbook of Agile Software Craftsmanship. Writing bad code faster implies considerable pains for the subsequent code readers.
To investigate the influence of AI-assistance GitClear used its Diff Delta tool to analyze 153 million changed lines of code, authored between January 2020 and December 2023 and projected the trends for 2024:
GitClear noted that since the introduction of Copilot in 2022 the percentage of 'added code' and 'copy/pasted code' had increased in proportion to 'updated,' 'deleted,' and 'moved 'code commenting:
In this regard, AI-generated code resembles an itinerant contributor, prone to violate the DRY-ness [don't repeat yourself] of the repos visited."
What GitClear finds most concerning is the increase in "churn" which it defines as the percentage of lines of code that are reverted or updated less than two weeks after being authored, showing that it is projected to double in 2024 compared to its 2021, pre-AI baseline:
The paper concludes:
There's no question that, as AI has surged in popularity, we have entered an era where code lines are being added faster than ever before. The better question for 2024: who's on the hook to clean up the mess afterward?"
Hopefully the answer to this is developers aided by Copilot.
After all, we are looking to AI to take on the spade work and that includes mucking out the stables as well as providing fresh fodder.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Wednesday, 31 January 2024 )