Codeacy has launched a free benchmarking survey to help engineering teams measure the risk profile of their AI coding workflows when using tools like GitHub Copilot, Cursor, or Claude and compare it against the industry standard. 

Codacy is know for its automated application security and code quality solutions and, as we reported in July, for Codacy Guardrails, a proprietary IDE plugin that automatically repairs security and quality violations in AI-generated code before it is even viewed by the user, allowing organizations to enforce compliance from the moment of code inception. 

As Google's recent DORA Report revealed, over 90% of developers now use AI tools at work. Even so most organizations do not have have clear AI-use policies and controls in place, leading to what Codacy perceives as a gaping "AI Governance Gap":

It points out:

AI coding assistants aren’t like other productivity tools. They generate and modify source code, touch production systems, and can leak sensitive data or pull unverified dependencies.

It’s a new attack surface hiding inside your development workflow, so a clear governance plan and security controls aren't optional.

This spurred Codacy to create the AI Coding Risk Assessment, a survey comprising 24 questions that measures the security and compliance posture of an organization’s AI-assisted development workflows.

Jaime Jorge, CEO and Co-founder of Codacy explains:

"After speaking with leading AI industry figures, we observed a need for a unified, data-backed resource. That's why we created this benchmark. It helps companies identify where they stand, compare themselves to the market, and take concrete, actionable steps to leverage AI at scale."

Completing the survey takes about 10 minutes and the very act of doing so will make many respondents recognise shortcomings in their current AI-coding security as they are asked questions relating to policy and governance; security and risk management and culture and training.

There survey provides immediate feedback by way a score from 0 to 100: 

On this scale a score of 0-25 is  deemed Critical,  26-50 Needs Work, 51-75 is Good and 75-100 is excellent.

Next comes a benchmark that allows them to see exactly how their company's practices compare to others in the industry:  

Despite being firmly in the "Needs Work" quartile and being is Flagged as "High Risk", the score of 40 is only 3 points below the industry average. Separate scores are now shown for  Compliance, where a score of 37 is just one point below the average; and Security, where a score of 47 is well below the average. So this organisation really needs the help of the AI Governance and Security checklist the Codacy follows up with to address gaps.

Codacy can also provide further advice for remediation which is emailed on request. This comprises as a list of Key Recommendations together with a personalized, AI-generated, report, setting out the scores and industry comparison along with "Actionable Recommendations" to improve both compliance and security, such as:

Immediately implement automated scanning and DAST scanning for AI systems, as these are critical security controls currently missing, contributing to below-average security performance.

Clearly there is an intention to promote the adoption of Coadacy Guardrails here. However the report also has Recommended Reading from various sources. 

Helpfully the report includes the questions and your answers which can help organisations track their progress - as of course will retaking the survey. This means that we should see an improvement  in the Industry Average scores for both Security and Compliance in due course.

It is entirely free to participate in the survey and access the AI Governance Checklist and benchmark data, visit: https://ai-risk.codacy.com/. 

More Information

Benchmark The Risk Profile of Your AI Coding Posture

Risk Assessment

Related Articles

Codacy Guardrails For Secure AI-Generated Code

DORA Report Reveals Widespread Reliance On AI 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info