|Does Strong Typing Help?|
|Written by Mike James|
|Wednesday, 18 October 2017|
We all have our opinions on static strong typing versus dynamic or even no real typing at all, but hard evidence of what works is thin on the ground. Now some new research gives us some facts to throw into the debate.
How can you prove, or at least get closer to the truth about whether, a particular technique reduces bugs or not? You can't really compare code bases created with and without the technique and then count the bugs because too much else varies - the programmers, the task. the bug detection rate. and so on.
Zheng Gao and Earl T. Barr at University College London and Christian Bird at Microsoft Research have a good idea how to do the job. As most projects use a source management system of some sort, it should be possible to go back to the code just before a bug was fixed and see if enabling type checking finds the bug in the unfixed code.
So the method is roughly:
This basic idea is that if the bug is flagged by the type-checking tool then it would have been found by the programmers before it got into the code base and would never have existed.
So what was the conclusion?
Both type checking approaches, Flow and TypeScript, detected 15% of the bugs. That is, out of 400 bugs Flow detected 57 and TypeScript detected 3 that Flow missed and vice versa.
This may not seem a great deal, but as the paper points out this is a conservative estimate of the bugs that strong typing might detect. The 400 bugs had survived testing and code reviews and so were slightly harder than bugs that were removed earlier in the process. Even so the researchers quote an unnamed engineering manager at Microsoft's response:
“That’s shocking. If you could make a change to the way we do development that would reduce the number of bugs being checked in by 10% or more overnight, that’s a no-brainer. Unless it doubles development time or something, we’d do it.”
Well this is exactly the point. We do have some hard facts to add to the debate, but the issue isn't how many bugs strong typing finds it is also a question of how much it increases the effort of producing the code. There is also the question of could we put that increased effort to better use.
There is also the point that Flow in particular is a static type checking tool that can make use of type inference to check the code base. A language doesn't have to be strongly typed to be type-checked in this sense. In addition type-checking is just one form of static code analysis and you can ask the question why just check type?
For example, the histogram of bugs that were undetectable based on type reveals types of bug that would have a chance of showing on a static code analysis:
It seems that we still need more data and the arguments can continue.
In a world where it is still respectable to claim to code in a text editor, no matter how clever, and use the compiler from the command line, it seems that there is a great deal that can be done to improve the programming environment.
We need more tools, not necessarily more restrictive languages.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Wednesday, 18 October 2017 )|