[Freeswitch-users] The FreeSWITCH project and Static Analysis

[Michael Giagnocavo]

How did PVS fare against MSVC’s /analyze?

I’d also be rather suspicious if in those 3.2m or 875k lines only generated some minor Windows-only bugs. What about the results from e.g. Coverity?

-Michael

From: freeswitch-users-bounces at lists.freeswitch.org [mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Ken Rice
Sent: Tuesday, 8 December, 2015 11:26
To: freeswitch-users at lists.freeswitch.org
Subject: [Freeswitch-users] The FreeSWITCH project and Static Analysis


New Post on freeswitch.org from Kathleen King
check it out at http://ift.tt/1R7Yh7U
The FreeSWITCH project and Static Analysis

The FreeSWITCH project is nearly ten years old, and the FreeSWITCH git repo has commits from about 214 different authors and over 3.2 million lines of code with 875k of those lines under the src directory. Some of the maintenance challenges associated with such a large software project include: detecting and resolving human errors such as typos, logic inversions, and dangerous formatting. Implementing code review is a must, and there are different techniques common to the industry used to reduce the defect density and standardize the code format: autobuilding against multiple compilers, routine testing, and static code analysis. The core development team at FreeSWITCH uses all three techniques.

Both autobuilding and routine testing can be applied with in-house system workflows. Routinely building the packages against different compilers allows for consistent tracking to make sure additional commits won’t break existing code in any of the prepackaged builds. This also allows for consistent handling of packages for multiple operating systems. By autobuilding against different compilers, we can make sure that a commit for one set of packages doesn’t break the builds for the others. Routine testing is another viable option for code review, and routinely testing and implementing a bug tracking system allows the community members to report bugs found in unique environmental circumstances. Open-source software relies on many different eyes to keep bugs shallow, and this practice opens up different configurations and applications of the software for a more thorough testing. Each year hundreds of tickets are opened on the FreeSWITCH project JIRA, and the developers work tirelessly to address all of them.

Static analyzers can scan thousands to millions of lines of code without getting tired and usually don’t require many manual steps to run. The relationship between a project’s developers and the creators of a static code analyzer can be a symbiotic one. The analyzer works by using a database of multiple tiers of positive and negative heuristics. First, it runs the low cost patterns against the entire code base to generate a large list of possible issues, then runs more accurate and higher cost patterns against the bug candidates to reduce the number of false positives, and finally evaluates the severity and more accurately classifies the issues. Once the analyzer has completed its run, it requires an experienced software developer familiar with the code base to review each issue reported.

Most static analyzers are built to report possible candidates in the first pass, and thus immature analyzers are perceived to red flag everything. They tend to create a lot of noise by reporting a large number of false positives and misclassifying the severity of issues. After the developers for the software being analyzed have reviewed the results of the analysis, they can give specific examples of why they determined it to be a false positive which can be used to improve the static analyzer’s heuristics. As the database matures, the quality of the negative heuristics improves and reduces the volume of false positives. The advantage here is that each report triaged leads to a commit resolving a bug or an improvement to the analyzer.

The team over at Program Verification Systems have built a static analyzer for C/C++ code that integrates into Microsoft Visual Studio. According to their website, the program allows the user to scan of lines of code to locate various typos and other errors. Their analyzer supports C/C++, C++/CLI, and C++/CX with support for C# language coming soon. The PVS-Studio is also available as a standalone utility through the distribution packages which allows for viewing the analysis logs on a machine without Visual Studio. It can also be used to track multiple sub-builds and analyze non-standard build systems. The reports for the open-source projects that have been analyzed with this software can be found on their website in the Checked Projects<%22http:/ift.tt/1dJEvHW> section.

The FreeSWITCH team ran the open-source FreeSWITCH project through the PVS analyzer. A decent majority of the issues reviewed were determined to be minor Windows-specific bugs not previously flagged by compilers currently implemented by the team. The team is continuing to review and resolve the alerts from the analysis and have integrated this analyzer into the code review workflow. They look forward to continuing this symbiotic relationship with the goal of improving the quality of software.

If you would like to replicate the results you can use the following steps.

  *   Set up an instance of Microsoft Windows 10, install Microsoft Visual Studio 2015, and install the analyzer from the http://ift.tt/1R7WpMh website.
[(1)MSVS_open]<http://ift.tt/1Ng86KQ>

  *   Create a new FreeSWITCH project.
[(2)Create_project]<http://ift.tt/1Ng89qg>

  *   Clone the FreeSWITCH repo into your project.
[(3)Clone]<http://ift.tt/1Ng89qi>

  *   Open the FreeSWITCH project.
[(4)Load_project]<http://ift.tt/1Ng86L6>

  *   Set the debug to ‘x64’ if not already set.
[(5)Change_debug]<http://ift.tt/1Ng89Gz>

  *   Click the PVS studio drop down box and select “check solution” to run the analyzer.
[(6)Run_PVS]<http://ift.tt/1Ng89GF>

  *   Settle in and wait for the results.
[(7)Final]<http://ift.tt/1Ng89GH>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20151209/e4cf2e91/attachment-0001.html