As software grows larger and larger, there is no chance that people are able to manually audit code. Just like people make syntax mistakes, people make semantics mistakes, mis-design and mis-understand design patterns. It is not possible to manually re-audit huge projects after updates, corrections, "corrections" and refactoring. Nor is it possible to reaudit huge projects after the knowledge about security vulnerabilities has improved, new types of vulnerabilities have been discovered, new types of attacks have been invented. Just like it is not economically possible to audit, whether the manually written macro-assembler code has been written with correct mnemonic names, operand numbers, operand names, it is not economically possible to audit, whether huge amount of high level programming language code has been written not just syntactically correctly, but also correctly in terms of known attacks. That is to say, automated software analysis is the only way to conduct audits. The automated software analysis may contain sandboxed runtime tests in addition to the formal verification, model-checking, whatever-the-fancy-word-or-method.
That holds for both, defense and offense. That is to say, both, malware authors and scale-ware authors have to look for security flaws in an automated fashion. The side that has smarter tools and/or more computing power, wins the race of finding flaws. The side that is smarter at writing software, wins the race of using the knowledge about the found flaws. The race will also be a bit hit and miss, trial and error, because it might happen that in a situation, where flaws F_1 and F_2 are known to both sides, the defense side might spend majority of its resources on the removal of F_1, but the attack side might spend majority of its resources on F_2.
Business wise it means that cloud service providers will love software developers, because software developers love to have their whole project analyzed in a fraction of a second and only huge computer clusters can do that. Services like Travis-CI.org, drone.io and various Jenkins hosting services will probably flourish.
My Dream Software Development Workstation, as of 2014_10
As of 2014_10 my (email@example.com) dream about software development tools is that there is a private cluster, where software analysis tools have been deployed with some Docker-like tool. Build results of a project will always reside outside the source folder. The project source folder at my desktop computer is actually a symbolic link to a local folder named "gateway2cluster" and the "gateway2cluster" is actually an sshfs link to a folder at the cluster. JetBrains IDE and KomodoEdit allow custom Bash scripts to be assigned to custom key combinations. One of those custom Bash scripts might request the analysis software that runs on the cluster to build the project and analyze it. The analysis results might be displayed as an auto-refreshing AJAX based web page, which in turn can be displayed by using any leftover, old, computer that has a comfortably big screen and a capability to run a proper, modern, web browser. The view at the web page might be automatically updated, synchronized, according to a http based API, which might be accessed by using wget, which might be triggered through a custom Bash script, which might be part of a custom IDE command just like the JumpGUID project has. :-D
Long story short: the only things that stop me from having my dream work-station are a pile of high-end hard-ware and the software analysis tools that can be deployed to the cluster. OK, may be some university would allow me to try out their cluster and there might be even some other options, but as of 2014_10 I still do not have the proper software. (I have to admit, it's kind of nice to have _only_ that problem. If I were a sea-man and dreamt of a perfect ship, not to mention owning one, then my chances of having my dream about my main tool of trade come true would probably be far shoddier than that.) :-D
Update on 2015_02_16
It turns out that as of 2015_02_16 there is a cloud service, checkmarx.com, that claims to offer security oriented automated code analysis services.
Update on 2016_06_30
SonarQube seems to be another similar project.
Update on 2016_07_10
Update on 2016_11_05
Update on 2016_11_06
Update on 2016_12_06
A little while after seeing a presentation by Robby Findler about the Racket language I came to a thought that may be a pre-compilation step that translates a program with security flaws to a program that does not have the specified types of security flaws might be even feasible, even, if it takes a whole compute-cluster and a lot of electricity to do it. Security flaw types might be described a bit like regular expressions are described for text search: in some domain specific programming language. Each operating system distribution might have its own set of regex-analogues, describing a type of security flaw that is considered a flaw by the developers of the given operating system distribution. Each operating system distribution might also have their own set of auto-fixes that use compute resources according to the preferences of the operating system distribution developers. The main issue that caught my attention is that even if there is a way to find all those type of properties that are classified as flaws by the search initiator, the flaws can have different fixes, the original author of some very popular software package might refuse to fix the flaw as a matter of his/her taste and preferences, the original developers might literally be dead, retarded due to age related brain damage, too busy with other projects or personal issues, in the case of typical corporate projects just plain indifferent about their creations, etc.
The reason, why sandboxing and running the application Joyent SmartOS style in a freshly cloned sandbox that gets intentionally destroyed regularly does not work is that even if the malware is not able to break out of the sandbox and it is not able to "phone home/dropbox", it can mis-process data conditionally. For example, a search engine that runs in a sandbox and has read-only access to a database at some other sandbox might refuse to return search results about some censored topics. Another case is that the software in the sandbox might crash on some specific data, leading to an off-line DoS attack that can not be fixed by destroying the sandbox with everything in it and re-cloning the sandbox.
The main beauty of the program translation based approach is that a small team can delegate bugfixing to computers, reap the benefits of the work of the features-only type of sloppy-developers in "wikipedia style", re-use some of their crappy components. It is a terribly ambitious project, but it's so ambitious that a data-flow analysis based auto-optimization and may be even algorithmic auto-optimization might be THE SIMPLE TEST CASE. An example of an algorithmic auto-optimization is that when the translator detects bubble-sort, then it replaces it with quick-sort according to one of its heuristics. The first language to try is the C. Low hanging fruits for the C-to-C translator might be the various "undefined behaviour" cases. The K-Framework is a project to study in addition to the Racket.
For me personally the translation based pre-compilaton step gives a lot of psychological relief, because it gives hope that it is possible to deliver proper software that the people, who pay only for features, actually want and are willing to pay for. Until that translation technology is ready, my options for third party components are very limited and that also limits the set of people, whom I'm able to serve without feeling ashamed of my deliverables. I know that I would never get caught with shoddy craftsmanship, specially given that the Microsoft-s and alike deliver the "blue screens of death" and other crap all the time, to the extent that people have become numb to crappy software, but the fact that clients do not know any better is really no excuse for a proper craftsman to lower its standards. There is also the issue that how can I ever ask others to do their job properly, if I self deliver CRAP and strive only to give my clients psychological satisfaction instead of the actual, proper, goods? Oh, they'll pay me, happily even, but it disgusts me to take money without proper delivery. I prefer to make my clients unhappy, if that is what it takes to get them to use proper software or to save the rest of their money from being spent on a software project that does not solve their problems. That partly explains, why my options, what I can deliver and to whom, are limited to projects that, given the state of the technology, I have at least a misguided belief that I can deliver a more-or-less decent service. I like the idea from medicine that says, if one can not do it properly oneself, then to avoid doing harm by doing a shoddy job one should refer the patient to somebody else. (I'm not saying that I have never violated that principle, nor am I saying that I will never violate that principle. I'm only saying that I thrive to follow that principle, even, if I fail miserably. Besides, if I'm not satisfied with the way others do their job, then I might as well take the risky project on myself, because if I'm not worse than others then no matter how bad the outcome is, I am not getting my benefits at the expense of my client. I think that it's sufficient, if the client candidate is informed of the probability of failure before accepting the project. A recommendation to spend the money on something else in stead of a software development project that has a high probability of failing or at least not producing the results that solve the original problem also helps.)
All practical languages are Turing Complete, which means that at least in theory any practical programming language can be translated to any other practical programming language. The translation result can be a mess that I as a human am not capable of comprehending, but for computers it will work and that's good enough for an output of a pre-compilation step. To maintain only a single set of flaw-descriptions and flaw-fixing-solutions and to make some single-threaded software benefit from multi-core-CPU-s, the architecture might be to translate applications of all other programming languages to ParaSail(my fork) applications, then apply the automated flaw-fixing in a form of a ParaSail-2-ParaSail translation and then compile the generated ParaSail code.
Update on 2016_12_08
Compilers, translators, test automation software, formal verification software, Integrated Development Environments (IDEs) and their various plugins can contain malware, which might plant malware to projects that are developed with them. As of 2016_12_08 I do not know, how to overcome that issue, but the way to mitigate that problem is to have the code analyzed by different code analysis servers that are owned by different parties. Open source software is public anyway, so it can be shared fully with anybody for analysis. The owners of open source hosting sites like GitHub and Sourceforge might be attacked by placing repository content modifying malware to their servers. A mitigating, not solving, countermeasure for that is to never download code from one's own project publishing repository, overwrite the public repository content with a privately maintained copy at every release. The idea of the mitigating countermeasure is based on genetic diversity. Part of the genetic diversity can be obtained by keeping available old systems (hardware + operating systems) that probabilistically lack the components that the malware depends on. Hardware that is no longer in production seems to be a fine candidate for that. (And I was mistakenly coursing the OpenBSD people for tinkering with out-of-production old crap in stead of making their operating system run on the hardware that people can buy from local computer shops.)
May be one idea is to fight fire with fire: if the attack is probabilistic, then the defense might also rely on some probabilistic phenomena. For example, in the case of control systems in stead of using voting like it was done in the Buran space shuttle control system(archival copy), there might be some analog circuit, which is literally assembled from discrete electronic components, at the entry point of every controllable module and that analog circuit selects on-off bit values randomly from multiple control computers, switching the control computer once per 2 minutes. The software of all computers contains branches for cases, when the control computer that some subsystem has selects to listen, does something awful that the other control computers do not approve. The control computers have a way to negotiate with each other about the common strategy, but each of the control computers uses a negotiation strategy that takes to account the Byzantine Generals Problem. The analogue electronics of each of the controllable modules might inform all of the control computers ahead of time about its next choice of control computer. The analogue electronics of each controllable module might also tell all of the control computers the onoff bits of all of the computers, countering the situation, where one control computer might lie to another control computer about its on-off bits at some controllable module. A compromise solution might be that the feedback at controllable modules might be implemented by an independent MCU-system, but the control computer selection must be primitive, analogue, made out of discrete electronic components.
Update on 2017_12_30
Update on 2019_06_19