What is open source?
Open source software is software whose source code has been made publicly available by its copyright holder. Under a true open source license, the software is developed collaboratively, and other programmers can look at, modify, or use the code their own purposes. This "pure" open source model is often referred to as FOSS (free and open source software).
A variant of open source is "source available", which means that no permission is granted to modify or otherwise use the code, but that it is available for inspection. For security purposes this is just as good as true open source, so when I refer to "open source" in this article, I include code that is "source available".
What is closed source?
Most software is written and developed by commercial companies. Understandably, these companies are keen not to have others stealing their hard work or trade secrets, so they hide their code away from prying eyes using encryption, and any attempt to use or modify the code without permission will result in lawsuits or worse.
So what is the problem?
As I say, this is all quite understandable, but when it comes to security it presents a major problem. If no-one can see the details of what a program does, how can we know that it is not doing something malicious? Basically we can’t, so we simply have to trust the company involved, which is something us paranoid security types are loath to do (with good reason).
Why is open source the best solution?
If code is open source then it can be independently examined and audited by anyone qualified to do so, in order to check that there are no backdoors, vulnerabilities, or other security issues. Open source is not a perfect solution (see below), but it is the only way to have to verify that software is only doing exactly what it is supposed to be doing.
Even if the code has not been audited, the very fact that it freely available to be audited provides a strong indication that it can be trusted, as it is unlikely that developers would include malicious code and then leave it open to be discovered by anyone who cares to look.
Not a perfect solution…
Unfortunately, there are a limited number of individuals with both the skills and time to audit open source software (usually for free), which means that the vast majority of open source programs have not been audited.
This problem is compounded by the fact that many open source programs are extremely complex, containing thousands upon thousands of lines of code, so even if they have been audited, it is entirely possible that the auditors missed a problem (especially if malicious code has been deliberately concealed).
But…
Open source, therefore, does not guarantee a program is "clean", but it is nevertheless the best guarantee that we have (or can have) that this is so. The alternative is closed source, which provides no guarantees whatsoever.
Always verify open source programs
So open source is great for security. Yay! But how can you know for sure that the open source program you just downloaded hasn’t been tampered with in some way?
This may sound like whacko paranoid conspiracy fantasy thinking, but in February 2016 the website of one of the most popular versions of the Linux open source Operating System, Linux Mint, was hacked, and a compromised version of the OS was made available to downloaders,
"Hackers made a modified Linux Mint ISO, with a backdoor in it, and managed to hack our website to point to it."
The infected Linux ISO images installed the complete OS with the Internet Relay Chat (IRC) backdoor Tsunami, which gave the attackers access to users’ system via IRC servers. So the threat is very real.
In this case, downloaders who bothered to checksum the file’s MD5 hash would have spotted the deception, but such hash checks are not a reliable protection because if a website can be hacked in the first place, it is trivial to replace the published checksum with a false one that verifies the comprised file.
Much better is for developers to digitally sign their software so that users can verify the origin of a file (the Mint developers were very lax in this regard, as their software was not digitally signed, and even the MD5 hash function that was used is known to be broken!)
Please see my article on Digital signatures – why and how you should use them for more information. Unfortunately, verifying digital signatures is something of a pain, but is necessary if you care about security.
I should also note that, ideally, all software should be digitally signed and verified, but because open source code can be freely modified by anyone, it is easier to tamper with than closed source code. It is therefore particularly important to verify open source programs.
Open Source: Conclusion
Open source is not a perfect solution, but it provides the best (and only!) guarantee possible that software can be trusted. The alternative is closed source, which provides no guarantee whatsoever (other than blind faith in the company, which is a faith tech companies do not deserve).