So basically, FOSS software will save humanity?
/g/ was right all along
FOSS is a step in the right direction but it's far from the full solution. As I said, the risk doesn't just lie in the system being coded in a biased or flawed way. Your algorithm can have the cleanest source code imaginable, but if the data or methods it relied on for machine learning are skewed then you're still going to run into all the bad shit with the flaws being even harder to detect. FOSS also has some potential drawbacks. While it's becoming more popular, there's still issues with its economic viability in all sectors. It's one thing providing a license to your software online and making the code available under some sort of copyright / copyleft model, but selling an actual full system (with hardware often included) to police, governments or major data analytics companies? That's something else entirely. Intellectual property rights and proprietary software are entangled in all of this and we're years away from seeing an all FOSS society.
It's also unclear how effective this would actually be. These extremely advanced analytical systems are incredibly complex and can consist out of millions of lines of code written in such a way that only a small amount of experts are able to even understand how it works. Just dumping the source code is akin to a company under tax fraud investigation giving the IRS 500,000 pages of undated and scrambled records with the message "knock yourselves out, it's all in there". Further cooperation and assistance will definitely be needed.
Complete openness might not be in everyone's best interests either. In the context of policing and criminal justice, it's easy to see why this would be the case. If you have an AI that is used to stop criminals or track down and identify suspects, it might not be the best idea to reveal the secrets as people can otherwise game the system to stay under the radar once they know exactly how it all works.