Email Tips to Avoid False Positives
In the pursuit of high levels of cyber threat detection, the topic of false positives is often forgotten in the information security industry. Indeed, this is a very inconvenient topic that some developers try to ignore (or solve in dubious ways) - until the first serious incident that can paralyze the work of customers. Unfortunately, such incidents do happen. And unfortunately, only then comes the understanding that high-quality protection against cyber threats is not only prevention, but also a low level of false positives.
Despite the seeming simplicity of the topic of minimizing false positives, there are, in fact, many complexities and pitfalls in it that require significant investments, technological maturity, and resources from developers.
The two main reasons for false positives are
(i) software, hardware and human errors on the developer's side
(ii) the variety of legal (“clean”) software that passes security checks. The last reason requires clarification. Indeed, programs are written by millions of people of various qualifications (from student to professional) around the world, using different platforms and standards. Each author has his own unique style, while the "handwriting" of the program code sometimes really resembles a malicious file, which is what security technologies work, in particular, based on behavioral analysis and machine learning.
Without taking into account this specificity and without introducing special technologies to minimize false positives, developers risk ignoring the "do no harm" principle. And this, in turn, leads to monstrous consequences (especially for corporate customers), comparable to the losses from the malware itself.
You can use thetwo-tiered quality control system to minimize false positives:
(i) design level control
(ii) detection method release control, and
Let's take a closer look at each level of the system.
Design-level control
One of the main principles in development is that every technology, product or process should a priori contain mechanisms to minimize the risks of false positives and related failures. As you know, errors at the design level are the costliest, because their full correction may require reworking the entire algorithm.
When developing or improving technology for detecting cyber threats based on machine learning, it is necessary to make sure that the technology was trained on significant collections of clean files of different formats. Also, the processes of work, training and test collections of each technology need to be constantly updated with actual clean files. These products should have built-in mechanisms to minimize false positives on files critical to the operating system.
However, in addition to technologies and products, there is the notorious human factor.
A virus analyst, an expert system developer, a data analyst can make mistakes at any stage, so there is also a place for a variety of blocking checks, as well as hints / warnings / prohibitions in case of dangerous actions detected by automatic systems.
Control upon release of the detection method
New methods of detecting cyber threats go through several more stages of testing before being delivered to users.The most important protective barrier is an infrastructural false positive testing system that works with two collections.
The first collection (critical set) consists of files of popular operating systems of different platforms and localizations, updates to these operating systems, office applications, drivers and our own products. This set of files is regularly updated.
The second collection contains a dynamically generated set of files made up of the currently most popular files. The size of this collection was chosen in such a way as to find a balance between the volume of scanned files (as a result, the number of servers), the time of such a scan (and, therefore, the time it takes to deliver the detection methods to users) and the maximum potential damage in the event of a false positive.
Both collections are now over 120 million files (about 50Tb of data). Considering that they are scanned every hour with each release of database updates, we can say that the infrastructure checks more than 1.2Pb of data for false positives per day.
For example, behavioral detection allows you to block a malicious program that displayed elements of malicious behavior during operation. To avoid false positives on the behavior of a clean file, we created a "farm" of computers that implement a variety of user scenarios.The farm presents various combinations of operating systems and popular software. Before the release of new non-signature detection methods, they are tested in dynamics on this "farm" in typical and special scenarios.
Finally, we cannot fail to mention the need to check for false positives for the web content scanner. Incorrect blocking of the website can also lead to interruptions in the work on the customer's side, which is unacceptable in our work.
To minimize these incidents, automated systems daily download relevant content from 10,000 of the most popular Internet sites and scan their content with our technologies to check for false positives. To achieve the most accurate results, content is downloaded by real browsers of the most common versions, and proxies are also used to exclude the provision of geo-dependent content.
Prevention is better than cure
Not everything can be foreseen, and even having foreseen everything, it would be good to know how all these measures will work in practice. You don't have to wait for a real incident to do this - you can simulate it.
You can regularly conduct internal drills to test the readiness of personnel and the effectiveness of methods to prevent false alarms. The exercise boils down to a full-fledged simulation of various "combat" scenarios in order to verify that all systems and experts are operating as planned. The exercise involves several divisions of the technical and service departments at once, is scheduled for a weekend and is conducted on the basis of a carefully thought-out scenario. After the exercise, an analysis of the work of each department is carried out, documentation is improved, changes are made to systems and processes.
Sometimes, during the learning process, a new risk can be identified that was not previously noticed. Regular brainstorming of potential problems in technology, processes and products can help identify such risks more systematically. After all, technologies, processes and products are constantly evolving, and any change brings new risks.
Finally, for all incidents, risks and problems uncovered by the exercises, systematic work is underway to eliminate the root causes.
Needless to say, all systems responsible for quality control are duplicated and supported around the clock by a team of on-duty administrators. Failure of one link leads to a transition to a redundant and prompt correction of the failure itself.
Conclusion
It is impossible to completely avoid false positives, but you can greatly reduce the likelihood of their occurrence and minimize the consequences. Yes, this requires significant investment, technological maturity and resources from the developer. But these efforts keep users and enterprise customers running smoothly. Such an effort is necessary and part of the responsibilities of every reliable cybersecurity vendor.
Instead of relying on a single security technology, try using a multi-layered security system. Protection against false positives can be arranged in a similar way - it is also multi-level and duplicated many times. There is no other way, because we are talking about high-quality protection of customers' infrastructure.
At the same time, manage to find and maintain an optimal balance between the highest level of protection against cyber threats and a very low level of false positives. Quality is not a result achieved once. This is a process that requires constant monitoring and improvement, especially in conditions when the cost of a possible mistake is a violation of the customer's business processes.