Automatically identifying and categorising security-threatening computer bugs

Research Article published on 10 July 2025

Great progress has been made in automated techniques for discovering computer bugs, to the point of being able to detect thousands of them. Unfortunately, it remains impossible to correct them all due to a lack of human resources, which brings in a new challenge: to identify and prioritise the resolution of the most damaging vulnerabilities to IT security. Scientists at the Systems and Technology Integration Laboratory (List - Univ. Paris-Saclay/French Alternative Energies and Atomic Energy Commission, CEA) have put forward automatic methods based on formal approaches to categorise the criticality of a bug.

While computers, programmes and software are now commonplace, the need to protect them from potential malicious acts has never been greater. Getting round these attacks lies right at the heart of cybersecurity issues; one of the strategies relies on the discovery of computer bugs and their vulnerabilities, which attackers use to carry out their actions.

In fact, when a programme runs on a computer, it sometimes writes or reads further than the space allocated in the buffer memory for the activity in question. In the best-case scenario, this bug - known as buffer overflow - causes the programme to freeze. In the worst case, attackers use it to hijack the programme by writing new instructions or reading elements that should remain secret. "This category of bugs is quite prevalent in programming languages such as C, and are among the most severe vulnerabilities," says Sébastien Bardin, a researcher at the Systems and Technology Integration Laboratory (List - Univ. Paris-Saclay/CEA), which is working on the matter. It is therefore essential to identify these vulnerabilities - and others - and correct them before an attacker takes advantage of them.

Fortunately, great progress has been made in the last ten years or so in automated techniques for discovering bugs in programmes. This is the case for intensive automated testing techniques - which generate a large number of tests in an attempt to find bugs - notably used by major players such as Microsoft and Google.
 

Identifying a dangerous bug

However, these advances have a downside: they identify too many vulnerabilities in the programmes. "Thousands of bugs that are still open have been found in the Linux kernel or Firefox, for example. Except that, out of 1,000 open bugs, only a small proportion can realistically be corrected. The question is which ones to correct first," explains Sébastien Bardin.

A number of techniques already exist for this purpose, such as CVSS (Common Vulnerability Scoring System), which assign a score to different bugs. But CVSS scores are based on manual analysis and overly simple vulnerability categories. For example, all buffer overflows are considered significant vulnerabilities. "This isn't really sufficient, because it's not specific enough - there are still too many vulnerabilities categorised as critical - and above all it still requires human intervention, which is the limiting resource. We'd like to be able to intervene automatically, before a human expert has analysed the problem."

The work of Sébastien Bardin and his colleagues fits into this context: categorising the danger presented by a vulnerability when attacked. "We were among the first to question the importance of bugs, not just their presence. It's a huge change in the way we look at them," says Sébastien Bardin.

If this hadn't been done before, it's because the job isn't an easy one. In fact, you need to be able to define what a dangerous bug is, and find the elements that characterise a significant bug among a host of other elements. But "it's a fairly open problem. The most important thing is to establish the right definitions for criticality, then automate the analysis."
 

Two angles of attack to categorise vulnerability

To gauge the danger of a bug, Sébastien Bardin's team takes into account two significant notions: the replicability of an attack and the attacker's control over it. In the first case, an attack is said to be more or less "replicable" depending on the hacker's ability to carry it out with no resistance. Sébastien Bardin takes the example of a PDF reader installed on any computer. If the software mishandles certain system inputs, i.e. certain data sent by a device, "the attacker can send you a PDF that has exactly the right flaws to corrupt the PDF reader's execution, and with one thing leading to another, take control of your system." This attack is replicable - all PDF reader users are vulnerable - and can be triggered at will.

Conversely, other vulnerabilities arise from a combination of circumstances, for example when the PDF reader has a defect in a certain operating system, hardware or network configuration. Here, for the attack to succeed, the attacker has to be lucky and the target's system has to be in the right parameter configuration. This is why Sébastien Bardin's team has introduced the notion of robust accessibility, which distinguishes bugs whose inputs do not depend on circumstances external to the attacker. However, this categorisation is almost too clear-cut: an attack may also depend on a frequent configuration of parameters, and the attacker may therefore often succeed. "We have refined this notion to characterise whether an attacker is fairly likely to achieve his goals."

The control that a bug gives an attacker is the other notion that Sébastien Bardin and his colleagues are looking into to better assess a bug's vulnerability. "Generally speaking, the more a vulnerability allows the attacker to write values in certain places in the system, the more opportunities it gives him to act." In the case of the buffer overflow bug, the danger is not the same depending on whether the attacker is able to write just one value or thousands of them. To analyse this, "we look at the inputs that cause the programme to crash, then follow the execution traces to calculate a representation of the control the bug gives the attacker."

Once the criticality levers have been identified and the definitions established, Sébastien Bardin's team builds approaches based on formal methods to implement automatic programme analyses, enriched by the ability to assess the criticality of a bug. These formal methods have the advantage of providing mathematical guarantees for bug detection and characterisation. From a mathematical point of view, finding a bug means finding a solution to an equation that derives from the programme's execution paths. To assess replicability and attacker control, the team introduces special ways to manipulate the different solutions to these equations.
 

Significant progress in cybersecurity

"There may be other ways of refining the measurement of vulnerability, in addition to the two concepts mentioned above. Identifying them is clearly one of the scientific difficulties of the job," continues Sébastien Bardin.

To find out exactly what makes a bug critical, the scientists work in close collaboration with industry experts, such as the French national agency for information systems security (Agence nationale de la sécurité des systèmes d'information - Anssi). "It's really an iterative research process. Manufacturers tell us about their problems, and we think about how we can solve them. Afterwards, they give us feedback on our technique and what it brings to their concrete cases." It's a way of getting closer to the reality on the ground.
 

Références :