Software Engineering & Evolution

The concept of “modernity” has been a topic of discussion in many fields, including philosophy, sociology and technology. In the software engineering context, modernity can be defined as the extent to which the source code of a software system utilises new features and capabilities of the programming language it is written in. In software evolution, we define modernity as a scale of measuring the age of a codebase, expressed as language levels/versions. For instance, your Python code can have a modernity of 3.11, if it runs fine with Python 3.11 but fails with an error with Python 3.10.

To investigate it in more detail, we introduce modernity signatures as vectors of values, each representing one language version: the higher the corresponding value, the more of language features from that version are used in the code. This has previously been done for PHP, Python, Rust, JavaScript, as well as for Java (technically, the project for Java only computed weighted abstract syntax trees, but converting them to modernity signatures is just a matter of traversal). A long, internationally peer-reviewed paper was published about this at a highly ranked scientific journal recently: https://doi.org/10.1016/j.jss.2024.111973

There are two ways to take this project:

• using existing implementations as inspiration, apply the method to another language and observe the differences (there were some differences between PHP and Python, so most probably if you take a completely different language like Kotlin, Rascal, Haskell or CSS, the differences will be substantial and interesting to investigate and explain);
• consider existing implementations, generalising their design choices, and propose conceptual improvements on visualisation, interpretation, summarisation, pattern detection, etc; then run experiments to show what the effect is (if any) of choosing one way or the other.

As an example of the first way, you can read the Chris Bleeker's thesis; as an example of the second way, Cris Zubcu's.

Get in touch with your potential future supervisors to discuss details!

Supervisors: Milan Lopuhaä - Zwakenberg, Ömer Sayilir

In SW engineering, code metrics tell something about the complexity of a program. Various metrics exist, ranging form simple ones like the number of lines of code, to advanced ones like cyclomatic complexity.

Code metrics can tell something about the code quality. This is extremely important: good code is easy to maintain, refactor, and port. Bad code costs time and money and frustration.
However, for software models, few of such metrics exist, even though such metrics could be equally useful for models as for code. Thus, the idea of this project is to come up with model metrics and evaluate if they make sense.

Note that various models could be studied: finite state automata, UML models, fault trees, etc.
Even though no company is involved, this is relevant for industry; we can involve companies later.

Tasks:

• Study the state of the art
• Define model metrics
• Evaluate on relevant case studies or examples: do these metrics really tell us what we want?

Supervisor: Fernando Castor

Generative AI and coding assistants are increasingly integrated into modern software development, particularly for code generation. While these tools can accelerate development workflows and reduce manual effort, they often produce extra and/or irrelevant output that must be removed during post-processing. This introduces additional cognitive load for developers and may lead to increased computational and energy costs. We refer to this behaviour as “babbling”.

Previous research [1] has identified three common forms of excessive generation: continuous generation of unnecessary functions, inclusion of unwanted test code, and the generation of redundant or excessive comments. Furthermore, our preliminary study [2] indicates that babbling tends to occur toward the end of the generation process, and that certain models appear to be more prone to this behaviour than others. Building on these findings, we introduce Babbling Suppression, a method that integrates test execution into the LLM generation process by evaluating intermediate outputs and terminating generation once a solution passes all tests. This approach reduces unnecessary token generation without compromising model accuracy. However, these findings are based on a limited set of smaller models (3–7B parameter transformers) and are limited to Python and Java code generation tasks. The goal of this project is therefore to deepen and broaden the understanding of babbling behaviour, and to use these insights to propose improvements to the design of Babbling Suppression.

Therefore, the primary objective of this project is:

• To analyse and systematically categorise the types of excessive tokens generated by LLMs, identifying recurring patterns of babbling as well as common errors in generated code.

The project may further include the following extensions:

• Expanding the analysis to larger and more advanced models. For example, including scaled variants of CodeLlama and Qwen3, as well as widely used AI coding assistants such as Claude and GitHub Copilot.

• Generalising the findings beyond Python and Java by investigating its occurrence across a broader range of programming languages.

References:

[1]: Lianghong Guo, Yanlin Wang, Ensheng Shi, Wanjun Zhong, Hongyu Zhang, Jiachi Chen, Ruikai Zhang, Yuchi Ma, and Zibin Zheng. 2024. When to Stop? Towards Efficient Code Generation in LLMs with Excess Token Prevention. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024). Association for Computing Machinery, New York, NY, USA, 1073–1085. https://doi.org/10.1145/3650212.3680343

[2]: Lola Solovyeva and Fernando Castor. 2026. Towards Green AI: Decoding the Energy of LLM Inference in Software Development. arXiv: https://arxiv.org/abs/2602.05712