Paper: An Extensive Empirical Study on Code Translation Technique

LLMs do a pretty good job of translating programming languages, but things
you would expect to be hard, like translating from dynamically to
statically typed languages, are indeed hard.


Abstract
Automated code translation is increasingly important for software
evolution, yet the relative strengths and limitations of learning-based
and large language model (LLM)-based techniques remain insufficiently
understood. To address this gap, we conduct a large-scale empirical study
comparing representative code translation techniques across methodological
paradigms and translation granularities. We evaluate learning-based
methods, LLM-based methods, and general-purpose LLMs on multilingual
method-level and class-level benchmarks involving multiple programming
languages. Our analysis considers executable correctness, code similarity,
translation direction, translation granularity, and failure patterns. The
results show that LLMs and LLM-based methods generally outperform
learning-based methods in method-level correctness, although similarity
metrics alone do not reliably reflect functional correctness. Translation
direction substantially affects performance, particularly when translating
between languages with different type-system characteristics. Class-level
translation remains considerably more difficult than method-level
translation because it requires preserving global semantics, interfaces,
member relationships, and cross-method dependencies. Our error analysis
further shows that static semantic errors and logical errors are the
primary challenges in existing code translation systems. These findings
provide empirical evidence and practical guidance for developing more
robust, type-aware, structure-aware, and context-aware code translation
techniques.


https://arxiv.org/abs/2608.20776


Regards,
John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

Return to the comp.compilers page.
Search the comp.compilers archives again.