Re: Mysterious triple test failures

Kaz Kylheku <>
Sat, 3 May 2014 22:19:06 +0000 (UTC)

          From comp.compilers

Related articles
Mysterious triple test failures (Nils M Holm) (2014-05-03)
Re: Mysterious triple test failures (Kaz Kylheku) (2014-05-03)
Re: Mysterious triple test failures mrs@Kithrup.COM (2014-05-04)
Re: Mysterious triple test failures (Nils M Holm) (2014-05-04)
Tripe test failure update (Nils M Holm) (2014-05-04)
Re: Mysterious triple test failures (2014-05-04)
Re: Triple test failure update (Chris Dodd) (2014-05-23)
Re: Triple test failure update (Nils M Holm) (2014-05-23)
[1 later articles]
| List of all articles for this month |

From: Kaz Kylheku <>
Newsgroups: comp.compilers
Date: Sat, 3 May 2014 22:19:06 +0000 (UTC)
Organization: NNTP Server
References: 14-05-003
Keywords: debug
Posted-Date: 03 May 2014 19:18:54 EDT

On 2014-05-03, Nils M Holm <> wrote:
> In the recent months, several people who ported and/or used my
> SubC compiler[1] reported that the triple test failed, but the
> compiler still worked fine.

If you don't understand it, who can? Haha.

This isn't real-time-event-processing software with nonrepeatable
inputs and scheduling patterns. :)

> I am running the triple test as follows:
> Subc sources ---[ third-party C compiler ]---> stage-0 SubC
> Subc sources ---[ stage-0 SubC ]---> stage-1 SubC
> Subc sources ---[ stage-1 SubC ]---> stage-2 SubC
> (where x---[y]--->z means "compile x with y to z"). Finally the
> stage-1 and stage-2 compilers are compared. When they are equal,
> the triple test is passed.
> Now it has happened that this test failed, resulting in stage-1
> and stage-2 compiler of identical size, but with tens of thousands
> of differences.

It means you have some uninitialized variable or something which leads
the third-party-C-compiled SubC to write different code from the
SubC-compiled SubC.

> However, both compilers seem to work fine.

Do these tests execute with zero errors under Valgrind on a Linux box?

> Further tests have shown that the assembly language output of
> the differing compilers is identical.

But that of stage-0 and stage-1 isn't identical: they produce
different output when compiling SubC. Necessarily so: the stages use
completely different compilers. But those outputs, in turn, have a different
behavior: they produce different code!

If you were to iterate and make a stage-3 with stage-2, it would probably come
out identical to stage-2. I.e. there is a "fixed point" when you feed
the compiler into itself.

> This happened on 386-based OpenBSD and x86-64-based Linux systems.
> So, what is going on here? Does anybody have an explanation?
> And: to work around the problem, I thought about comparing the
> assembly output of the compilers instead of the binaries. This
> should yield the same results, shouldn't it?

Only if the assembly stage is responsible for the differences.

> Please let me know what you think!

Disable all special code-generation options like optimizations.
If that makes the problem go away, re-introduce them one by one until
a difference shows up.

A bug could be causing some optimization or other code transformation decision
to be done differently. If such a decision is semantics-preserving
(which such decisions often are) then the differing variants of the
translated code behave the same way.

It still points to a bug, which is why it's smart to do that comparison between
the stages.

You have a simple case here of a program compiled by two different C compilers
producing different output. How does that happen? I mean, let me count
the ways, right? Dependencies on orders of evaluation, uninitialized
storage, or numerous other "undefined behaviors". This is why a sweep with
a debugging tool like Valgrind wouldn't hurt.

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.