Re: Please help with testing object oriented compilers (C++)

amin@unirsvl.rsvl.unisys.com (Amin Kassem ) writes:
]I am seeking an advice on Object Oriented FUNCTIONALITY testing. My group
]is about to embark into a new territory which we have no prior experience.
]
]Our primary objective is to develop test plans, test specifications and
]test cases for a suite of compilers, linkers, databaseses operating
]systems, etc..., report and keep track of defects.
]
]So far we have done a very good job of uncovering many software defects.
]However, we are about to venture into a new territory. It is called
]Object Orientation.
]
]As you can tell, we have always worked with a structured and procedural
]approach to our testing. My group has no prior experience with any object
]oriented language. A class is in the works to teach them the basics of
]C++.
]
]I am seeking those who have had experience with FUNCTION TESTING of an
]object oriented language, especially C++.


I've not done anything formally, but I have done some thought about what
changes are required for test methods to test object-oriented software
elements (in the context of trying to come up with ways to assist in the
debugging of parallel object-oriented programs).


Before I start, I should warn: C++ has a *lot* of complexity which makes
doing this harder. I've thought about it from the perspective of languages
like Eiffel and Modula-3, where you don't have to worry so much about type
errors. (In fact, you don't have to worry about them at *all* in the
general case. Eiffel is completely type-safe, and Modula-3 is type-safe
outside of unsafe modules.)


The central difference, in my opinion, between automatic testing of
traditional structures programs and object-oriented programs is largely
one of philosophy.


In a structured program, tests attempt to put the entire program through
its paces, seeing if any problems are revealed. The goal is to try to make
the program go through all execution paths that might occur in real use,
and insure that nothing unproper happens (where unproper is defined as:
generate incorrect results, follow wrong paths, or cause a halt through
erroneous execution [ie, segfault]).


In an object-oriented program, this kind of testing is counter to the
philosophy of design of the system, and downright wrong.


Object-oriented programs are made up of software components (generally
classes), which are intended to be general, reusable tools for doing some
job. Rather than testing a system as a whole, testing for object-oriented
programs should be done on a class-wise basis.


In a particular system, some method of a class may not get used. In a
conventional testing environment, that method probably would not get
tested, since it was dead code. But if you validate the class as a part of
this system, then the next user adopts the class as a valid component, and
uncovers bugs in the untested routine.


So an object-oriented test system should be trying to do tests to prove
the complete validity of a system, with each class being tested
independent of the system. (Or partially independent. Obviously, a class
cannot be tested without its clients.)


That's my philosophy on it. I'll try to answer the questions where my
answers will be somewhat meaningful.


]2- What areas, if any, need special/careful attention?


Since you're working with C++, there are two areas in C++ that need
special attention. Type analysis, and memory management.


The C++ type system has enough holes that most large C++ programs will
contain at least one or two type errors. These tend to be the hardest
errors for a programmer to discover. A good automatic testing system
should do as much type analysis as possible to uncover these.


C++ programmers also often have problems in their memory allocation code.
I've yet to see a single C++ program whose first release didn't have
memory leaks. And at least half of the libraries I've used in C++ have
intermittent problems with hanging pointers.


]3- In addition to the C++ standard and the commercial validation suites,
] is there, or has anyone developed a list of objectives or list of
] items/areas that must be tested in C++? Who? Where? What?


I haven't looked at it, but the GNU folks have been doing a lot of work on
a validation suite for C++ compilers. They tend to have some pretty bright
people, so that their discussions would probably be a useful place to
start. Try checking the archives for GNU groups.


<MC>
--
[ Mark Craig Carroll <MC> ]
[ U of Delaware, CIS Dept ]
[ PGP key available, ]
[ finger carroll@udel.edu ]
--

Return to the comp.compilers page.
Search the comp.compilers archives again.