Re: Weeding out unwanted include files

: Along these same lines, a tool which would generate a symbol usage DAG
: for an entire (very large) program could be extremely valuable.
: For example:
: ----bar.h-------
: #define SOME_VALUE 8
: ----foo.c-------
: include "bar.h"
: int global1;
: foo() { global1 = SOME_VALUE; }
: main() { if (global1 == foo()) .... }
: Would produce something like this:
: foo.c:main() -> foo.c:foo()
: foo.c:global1;
: foo.c:foo() -> foo.c:global1;
: bar.h:SOME_VALUE
: So does anyone have some time to modify gcc and a linker???


Well, I don't, but spending a percent or two of the effort yields
(IMHO,YMMV) more than 80% of the benefit. In particular, many of the
interesting cross-file references can be determined using simple
heuristics, rather than a compiler-level analysis. And using a simple
token scanner and a few heuristics has the advantage that it will "see"
preprocessor tokens without needing to go to special trouble.


For the above example, the appended perl script yields


                        foo.c -> bar.h: SOME_VALUE


(because it is only chasing references made cross-file, and so ignores
the main->foo call).


: [There are plenty of C cross-referencers, though I don't know of any that
: deal well with preprocessor symbols. -John]


Just for fun, I'll also append a simple cross-references script. I've
used the perl include-references script and the awk/tcl xref-browsing
script with success on programs of many megabytes of source code. A lot
of their charm is in their simplicity. The lack of source-syntax
sensitivity is (for me) actually a plus, since eg, it shows me where
variables, parameters, function names, and the such are mentioned
inside comments.


I also note that another (more mature and powerful in some ways) tool
that will ease cross-referencing tasks including preprocessing tokens is
mkid, from comp.sources.unix, volume 24. I have also seen references to
other inter-include-file reference tools in perl, but I don't have a
definite reference to them.


I end up using the following scripts daily, when dealing with
code I'm not familar with.


--- include-ref
#!/usr/bin/env perl
#
# usage include-ref c-filename [include-directories]
#
# Quick hack to help in checking references from .c or .h files
# to .h files included. It has many areas that could be improved,
# but even in its current crude form (perhaps akin to the famous
# one-liner spell checking hack), it is arguably useful.
#
# The notion is to NOT preprocess, but find as many of the files
# as possible (in order to preserve the preprocessor names), and then
# see where each name is first found, assume it is defined in the file
# it is first found in, and finally compute references from file
# to file.
#
# This method gives spurrious results, for overlooked keywords, for
# unfound .h files, for #ifdef problems, struct field names, formal
# names in prototypes, and many more, but it gives good enough answers
# to be useful as a first approximation.
#
# global state: %file, %reference, %token, $current_file, $root_file
# $in_comment, %ignore


# here we could fill in some of the more common identifiers we
# don't want to keep track of, such as keywords, common local identifiers,
# and so on and on (incomplete, haphazard list; flesh out as needed... )


foreach $x (split(/\s+/,"
      if then else while for switch case do return main
      void int long short double float struct union extern const volatile
      char typedef unsigned signed define
      ifdef ifndef elif endif
      i j k p
")) {$ignore{$x} = 1;}


$root_file = shift(@ARGV);
push(@ARGV,"/usr/include");
do process_file($root_file);


foreach $x (sort keys %reference) {
        print $x, "\n";
}


sub process_file {
        local($old_file);
        local($fh);
        local($x);
        return if $file{$_[0]};
        $old_file = $current_file;
        $current_file = $_[0];
        $file{$current_file} = 1;
        if (! -f $current_file){
                foreach $x (@ARGV){
                        last if (-f ($current_file = "$x/$_[0]"));
                }
        }
        if( !open($fh=$current_file,$current_file) ){
                print "Can't open $_[0]\n";
        } else {
                while(<$fh>){
                        chop;
                        if( (!$in_comment) && /^\s*#\s*include\s*["<].*[">]/ ){
                                s/^\s*#\s*include\s*["<](.*)[">]/\1/;
                                do process_file($_);
                                $in_comment = 0; # just in case
                        } else {
                                do process_line($_);
                        }
                }
                close($fh);
        }
        $current_file = $old_file;
}


sub process_line {
        foreach $x (do tokenize($_[0])){
                next if ($ignore{$x});
                $token{$x} || ($token{$x} = $current_file);
                next if ($token{$x} eq $current_file);
                $reference{"$current_file -> $token{$x}: $x"} = 1;
        }
}


sub tokenize { # inadequate comment and quote processing...
                                # anybody know a way to write a qick lexer in perl for
                                # "" and '' strings (with escapes), and comments?
        $line = $_[0];
        $line = ("/* " . $line) if ($in_comment);
        $line =~ s/\/\*.+\*\// /;
        $in_comment = ($line =~ /\/\*/);
        $line =~ s/\/\*.*$/ /;
        $line =~ s/"[^"]*"/ /g;
        $line =~ s/'[^']*'/ /g;
        return grep(!/(^[0-9]|^\s*$)/,split(/[^a-zA-Z_0-9]+/,$line));
}
--- mxref
#!/bin/sh
#
# usage: mxref infile1 infile2 ... >outfile
#
# create index lines for all tokens in the given files
nawk '{
        sub("$"," ")
        gsub("[^a-zA-Z0-9_]+"," ")
        sub("^ ","")
        gsub(" "," " FILENAME " " sprintf("%3d",FNR) "\n")
        printf "%s",$0
}' ${1+"$@"} | egrep '.....* .* ' | sort -T .
--- vxref
#!/usr/bin/env wish-f
#
# usage: vxref file
#
# view an xref compiled with mxref


set idxf [lindex $argv 0]


frame .panel
button .panel.quit -text quit -command {destroy .}
button .panel.help -text help -command {
.display.reftext delete 1.0 end
.display.reftext insert 1.0 {
To start out, X-select a token on any X window, and click the
"idx" button. Index lines for that token will be inserted in the
left window pane.


Move the text cursor to any of the index lines in the left pane,
and click on the "ref" button, and the text where reference occurs
will be displayed in the right window pane. Simply right-click
on an index line is an abbreviation for moving the cursor and
clicking on "ref".


The "clr" button empties the index pane.


The "tcl" button executes the current X selection as a tcl script.


You can launch your favorite editor, positioned to the file and
line displayed in the "ref" pane, by defining the "launch-edit"
and "launch-view" procedures in your $HOME/.env-vxrefrc file.
For example, to launch vi:


        proc launch-edit {file line} {
                exec xterm -e vi -c "$line" $file & }
        proc launch-view {file line} {
                exec xterm -e view -c "$line" $file & }


or emacs (via the emacs client):


        proc launch-edit {file line} {
                exec emacsclient +$line $file & }


In general, any customization can be added to the .env-vxrefrc file.
For example, to select a small, 10-point font for the two primary
text display windows:


        .display.idxtext configure -font *courier*medium*r*normal*100*
        .display.reftext configure -font *courier*medium*r*normal*100*


In general, any wish tcl commands can be used in the rc file.
The widget names initially available for customization are


        widget name widget type
        ----------- -----------
        .panel frame
        .panel.quit button
        .panel.help button
        .panel.tcl button
        .panel.clr button
        .panel.idxshrink button
        .panel.idx button
        .panel.idxgrow button
        .panel.ref button
        .panel.le button
        .panel.lv button
        .panel.fname text
        .display frame
        .display.idxscroll scrollbar
        .display.idxtext text
        .display.refscroll scrollbar
        .display.reftext text
}}
button .panel.tcl -text tcl -command {eval [selection get]}
button .panel.clr -text clr -command {.display.idxtext delete 1.0 end}
button .panel.all -text all -command {
        .display.idxtext delete 1.0 end
        .display.idxtext insert 1.0 [exec cat $idxf]\n
}
button .panel.idx -text idx -command {
        set w [selection get]
        .display.idxtext insert 1.0 [exec look $w $idxf]\n
        .display.idxtext yview 0
}
button .panel.ref -text ref -command {
        set r [.display.idxtext get "insert linestart" "insert lineend"]
        set f [lindex $r 1]
        set l [expr [lindex $r 2]-1]
        .display.reftext delete 1.0 end
        .display.reftext insert 1.0 [exec grep -n {^} $f]\n
        .display.reftext yview $l
        .panel.fname delete 1.0 end
        .panel.fname insert 1.0 $f
}
button .panel.le -text edit -command {
        launch-edit [.panel.fname get 1.0 end] \
                                [expr int([.display.reftext index @1,1])]
}
button .panel.lv -text view -command {
        launch-view [.panel.fname get 1.0 end] \
                                [expr int([.display.reftext index @1,1])]
}
proc launch-view {file line} {
        error "view command not defined"
}
proc launch-edit {file line} {
        error "edit command not defined"
}
text .panel.fname -height 1 -width 60 -borderwidth 4 -relief ridge


button .panel.idxshrink -text < -command {
        .display.idxtext configure -width \
                [expr [lindex [.display.idxtext configure -width] 4]-10]
}
button .panel.idxgrow -text > -command {
        .display.idxtext configure -width \
                [expr [lindex [.display.idxtext configure -width] 4]+10]
}


pack .panel.quit -side left
pack .panel.help -side left
pack .panel.tcl -side left
pack .panel.clr -side left
pack .panel.idxshrink -side left
pack .panel.idx -side left
pack .panel.idxgrow -side left
pack .panel.ref -side left
pack .panel.le -side left
pack .panel.lv -side left
pack .panel.fname -side left


frame .display
scrollbar .display.idxscroll -command {.display.idxtext yview}
text .display.idxtext -width 40 -height 40 \
          -yscroll {.display.idxscroll set}
bind .display.idxtext <ButtonRelease-3> {
        .display.idxtext mark set insert @%x,%y
        .panel.ref invoke
}
scrollbar .display.refscroll -command {.display.reftext yview}
text .display.reftext -width 80 -height 40 \
          -yscroll {.display.refscroll set}
pack .display.idxscroll -side left -fill y
pack .display.idxtext -side left
pack .display.refscroll -side left -fill y
pack .display.reftext -side left


pack .panel -side top -anchor w
pack .display -side top


wm minsize . 100 100


catch {source $env(HOME)/.env-vxrefrc}


--
Wayne Throop throopw%sheol.uucp@dg-rtp.dg.com
                              throop@aur.alcatel.com
--

Return to the comp.compilers page.
Search the comp.compilers archives again.