[H-GEN] gnu make command-line target ordering problems?

Martin Pool martinp at mincom.com
Mon Oct 18 01:29:53 EDT 1999


[ Humbug *General* list - semi-serious discussions about Humbug and
Unix-related topics. ]

Ben Carlyle wrote:

> Anthony Towns wrote:
> > > I think the simplest way to state my case is this:
> > > The parallel execution of command-line targets violates the make
> > > commands are processed in order.
> 
> > Similary, the parallel execution of dependent targets violates the
> > rule that dependents are processed in order. The real question is
> > whether this is particularly useful.
> 
> Is your in-order processing rule documented?  <curious>

I don't know if it's documented anywhere.  I know it's certainly a
"rule" many people have in their heads, though it's not exactly correct:
this is quite similar to the idea that command-line targets are executed
in series.

> > > If the parallel make does not
> > > process the command-line in order, then it has different behaviour
> > > characteristics to the sequential make.
> > Similarly, if the parallel make does not process the dependent targets
> > inorder, then it has different behavioural characteristics to the
> > sequential make.
> 
> The principal behind parallel builds is that in a Makefile with fully
> expressed dependancies, the behavioural characteristics of sequential
> and parallel make are identical.  Modern makefiles are to maximise
> visibility of the dependancy tree to make in order to allow maximum
> concurrent processing.

We seem to be in agreement that in a properly-specified Makefile,
sequential and parallel makes will both produce correct results.  As
with concurrent programming in general, when you're using parallel make
you have to be more careful or you will get hard-to-debug problems. 
This doesn't imply that parallelism should not be offered.

> > > The command-line should
> > > not specify a list of depedancies, but a list of targets to be
> > > executed one after the other.

Saying "execute a target" is not really the zen of make.  "Seek a goal"
tends to give better designs.

> > Or, alternately, it should specify a list of goals, all of which have
> > to be made. If you want to build your system, you should have a single
> > goal that does just that. If you want to clean your system, you should
> > have a single goal that does that.
>
> As I do have.
 
> > Having a goal `build' that only works when you've already done a `clean',
> > is broken.
> 
> In my own defense I ask you to note what happens when you modify a
> makefile.  

Personally, I'm about equally like to:

 * make things depend on the makefile

 * use automake/autoconf and have make work out for itself when to
rebuild things

 * manually do a "make clean" after changing the configuration

As a matter of habit, I write "make clean && make -j2 all", never "make
clean all".  It's just better; if it saves you confusion once it's worth
it.

> Suddenly your targets are not identical to those that
> correspond to that makefile.  Do you have every single target in
> your system with a dependancy listed on the makefile?

Note that some makes and make idioms will automatically generate this
for all targets; in particular they can recompile files *if the command
line used to compile them has changed*.  I think recent Linux kernels do
this, though I might be wrong; and also that PMAKE has this built-in.

> Personally I do not.  In some cases I prefer clarity in makefiles over
> fully-defined dependancies.
> There are often reasons to do a make clean, then make build, especially
> when makefile development is being performed.

So: I would say, typing "make clean && make all" is easier than hacking
make.  It will also work on both Sun and GNU.
 
> > > Parallelism should be left to the defined make dependancies, not
> > > assumed from the commandline.  

Hear hear.

> Firstly, I am controlling a centralised rule set that is
> included by each makefile.  The rule set has to be strict
> enough to keep the makefiles simple, but also versitile
> enough to make everything from C and C++ code to gperf
> and yacc and shared libraries and static libraries to
> relocatable objects that are linked at runtime.  I'm currently
> looking at around 3000 lines of included makefile.  The makefiles
> themselves add up to an additional 30klocs.  It's a complicated
> system with complicated dependancy structures that can either
> be expressed in every rule at a large software engineering
> and documentation cost, or the make system can be encapsulated
> into "super-targets" with simple interdependancies.  My system
> is an attempt almost at namespaces in Makefiles.  By wrapping
> up dependancies into blocks which have clear orderings I can
> simplify the overall structures.

There's a very good paper somewhere called "Recursive make Considered
Harmful", which talks about makefile design patters.

Our approach, on a medium-complexity Java system (>250kloc, ~60min to
build), is:

  * People working on particular packages can use whatever system they
like: personally I run Jikes by hand on the files I've changed.

  * For releases and daily builds we use a homegrown script that
clobbers the output and rebuilds everything.  We don't do any dependency
analysis, because we actually want to make sure everything is built from
scratch.

This is a tolerable -- not great, but at least reliable -- solution to a
problem where Make has lots of problems, mainly because Java doesn't fit
the usual .h/.c/.o dependencies.

> That's not a bad idea.  In my CM tool the only simple way to do that
> is to completely delete the work areas, then sync them back out of
> the database.  This takes an hour or two for a system of this size,
> so until I have a better solution I'll continue to use make for this
> purpose.

Oh, and we always put build output in a separate directory from the
source, so cleaning up is a simple "rm -rf classes gen lib" away (or
Ctrl+Esc P W Down Down Enter Click Scroll Click Scroll Click Click Del
Wait Click Click Del Click Wait Click Click Del Click <sigh>).

-- 
 /\\\  Mincom | Martin Pool          | martinp at mincom.com
// \\\        | Software Engineer    | Phone: +61 7 3303-3333
\\ ///        | Mincom Ltd.          | 
 \///         | Teneriffe, Brisbane  | Speaking for myself only

--
This is list (humbug) general handled by majordomo at lists.humbug.org.au .
Postings only from subscribed addresses of lists general or general-post.



More information about the General mailing list