muddle and its directories¶
About this chapter¶
muddle is three or four different things, entwined:
- It is a command line tool, providing useful tools for managing build trees on Linux.
- It is a Python package,
muddled
- It is a simple dependency tracking and analysis tool, with its state made evident via the file structure.
- It is a binding of the above to the common tasks required for managing, building and preparing deployments of Linux software from common sources to multiple architectures and platforms.
In a perfect world (perhaps the eventual “muddle 3”) these would be better separated. In particular, it would be nice if it was obvious that muddle can be adapted for things other than Linux build trees.
This chapter is meant to be a longer and more discursive introduction to some of the basics of muddle, with a concentration on how the muddle build tree works as a directory tree, and with worked examples.
If you are just going to use muddle to check out and build an existing build tree, then this is probably much too much information, although the short section An overview of the directory structure may be useful.
If you are going to be developing new muddle builds, then this chapter should enable you to do so more effectively.
If you are going to be reading the muddle source code, or developing muddle itself, then it should explain much of the intent behind muddle’s workings.
Notation in this chapter¶
Directory names are consistently shown with a trailing
/
, as byls -F
. Since I’m talking about directories and files a lot, I think this makes it easier to tell which is which.The “tree” listings of directory structures are produced with the command
tree
, normally as:$ tree -aF -I .svn --noreport
i.e., showing files and directories that start with a dot, suffixing directory names with
/
, not showing.svn/
directories, and not reporting on the number of files and directories. In a few places the “stub” of a.svn/
directory (that is, just its name) may be shown - this should be obvious from context.When referrring to the parts of a label (
type:(domain)name{role}/tag
) it is useful to keep the punctuation that indicates that part (so,type:
,(domain)
,name
(no punctuation!),{role}
and/tag
).Remember that, in the current scheme of things, only
package:
labels use{role}
, and “normal” builds do not use(domain)
.
An overview of the directory structure¶
Normal practice is to keep a muddle build in its own directory. This is not a requirement, but it helps keep it more obvious that it is a build.
The muddle directories are as follows:
.muddle/ : | contains information about the build state, and signifies that this is a muddle build. |
---|---|
src/ : | contains the build description and any other checkouts. |
obj/ : | generated by muddle to hold the results of building packages. |
install/ : | generated by muddle to hold the results of “installing” packages (this will be explained later on). |
deploy/ : | generated by muddle to hold the results of “deploying” things
in install/ . |
versions/ : | generated by muddle to hold the result of muddle stamp version . |
domains/ : | present if there are sub-domains, muddle will create this directory if it is needed. |
A simple example build¶
The muddle repository has a variety of example build descriptions, of varying complexity.
Perhaps inevitably, not all of the examples are maintained as well as they should be. If you do find problems with any of them, please raise an issue about it on the muddle issues page.
We shall use the “cpio” example.
We start with a new, empty directory:
$ mkdir example $ cd example
and then use the muddle init
command:
$ muddle init svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio builds/01.py > Make directory /home/tibs/sw/m3/example/.muddle Initialised build tree in /home/tibs/sw/m3/example Repository: svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio Build description: builds/01.py Checking out build description .. > Make directory /home/tibs/sw/m3/example/src > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/builds builds A builds/01.py Checked out revision 458. > Make directory /home/tibs/sw/m3/example/.muddle/tags/checkout/builds Done.
This leaves us with two new directories:
$ ls -AF .muddle/ src/
Or, in more detail (with the .svn/
directory in src/builds/
shown as a
“stub”):
. |-- .muddle/ | |-- Description | |-- RootRepository | `-- tags/ | `-- checkout/ | `-- builds/ | `-- checked_out `-- src/ `-- builds/ |-- 01.py |-- 01.pyc `-- .svn/
Whilst the name and content of the src/builds
directory might differ, this
is the normal state of a muddle build tree after the init
command.
The .muddle
directory¶
The .muddle/
directory is, in many ways, the heart of the muddle build
tree.
In the first place, it identifies a directory as the top-level of a muddle
build. Indeed, the muddle command line tool looks in the current directory,
and then upwards through the filesystem, to find a .muddle/
directory.
If it cannot find one, then it decides that it is not in a muddle build tree,
and behaves accordingly (depending on what action it was asked to do).
The .muddle/
directory always contains at least two files and one
directory:
The Description
file¶
$ cat .muddle/Description builds/01.py
which should be recognisable from the init
command. It tells muddle where
its build description was checked out (in the src/
directory, since that
is where checkouts go).
You should not normally need to edit this file, but if you do, the next time you run muddle, it will believe that the build description is in the new location.
The RootRepository
file¶
$ cat .muddle/RootRepository svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio
Again, this should be recognisable from the init
command. This specifies
the default repository, which muddle will assume is being used for checkouts
unless the build description says otherwise.
You should not normally need to edit this file either, but again muddle will believe its contents after a change.
The tags/
directory¶
The tags/
directory contains the current state of the build - it is
essentially a database of “labels satisfied”, stored in the filesystem.
In the current state:
.muddle/tags `-- checkout/ `-- builds/ `-- checked_out
we can see that the label checkout:builds/checked_out
has been “built” (or
has had its target satisfied). This corresponds to having checked out the
builds
checkout, which is indeed what has been done. We shall see later
that it is possible to change the build state by deleting or “touch
“ing
tag files.
The checked_out
file created by muddle does have content - it contains
a timestamp:
$ cat .muddle/tags/checkout/builds/checked_out 2010-09-28 11:14:34
but this content is never actually used for anything, and if you do add a tag
file, it is quite sufficient to use touch
to create an empty file.
If we ask muddle what checkouts it knows about:
$ muddle query checkouts builds cpio_co
we see two checkouts. The absence of any tag files for the “cpio_co” checkout tells us that it hasn’t been checked out yet.
Other contents¶
There may also be other things in the .muddle/
directory, notably an
instructions/
directory (used for deployment instructions), and an
am_subdomain
file (described when we talk about domains), but we shall
ignore these for now.
The src/builds
directory¶
The other directory present is the src/
directory, which, so far, contains
a single subdirectory for the build description.
`src/ `-- builds/ |-- 01.py |-- 01.pyc `-- .svn/
The mechanics of the build - the build description¶
The build description is one or more Python files, which the muddle command runs before doing anything else.
(A few muddle commands do not need a build description, obviously includinginit
, but these are the exception.)
For various reasons, the main build description file is traditionally
called src/builds/01.py
. muddle automatically adds the directory
containing the main build description (the file named in
.muddle/Description
) to the PYTHONPATH for the build description. Put more
simply, the build description can import
other Python files in the same
directory, which makes it easy to split build descriptions if necessary.
This example build has a fairly simple build description, in src/builds/01.py
:
#! /usr/bin/env python # # An example of how to build a cpio archive as a # deployment - e.g. for a Linux initrd. import muddled import muddled.pkgs.make import muddled.deployments.cpio import muddled.checkouts.simple def describe_to(builder): # Checkout .. muddled.checkouts.simple.relative(builder, "cpio_co") muddled.pkgs.make.simple(builder, "pkg_cpio", "x86", "cpio_co") muddled.deployments.cpio.deploy(builder, "my_archive.cpio", {"x86": "/"}, "cpio_dep", [ "x86" ]) builder.add_default_role("x86") builder.by_default_deploy("cpio_dep") # End file.
Briefly, muddle looks for a function called describe_to()
in the build
description, and runs it to define the build.
This particular example is quite old, and I’m not sure we’d write it quite like that any more. Regardless, it says the following things:
- there is a checkout called “cpio_co”, which is checked out from the default
repository (as defined in
.muddle/RootRepository
). - the package named
pkg_cpio
in rolex86
is built from that checkout - this will, by default, be done using aMakefile
in the checkout directory. - when deploying the final results of the build, a CPIO archive shall be created, with the role “x86” going at the root of the filesystem in that CPIO archive. The name of this deployment is “cpio_dep”.
- the default package role to build is “x86”
- the default thing to deploy is the “cpio_dep” deployment.
Some naming conventions¶
There are some strong naming conventions associated with muddle build trees:
The build description lives in a checkout called
src/builds/
All packages build out-of-tree (that is, in the
obj/
directory, which we shall describe later), so that thesrc/
directory only contains the checkout files as-checked-out.If there is a checkout containing “helper” makefiles and other associated things, used to build checkouts in other directories, it should be called
src/helpers/
.(For instance, if one is building kbus, and downloading it directly from its google code repository each time, then it is convenient to put the muddle-specific makefile in a different checkout, possibly as
src/helpers/Makefile.kbus
)There was an earlier convention of using the name
src/builders/
for such a directory, but this is visually confusing, and does not work well with tab-completion at the bash prompt.Checkout names should reflect the name of the external package being built, and generally also the version. So, for instance
screen-1.0.3
.Package names should be more general - if a package is built from
screen-1.0.3
, it would normally be namedscreen
. This allows for a later change in the build to use (for instance) checkoutscreen-1.0.4
instead.Deployment names should reflect the purpose of the deployment - for instance,
firmware
versuskernel
, and so on. This is very much dependent on the purpose of the build itself.
Note
This particular build is not really following the naming convention
for checkouts, packages and deployments as described above. This is because
it is trying to make it very clear which name belongs to which label type
(thus co_cpio
, pkg_cpio
and dep_cpio
)
It is also moderately traditional to call the main Python file for a build
description 01.py
(or 02.py
if it is version 2, and so on). This is
not, however, a strong convention, it just makes the “main” file easy to
spot.
Introspection of the dependency tree¶
muddle does have some ability to display the dependency tree that is generated from the build description, although it is not as user-friendly as we would like (and, ultimately, there should be graphical tools).
The simplest tool for dumping the dependency rules is the depends
command
(see muddle help depends
for more details on what it does).
The following shows the dependency rules described directly by the build description above:
$ muddle depends user-short ----- checkout:builds/changes_committed <-VcsCheckoutBuilder-- [ checkout:builds/up_to_date[T] ] checkout:builds/changes_pushed <-VcsCheckoutBuilder-- [ checkout:builds/changes_committed ] checkout:builds/pulled <-VcsCheckoutBuilder-- [ checkout:builds/checked_out, checkout:builds/up_to_date[T] ] checkout:builds/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:builds/checked_out ] checkout:cpio_co/changes_committed <-VcsCheckoutBuilder-- [ checkout:cpio_co/up_to_date[T] ] checkout:cpio_co/changes_pushed <-VcsCheckoutBuilder-- [ checkout:cpio_co/changes_committed ] checkout:cpio_co/pulled <-VcsCheckoutBuilder-- [ checkout:cpio_co/checked_out, checkout:cpio_co/up_to_date[T] ] checkout:cpio_co/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:cpio_co/checked_out ] deployment:cpio_dep/deployed <-CpioDeploymentBuilder-- [ package:*{x86}/postinstalled ] package:pkg_cpio{x86}/built <-MakeBuilder-- [ package:pkg_cpio{x86}/configured ] package:pkg_cpio{x86}/configured <-MakeBuilder-- [ package:pkg_cpio{x86}/preconfig ] package:pkg_cpio{x86}/installed <-MakeBuilder-- [ package:pkg_cpio{x86}/built ] package:pkg_cpio{x86}/postinstalled <-MakeBuilder-- [ package:pkg_cpio{x86}/installed ] package:pkg_cpio{x86}/preconfig <-MakeBuilder-- [ checkout:cpio_co/checked_out ] -----
Each line above shows three things:
- a target label.
- an arrow containing the name of the action to take to “satisfy” or “build” that label (nb: in the current trunk of muddle, this is not present).
- a list of the labels that must be “satisfied” or “built” before that action can be performed.
It is sorted by target label, which unfortunately is not too much help, but
one can see that, for instance, the deployment:cpio_dep/deployed
label
depends on package:*{x86}/postinstalled
, where the *
is a wildcard
over all package names in the {x86}
role.
A shorter and perhaps more helpful representation of that can be provided
using the query deps
command:
$ muddle query deps deployment:cpio_dep/deployed Build order for deployment:cpio_dep/deployed .. checkout:cpio_co/checked_out package:pkg_cpio{x86}/preconfig package:pkg_cpio{x86}/configured package:pkg_cpio{x86}/built package:pkg_cpio{x86}/installed package:pkg_cpio{x86}/postinstalled deployment:cpio_dep/deployed
This shows all of the labels that must be built before
deployment:cpio_dep/deployed
, and in what order they will be built.
Checking out the source code¶
Normally, after the init
command, one would just use a “bare” muddle
command to perform the rest of the checkouts, build them (as they are checked
out), and do any other steps indicated by the build description. However,
since I’m interested in the various stages of the build tree, I shall do all
of these things one by one.
So we start by checking out the actual source for our build:
$ muddle checkout _all > Building checkout:cpio_co/checked_out > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/cpio_co cpio_co A cpio_co/hello_world.c A cpio_co/Makefile Checked out revision 458. > Make directory /home/tibs/sw/m3/example/.muddle/tags/checkout/cpio_co
After doing this, we’ve now gained a new tag in the .muddle/
directory:
`-- .muddle/ |-- Description |-- RootRepository `-- tags/ `-- checkout/ |-- builds/ | `-- checked_out `-- cpio_co/ `-- checked_out
indicating that we have successfully checked out the cpio_co
checkout,
and the source files for our missing checkout are now present in the src/
directory:
`-- src/ |-- builds/ | |-- 01.py | |-- 01.pyc | `-- .svn/ `-- cpio_co/ |-- hello_world.c |-- Makefile `-- .svn/
(whilst I’m showing the .svn/
directories here, I’ve removed the listing
of their contents).
The mechanics of the build - the makefile¶
This build only has a single checkout, which produces a single package. As
such it also has a single makefile, src/cpio_co/Makefile
:
# Makefile for cpio_co # # Just dumps some compiled C into the right place. INSTALL=install all: $(CC) -o $(MUDDLE_OBJ)/hello_world hello_world.c install: if [ ! -d $(MUDDLE_INSTALL)/bin ]; then mkdir $(MUDDLE_INSTALL)/bin; fi $(INSTALL) -m 0755 $(MUDDLE_OBJ)/hello_world $(MUDDLE_INSTALL)/bin/hello_world $(INSTALL) -m 0644 hello_world.c $(MUDDLE_INSTALL)/hello_world.c config: @echo Nothing to do clean: rm -f $(MUDDLE_OBJ)/hello_world distclean: clean @echo Distclean is just a clean # end file.
We saw when querying the builds dependencies (in Introspection of the
dependency tree) that various dependency rules had MakeBuilder
as their
action. This action knows what targets in a makefile to call in order to build
a particular package:
label tag.
The target label tags (the /xxx
at the end of a label) correspond to
makefile targets as follows:
Target tag Makefile target /preconfig <none> /configured config /built all /installed install /postinstalled <none>
Target tag Makefile target /clean clean /distclean distclean
It can be useful to think of a package as “progressing” from /preconfig
through to /postinstalled
.
preconfig
does not correspond to a make target, and there is no default
action for it. Things that depend on a package existing (but nothing else)
will depend on the /preconfig
tagged package label.
There is no action defined by default for /preconfig
. That doesn’t stop a
particular build from defining one - for instance, ExpandingMakeBuilder
is a
MakeBuilder subclass which expands a .tgz
file into source files for the
later stages to compile and install (see muddled.pkgs.make
).
Although there is no make target associated with /postinstalled
, it is at
this stage that pkg-config
files are rewritten (if requested), and this
is the tag that should be used when another package depends on something
being “fully” built.
Building a package¶
Whilst it is possible for a package to be built from more than one checkout, this is relatively uncommon, and it is quite usual for a package to correspond directly to a checkout. As it does in this case.
If one tries to build a package that doesn’t exist, one will be told so:
$ muddle build cpio Building package:cpio{x86}/postinstalled There is no rule to build label package:cpio{x86}/postinstalled
It’s simple to find out what packages there are:
$ muddle query packages pkg_cpio
So let’s build it:
$ muddle build pkg_cpio Building package:pkg_cpio{x86}/postinstalled > Building package:pkg_cpio{x86}/preconfig > Make directory /home/tibs/sw/m3/example/obj/pkg_cpio/x86 > Make directory /home/tibs/sw/m3/example/install/x86 > Make directory /home/tibs/sw/m3/example/.muddle/tags/package/pkg_cpio > Building package:pkg_cpio{x86}/configured > make -f Makefile config Nothing to do > Building package:pkg_cpio{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world hello_world.c > Building package:pkg_cpio{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world /home/tibs/sw/m3/example/install/x86/bin/hello_world install -m 0644 hello_world.c /home/tibs/sw/m3/example/install/x86/hello_world.c > Building package:pkg_cpio{x86}/postinstalled
We now have two new directories:
$ ls -AFt install/ obj/ src/ .muddle/
The src/
directory has not changed.
The .muddle/
directory has gained some new tags in
.muddle/tags/package/pkg_cpio/
, reflecting the new state of the build:
.muddle/ |-- Description |-- RootRepository `-- tags/ |-- checkout/ | |-- builds/ | | `-- checked_out | `-- cpio_co/ | `-- checked_out `-- package/ `-- pkg_cpio/ |-- x86-built |-- x86-configured |-- x86-installed |-- x86-postinstalled `-- x86-preconfig
The obj/
directory contains the results of building packages. Thus we have
a new directory called obj/pkg_cpio/x86
.
obj/ `-- pkg_cpio/ `-- x86/ `-- hello_world*
Specifically, within obj/
there is a directory named after each package,
within which there is a directory named after each role for that package. For
convenience, the makefile can use the environment variable $(MUDDLE_OBJ)
to refer to <root_dir>/obj/pkg_cpio/x86
(<root_dir>
indicates the
absolute path to the top-level build directory).
There would normally be more use of subdirectories such as
$(MUDDLE_OBJ)/bin
in a “real” build, and muddle provides some other
environment variables to help with these.
The install/
contains the results of installing packages. Each {role}
gets a separate directory in install/
:
install/ `-- x86/ |-- bin/ | `-- hello_world* `-- hello_world.c
Again, the makefile can use the environment variable $(MUDDLE_INSTALL)
to
refer to this role-specific directory.
As an aside, using theMUDDLE
environment variables in this way means that if two packages (e.g.,package:a{b}/*
versuspackage:c{d}/*
, or evenpackage:a{b}/*
versuspackage:a{d}/*
) both use the same checkout, and thus the same makefile, the results of building the two packages will still end up in different, predictable directories.
It is possible to find out the entire environment muddle passes to a makefile
with the makeenv
query:
$ muddle query makeenv package:pkg_cpio{x86}/built MUDDLE=/home/tibs/sw/m3/muddle3_labels/muddle/muddled/__main__.py MUDDLE_INCLUDE_DIRS= MUDDLE_INSTALL=/home/tibs/sw/m3/example/install/x86 MUDDLE_INSTRUCT=/home/tibs/sw/m3/muddle3_labels/muddle/muddled/__main__.py instruct pkg_cpio{x86} MUDDLE_KIND=package MUDDLE_LABEL=package:pkg_cpio{x86}/built MUDDLE_LD_LIBRARY_PATH= MUDDLE_LIB_DIRS= MUDDLE_NAME=pkg_cpio MUDDLE_OBJ=/home/tibs/sw/m3/example/obj/pkg_cpio/x86 MUDDLE_OBJ_INCLUDE=/home/tibs/sw/m3/example/obj/pkg_cpio/x86/include MUDDLE_OBJ_LIB=/home/tibs/sw/m3/example/obj/pkg_cpio/x86/lib MUDDLE_OBJ_OBJ=/home/tibs/sw/m3/example/obj/pkg_cpio/x86/obj MUDDLE_PKGCONFIG_DIRS= MUDDLE_PKGCONFIG_DIRS_AS_PATH= MUDDLE_ROLE=x86 MUDDLE_ROOT=/home/tibs/sw/m3/example MUDDLE_SRC=/home/tibs/sw/m3/example/src/cpio_co MUDDLE_TAG=built MUDDLE_TARGET_LOCATION=/ MUDDLE_UNINSTRUCT=/home/tibs/sw/m3/muddle3_labels/muddle/muddled/__main__.py instruct pkg_cpio{x86}
Builds specifying cross-compilation toolchains and other options may have much longer environments passed down.
Deployment¶
Deployment is the process of copying the files for the various roles from the
install/
directories to the deploy/
directories.
Deployment is commonly the stage that prepares the “blob” that will be put onto the final device (if one is building an embedded device), and perhaps also aggregates the tools for doing so.
In this build, we are producing a CPIO archive.
(CPIO is a very old Unix archive format. For our purposes (“us” being muddle), its advantage is that it is possible to flag files within the archive with particular permission bits, to request creationg of device nodes, and other thing that would require superuser privileges if done “live” in the build tree.)
In this build we only have a single deployment target, so we don’t need to specify its name.
$ muddle deploy Building deployment:cpio_dep/deployed > Building deployment:cpio_dep/deployed > Make directory /home/tibs/sw/m3/example/deploy/cpio_dep Collecting package:*{x86}/* for deployment to / .. h = ---Roots--- / -> [ / (fs /home/tibs/sw/m3/example/install/x86) mode = 40755 uid = 7007 gid = 7007 kids = /bin /hello_world.c] ---Map--- /bin/hello_world -> [ /bin/hello_world (fs /home/tibs/sw/m3/example/install/x86/bin/hello_world) mode = 100755 uid = 7007 gid = 7007 kids = ] / -> [ / (fs /home/tibs/sw/m3/example/install/x86) mode = 40755 uid = 7007 gid = 7007 kids = /bin /hello_world.c] /bin -> [ /bin (fs /home/tibs/sw/m3/example/install/x86/bin) mode = 40755 uid = 7007 gid = 7007 kids = /bin/hello_world] /hello_world.c -> [ /hello_world.c (fs /home/tibs/sw/m3/example/install/x86/hello_world.c) mode = 100644 uid = 7007 gid = 7007 kids = ] --- base = / Scanning instructions for role x86, domain None .. > Writing /home/tibs/sw/m3/example/deploy/cpio_dep/my_archive.cpio .. > Packing / .. > Packing /bin .. > Packing /bin/hello_world .. > Packing /hello_world.c .. > Packing TRAILER!!! .. > Make directory /home/tibs/sw/m3/example/.muddle/tags/deployment/cpio_dep
That leaves us with a new deploy/
directory:
$ ls -AFt deploy/ install/ obj/ src/ .muddle/
containing our CPIO archive:
deploy/ `-- cpio_dep/ `-- my_archive.cpio
and the appropriate tag has been set in the .muddle/
directory tree:
.muddle/tags/deployment/ `-- cpio_dep/ `-- deployed
Rebuilding things¶
muddle provides a convenience command to rebuild a package:
$ muddle rebuild pkg_cpio Killing package:pkg_cpio{x86}/built Clearing tags: package:pkg_cpio{x86}/built package:pkg_cpio{x86}/installed package:pkg_cpio{x86}/postinstalled package:pkg_cpio{x86}/built Building package:pkg_cpio{x86}/postinstalled > Building package:pkg_cpio{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world hello_world.c > Building package:pkg_cpio{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world /home/tibs/sw/m3/example/install/x86/bin/hello_world install -m 0644 hello_world.c /home/tibs/sw/m3/example/install/x86/hello_world.c > Building package:pkg_cpio{x86}/postinstalled
As you can see, this “kills” (deletes) the tags saying that the package has
been built, installed and postinstalled, and then rebuilds the
/postinstalled
tag. This does not, however, do anything about things that
depend on this package.
It is frequently more useful to use the distrebuild
command, which is a
conflation of the distclean
and build
commands:
$ muddle distrebuild pkg_cpio Building: package:pkg_cpio{x86}/distclean .. > Building package:pkg_cpio{x86}/distclean[T] > make -f Makefile distclean rm -f /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world Distclean is just a clean Killing: package:pkg_cpio{x86}/preconfig .. Clearing tags: package:pkg_cpio{x86}/preconfig package:pkg_cpio{x86}/configured package:pkg_cpio{x86}/preconfig package:pkg_cpio{x86}/installed package:pkg_cpio{x86}/postinstalled package:pkg_cpio{x86}/built Building package:pkg_cpio{x86}/postinstalled > Building package:pkg_cpio{x86}/preconfig > Building package:pkg_cpio{x86}/configured > make -f Makefile config Nothing to do > Building package:pkg_cpio{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world hello_world.c > Building package:pkg_cpio{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world /home/tibs/sw/m3/example/install/x86/bin/hello_world install -m 0644 hello_world.c /home/tibs/sw/m3/example/install/x86/hello_world.c > Building package:pkg_cpio{x86}/postinstalled
The distclean
also removes the tags for any packages that depended upon
this package (not very obvious in this case where we only had one, of course).
(Beware - it unsets the state of other packages, but not the deployment.)
You can also, quite legally, do the same thing by direct manipulation of
the tag files in the .muddle/
directories:
$ rm .muddle/tags/package/pkg_cpio/* $ muddle build pkg_cpio Building package:pkg_cpio{x86}/postinstalled > Building package:pkg_cpio{x86}/preconfig > Building package:pkg_cpio{x86}/configured > make -f Makefile config Nothing to do > Building package:pkg_cpio{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world hello_world.c > Building package:pkg_cpio{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world /home/tibs/sw/m3/example/install/x86/bin/hello_world install -m 0644 hello_world.c /home/tibs/sw/m3/example/install/x86/hello_world.c > Building package:pkg_cpio{x86}/postinstalled
That is, muddle doesn’t care if you remove the tags directly, instead of via the command line tool.
You can even do things like:
$ rm .muddle/tags/checkout/builds/checked_out $ muddle checkout _all > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/builds builds Checked out revision 459.
This is quite deliberate. Whilst the muddle command line tool provides a
variery of useful shorthand commands (such as “distclean”, “distrebuild” and
“redeploy”), it is quite possible and even sensible to exert finer control
over a build by deleting (or, indeed, “touch”ing) tag files in the .muddle
directory.
Do what I say, not what I do¶
Please note that although I have been explicitly requesting indiviual builds, rebuilds, deployments and so on above, this is not necessarily the normal way of using muddle. The muddle command line tool is carefully designed so that it normally does “the right thing”, according to which directory you are in.
So, if one is at the top level of the build, and has checked out everything, it is more colloquial just to give the muddle command with no arguments than to be overly specific.
This is perhaps most easily shown by, well, showing it.
We can revert to the just checked out state quite easily, by just removing
everything except the .muddle/
and src/
directories:
$ ls -AF deploy/ install/ .muddle/ obj/ src/ $ rm -rf deploy/ install/ obj/ $ rm -rf .muddle/tags/package/ .muddle/tags/deployment/
Once we’re back to the stage just after init
, we can just do:
$ muddle Building deployment:cpio_dep/deployed > Building package:pkg_cpio{x86}/preconfig ... > Building package:pkg_cpio{x86}/configured ... > Building package:pkg_cpio{x86}/built ... > Building package:pkg_cpio{x86}/installed ... > Building package:pkg_cpio{x86}/postinstalled > Building deployment:cpio_dep/deployed ... > Writing /home/tibs/sw/m3/example/deploy/cpio_dep/my_archive.cpio .. ... > Make directory /home/tibs/sw/m3/example/.muddle/tags/deployment/cpio_dep
(I’ve shortened the log above, because by now it should be all too familiar.)
Contrariwise, if we clean it all up again:
$ rm -rf deploy/ install/ obj/ $ rm -rf .muddle/tags/package/ .muddle/tags/deployment/
and go into a particular checkout directory:
$ pushd src/cpio_co $ muddle Killing package:pkg_cpio{x86}/built Clearing tags: package:pkg_cpio{x86}/built package:pkg_cpio{x86}/installed package:pkg_cpio{x86}/postinstalled package:pkg_cpio{x86}/built Building package:pkg_cpio{x86}/* > Building package:pkg_cpio{x86}/distclean[T] > Make directory /home/tibs/sw/m3/example/obj/pkg_cpio/x86 > Make directory /home/tibs/sw/m3/example/install/x86 > make -f Makefile distclean rm -f /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world Distclean is just a clean > Building package:pkg_cpio{x86}/clean[T] > make -f Makefile clean rm -f /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world > Building package:pkg_cpio{x86}/preconfig > Make directory /home/tibs/sw/m3/example/.muddle/tags/package/pkg_cpio > Building package:pkg_cpio{x86}/configured > make -f Makefile config Nothing to do > Building package:pkg_cpio{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world hello_world.c > Building package:pkg_cpio{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example/obj/pkg_cpio/x86/hello_world /home/tibs/sw/m3/example/install/x86/bin/hello_world install -m 0644 hello_world.c /home/tibs/sw/m3/example/install/x86/hello_world.c > Building package:pkg_cpio{x86}/postinstalled $ popd
That is, muddle only rebuilds the packages that depend on this checkout. See
muddle help build
and its friends for more information on this.
People who like to cd
around the filesystem a lot will probably
find this intuitive. Those of us who prefer to sit at the top-level and work
from there (not necessarily any more sensible, of course) are more likely just
to do:
$ muddle distrebuild pkg_cpio
(yes, this has identical effect, and no, by now I’m not going to include the listing yet again).
Note
As an aside, if you want to preserve the source of a build, then (if there aren’t any domains) it is sufficient to do:
$ tar -zcvf build.tgz .muddle/Description .muddle/RootRepository .muddle/tags/checkout src
However, muddle provides an alternative that will work in all cases, including when there are subdomains:
$ muddle distribute --with-vcs _source_release ../save_dir
$ cd ..
$ tar -zcvf build.tgz save_dir
(where ``save_dir`` should be replaced by something more informative).
Adding a new checkout/package¶
Let’s suppose we want to add a new package, based on a new checkout. There is more than one way to do this, but the following shows a sensible approach.
We shall create our new package by just copying one we’ve already got:
$ cp -a src/cpio_co src/fred
We must remember to remove the outdated repository information:
$ rm -rf src/fred/,svn/
And make this checkout do something different:
$ mv src/fred/hello_world.c src/fred/bye_world.c $ sed -e s/hello_world/bye_world/g --in-place src/fred/Makefile $ sed -e s/Hello/Byebye/g --in-place src/fred/bye_world.c
muddle has no knowledge of this checkout yet, so we need to add it to the
build description. We can simply edit src/builds/01.py
to add the lines:
muddled.checkouts.simple.relative(builder, "fred")
and:
muddled.pkgs.make.simple(builder, "fred", "x86", "fred")
(yes, that leaves us with a package and checkout with the same name, but that’s not a problem, and is closer to the more conventional usage).
That’s enough to give us:
$ muddle query checkouts builds cpio_co fred
and:
$ muddle query deps deployment:cpio_dep/deployed Build order for deployment:cpio_dep/deployed .. checkout:fred/checked_out checkout:cpio_co/checked_out package:pkg_cpio{x86}/preconfig package:fred{x86}/preconfig package:pkg_cpio{x86}/configured package:fred{x86}/configured package:fred{x86}/built package:pkg_cpio{x86}/built package:fred{x86}/installed package:pkg_cpio{x86}/installed package:pkg_cpio{x86}/postinstalled deployment:cpio_dep/deployed
However, if we were to try to build:
$ muddle Building deployment:cpio_dep/deployed > Building checkout:fred/checked_out > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/fred fred svn: URL 'http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/fred' doesn't exist Can't build deployment:cpio_dep/deployed - Command 'svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/fred fred' execution failed - 1
This makes sense, because (a) we have not told muddle that the new checkout is present in its directory structure, and (b) we have not actually checked it in to the subversion repository it wants to look in.
Since we’ve not yet finished developing this new package, we don’t want to check it in to the repository yet (and, in particular in our case, we do not want to permanently add it to our example).
So we need to make muddle believe that it has been checked out.
As you might expect, there are two ways to do this. The one that you may
guess is to manipulate the .muddle/`
directory structure directly - so we
could just do:
$ mkdir .muddle/tags/checkout/fred $ touch .muddle/tags/checkout/fred/checked_out
However, that’s a bit clumsy to type, so muddle provides a convenient command for asserting a particular tag (just assuming you are comfortable with the label syntax):
$ muddle assert checkout:fred/checked_out > Make directory /home/tibs/sw/m3/example3/.muddle/tags/checkout/fred
and now the tag file is present:
$ ls .muddle/tags/checkout/fred checked_out
and we can then do:
$ muddle Building deployment:cpio_dep/deployed > Building package:fred{x86}/preconfig > Make directory /home/tibs/sw/m3/example3/obj/fred/x86 > Make directory /home/tibs/sw/m3/example3/.muddle/tags/package/fred > Building package:fred{x86}/configured > make -f Makefile config Nothing to do > Building package:fred{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example3/obj/fred/x86/bye_world bye_world.c > Building package:fred{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example3/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example3/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example3/obj/fred/x86/bye_world /home/tibs/sw/m3/example3/install/x86/bin/bye_world install -m 0644 bye_world.c /home/tibs/sw/m3/example3/install/x86/bye_world.c > Building deployment:cpio_dep/deployed > Make directory /home/tibs/sw/m3/example3/deploy/cpio_dep Collecting package:*{x86}/* for deployment to / .. h = ---Roots--- / -> [ / (fs /home/tibs/sw/m3/example3/install/x86) mode = 40755 uid = 7007 gid = 7007 kids = /bin /hello_world.c /bye_world.c] ---Map--- /bin/bye_world -> [ /bin/bye_world (fs /home/tibs/sw/m3/example3/install/x86/bin/bye_world) mode = 100755 uid = 7007 gid = 7007 kids = ] /bin -> [ /bin (fs /home/tibs/sw/m3/example3/install/x86/bin) mode = 40755 uid = 7007 gid = 7007 kids = /bin/bye_world /bin/hello_world] /hello_world.c -> [ /hello_world.c (fs /home/tibs/sw/m3/example3/install/x86/hello_world.c) mode = 100644 uid = 7007 gid = 7007 kids = ] /bin/hello_world -> [ /bin/hello_world (fs /home/tibs/sw/m3/example3/install/x86/bin/hello_world) mode = 100755 uid = 7007 gid = 7007 kids = ] /bye_world.c -> [ /bye_world.c (fs /home/tibs/sw/m3/example3/install/x86/bye_world.c) mode = 100644 uid = 7007 gid = 7007 kids = ] / -> [ / (fs /home/tibs/sw/m3/example3/install/x86) mode = 40755 uid = 7007 gid = 7007 kids = /bin /hello_world.c /bye_world.c] --- base = / Scanning instructions for role x86, domain None .. > Writing /home/tibs/sw/m3/example3/deploy/cpio_dep/my_archive.cpio .. > Packing / .. > Packing /bin .. > Packing /bin/bye_world .. > Packing /bin/hello_world .. > Packing /hello_world.c .. > Packing /bye_world.c .. > Packing TRAILER!!! .. > Make directory /home/tibs/sw/m3/example3/.muddle/tags/deployment/cpio_dep
This just leaves actually adding the new package to revision control somewhere - in this case, the build description is saying (implicitly) that this package is stored in the same subversion repository as the rest of the example, and so one would need to add it there by the normal means (which is a topic for another time).
(The fact that this is much easier to do for distributed revision control systems like bazaar, mercurial or git is in itself a good reason for using them!)
Version stamps¶
Version stamps allow one to produce a simple text file (actually, an INI
file) representing the current state of a build. This can be useful for
various purposes, but its main intent is to allow saving a build state so that
it can be accurately recreated at a later date (for instance, so one can
rebuild an earlier release to customers).
The muddle stamp
and unstamp
commands have documentation via muddle
help stamp
and help unstamp
.
However, it is worth giving a quick example of creating a version stamp:
$ muddle stamp version Finding all checkouts... found 2 Processing Svn checkout 'builds' Processing Svn checkout 'cpio_co' Creating directory /home/tibs/sw/m3/example/versions Writing to /home/tibs/sw/m3/example/versions/_temporary.stamp Wrote revision data to /home/tibs/sw/m3/example/versions/_temporary.stamp File has SHA1 hash 189085d413217cb1865868b5d9916085d22e6f50 Renaming /home/tibs/sw/m3/example/versions/_temporary.stamp to /home/tibs/sw/m3/example/versions/01.stamp
As indicated above, we now have a new versions/
directory:
$ ls -AFt versions/ deploy/ install/ obj/ src/ .muddle/
containing the stamp file:
versions `-- 01.stamp
Modern build descriptions are encouraged to specify a build name (this is as simple as adding a line of the form:
builder.build_name = 'ExampleBuild'
to the Python code). If this is given, then the version stamp file will use
that build name as its filename, otherwise it defaults to the filename of the
main build description file (01
in this case).
The versions/
directory is suitable for putting into a version control
system, as one might expect, and future versions of muddle are likely to
provide more support for handling this (see issue 117 for some ideas on
this).
The contents of a stamp file is deliberately human readable:
[ROOT] description = builds/01.py repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio [CHECKOUT builds] name = builds relative = builds repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio revision = 458 [CHECKOUT cpio_co] name = cpio_co relative = cpio_co repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio revision = 458
As normal. muddle will not stop you writing or editing stamp files. This may conceivably be useful.
Domains¶
Domains are part of advanced muddle use, and are probably not relevant to most builds. Regardless, it is probably worth reading at least What domains are for from this section, as useful background, and so that you can recognise when they might be applicable.
The way domains work follows naturally from the rest of muddle, but the implementation is still being tidied up - hence themuddle3_label
branch.
What domains are for¶
Domains allow one to include one build description inside another, in much the same way as a Python module can import another.
A simple example might be when building a system with WebKit and X11 support. We might have two builds, constructed as follows:
def UI_Build: import WebKit_build import X11_build
and:
def Login_build: import X11_build
Alternatively, we might have some high-level software for handling internet television, and want to build two systems on different architectures. In this case, our contrasting builds might be:
def IPTV_stack: import ARM_based_stack
and:
def IPTV_stack: import MIPS_based_stack
How domains work¶
A build includes another build (a subdomain) using the include_domain()
function. This takes the same arguments as the muddle init
command (that
is, a repository specification and a build description therein), plus a name
for the domain.
The following sequence of actions is then performed:
- Create a directory called
domains/<name>/
, where<name>
is the name of the new domain. cd
into that new directory, and perform (the equivalent of) a muddleinit
command therein, using the repository specification and build description given.- Read in this new build, giving a stand-alone build tree datastructure.
- Find all of the labels in the new build tree datastructures, and change
them to add the domain name. So, for instance,
checkout:fred/checked_out
would becomecheckout:(<name>)fred/checked_out
. - Incorporate this amended build tree datastructure into the original (“top level”) build.
- Create a
.muddle/am_subdomain
file in the domain. This allows muddle to distinguish the actual (final) top-level build from any subdomains within it, which is a useful optimisation.
Note that the build description for a domain does not itself know that it is not the top-level of a muddle build. Indeed, this is an important property of domains - it means that any build description can potentially be included in any other.
For most purposes, the subdomain works just as if it were a “normal” top-level
build. In particular, it uses its own .muddle/
directory to record its
build state, and writes things to its own obj/
and install/
directories.
Also note that, as a consequence of the way domains work (and this is a good thing), it is only possible for a build to refer to labels in subdomains, not to those in sibling or parent build trees.
To allow for more flexibility in how domains are used, there are tools in the
muddled
package for manipulating the merged label-space. For instance, if one
is using subdomains that originated as two different set-top-box builds, both
might provide a linux kernel, and the top-level build probably wants to amend
the labels such that both subdomains use one of the kernel packages
(broadly, saying “this label should be used instead of that label”).
A worked example of using domains¶
So, we shall start in a new empty directory:
$ cd .. $ mkdir example2 $ cd example2
and start with the same build that we used before:
$ muddle init svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio builds/01.py > Make directory /home/tibs/sw/m3/example2/.muddle Initialised build tree in /home/tibs/sw/m3/example2 Repository: svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio Build description: builds/01.py Checking out build description .. > Make directory /home/tibs/sw/m3/example2/src > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/builds builds A builds/01.py Checked out revision 459. > Make directory /home/tibs/sw/m3/example2/.muddle/tags/checkout/builds Done.
So, just as before, we now have the following directory structure:
. |-- .muddle/ | |-- Description | |-- RootRepository | `-- tags/ | `-- checkout/ | `-- builds/ | `-- checked_out `-- src/ `-- builds/ |-- 01.py `-- 01.pyc
We can now edit the src/builds/01.py
file to include a subdomain as part of
the build. As described above, we need to specify how to retrieve the domain,
giving the same information as for the “muddle init” command. We shall call
our domain b
(after the example we’re taking it from):
from muddled.mechanics import include_domain include_domain(builder, domain_name = "b", domain_repo = "svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b", domain_desc = "builds/01.py")
We also have to say how to include the result into our deployment - a simple way to do this is:
import muddled.deployments.collect as collect collect.deploy(builder, "everything") collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "", domain = None) collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "usr", domain = "b")
and then deploy the new deployment “everything”, instead of the old “cpio_dep”. Note that we’re not deploying to a CPIO archive this time, but just as normal files.
We should probably also change the introductory comment to:
# An example of building with a subdomain
So the “top level” build description is now:
#! /usr/bin/env python # # An example of building with a subdomain import muddled import muddled.pkgs.make import muddled.deployments.cpio import muddled.checkouts.simple import muddled.deployments.collect as collect from muddled.mechanics import include_domain def describe_to(builder): # Checkout .. muddled.checkouts.simple.relative(builder, "cpio_co") muddled.pkgs.make.simple(builder, "pkg_cpio", "x86", "cpio_co") include_domain(builder, domain_name = "b", domain_repo = "svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b", domain_desc = "builds/01.py") collect.deploy(builder, "everything") collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "", domain = None) collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "usr", domain = "b") builder.add_default_role("x86") builder.by_default_deploy("everything") # End file.
If we now do a “checkout _all”:
$ muddle checkout _all > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle Initialised build tree in /home/tibs/sw/m3/example3/domains/b Repository: svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b Build description: builds/01.py Checking out build description .. > Make directory /home/tibs/sw/m3/example3/domains/b/src > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/b/builds builds A builds/01.py Checked out revision 459. > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle/tags/checkout/builds There is no rule to build label checkout:b_co/checked_out > Building checkout:cpio_co/checked_out > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/cpio_co cpio_co A cpio_co/hello_world.c A cpio_co/Makefile Checked out revision 459. > Make directory /home/tibs/sw/m3/example3/.muddle/tags/checkout/cpio_co
giving us:
$ ls -AF domains/ .muddle/ src/
or, in more detail (omitting .svn
directories):
. |-- domains/ | `-- b/ | |-- .muddle/ | | |-- am_subdomain | | |-- Description | | |-- RootRepository | | `-- tags/ | | `-- checkout/ | | `-- builds/ | | `-- checked_out | `-- src/ | `-- builds/ | |-- 01.py | `-- 01.pyc |-- .muddle/ | |-- Description | |-- RootRepository | `-- tags/ | `-- checkout/ | |-- builds/ | | `-- checked_out | `-- cpio_co/ | `-- checked_out `-- src/ |-- builds/ | |-- 01.py | `-- 01.pyc `-- cpio_co/ |-- hello_world.c `-- Makefile
Apart from the am_subdomain
file in its .muddle/
directory, the
domains/b/
directory looks like a perfectly normal build.
If we then do:
$ muddle build _all Building package:(b)pkg_b{x86}/postinstalled package:pkg_cpio{x86}/postinstalled > Building checkout:(b)b_co/checked_out > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/b/b_co b_co A b_co/hello_world.c A b_co/Makefile Checked out revision 459. > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle/tags/checkout/b_co > Building package:(b)pkg_b{x86}/preconfig > Make directory /home/tibs/sw/m3/example3/domains/b/obj/pkg_b/x86 > Make directory /home/tibs/sw/m3/example3/domains/b/install/x86 > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle/tags/package/pkg_b > Building package:(b)pkg_b{x86}/configured > make -f Makefile config Nothing to do > Building package:(b)pkg_b{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example3/domains/b/obj/pkg_b/x86/hello_world hello_world.c > Building package:(b)pkg_b{x86}/installed > make -f Makefile install install -m 0755 /home/tibs/sw/m3/example3/domains/b/obj/pkg_b/x86/hello_world /home/tibs/sw/m3/example3/domains/b/install/x86/hello_world > Building package:(b)pkg_b{x86}/postinstalled > Building package:pkg_cpio{x86}/preconfig > Make directory /home/tibs/sw/m3/example3/obj/pkg_cpio/x86 > Make directory /home/tibs/sw/m3/example3/install/x86 > Make directory /home/tibs/sw/m3/example3/.muddle/tags/package/pkg_cpio > Building package:pkg_cpio{x86}/configured > make -f Makefile config Nothing to do > Building package:pkg_cpio{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example3/obj/pkg_cpio/x86/hello_world hello_world.c > Building package:pkg_cpio{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example3/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example3/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example3/obj/pkg_cpio/x86/hello_world /home/tibs/sw/m3/example3/install/x86/bin/hello_world install -m 0644 hello_world.c /home/tibs/sw/m3/example3/install/x86/hello_world.c > Building package:pkg_cpio{x86}/postinstalled
we end up with the subdomain built in its directory structure:
domains/ `-- b/ |-- install/ | `-- x86/ | `-- hello_world* |-- .muddle/ | |-- am_subdomain | |-- Description | |-- RootRepository | `-- tags/ | |-- checkout/ | | |-- b_co/ | | | `-- checked_out | | `-- builds/ | | `-- checked_out | `-- package/ | `-- pkg_b/ | |-- x86-built | |-- x86-configured | |-- x86-installed | |-- x86-postinstalled | `-- x86-preconfig |-- obj/ | `-- pkg_b/ | `-- x86/ | `-- hello_world* `-- src/ |-- b_co/ | |-- hello_world.c | `-- Makefile `-- builds/ |-- 01.py `-- 01.pyc
and the top-level build in its:
`-- install/ | `-- x86/ | |-- bin/ | | `-- hello_world* | `-- hello_world.c |-- .muddle/ | |-- Description | |-- RootRepository | `-- tags/ | |-- checkout/ | | |-- builds/ | | | `-- checked_out | | `-- cpio_co/ | | `-- checked_out | `-- package/ | `-- pkg_cpio/ | |-- x86-built | |-- x86-configured | |-- x86-installed | |-- x86-postinstalled | `-- x86-preconfig |-- obj/ | `-- pkg_cpio/ | `-- x86/ | `-- hello_world* `-- src/ |-- builds/ | |-- 01.py | `-- 01.pyc `-- cpio_co/ |-- hello_world.c `-- Makefile
If we ask after packages, we have both sets:
$ muddle query packages pkg_b pkg_cpio
(this should arguably report the domain of each package name as well - there should probably be an option to select this).
If we ask what domains we have, we get:
$ muddle query domains b
(the current printout of domains includes the “empty” or top-level domain in its listing, which is slightly unobvious, and will probably be fixed at some time).
Our dependency rules are now extended to include those from the subdomain as well:
$ muddle depend user-short ----- checkout:builds/changes_committed <-VcsCheckoutBuilder-- [ checkout:builds/up_to_date[T] ] checkout:builds/changes_pushed <-VcsCheckoutBuilder-- [ checkout:builds/changes_committed ] checkout:builds/pulled <-VcsCheckoutBuilder-- [ checkout:builds/checked_out, checkout:builds/up_to_date[T] ] checkout:builds/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:builds/checked_out ] checkout:cpio_co/changes_committed <-VcsCheckoutBuilder-- [ checkout:cpio_co/up_to_date[T] ] checkout:cpio_co/changes_pushed <-VcsCheckoutBuilder-- [ checkout:cpio_co/changes_committed ] checkout:cpio_co/pulled <-VcsCheckoutBuilder-- [ checkout:cpio_co/checked_out, checkout:cpio_co/up_to_date[T] ] checkout:cpio_co/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:cpio_co/checked_out ] checkout:(b)b_co/changes_committed <-VcsCheckoutBuilder-- [ checkout:(b)b_co/up_to_date[T] ] checkout:(b)b_co/changes_pushed <-VcsCheckoutBuilder-- [ checkout:(b)b_co/changes_committed ] checkout:(b)b_co/pulled <-VcsCheckoutBuilder-- [ checkout:(b)b_co/checked_out, checkout:(b)b_co/up_to_date[T] ] checkout:(b)b_co/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:(b)b_co/checked_out ] checkout:(b)builds/changes_committed <-VcsCheckoutBuilder-- [ checkout:(b)builds/up_to_date[T] ] checkout:(b)builds/changes_pushed <-VcsCheckoutBuilder-- [ checkout:(b)builds/changes_committed ] checkout:(b)builds/pulled <-VcsCheckoutBuilder-- [ checkout:(b)builds/checked_out, checkout:(b)builds/up_to_date[T] ] checkout:(b)builds/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:(b)builds/checked_out ] deployment:everything/deployed <-CollectDeploymentBuilder-- [ package:*{x86}/postinstalled, package:(b)*{x86}/postinstalled ] package:pkg_cpio{x86}/built <-MakeBuilder-- [ package:pkg_cpio{x86}/configured ] package:pkg_cpio{x86}/configured <-MakeBuilder-- [ package:pkg_cpio{x86}/preconfig ] package:pkg_cpio{x86}/installed <-MakeBuilder-- [ package:pkg_cpio{x86}/built ] package:pkg_cpio{x86}/postinstalled <-MakeBuilder-- [ package:pkg_cpio{x86}/installed ] package:pkg_cpio{x86}/preconfig <-MakeBuilder-- [ checkout:cpio_co/checked_out ] package:(b)pkg_b{x86}/built <-MakeBuilder-- [ package:(b)pkg_b{x86}/configured ] package:(b)pkg_b{x86}/configured <-MakeBuilder-- [ package:(b)pkg_b{x86}/preconfig ] package:(b)pkg_b{x86}/installed <-MakeBuilder-- [ package:(b)pkg_b{x86}/built ] package:(b)pkg_b{x86}/postinstalled <-MakeBuilder-- [ package:(b)pkg_b{x86}/installed ] package:(b)pkg_b{x86}/preconfig <-MakeBuilder-- [ checkout:(b)b_co/checked_out ] -----
and our deployment has the expected build order:
$ muddle query deps deployment:everything/deployed Build order for deployment:everything/deployed .. checkout:cpio_co/checked_out checkout:(b)b_co/checked_out package:(b)pkg_b{x86}/preconfig package:pkg_cpio{x86}/preconfig package:(b)pkg_b{x86}/configured package:pkg_cpio{x86}/configured package:(b)pkg_b{x86}/built package:pkg_cpio{x86}/built package:(b)pkg_b{x86}/installed package:pkg_cpio{x86}/installed package:(b)pkg_b{x86}/postinstalled package:pkg_cpio{x86}/postinstalled deployment:everything/deployed
Which means that when we deploy:
$ muddle deploy Building deployment:everything/deployed > Building deployment:everything/deployed > Make directory /home/tibs/sw/m3/example3/deploy/everything Copying /home/tibs/sw/m3/example3/install/x86/ to /home/tibs/sw/m3/example3/deploy/everything/ Copying /home/tibs/sw/m3/example3/domains/b/install/x86/ to /home/tibs/sw/m3/example3/deploy/everything/usr > Make directory /home/tibs/sw/m3/example3/.muddle/tags/deployment/everything
we end up with everything deployed in the deploy/
directory at the
top-level:
deploy/ `-- everything/ |-- bin/ | `-- hello_world* |-- hello_world.c `-- usr/ `-- hello_world*
bin/hello_world
comes from the top-level build, and usr/hello_world
from the subdomain. The hello_world.c
is also from the top-level build
(the example Makefile for the cpio
package copies the source file into the
install/
directory).
Version stamps and domains¶
Version stamps work with domains as well, although one may only generate them for the top-level in a particular build tree.
Thus we can try:
$ muddle stamp version Finding all checkouts... found 4 Processing Svn checkout 'builds' builds: 'svnversion' reports checkout has revision '459M' Processing Svn checkout 'cpio_co' Processing Svn checkout '(b)b_co' Processing Svn checkout '(b)builds' Found domains: set([DomainTuple(name='b', repository='svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b', description='builds/01.py')]) Unable to work out revision ids for all the checkouts - although we did work out 3 of 4 Problems were: * builds: 'svnversion' reports checkout has revision '459M' Problems prevent writing version stamp file
OK, that makes sense, since we had indeed edited that checkout, and had not commited the changes. Luckily, it is sufficient for our purposes to save the state without that edit:
$ muddle stamp save -force Forcing original revision ids when necessary Finding all checkouts... found 4 Processing Svn checkout 'builds' builds: 'svnversion' reports checkout has revision '459M' Processing Svn checkout 'cpio_co' Processing Svn checkout '(b)b_co' Processing Svn checkout '(b)builds' Found domains: set([DomainTuple(name='b', repository='svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b', description='builds/01.py')]) Unable to work out revision ids for all the checkouts - although we did work out 3 of 4 Problems were: * builds: 'svnversion' reports checkout has revision '459M' Writing to working.stamp Wrote revision data to working.stamp File has SHA1 hash 6425284f002c81c9849f2f23add025a710207509 Renaming working.stamp to 6425284f002c81c9849f2f23add025a710207509.partial
and that at least shows us that the stamp file does record details of any subdomains:
[ROOT] description = builds/01.py repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio [DOMAIN b] description = builds/01.py name = b repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b [CHECKOUT (b)b_co] domain = b name = b_co relative = b_co repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b revision = 459 [CHECKOUT (b)builds] domain = b name = builds relative = builds repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b revision = 459 [CHECKOUT cpio_co] name = cpio_co relative = cpio_co repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio revision = 459 [PROBLEMS] problem1 = builds: 'svnversion' reports checkout has revision '459M'