mozbuild.frontend package

Submodules

mozbuild.frontend.context module

This module contains the data structure (context) holding the configuration from a moz.build. The data emitted by the frontend derives from those contexts.

It also defines the set of variables and functions available in moz.build. If you are looking for the absolute authority on what moz.build files can contain, you’ve come to the right place.

class mozbuild.frontend.context.AbsolutePath(context, value=None)

Bases: mozbuild.frontend.context.Path

Like Path, but allows arbitrary paths outside the source and object directories.

class mozbuild.frontend.context.Context(allowed_variables={}, config=None, finder=None)

Bases: mozbuild.util.KeyedDefaultDict

Represents a moz.build configuration context.

Instances of this class are filled by the execution of sandboxes. At the core, a Context is a dict, with a defined set of possible keys we’ll call variables. Each variable is associated with a type.

When reading a value for a given key, we first try to read the existing value. If a value is not found and it is defined in the allowed variables set, we return a new instance of the class for that variable. We don’t assign default instances until they are accessed because this makes debugging the end-result much simpler. Instead of a data structure with lots of empty/default values, you have a data structure with only the values that were read or touched.

Instances of variables classes are created by invoking class_name(), except when class_name derives from ContextDerivedValue or SubContext, in which case class_name(instance_of_the_context) or class_name(self) is invoked. A value is added to those calls when instances are created during assignment (setitem).

allowed_variables is a dict of the variables that can be set and read in this context instance. Keys in this dict are the strings representing keys in this context which are valid. Values are tuples of stored type, assigned type, default value, a docstring describing the purpose of the variable, and a tier indicator (see comment above the VARIABLES declaration in this module).

config is the ConfigEnvironment for this context.

add_source(path)

Adds the given path as source of the data from this context.

all_paths

Returns all paths ever added to the context.

error_is_fatal

Returns True if the error function should be fatal.

pop_source()

Get back to the previous current path for the context.

push_source(path)

Adds the given path as source of the data from this context and make it the current path for the context.

relsrcdir

source_stack

Returns the current stack of pushed sources.

srcdir

update(iterable={}, **kwargs)

Like dict.update(), but using the context’s setitem.

This function is transactional: if setitem fails for one of the values, the context is not updated at all.

mozbuild.frontend.context.ContextDerivedTypedHierarchicalStringList

Specialized HierarchicalStringList for use with ContextDerivedValue types.

mozbuild.frontend.context.ContextDerivedTypedList

Specialized TypedList for use with ContextDerivedValue types.

mozbuild.frontend.context.ContextDerivedTypedListWithItems

Specialized TypedList for use with ContextDerivedValue types.

mozbuild.frontend.context.ContextDerivedTypedRecord

Factory for objects with certain properties and dynamic type checks.

This API is extremely similar to the TypedNamedTuple API, except that properties may be mutated. This supports syntax like:

VARIABLE_NAME.property += [

‘item1’, ‘item2’,

]

class mozbuild.frontend.context.ContextDerivedValue

Bases: object

Classes deriving from this one receive a special treatment in a Context. See Context documentation.

mozbuild.frontend.context.DependentTestsEntry

alias of _TypedRecord

mozbuild.frontend.context.Enum(*values)

class mozbuild.frontend.context.Files(parent, pattern=None)

Bases: mozbuild.frontend.context.SubContext

Metadata attached to files.

It is common to want to annotate files with metadata, such as which Bugzilla component tracks issues with certain files. This sub-context is where we stick that metadata.

The argument to this sub-context is a file matching pattern that is applied against the host file’s directory. If the pattern matches a file whose info is currently being sought, the metadata attached to this instance will be applied to that file.

Patterns are collections of filename characters with / used as the directory separate (UNIX-style paths) and * and ** used to denote wildcard matching.

Patterns without the * character are literal matches and will match at most one entity.

Patterns with * or ** are wildcard matches. * matches files at least within a single directory. ** matches files across several directories.

foo.html

Will match only the foo.html file in the current directory.

*.jsm

Will match all .jsm files in the current directory.

**/*.cpp

Will match all .cpp files in this and all child directories.

foo/*.css

Will match all .css files in the foo/ directory.

bar/*

Will match all files in the bar/ directory and all of its children directories.

bar/**

This is equivalent to bar/* above.

bar/**/foo

Will match all foo files in the bar/ directory and all of its children directories.

The difference in behavior between * and ** is only evident if a pattern follows the * or **. A pattern ending with * is greedy. ** is needed when you need an additional pattern after the wildcard. e.g. **/foo.

VARIABLES = {u'BUG_COMPONENT': (<class 'mozbuild.util.TypedTuple'>, <type 'tuple'>, u"The bug component that tracks changes to these files.\n\n Values are a 2-tuple of unicode describing the Bugzilla product and\n component. e.g. ``('Core', 'Build Config')``.\n "), u'FINAL': (<type 'bool'>, <type 'bool'>, u'Mark variable assignments as finalized.\n\n During normal processing, values from newer Files contexts\n overwrite previously set values. Last write wins. This behavior is\n not always desired. ``FINAL`` provides a mechanism to prevent\n further updates to a variable.\n\n When ``FINAL`` is set, the value of all variables defined in this\n context are marked as frozen and all subsequent writes to them\n are ignored during metadata reading.\n\n See :ref:`mozbuild_files_metadata_finalizing` for more info.\n '), u'IMPACTED_TESTS': (<class 'mozbuild.frontend.context._TypedRecord'>, <type 'list'>, u"File patterns, tags, and flavors for tests relevant to these files.\n\n Maps source files to the tests potentially impacted by those files.\n Tests can be specified by file pattern, tag, or flavor.\n\n For example:\n\n with Files('runtests.py'):\n IMPACTED_TESTS.files += [\n '**',\n ]\n\n in testing/mochitest/moz.build will suggest that any of the tests\n under testing/mochitest may be impacted by a change to runtests.py.\n\n File patterns may be made relative to the topsrcdir with a leading\n '/', so\n\n with Files('httpd.js'):\n IMPACTED_TESTS.files += [\n '/testing/mochitest/tests/Harness_sanity/**',\n ]\n\n in netwerk/test/httpserver/moz.build will suggest that any change to httpd.js\n will be relevant to the mochitest sanity tests.\n\n Tags and flavors are sorted string lists (flavors are limited to valid\n values).\n\n For example:\n\n with Files('toolkit/devtools/*'):\n IMPACTED_TESTS.tags += [\n 'devtools',\n ]\n\n in the root moz.build would suggest that any test tagged 'devtools' would\n potentially be impacted by a change to a file under toolkit/devtools, and\n\n with Files('dom/base/nsGlobalWindow.cpp'):\n IMPACTED_TESTS.flavors += [\n 'mochitest',\n ]\n\n Would suggest that nsGlobalWindow.cpp is potentially relevant to\n any plain mochitest.\n ")}

static aggregate(files)

Given a mapping of path to Files, obtain aggregate results.

Consumers may want to extract useful information from a collection of Files describing paths. e.g. given the files info data for N paths, recommend a single bug component based on the most frequent one. This function provides logic for deriving aggregate knowledge from a collection of path File metadata.

Note: the intent of this function is to operate on the result of mozbuild.frontend.reader.BuildReader.files_info(). The mozbuild.frontend.context.Files() instances passed in are thus the “collapsed” (__iadd__``ed) results of all ``Files from all moz.build files relevant to a specific path, not individual Files instances from a single moz.build file.

asdict()

Return this instance as a dict with built-in data structures.

Call this to obtain an object suitable for serializing.

class mozbuild.frontend.context.FinalTargetValue

Bases: mozbuild.frontend.context.ContextDerivedValue, unicode

class mozbuild.frontend.context.InitializedDefines(context, value=None)

Bases: mozbuild.frontend.context.ContextDerivedValue, collections.OrderedDict

mozbuild.frontend.context.ManifestparserManifestList

alias of _OrderedListWithAction

class mozbuild.frontend.context.ObjDirPath(context, value=None)

Bases: mozbuild.frontend.context.Path

Like Path, but limited to paths in the object directory.

mozbuild.frontend.context.OrderedListWithAction(action)

Returns a class which behaves as a StrictOrderingOnAppendList, but invokes the given callable with each input and a context as it is read, storing a tuple including the result and the original item.

This used to extend moz.build reading to make more data available in filesystem-reading mode.

mozbuild.frontend.context.OrderedSourceList

alias of _TypedList

class mozbuild.frontend.context.Path(context, value=None)

Bases: mozbuild.frontend.context.ContextDerivedValue, unicode

Stores and resolves a source path relative to a given context

This class is used as a backing type for some of the sandbox variables. It expresses paths relative to a context. Supported paths are:

• ‘/topsrcdir/relative/paths’
• ‘srcdir/relative/paths’
• ‘!/topobjdir/relative/paths’
• ‘!objdir/relative/paths’
• ‘%/filesystem/absolute/paths’

join(*p)

ContextDerived equivalent of mozpath.join(self, *p), returning a new Path instance.

class mozbuild.frontend.context.PathMeta

Bases: type

Meta class for the Path family of classes.

It handles calling __new__ and __init__ with the right arguments in cases where a Path is instantiated with another instance of Path instead of having received a context.

It also makes Path(context, value) instantiate one of the subclasses depending on the value, allowing callers to do standard type checking (isinstance(path, ObjDirPath)) instead of checking the value itself (path.startswith(‘!’)).

mozbuild.frontend.context.ReftestManifestList

alias of _OrderedListWithAction

class mozbuild.frontend.context.RenamedSourcePath(context, value)

Bases: mozbuild.frontend.context.SourcePath

Like SourcePath, but with a different base name when installed.

The constructor takes a tuple of (source, target_basename).

This class is not meant to be exposed to moz.build sandboxes as of now, and is not supported by the RecursiveMake backend.

target_basename

class mozbuild.frontend.context.SourcePath(context, value)

Bases: mozbuild.frontend.context.Path

Like Path, but limited to paths in the source directory.

class mozbuild.frontend.context.SubContext(parent)

Bases: mozbuild.frontend.context.Context, mozbuild.frontend.context.ContextDerivedValue

A Context derived from another Context.

Sub-contexts are intended to be used as context managers.

Sub-contexts inherit paths and other relevant state from the parent context.

class mozbuild.frontend.context.TemplateContext(template=None, allowed_variables={}, config=None)

Bases: mozbuild.frontend.context.Context

mozbuild.frontend.context.TypedListWithAction(typ, action)

Returns a class which behaves as a TypedList with the provided type, but invokes the given given callable with each input and a context as it is read, storing a tuple including the result and the original item.

This used to extend moz.build reading to make more data available in filesystem-reading mode.

mozbuild.frontend.context.WptManifestList

alias of _TypedListWithAction

mozbuild.frontend.context.cls

alias of Files

mozbuild.frontend.data module

Data structures representing Mozilla’s source tree.

The frontend files are parsed into static data structures. These data structures are defined in this module.

All data structures of interest are children of the TreeMetadata class.

Logic for populating these data structures is not defined in this class. Instead, what we have here are dumb container classes. The emitter module contains the code for converting executed mozbuild files into these data structures.

class mozbuild.frontend.data.AndroidAssetsDirs(context, paths)

Bases: mozbuild.frontend.data.ContextDerived

Represents Android assets directories.

paths

class mozbuild.frontend.data.AndroidEclipseProjectData(name)

Bases: object

Represents an Android Eclipse project.

add_classpathentry(path, srcdir, dstdir, exclude_patterns=[], ignore_warnings=False)

assets

extra_jars

filtered_resources

included_projects

is_library

libs

manifest

name

package_name

recursive_make_targets

referenced_projects

res

class mozbuild.frontend.data.AndroidExtraPackages(context, packages)

Bases: mozbuild.frontend.data.ContextDerived

Represents Android extra packages.

packages

class mozbuild.frontend.data.AndroidExtraResDirs(context, paths)

Bases: mozbuild.frontend.data.ContextDerived

Represents Android extra resource directories.

Extra resources are resources provided by libraries and including in a packaged APK, but not otherwise redistributed. In practice, this means resources included in Fennec but not in GeckoView.

paths

class mozbuild.frontend.data.AndroidResDirs(context, paths)

Bases: mozbuild.frontend.data.ContextDerived

Represents Android resource directories.

paths

class mozbuild.frontend.data.BaseConfigSubstitution(context)

Bases: mozbuild.frontend.data.ContextDerived

Base class describing autogenerated files as part of config.status.

input_path

output_path

relpath

class mozbuild.frontend.data.BaseDefines(context, defines)

Bases: mozbuild.frontend.data.ContextDerived

Context derived container object for DEFINES/HOST_DEFINES, which are OrderedDicts.

defines

get_defines()

update(more_defines)

class mozbuild.frontend.data.BaseLibrary(context, basename)

Bases: mozbuild.frontend.data.Linkable

Generic context derived container object for libraries.

basename

import_name

lib_name

refs

class mozbuild.frontend.data.BaseProgram(context, program, is_unit_test=False)

Bases: mozbuild.frontend.data.Linkable

Context derived container object for programs, which is a unicode string.

This class handles automatically appending a binary suffix to the program name. If the suffix is not defined, the program name is unchanged. Otherwise, if the program name ends with the given suffix, it is unchanged Otherwise, the suffix is appended to the program name.

DICT_ATTRS = set([u'relobjdir', u'install_target', u'KIND', u'program'])

program

class mozbuild.frontend.data.BaseSources(context, files, canonical_suffix)

Bases: mozbuild.frontend.data.ContextDerived

Base class for files to be compiled during the build.

canonical_suffix

files

class mozbuild.frontend.data.BrandingFiles(sandbox, files)

Bases: mozbuild.frontend.data.FinalTargetFiles

Sandbox container object for BRANDING_FILES, which is a HierarchicalStringList.

We need an object derived from ContextDerived for use in the backend, so this object fills that role. It just has a reference to the underlying HierarchicalStringList, which is created when parsing BRANDING_FILES.

install_target

class mozbuild.frontend.data.ChromeManifestEntry(context, manifest_path, entry)

Bases: mozbuild.frontend.data.ContextDerived

Represents a chrome.manifest entry.

entry

path

class mozbuild.frontend.data.ClassPathEntry

Bases: object

Represents a classpathentry in an Android Eclipse project.

dstdir

exclude_patterns

ignore_warnings

path

srcdir

class mozbuild.frontend.data.ConfigFileSubstitution(context)

Bases: mozbuild.frontend.data.BaseConfigSubstitution

Describes a config file that will be generated using substitutions.

class mozbuild.frontend.data.ContextDerived(context)

Bases: mozbuild.frontend.data.TreeMetadata

Build object derived from a single Context instance.

It holds fields common to all context derived classes. This class is likely never instantiated directly but is instead derived from.

config

context_all_paths

context_main_path

defines

install_target

objdir

relativedir

relobjdir

srcdir

topobjdir

topsrcdir

class mozbuild.frontend.data.ContextWrapped(context, wrapped)

Bases: mozbuild.frontend.data.ContextDerived

Generic context derived container object for a wrapped rich object.

Use this wrapper class to shuttle a rich build system object completely defined in moz.build files through the tree metadata emitter to the build backend for processing as-is.

wrapped

class mozbuild.frontend.data.Defines(context, defines)

Bases: mozbuild.frontend.data.BaseDefines

class mozbuild.frontend.data.DirectoryTraversal(context)

Bases: mozbuild.frontend.data.ContextDerived

Describes how directory traversal for building should work.

This build object is likely only of interest to the recursive make backend. Other build backends should (ideally) not attempt to mimic the behavior of the recursive make backend. The only reason this exists is to support the existing recursive make backend while the transition to mozbuild frontend files is complete and we move to a more optimal build backend.

Fields in this class correspond to similarly named variables in the frontend files.

dirs

class mozbuild.frontend.data.ExampleWebIDLInterface(context, name)

Bases: mozbuild.frontend.data.ContextDerived

An individual WebIDL interface to generate.

name

class mozbuild.frontend.data.Exports(sandbox, files)

Bases: mozbuild.frontend.data.FinalTargetFiles

Context derived container object for EXPORTS, which is a HierarchicalStringList.

We need an object derived from ContextDerived for use in the backend, so this object fills that role. It just has a reference to the underlying HierarchicalStringList, which is created when parsing EXPORTS.

install_target

class mozbuild.frontend.data.ExternalLibrary

Bases: object

Empty mixin for libraries built by an external build system.

class mozbuild.frontend.data.ExternalSharedLibrary(context, basename, real_name=None, is_sdk=False, soname=None, variant=None, symbols_file=False)

Bases: mozbuild.frontend.data.SharedLibrary, mozbuild.frontend.data.ExternalLibrary

Context derived container for shared libraries built by an external build system.

class mozbuild.frontend.data.ExternalStaticLibrary(context, basename, real_name=None, is_sdk=False, link_into=None, no_expand_lib=False)

Bases: mozbuild.frontend.data.StaticLibrary, mozbuild.frontend.data.ExternalLibrary

Context derived container for static libraries built by an external build system.

class mozbuild.frontend.data.FinalTargetFiles(sandbox, files)

Bases: mozbuild.frontend.data.ContextDerived

Sandbox container object for FINAL_TARGET_FILES, which is a HierarchicalStringList.

We need an object derived from ContextDerived for use in the backend, so this object fills that role. It just has a reference to the underlying HierarchicalStringList, which is created when parsing FINAL_TARGET_FILES.

files

class mozbuild.frontend.data.FinalTargetPreprocessedFiles(sandbox, files)

Bases: mozbuild.frontend.data.ContextDerived

Sandbox container object for FINAL_TARGET_PP_FILES, which is a HierarchicalStringList.

We need an object derived from ContextDerived for use in the backend, so this object fills that role. It just has a reference to the underlying HierarchicalStringList, which is created when parsing FINAL_TARGET_PP_FILES.

files

class mozbuild.frontend.data.GeneratedEventWebIDLFile(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual .webidl source file.

basename

class mozbuild.frontend.data.GeneratedFile(context, script, method, outputs, inputs, flags=())

Bases: mozbuild.frontend.data.ContextDerived

Represents a generated file.

flags

inputs

method

outputs

script

class mozbuild.frontend.data.GeneratedSources(context, files, canonical_suffix)

Bases: mozbuild.frontend.data.BaseSources

Represents generated files to be compiled during the build.

class mozbuild.frontend.data.GeneratedWebIDLFile(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual .webidl source file that is generated from build rules.

basename

class mozbuild.frontend.data.HostDefines(context, defines)

Bases: mozbuild.frontend.data.BaseDefines

class mozbuild.frontend.data.HostLibrary(context, basename)

Bases: mozbuild.frontend.data.HostMixin, mozbuild.frontend.data.BaseLibrary

Context derived container object for a host library

KIND = u'host'

class mozbuild.frontend.data.HostMixin

Bases: object

defines

class mozbuild.frontend.data.HostProgram(context, program, is_unit_test=False)

Bases: mozbuild.frontend.data.HostMixin, mozbuild.frontend.data.BaseProgram

Context derived container object for HOST_PROGRAM

KIND = u'host'

SUFFIX_VAR = u'HOST_BIN_SUFFIX'

class mozbuild.frontend.data.HostSimpleProgram(context, program, is_unit_test=False)

Bases: mozbuild.frontend.data.HostMixin, mozbuild.frontend.data.BaseProgram

Context derived container object for each program in HOST_SIMPLE_PROGRAMS

KIND = u'host'

SUFFIX_VAR = u'HOST_BIN_SUFFIX'

class mozbuild.frontend.data.HostSources(context, files, canonical_suffix)

Bases: mozbuild.frontend.data.HostMixin, mozbuild.frontend.data.BaseSources

Represents files to be compiled for the host during the build.

class mozbuild.frontend.data.IPDLFile(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual .ipdl source file.

basename

class mozbuild.frontend.data.InstallationTarget(context)

Bases: mozbuild.frontend.data.ContextDerived

Describes the rules that affect where files get installed to.

enabled

is_custom()

Returns whether or not the target is not derived from the default given xpiname and subdir.

subdir

target

xpiname

class mozbuild.frontend.data.JARManifest(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual JAR manifest file and how to process it.

This class isn’t very useful for optimizing backends yet because we don’t capture defines. We can’t capture defines safely until all of them are defined in moz.build and not Makefile.in files.

path

class mozbuild.frontend.data.JavaJarData(name, sources=[], generated_sources=[], extra_jars=[], javac_flags=[])

Bases: object

Represents a Java JAR file.

A Java JAR has the following members:

• sources - strictly ordered list of input java sources
• generated_sources - strictly ordered list of generated input java sources
• extra_jars - list of JAR file dependencies to include on the javac compiler classpath
• javac_flags - list containing extra flags passed to the javac compiler

extra_jars

generated_sources

javac_flags

name

sources

class mozbuild.frontend.data.Library(context, basename, real_name=None, is_sdk=False)

Bases: mozbuild.frontend.data.BaseLibrary

Context derived container object for a library

KIND = u'target'

is_sdk

class mozbuild.frontend.data.Linkable(context)

Bases: mozbuild.frontend.data.ContextDerived

Generic context derived container object for programs and libraries

lib_defines

link_library(obj)

link_system_library(lib)

linked_libraries

linked_system_libs

exception mozbuild.frontend.data.LinkageWrongKindError

Bases: exceptions.Exception

Error thrown when trying to link objects of the wrong kind

class mozbuild.frontend.data.LocalInclude(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual local include path.

path

class mozbuild.frontend.data.ObjdirFiles(sandbox, files)

Bases: mozbuild.frontend.data.ContextDerived

Sandbox container object for OBJDIR_FILES, which is a HierarchicalStringList.

files

install_target

class mozbuild.frontend.data.ObjdirPreprocessedFiles(sandbox, files)

Bases: mozbuild.frontend.data.ContextDerived

Sandbox container object for OBJDIR_PP_FILES, which is a HierarchicalStringList.

files

install_target

class mozbuild.frontend.data.PerSourceFlag(context, file_name, flags)

Bases: mozbuild.frontend.data.ContextDerived

Describes compiler flags specified for individual source files.

file_name

flags

class mozbuild.frontend.data.PreprocessedTestWebIDLFile(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual test-only .webidl source file that requires preprocessing.

basename

class mozbuild.frontend.data.PreprocessedWebIDLFile(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual .webidl source file that requires preprocessing.

basename

class mozbuild.frontend.data.Program(context, program, is_unit_test=False)

Bases: mozbuild.frontend.data.BaseProgram

Context derived container object for PROGRAM

KIND = u'target'

SUFFIX_VAR = u'BIN_SUFFIX'

class mozbuild.frontend.data.RustRlibLibrary(context, basename, crate_name, rlib_filename, link_into)

Bases: mozbuild.frontend.data.Library

Context derived container object for a Rust rlib

class mozbuild.frontend.data.SdkFiles(sandbox, files)

Bases: mozbuild.frontend.data.FinalTargetFiles

Sandbox container object for SDK_FILES, which is a HierarchicalStringList.

We need an object derived from ContextDerived for use in the backend, so this object fills that role. It just has a reference to the underlying HierarchicalStringList, which is created when parsing SDK_FILES.

install_target

class mozbuild.frontend.data.SharedLibrary(context, basename, real_name=None, is_sdk=False, soname=None, variant=None, symbols_file=False)

Bases: mozbuild.frontend.data.Library

Context derived container object for a shared library

COMPONENT = 2

DICT_ATTRS = set([u'install_target', u'soname', u'basename', u'relobjdir', u'lib_name', u'import_name'])

FRAMEWORK = 1

MAX_VARIANT = 3

soname

symbols_file

variant

class mozbuild.frontend.data.SimpleProgram(context, program, is_unit_test=False)

Bases: mozbuild.frontend.data.BaseProgram

Context derived container object for each program in SIMPLE_PROGRAMS

KIND = u'target'

SUFFIX_VAR = u'BIN_SUFFIX'

class mozbuild.frontend.data.Sources(context, files, canonical_suffix)

Bases: mozbuild.frontend.data.BaseSources

Represents files to be compiled during the build.

class mozbuild.frontend.data.StaticLibrary(context, basename, real_name=None, is_sdk=False, link_into=None, no_expand_lib=False)

Bases: mozbuild.frontend.data.Library

Context derived container object for a static library

link_into

no_expand_lib

class mozbuild.frontend.data.TestHarnessFiles(sandbox, files)

Bases: mozbuild.frontend.data.FinalTargetFiles

Sandbox container object for TEST_HARNESS_FILES, which is a HierarchicalStringList.

install_target

class mozbuild.frontend.data.TestManifest(context, path, manifest, flavor=None, install_prefix=None, relpath=None, dupe_manifest=False)

Bases: mozbuild.frontend.data.ContextDerived

Represents a manifest file containing information about tests.

default_support_files

deferred_installs

directory

dupe_manifest

external_installs

flavor

install_prefix

installs

manifest

manifest_obj_relpath

manifest_relpath

path

pattern_installs

tests

class mozbuild.frontend.data.TestWebIDLFile(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual test-only .webidl source file.

basename

class mozbuild.frontend.data.TreeMetadata

Bases: object

Base class for all data being captured.

to_dict()

class mozbuild.frontend.data.UnifiedSources(context, files, canonical_suffix, files_per_unified_file=16)

Bases: mozbuild.frontend.data.BaseSources

Represents files to be compiled in a unified fashion during the build.

have_unified_mapping

unified_source_mapping

class mozbuild.frontend.data.VariablePassthru(context)

Bases: mozbuild.frontend.data.ContextDerived

A dict of variables to pass through to backend.mk unaltered.

The purpose of this object is to facilitate rapid transitioning of variables from Makefile.in to moz.build. In the ideal world, this class does not exist and every variable has a richer class representing it. As long as we rely on this class, we lose the ability to have flexibility in our build backends since we will continue to be tied to our rules.mk.

variables

class mozbuild.frontend.data.WebIDLFile(context, path)

Bases: mozbuild.frontend.data.ContextDerived

Describes an individual .webidl source file.

basename

class mozbuild.frontend.data.XPIDLFile(context, source, module, add_to_manifest)

Bases: mozbuild.frontend.data.ContextDerived

Describes an XPIDL file to be compiled.

add_to_manifest

basename

module

source_path

mozbuild.frontend.emitter module

class mozbuild.frontend.emitter.TreeMetadataEmitter(config)

Bases: mach.mixin.logging.LoggingMixin

Converts the executed mozbuild files into data structures.

This is a bridge between reader.py and data.py. It takes what was read by reader.BuildReader and converts it into the classes defined in the data module.

LIBRARY_NAME_VAR = {u'host': u'HOST_LIBRARY_NAME', u'target': u'LIBRARY_NAME'}

emit(output)

Convert the BuildReader output into data structures.

The return value from BuildReader.read_topsrcdir() (a generator) is typically fed into this function.

emit_from_context(context)

Convert a Context to tree metadata objects.

This is a generator of mozbuild.frontend.data.ContextDerived instances.

summary()

mozbuild.frontend.gyp_reader module

class mozbuild.frontend.gyp_reader.GypContext(config, relobjdir)

Bases: mozbuild.frontend.context.TemplateContext

Specialized Context for use with data extracted from Gyp.

config is the ConfigEnvironment for this context. relobjdir is the object directory that will be used for this context, relative to the topobjdir defined in the ConfigEnvironment.

mozbuild.frontend.gyp_reader.encode(value)

mozbuild.frontend.gyp_reader.read_from_gyp(config, path, output, vars, non_unified_sources=set([]))

Read a gyp configuration and emits GypContexts for the backend to process.

config is a ConfigEnvironment, path is the path to a root gyp configuration file, output is the base path under which the objdir for the various gyp dependencies will be, and vars a dict of variables to pass to the gyp processor.

mozbuild.frontend.mach_commands module

mozbuild.frontend.reader module

Read build frontend files into data structures.

In terms of code architecture, the main interface is BuildReader. BuildReader starts with a root mozbuild file. It creates a new execution environment for this file, which is represented by the Sandbox class. The Sandbox class is used to fill a Context, representing the output of an individual mozbuild file. The

The BuildReader contains basic logic for traversing a tree of mozbuild files. It does this by examining specific variables populated during execution.

class mozbuild.frontend.reader.BuildReader(config, finder=<mozpack.files.FileFinder object>)

Bases: object

Read a tree of mozbuild files into data structures.

This is where the build system starts. You give it a tree configuration (the output of configuration) and it executes the moz.build files and collects the data they define.

The reader can optionally call a callable after each sandbox is evaluated but before its evaluated content is processed. This gives callers the opportunity to modify contexts before side-effects occur from their content. This callback receives the Context containing the result of each sandbox evaluation. Its return value is ignored.

all_mozbuild_paths()

Iterator over all available moz.build files.

This method has little to do with the reader. It should arguably belong elsewhere.

files_info(paths)

Obtain aggregate data from Files for a set of files.

Given a set of input paths, determine which moz.build files may define metadata for them, evaluate those moz.build files, and apply file metadata rules defined within to determine metadata values for each file requested.

Essentially, for each input path:

1. Determine the set of moz.build files relevant to that file by looking for moz.build files in ancestor directories.
2. Evaluate moz.build files starting with the most distant.
3. Iterate over Files sub-contexts.
4. If the file pattern matches the file we’re seeking info on, apply attribute updates.
5. Return the most recent value of attributes.

find_sphinx_variables()

This function finds all assignments of Sphinx documentation variables.

This is a generator of tuples of (moz.build path, var, key, value). For variables that assign to keys in objects, key will be defined.

With a little work, this function could be made more generic. But if we end up writing a lot of ast code, it might be best to import a high-level AST manipulation library into the tree.

read_mozbuild(path, config, descend=True, metadata={})

Read and process a mozbuild file, descending into children.

This starts with a single mozbuild file, executes it, and descends into other referenced files per our traversal logic.

The traversal logic is to iterate over the *DIRS variables, treating each element as a relative directory path. For each encountered directory, we will open the moz.build file located in that directory in a new Sandbox and process it.

If descend is True (the default), we will descend into child directories and files per variable values.

Arbitrary metadata in the form of a dict can be passed into this function. This feature is intended to facilitate the build reader injecting state and annotations into moz.build files that is independent of the sandbox’s execution context.

Traversal is performed depth first (for no particular reason).

read_relevant_mozbuilds(paths)

Read and process moz.build files relevant for a set of paths.

For an iterable of relative-to-root filesystem paths paths, find all moz.build files that may apply to them based on filesystem hierarchy and read those moz.build files.

The return value is a 2-tuple. The first item is a dict mapping each input filesystem path to a list of Context instances that are relevant to that path. The second item is a list of all Context instances. Each Context instance is in both data structures.

read_topsrcdir()

Read the tree of linked moz.build files.

This starts with the tree’s top-most moz.build file and descends into all linked moz.build files until all relevant files have been evaluated.

This is a generator of Context instances. As each moz.build file is read, a new Context is created and emitted.

summary()

test_defaults_for_path(ctxs)

exception mozbuild.frontend.reader.BuildReaderError(file_stack, trace, sandbox_exec_error=None, sandbox_load_error=None, validation_error=None, other_error=None, sandbox_called_error=None)

Bases: exceptions.Exception

Represents errors encountered during BuildReader execution.

The main purpose of this class is to facilitate user-actionable error messages. Execution errors should say:

• Why they failed
• Where they failed
• What can be done to prevent the error

A lot of the code in this class should arguably be inside sandbox.py. However, extraction is somewhat difficult given the additions MozbuildSandbox has over Sandbox (e.g. the concept of included files - which affect error messages, of course).

actual_file

main_file

sandbox_error

class mozbuild.frontend.reader.EmptyConfig(topsrcdir)

Bases: object

A config object that is empty.

This config object is suitable for using with a BuildReader on a vanilla checkout, without any existing configuration. The config is simply bootstrapped from a top source directory path.

class PopulateOnGetDict(default_factory, *args, **kwargs)

Bases: mozbuild.util.ReadOnlyDefaultDict

A variation on ReadOnlyDefaultDict that populates during .get().

This variation is needed because CONFIG uses .get() to access members. Without it, None (instead of our EmptyValue types) would be returned.

get(key, default=None)

class mozbuild.frontend.reader.MozbuildSandbox(context, metadata={}, finder=<mozpack.files.FileFinder object>)

Bases: mozbuild.frontend.sandbox.Sandbox

Implementation of a Sandbox tailored for mozbuild files.

We expose a few useful functions and expose the set of variables defining Mozilla’s build system.

context is a Context instance.

metadata is a dict of metadata that can be used during the sandbox evaluation.

add_android_eclipse_project_helper(name)

Add an Android Eclipse project target.

exec_file(path)

Override exec_file to normalize paths and restrict file loading.

Paths will be rejected if they do not fall under topsrcdir or one of the external roots.

recompute_exports()

Recompute the variables to export to subdirectories with the current values in the subdirectory.

exception mozbuild.frontend.reader.SandboxCalledError(file_stack, message)

Bases: mozbuild.frontend.sandbox.SandboxError

Represents an error resulting from calling the error() function.

exception mozbuild.frontend.reader.SandboxValidationError(message, context)

Bases: exceptions.Exception

Represents an error encountered when validating sandbox results.

class mozbuild.frontend.reader.TemplateFunction(func, sandbox)

Bases: object

class RewriteName(sandbox, global_name)

Bases: ast.NodeTransformer

AST Node Transformer to rewrite variable accesses to go through a dict.

visit_Name(node)

visit_Str(node)

TemplateFunction.exec_in_sandbox(sandbox, *args, **kwargs)

Executes the template function in the given sandbox.

mozbuild.frontend.reader.is_read_allowed(path, config)

Whether we are allowed to load a mozbuild file at the specified path.

This is used as cheap security to ensure the build is isolated to known source directories.

We are allowed to read from the main source directory and any defined external source directories. The latter is to allow 3rd party applications to hook into our build system.

mozbuild.frontend.reader.log(logger, level, action, params, formatter)

mozbuild.frontend.sandbox module

Python sandbox implementation for build files.

This module contains classes for Python sandboxes that execute in a highly-controlled environment.

The main class is Sandbox. This provides an execution environment for Python code and is used to fill a Context instance for the takeaway information from the execution.

Code in this module takes a different approach to exception handling compared to what you’d see elsewhere in Python. Arguments to built-in exceptions like KeyError are machine parseable. This machine-friendly data is used to present user-friendly error messages in the case of errors.

class mozbuild.frontend.sandbox.Sandbox(context, builtins=None, finder=<mozpack.files.FileFinder object>)

Bases: dict

Represents a sandbox for executing Python code.

This class provides a sandbox for execution of a single mozbuild frontend file. The results of that execution is stored in the Context instance given as the context argument.

Sandbox is effectively a glorified wrapper around compile() + exec(). You point it at some Python code and it executes it. The main difference from executing Python code like normal is that the executed code is very limited in what it can do: the sandbox only exposes a very limited set of Python functionality. Only specific types and functions are available. This prevents executed code from doing things like import modules, open files, etc.

Sandbox instances act as global namespace for the sandboxed execution itself. They shall not be used to access the results of the execution. Those results are available in the given Context instance after execution.

The Sandbox itself is responsible for enforcing rules such as forbidding reassignment of variables.

Implementation note: Sandbox derives from dict because exec() insists that what it is given for namespaces is a dict.

BUILTINS = {u'int': <type 'int'>, u'False': False, u'None': None, u'True': True, u'sorted': <function alphabetical_sorted at 0x7ff5048c5b90>}

exec_file(path)

Execute code at a path in the sandbox.

The path must be absolute.

exec_function(func, args=(), kwargs={}, path=u'', becomes_current_path=True)

Execute function with the given arguments in the sandbox.

exec_source(source, path=u'')

Execute Python code within a string.

The passed string should contain Python code to be executed. The string will be compiled and executed.

You should almost always go through exec_file() because exec_source() does not perform extra path normalization. This can cause relative paths to behave weirdly.

get(key, default=None)

pop_subcontext(context)

Pop a SubContext off the execution stack.

SubContexts must be pushed and popped in opposite order. This is validated as part of the function call to ensure proper consumer API use.

push_subcontext(context)

Push a SubContext onto the execution stack.

When called, the active context will be set to the specified context, meaning all variable accesses will go through it. We also record this SubContext as having been executed as part of this sandbox.

exception mozbuild.frontend.sandbox.SandboxError(file_stack)

Bases: exceptions.Exception

exception mozbuild.frontend.sandbox.SandboxExecutionError(file_stack, exc_type, exc_value, trace)

Bases: mozbuild.frontend.sandbox.SandboxError

Represents errors encountered during execution of a Sandbox.

This is a simple container exception. It’s purpose is to capture state so something else can report on it.

exception mozbuild.frontend.sandbox.SandboxLoadError(file_stack, trace, illegal_path=None, read_error=None)

Bases: mozbuild.frontend.sandbox.SandboxError

Represents errors encountered when loading a file for execution.

This exception represents errors in a Sandbox that occurred as part of loading a file. The error could have occurred in the course of executing a file. If so, the file_stack will be non-empty and the file that caused the load will be on top of the stack.

mozbuild.frontend.sandbox.alphabetical_sorted(iterable, cmp=None, key=<function <lambda>>, reverse=False)

sorted() replacement for the sandbox, ordering alphabetically by default.

Module contents