Utilities

class breze.arch.util.Model

Model class.

Intended as a base class for parameterized models providing a convenience method for compilation and a common interface.

We partition Theano variables for parametrized models in three groups. (1) The adaptable parameters, (2) external variables such as inputs and targets, the data (3) expressions composed out of the two, such as the prediction of a model or the loss resulting from those.

There are several “reserved” names for expressions.

• inpt: observations of a supervised or unsupervised model,
• target: desired outputs of a supervised model,
• loss: quantity to be optimized for fitting the parameters; might not refer to the criterion of interest, but instead to a regularzied objective.
• true_loss: Quantity of interest for the user, e.g. the loss without regularization or the empirical risk.

Overriding these names is possible in general, but is part of the interface and will lead to unexpected behaviour with functionality building upon this.

Lookup of variables and expressions is typically done in the following ways.

• as the variable/expression itself,
• as a string which is the attribute/key to look for in the ParameterSet

object/expression dictinary, - as a path along theese, e.g. the tuple ('foo', 'bar', 0) will identify .parameters.foo.bar[0] or .parameters['foo']['bar'][0] depending on the context.

Attributes

pars	(ParameterSet object) Holding the adaptable parameters of the object.
exprs	(dictionary) Containig the expressions. Out of convenience, the external variables are held in here as well.
updates	(dict) Containing update variables, e.g. due to the use of theano.scan.

Methods

function
var_exp_for_gpu

function(variables, exprs, mode=None, explicit_pars=False, givens=None, on_unused_input='raise', numpy_result=False)

Return a compiled function for the given exprs given variables.

Parameters:

variables : list of strings

Each string refers to an item in .exprs and is considered an input to the function.

exprs : (List of) Theano expression or string

Expressions for which to create the function. If a single expression is given, the function will return a single value; if a list is given, the result will be a tuple containing one element for each. An expression can either be a Theano expression or a string. In the latter case, the corresponding expression will be retrieved from .exprs.

mode : string or None, optional, default: None

Mode to use for compilation. Passed on to theano.function. See Theano documentation for details. If None, self.mode will be used.

explicit_pars: boolean, optional, default: False

If True, the first argument to the function is expected to be an array representing the adaptable parameters of the model.

givens : dictionary, optional, default: None

Dictionary of substitutions for compilation. Not passed on to theano.function, instead the expressions are cloned. See code for further details.

on_unused_input: string

Specifiy behaviour in case of unused inputs. Passed on to theano.function. See Theano documentation for details.

numpy_result : boolean, optional, default: False

If set to True, a numpy array is always returned, even if the computation is done on the GPU and a gnumpy array was more natural.

var_exp_for_gpu(variables, exprs, outputs=True)

Given variables and theano expressions built from these variables, return variables and expressions of the same form that are tailored towards GPU usage.

class breze.arch.util.ParameterSet

ParameterSet class.

This class provides functionality to group several Theano tensors of different sizes in a consecutive chunk of memory. The main aim of this is to allow a view on several tensors as a single long vector.

In the following, a (parameter) array refers to a concrete instantiation of a parameter variable (with concrete values) while a (parameter) tensor/variable refers to the symbolic Theano variable.

Initialization takes a variable amount of keyword arguments, where each has to be a single integer or a tuple of arbitrary length containing only integers. For each of the keyword argument keys a tensor of the shape given by the value will be created. The key is the identifier of that variable.

All symbolic variables can be accessed as attributes of the object, all concrete variables as keys. E.g. parameter_set.x references the symbolic variable, while parameter_set[‘x’] will give you the concrete array.

Attributes

n_pars	(integer) Total amount of parameters.
flat	(Theano vector) Flat one dimensional tensor containing all the different tensors flattened out. Symbolic pendant to data.
data	(array_like) Concrete array containig all the different arrays flattened out. Concrete pendant to flat.
views	(dict) All parameter arrays can be accessed by with their identifier as key in this dictionary.

Methods

alloc
declare
view

Nested Lists for Theano, etc.

breze.arch.util.flatten(nested)

Flatten nested tuples and/or lists into a flat list.

breze.arch.util.unflatten(tmpl, flat)

Nest the items in flat into the shape of tmpl.

breze.arch.util.theano_function_with_nested_exprs(variables, exprs, *args, **kwargs)

Creates and returns a theano.function that takes values for variables as arguments, where variables` may contain nested lists and/or tuples, and returns values for ``exprs, where again exprs may contain nested lists and/or tuples.

All other arguments are passed to theano.function without modification.

breze.arch.util.theano_expr_bfs(expr)

Generator function to walk a Theano expression graph in breadth first.

breze.arch.util.tell_deterministic(expr)

Return True iff no random number generator is in the expression graph.

GPU related utilities

breze.arch.util.cpu_tensor_to_gpu(tensor)

Given a tensor for the CPU return a tensor of the same type and name for the GPU.

breze.arch.util.cpu_tensor_to_gpu_nested(inpts, cache=None)

Given a list (of lists of...) CPU tensor variables return as list of the same types of corresponding GPU tensor varaibles.

Also return a dictionary containing all substitutions done. This can be provided to future calls to not make conversions multiple times.

breze.arch.util.cpu_expr_to_gpu(expr, unsafe=False)

Given a CPU expr return the same expression for the GPU.

If unsafe is set to True, subsequent function calls evaluating the expression might return arrays pointing at the same memory region.

breze.arch.util.cpu_expr_to_gpu_nested(inpts, unsafe=False)

Given a list (of lists of...) expressions, return expressions for the GPU.

If unsafe is set to True, subsequent function calls evaluating the expression might return arrays pointing at the same memory region.

breze.arch.util.garray_to_cudandarray_nested(lst)

breze.arch.util.gnumpy_func_wrap(f)

Wrap a function that accepts and returns CudaNdArrays to accept and return gnumpy arrays.

Other

breze.arch.util.get_named_variables(dct, name=True, overwrite=False, prefix='')

Return a dictionary with all the items from dct with only Theano variables/expressions.

If name is set to True, the variables will be named accordingly, however not be overwritten unless overwrite is True as well.

breze.arch.util.lookup(what, where, default=None)

Return where.what if what is a string, otherwise what. If not found return default.

breze.arch.util.lookup_some_key(what, where, default=None)

Given a list of keys what, return the first of those to which there is an item in where.

If nothing is found, return default.