quantdata.torch.ffw#

Fast Forward Network

class quantdata.torch.ffw.FeedForwardConfig(activation='Softplus', num_depth=0, num_width=16, use_bias=True)[source]#

Bases: object

Configuration for FeedForward. This is a separate class so it can be imported without importing torch.

The default is a one-layer linear network with a width of 16.

activation: str = 'Softplus'#
num_depth: int = 0#
num_width: int = 16#
use_bias: bool = True#
get_activation()[source]#

Retrieves the requested activation function

static linear_config(activation='Identity')[source]#

Returns a config for a simple one-layer linear network with the specified activation.

class quantdata.torch.ffw.FeedForward(env, num_input, num_output, config=FeedForwardConfig(activation='Softplus', num_depth=0, num_width=16, use_bias=True), *, use_final_bias=True, use_final_activation=False)[source]#

Bases: BaseModel

Dense network applies to list of inputs (instead of a stack).

Essentially this function does:

y: input as list, implicitly y=stack(y,axis=-1)

x = L0( y )
for d in num_depth:
    x = activation( x )
    x = Ld( x )
if use_final_activation:
    x = activation( x )
x = mask & x

where Li is a set of linear layers applied to each element in the list.

(c)copyright 2019, Hans Buehler

Parameters:
  • env (BaseEnvironment) – Environment of the model which includes the random generator, default floating point dtype, and whether to run invasive debugging.

  • num_input (int) – Number of inputs, which will be expected as list of the form [ (A,...,B) ]. Numerically equivalent to a stack of dimension (A,....B,num_input).

  • num_output (int | tuple[int]) – Last dimension of output ie output is (A,...,B,num_output). Set to zero to return (A,...,B). Accepts a tuple for multi-dimensional output, see torch.nn.Linear.

  • config (FeedForwardConfig) – User config of the network.

  • use_final_bias (bool) – whether the final linear layer adds a bias.

  • use_final_activation (bool) – Whether to apply activation after the final linear layer.

Config#

alias of FeedForwardConfig

forward(input, mask=None)[source]#

Compute masked fast forward network.

Parameters:
  • input (list[Tensor] | Tensor) – Either a list with num_input elements (A,...,B), or a torch.Tensor of dimension (A,...,B,num_input).

  • mask (Tensor | None) – A mask of shape (A,...,B) to apply to the input, or None. If provided, the mask is applied to each input element as input = torch.where(mask[...,None], input, 0). This can be used to zero out certain inputs without changing the shape of the output.