modelTest_ReturnSequences

Information

Test_ReturnSequences – 2-Output Linear Integrator RNN Example

This package demonstrates and validates the behavior of the returnSequences functionality for a recurrent neural network with two outputs.

The model SMArtInt.Internal.Utilities.SubModels.Tests.Test_ReturnSequences implements a simple linear SimpleRNN without bias. Each output channel has its own recurrent weight, allowing the two outputs to evolve independently.

The recurrence relation for each output is defined as:

h_i(t) = h_i(t-1) + W_i · x_i(t),   i = 1, 2

Consequently, each output acts as an independent discrete-time integrator, accumulating its respective input over time, scaled by its recurrent weight W_i.

Purpose of the Model

The main objective of this test model is to verify the functional difference between:

• returnSequences = true – The network outputs the full output sequence for both outputs.
• returnSequences = false – The network outputs only the final values for both outputs.

Because the network is linear and has two outputs, the expected behavior can be computed analytically. For constant input, each output grows linearly, at a rate determined by its recurrent weight W_i.

Test Scenario

A constant input signal is applied to both outputs over multiple time steps. In this test configuration, the two output channels use different recurrent weights:

• Output 1: W = 1
• Output 2: W = 2

The hidden states therefore evolve according to:

h_i(t) = h_i(t-1) + W_i · x_i(t),   i = 1, 2

With constant input, both outputs behave as linear integrators but grow at different rates depending on their respective weights.

For example, if x_i(t) = 1 for all time steps, the output sequences become:

Output 1 (W = 1): 1, 2, 3, …, 10
Output 2 (W = 2): 2, 4, 6, …, 20

This highlights that although both outputs follow the same integration principle, their growth differs due to their different recurrent weights.

Key Variables for Analysis

Sequence Output Variant (returnSequences = true)

• Network input (SeqTrue.runInterferenceHistory.flattenedHistory[1].inputFlattenTensor): Flattened test signal for both outputs.
• Network output (SeqTrue.y): Final values for both outputs.
• Network sequence output (SeqTrue.ySequences): Full output sequences for both outputs.

Final-State Variant (returnSequences = false)

• Network input (SeqFalse.runInterferenceHistory.flattenedHistory[1].inputFlattenTensor): Flattened test signal for both outputs.
• Network output (SeqFalse.y): Final value for both outputs.

When simulated, the most important variables are plotted automatically.

The final outputs of the returnSequences = false configuration must match the last values of the sequences from the returnSequences = true configuration for both outputs.

Expected Behavior

• Linear growth for each output when input is constant
• No decay or forgetting (no bias, linear update equation)
• Different growth rates based on W = 1 and W = 2
• Identical final values between the two configurations
• Independent but consistent accumulation for the two outputs

This model serves as a minimal, analytically verifiable benchmark for validating sequence handling and multi-output behavior in RNNs within the SMArtInt framework.

Components