The idea
TL;DR FB learns the successor representation (SR) through a low-rank bottleneck, but continuous dynamics are high-rank. Repeating actions acts as a low-pass filter: slow, room-scale modes survive, fast ones die out, and the SR becomes low-rank enough for FB to learn it stably.
Overview
Three ways to a low-rank SR
Three ways to make the SR low-rank: force a low rank, raise $\gamma$, or repeat actions. With exact dynamics all three work. When FB has to learn the SR in continuous space, only temporal abstraction gives a correct, long-horizon Q-function.
- Low-rank: a small bump at ★, then flat and patchy. No value reaches the other rooms.
- High $\gamma$: dark almost everywhere, with false peaks in the far rooms. Nothing points back to ★.
- Temporal abstraction: one smooth slope from ★ out through the doorways.
Abstract
Forward–backward (FB) representations provide a powerful framework for learning the successor representation (SR) in continuous spaces by enforcing a low-rank factorization. However, a fundamental spectral mismatch often exists between the high-rank transition dynamics of continuous environments and the low-rank bottleneck of the FB architecture, making accurate low-rank representation learning difficult. In this work, we analyze temporal abstraction as a mechanism to mitigate this mismatch. By characterizing the spectral properties of the transition operator, we show that temporal abstraction acts analogously to a low-pass filter that suppresses high-frequency spectral components. This suppression reduces the effective rank of the induced SR while preserving a formal bound on the resulting value function error. Empirically, we show that this alignment is a key factor for stable FB learning, particularly at high discount factors where bootstrapping becomes error-prone. Our results identify temporal abstraction as a principled mechanism for shaping the spectral structure of the underlying MDP and enabling effective long-horizon representations in continuous control.
1 Effect
How temporal abstraction reshapes Q and the SR
Left: the exact solution in the four-rooms grid world. Right: FB learned in the continuous version. Both use $\gamma = 0.95$. Click or drag on a map to move ★.
Loading data…
Look for: at $k = 1$ the learned Q (right) is patchy and doesn't match the exact one (left). Raise $k$: value spreads over the goal room and through the doorways, and the two maps start to agree.
Success rate of the learned agent
Same FB agent; only $k$ changes. Goal-reaching success over 50 episodes. Click a bar to jump to that $k$.
Why does repeating actions help? ↓
2 Mechanism
Temporal abstraction as a low-pass filter
Repeating an action $k$ times turns $P$ into $P^k$: same eigenvectors, eigenvalues $\lambda \to \lambda^k$. Slow, room-scale modes ($\lambda \approx 1$) survive; fast ones die out. Shown for the four-rooms grid world.
Look for: slide $k$. The grey dots ($k = 1$) spread over many modes; the blue dots collapse onto $\mu = 1$ except a few. Everything right of the FB rank line is what FB can't hold; the readout below shows how much of the SR that is. Hover a dot to see its mode on the grid.
Loading data…
Cite
Citation
@inproceedings{azad2026spectral,title = {Spectral Alignment in Forward-Backward Representations via Temporal Abstraction},author = {Azad, Seyed Mahdi B and Hoffmann, Jasper and Nematollahi, Iman and Zhu, Hao and Valada, Abhinav and Boedecker, Joschka},booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},year = {2026},}