Dataflow Soundness Problems with the Observable Runtime

The promise of reactivity in Observable Notebooks is that you don't need to worry about executing the cells, they recompute themselves intelligently when code changes, minimizing state churn and letting the user focus on the code. In this brief report we describe the invariants this implies: Determinism, Incremental Computation, and the situations in which the Observable Runtime breaks them.

While Observable fails complete soundness, we find that islands of soundness can be formed, and breaking some rules enables common patterns for application development. We conclude that repairing the Observable Runtime is probably not possible with a JavaScript based language, but that is a pragmatic choice for ecosystem compatibility. There is a door of opportunity open for a sound core interfacing with an unsound imperative shell, still within the Observable Runtime programming model.

Tom Larkworthy · Winter Substrates 2026, London · 2026-11-09

Background

Observable Notebooks are a web coding platform with an innovative MIT licensed reactive runtime applied to coding. Several projects have built upon its runtime [Larkworthy 2025] [Observable 2025] [Observable 2024] [Posit 2022]. The reactive runtime consists of reactive variables on a DAG that recompute when an ancestor changes, allowing fluid partial recomputation that minimizes state churn when compared to restarting a program.

The programming model is related to spreadsheet programming, functional reactive programming [Elliott & Hudak 1997] and live programming [Tanimoto 2013]. One specific claim is that you don't need to worry about rerunning cells manually like in a Jupyter notebook, nor be concerned about the order you run them in. The cells recompute in topological order and therefore remove that cognitive overhead from the programmer.

In this report we look at whether the claim holds up in general, and show several areas where it breaks down.

Invariants we want

Two invariants for reactive programming:

  • (D) Deterministic dataflow. The cells converge to the same values given the same inputs.
  • (I) Incremental computation. After an edit, recomputing a subset of cells leaves every value equal to what it would be if the dataflow had been executed from scratch [Acar 2005].

What they would buy:

  • Memoization: values could be cached to save computation.
  • Concurrency: parts that share nothing can run in any order, or elsewhere.

The Observable runtime gives neither (D) nor (I) for the programs we write.

Definitions

A simple model of dataflow is a graph G between cells V connected by dependency edges E.

G=(V,E)G = (V, E), with an edge u→vu \to v when uu is an input of vv.

Invariant 0: the computation graph is a DAG. This is enforced by the runtime, which rejects cycles with a RuntimeError("circular definition") (runtime.js:147-152 in @observablehq/runtime@6.0.0).

Cells are named functions of their declared inputs, holding a current value: C=(name,definition,value)C = (\mathit{name}, \mathit{definition}, \mathit{value}).

Theorem 0: the scheduler is sound when cells are hermetic

A cell is hermetic when its value is a pure function of its declared inputs. Ancestors, anc⁡(v)\operatorname{anc}(v) are the transitive inputs of the cell vv, the support is supp⁡(v)=anc⁡(v)∪{v}\operatorname{supp}(v) = \operatorname{anc}(v) \cup \{v\}, and desc⁡(c)\operatorname{desc}(c) are the downstream cells (descendants).

  • Lemma 1. If every cell is hermetic and i∉supp⁡(v)i \notin \operatorname{supp}(v), redefining ii cannot change vv.
  • Theorem 0 (scheduling). Redefining cc recomputes exactly ({c}∪desc⁡(c))∩reachable(\{c\} \cup \operatorname{desc}(c)) \cap \mathit{reachable}.
  • Together: in a hermetic program, a cell that is not recomputed could not have changed, so the scheduler alone gives (I).
  • Safe, but not minimal [Mokhov et al. 2018]: some unchanged values still recompute.
Theorem 0: an edit reruns its descendants3 / 4inputcomputedtest r = 1→ 1c = r + 2→ 3k = r * 2→ 2e = c * 10→ 30t2 = c == 2→ falset3 = k == 2→ truet1 = e == 20→ truethe runtime recomputes desc(c)…

Problem 1: action at a distance

A cell can change a value that is not downstream of it by writing into a shared reference. For example, a parent initialises list = [], a test checks isEmpty(list), and another cell does list.push("p").

Observable propagates only downstream. When the writer is edited, readers elsewhere keep values a fresh run would not produce, so property (I) is lost.

Rerunning the reader is not enough either: the old write remains in the shared object.

In graph terms the write is an edge that is not in GG. We call those hidden edges HH. A mutation of an input points back up the graph, so GG is a DAG and G∪HG \cup H is not.

Problem 1: the pusher runs before the test4 / 5holds a mutable objectwrites an inputtest l = []→ ["p"]push = l.length→ 1test = l.length == 0→ falsererun only the test: the old "p" is still in l → still false

Problem 2: the schedule causes non-determinism

Problem 1 assumed the pusher runs first. A parent initialises list = []. A test checks isEmpty(list). Another cell does list.push("p"). Should the test pass?

The test and the list mutation are siblings, and the ordering of execution is not determined by the topological sort. Therefore the test may or may not pass depending on the order of execution. This kind of problem can be fixed by artificially forcing an execution order.

A1: a stable order can be induced (e.g. order by label). Under A1 a synchronous program has one result, so (D) holds. This is an issue in the current Runtime but trivially fixable. Assume A1 from here on.

The real Observable Runtime has this problem because siblings are queued in the order of their parent's Set of outputs, and the queue is a stack (queue.pop(), runtime.js:141). A redefined cell is added to the end of the Set and so runs first, but on page reload cells are added in the order they are displayed on the page, so siblings run in reverse page order.

Problem 2: the schedule decides the answer3 / 3holds a mutable objectwrites an inputtesttest runs firstpush runs first l = []→ ["p"]test = l.length == 0→ truepush = l.push("p")→ 1l = []→ ["p"]push = l.push("p")→ 1test = l.length == 0→ falsesame program, leaves swapped: test is true on the left and false on the right

The DOM as a source of hidden edges

The Observable Runtime is interoperable with DOM APIs, which is a key building block for custom UIs. But the DOM itself is also a side channel for hidden edges.

hidden edge 1 — the page is shared state3 / 4computedtestthe page / outside document = (the page)→ title "b"w1 = document.title = "b"→ "b"w2 = document.title→ "a"t = w2 == "a"→ truethe write lands in the page — no edge in G

The mutation ball algorithm

If we cannot stop cells writing into shared objects, can we at least work out what to rerun after an edit? Bayes-Ball [Shachter 1998] answers a similar question for belief networks: a ball moves through the graph, and a rule at each node says whether it passes, bounces or stops. We start the ball at the cell that just changed.

By default the ball always moves down declared edges (the Observable Runtime's existing invalidation algorithm). Our new algorithm moves up only through a shared mutable reference: a cell that mutates its input sends the ball up to the cell that created the object, and from there downward to every descendant cell.

To restore sound incremental computation (I), we rerun from the topmost shared cells the ball bounced through: the creators of the shared objects. Rerunning a creator reruns all of its children, and one of them may dirty a different shared object (08), so the lift repeats until nothing changes. In 08 one round fixed the first fixture but not the second; iterating until nothing changed matched a fresh run on both.

The algorithm was inspired by Bayes-Ball's (local rules, linear traversal, reachability as the computation) but the rules are different, so we call it the mutation ball.

the ball: down always, up only through a mutable reference3 / 4computedholds a mutable objectwrites an inputtest m = new Map()h = freeze({k: 1})t1 = m.size == 0b = m.set(h.k, h)t2 = h.k == 1t3 = b.size == 1up to the parents: the Map passes it, the frozen value blocks it
a change entering a cycle from the side (08)3 / 4inputcomputedholds a mutable objectwrites an input o = new Map()d = 1→ 1o2 = new Map()t = d * 100 + o.size→ 100x = o2.set(o, 1).size→ 2b = o.set(t, 1)y = (x, o2.size)→ 2one round: lift o → t fixed, but x reruns and writes o2 again

Determining shared edges is hard

The ball needs every cell labelled mutated and shared or not. Unfortunately, computing whether something is mutated requires following potentially aliased references, which cannot be decided statically in JavaScript. So the mutation ball algorithm remains a thought experiment that cannot be realized on the Observable Runtime in general.

Recording writes at run time, with a Proxy on each input, would give real labels for the run observed but it does not generalize to Map, Set, #private or DOM fields.

Hardened JavaScript (SES) [Agoric 2024] can prevent access to names outside a cell's scope (e.g. window) but does not help with mutation of references passed to the cell as dependencies.

So we see no practical general repair of (I) for Observable Runtime programs as they are written.

recording writes at run time1 / 4computedholds a mutable objectwrites an inputtest owner = []writer = owner.push(1)a = owner.lengthb = owner.at(0)t1 = writer == 1t2 = a == 1t3 = b == 1edit a, which only reads the owner's value

Hermetic by construction

Hermetic cells can be built by construction instead of detected through analysis.

Copying on every input (structuredClone) prevents back-edges, at the price of a copy per edge per recompute (and not all things can be copied). It is an intriguing choice as it is the algorithm used for message passing to concurrent WebWorker runtimes [WHATWG], which also lack DOM access and other sources of hidden edges. Using structuredClone between cells could both help prove hermeticity and unblock multi-core processing on the Observable Runtime.

Alternatively a different authoring language could be used with statically verifiable properties.

clone on the edge3 / 4computedholds a mutable objectwrites an inputtest owner = []writer = owner.push(1)→ 1 (its copy)a = owner.lengthb = owner.at(0)t1 = writer == 1t2 = a == 1t3 = b == 1the write lands in the writer's copy; the owner never sees it

Perhaps functional core and the imperative shell?

There is an idiom in functional programming: functional core, imperative shell [Bernhardt 2012] keeps effects in a thin outer layer in order to build effectful systems like databases whilst retaining pure functional inner code. In a dataflow graph we could perhaps opt into an immutable core and have the unsound DOM and networking machinery located in a disconnected region.

the shell at both ends, the core between3 / 4inputcomputedtestthe page / outside viewof x = Inputs.text()→ " ab "g = (s) => s.trim()data = fetch(url)f = (s) => g(s).lengtht1 = g(" a ") == "a"t2 = f(" ab ") == 2y = f(x)→ 2z = data.lengthview = html`${y} ${z}`shell: inputs, glue cells (y = f(x)) and rendering; rerun or reboot

Conclusion

We want two things from a reactive runtime: deterministic dataflow (D) and sound incremental computation (I). The Observable runtime gives neither. Cells write into shared objects, which reaches cells that are not downstream, and the runtime only propagates downstream (Problem 1). Siblings run in an order the graph does not fix, so the same program can give different answers (Problem 2) although a stable stored order (A1) can fix this.

A hypothetical mutation ball algorithm describes what a sound repair must rerun. Its rules are simple, but it needs to know which values are shared and mutated, which doesn't seem calculable when the runtime language is JavaScript. We see no practical general repair.

The unsoundness is not an accident. The hidden edges in our corpus are counters, stores, logs and DOM nodes: the imperative shell of real applications. What might be possible is an opt-in boundary, a functional core where hermeticity is known by construction. JavaScript's structuredClone seems promising, as that function can be used to pass values to an isolate like a WebWorker with hermeticity built into the environment.

References

  1. Larkworthy, T. (2025). Lopecode. GitHub repository.
  2. Observable, Inc. (2025). Notebook Kit. GitHub repository.
  3. Observable, Inc. (2024). Observable Framework. Documentation.
  4. Posit Software (2022). Observable JS in Quarto. Quarto documentation.
  5. Elliott, C. & Hudak, P. (1997). Functional Reactive Animation. ICFP.
  6. Tanimoto, S. L. (2013). A Perspective on the Evolution of Live Programming. LIVE workshop, ICSE.
  7. Acar, U. A. (2005). Self-Adjusting Computation. PhD thesis, Carnegie Mellon University.
  8. Mokhov, A., Mitchell, N. & Peyton Jones, S. (2018). Build Systems à la Carte. ICFP.
  9. Shachter, R. D. (1998). Bayes-Ball: The Rational Pastime (for Determining Irrelevance and Requisite Information in Belief Networks and Influence Diagrams). UAI.
  10. Agoric & the Endo contributors (2024). Hardened JavaScript. Documentation.
  11. WHATWG (2026). HTML Living Standard: Safe passing of structured data. Living standard.
  12. Bernhardt, G. (2012). Functional Core, Imperative Shell. Destroy All Software screencast.
cite = ƒ(key)
bibliography = Object {lopecode: Object, notebookkit: Object, framework: Object, quarto: Object, elliott1997frp: Object, tanimoto2013live: Object, acar2005: Object, mokhov2018: Object, shachter1998: Object, hardenedjs: Object, whatwgclone: Object, bernhardt2012: Object}
dag = ƒ(spec, invalidation)
test_dag_draws_every_node_and_edge = "3 boxes, 2 edges"