AlgebraicJulia Tutorial

Applied Category Theory 2023

Owen Lynch

Topos Institute

Evan Patterson

Topos Institute

Outline

Tutorial in three parts:

  1. Category theory in Julia
  2. Dynamical systems and composition patterns
  3. Operadic composition of dynamical systems

Part 1: Category theory in Julia

Overview of part 1:

  1. Category theory in a dynamic language??!?!
  2. The Julia programming language
  3. Category theory in Julia

Category theory in a dynamic language

It’s not as cursed as you’d think.

– Owen Lynch, 2023

  • Curry-Howard correspondence makes a connection between typed lambda calculus and cartesian closed categories
  • This is not the only way of doing category theory on a computer
  • Many categorical things can be implemented without working in a faithful implementation of the typed lambda calculus:
    • Databases
    • Dynamical systems
    • Computer algebra

What is Julia?

Julia is a programming language that is…

  • Dynamic:

    Variables don’t have types, only values do.

  • Fast:

    Carefully written Julia is competitive with C or Fortran in performance.

  • Dependently typed:

    Although only values have types, the types of those values can depend on other values. For instance StaticVector{Float64, n} represents \(\mathbb{R}^n\).

  • Looks like math:

    \[ \lVert v \rVert^2 = v^T v \]

    magnitude(x::Vector) = x' * x

Why Julia?

Julia was built to solve the “two-language problem” in scientific computing.

  • Numerical algorithms written in C/C++/Fortran
  • High-level, dynamic interfaces written in Python/R

Interfacing the “two languages” is a complex endeavor beyond the reach of small projects

Julia is a single language that can achieve the speed of C/C++/Fortran while still having high-level interfaces a la Python/R.

How do you Julia? (1)

A simple implementation of the Euler algorithm for integrating a vector field

#   du : ℝⁿ -> ℝⁿ is the vectorfield
# init : ℝⁿ       is the initial state
#   dt : ℝ        is the time increment
#    m : ℕ        is the number of steps to run for
function euler(du::Function, init::Vector{Float64}, dt::Float64, m::Int)
  n = length(init)
  x = zeros(Float64, (m + 1, n))       # a matrix to store the path
  x[1,:] = init                        # initialize the first row
  for i in 1:m
    x[i+1,:] = x[i,:] + dt * du(x[i,:]) # write the next row
  end
  x                                    # return the path
end

Notable features:

  • Built-in numerical linear algebra
  • Auto-broadcasted arithmetic
  • Syntax familiar to a Python/MATLAB user

How do you Julia? (2)

# Simulate the trajectory of a 1kg cannonball in 2d
# initial position = (0,0)
# initial velocity = (5.0, 10.0)
# (d/dt)(x,v) = (v, (0, -9.8))
trajectory = euler(u -> [u[3], u[4], 0, -9.8], [0, 0, 5.0, 10.0], 0.1, 22)

plot(trajectory[:, 1], trajectory[:, 2])

Notable features:

  • Easy anonymous functions
  • Easy plotting (with package Plots.jl)

How do you Julia? (3)

A simple interpreter

struct CalcExpr{op}
  args::Vector{Union{CalcExpr, Float64}}
end

evaluate(n::Float64)      = n
evaluate(e::CalcExpr{:+}) = sum(evaluate.(e.args))
evaluate(e::CalcExpr{:*}) = prod(evaluate.(e.args))

evaluate(CalcExpr{:*}([2.0, 3.0, CalcExpr{:+}([1.0, 1.0])]))
12.0

Notable features:

  • Single-line function definition
  • First-class symbols (:blah, :*, :+)
  • The type of a struct can be parameterized by a value
  • Map f over xs with f.(xs)
  • Multiple dispatch allows multiple definitions of a function for different types

Where is Julia?

Downloads and documentation:

Julia community:

Category theory in Julia: FinSet

struct FinSet
  n::Int
end
struct FinFunction
  dom::FinSet
  codom::FinSet
  values::Vector{Int}
end

id(X::FinSet) = FinFunction(X, X, Int[i for i in 1:X.n])

compose(f::FinFunction, g::FinFunction) =
  FinFunction(f.dom, g.codom, Int[g.values[j] for j in f.values])

Category theory in Julia: FdVect

Simple version:

struct FdVectℝ
  dim::Int
end

id(V::FdVectℝ) = Float64[i == j ? 1.0 : 0.0 for i in 1:V.dim, j in 1:V.dim]

compose(M::Matrix{Float64}, N::Matrix{Float64}) = M * N


Fancier version with type parameter for scalar field:

struct FdVect{T<:Number}
  dim::Int
end

id(V::FdVect{T}) where T = T[i == j ? one(T) : zero(T)
                             for i in 1:V.dim, j in 1:V.dim]

compose(M::Matrix{T}, N::Matrix{T}) where T = M * N

Part 2: Dynamical systems and composition patterns

Compositional systems theory

In the categorical approach to systems theory, we separate:

  1. Interfaces for systems, specified by
    • a kind of monoidal category with extra structure, or
    • an operad
  2. Systems conforming to the interface, forming
    • a monoidal category of that kind, or
    • an algebra of the operad

Different kinds of systems can use the same kind of interface, enabling reuse in

  • mathematics
  • software

Composing dynamical systems

Dynamical systems can be composed in several different styles:

  1. As resource sharers:
    • Identifying shared states between systems
    • “Undirected” interface
  2. As machines:
    • Plugging outputs of one system into parameters of another
    • “Directed” interface
  3. As resource-sharing machines: combines both

We focus on resource sharers and their implementation in AlgebraicJulia.

What are resource sharers?

  • Continuous dynamical systems: systems of ODEs
  • Expose “resources” as their interface
  • Compose by “identifying” or “sharing” the values of resources
  • Derivative of a resource is the sum of the vectorfields from all systems using that resource

Category-theoretically, resource sharers will form

  • a hypergraph category, or equivalently
  • an algebra of the operad of undirected wiring diagrams

Resource sharers are dynamical systems that expose certain quantities that we call resources. They are composed by “sharing” resources. One might assume that “sharing” means “dividing between them”, however this is not the case. What it means for two machines to share a resource is that they both agree on the value of that resource.

For instance, if one machine represents the dynamics of a population of foxes and a population of rabbits, and another represents the dynamics of a population of hawks and a population of rabbits, then one composition of these machines might be a system which represents the dynamics of a population of hawks, a population of rabbits, and a population of foxes. The fox-rabbit system and the hawk-rabbit system agree on the overall population of rabbits.

However, both the fox-rabbit system and the hawk-rabbit system can both “tug” on the population of rabbits, by sending it down.

Interlude: computing with morphisms

  • When computing, as opposed to proving, with functions or other morphisms
    • we can’t treat them as black boxes
    • we need a finitary and introspectable representation
  • A few examples:
    • Functions between finite sets
    • Linear maps between finite dimensional vector spaces
  • AlgebraicJulia, unlike more formalized math libraries, focuses on concrete computations in categories
  • We contrast the two approaches as
    1. geometric: concerned with abstract spaces and the maps between them
    2. symbolic: concerned with symbolic representations of spaces and functions

One way of mathematically defining resource sharers would be to say that their interfaces were spaces. However, what does this actually mean on a computer? We could say that a space was a type. But in a dynamic language, this is not very helpful. Moreover, we really want some smooth structure on our spaces.

Our philosophy is that one should have a symbolic representation of a space. What symbolic representation? Well, one chooses a level of generality appropriate for the application. In this case, we mostly just care about \(\mathbb{R}^n\). Thus, denoting a space by a natural number or a finite set is an appropriate choice.

What then is a function between symbolic spaces \(\mathbb{R}^n\) and \(\mathbb{R}^m\)? One choice would be a Julia function that takes a length \(n\) vector to a length \(m\) vector. This is appropriate in some circumstances, but it is not introspectable, which makes some operations uncomputable. For instance, it is impossible to take a pushout or pullback of such functions.

A more introspectable, yet more restrictive choice would be matrices. These are much easier to compute with. However, for our purposes, an even simpler choice of maps works best: functions from \([m]\) to \([n]\).

Geometric definition of resource sharers

An interface for a geometric resource sharer is a space \(I\).

We think of the interface as specifying the resources that a dynamical system has available to share.


A geometric resource sharer on an interface \(I\) consists of

  • A space \(X\), the state space
  • A function \(\mathrm{expose} \colon X \to I\)
  • A vector field \(v \colon X \to TX\)

Symbolic definition of resource sharer

An interface for a symbolic resource sharer is a finite set \(I\).

This finite set symbolically represents the space \(\mathbb{R}^I\). We think of \(I\) as the set of ports or interface variables


A symbolic resource sharer on an interface \(I\) consists of

  • A finite set \(S\), the state variables
  • A function \(\mathrm{expose} \colon I \to S\)
  • A vector field \(v \colon \mathbb{R}^S \to \mathbb{R}^S\)

The function \(\mathrm{expose}\) symbolically represents the function \(\mathbb{R}^S \to \mathbb{R}^I\) given by precomposition with \(\mathrm{expose} \colon I \to S\).

We can keep a geometric definition of \(v\) because we will only need to call \(v\) and don’t need to introspect it.

Example: predator-prey interaction

A simple system that models

  • predation of a predator species, say foxes \(f\)
  • on a prey species, say rabbits \(r\)

is given geometrically by

\[ X = I = \mathbb{R}^2, \qquad \mathrm{expose} = \mathrm{id}_X, \]

or symbolically by

\[ X = I = \{f,r\}, \qquad \mathrm{expose} = \mathrm{id}_X. \]

The vector field is

\[ v(f,r) = (\beta f r, -\beta f r). \]

Example: diffusion in two chambers

Two interconnected chambers

\[\begin{align*} \dot{c}_1 &= \rho(c_2 - c_1) \\ \dot{c}_2 &= \rho(c_1 - c_2) \end{align*}\]

\[\begin{align*} X &= \{1,2\} \\ I &= \{1\} \end{align*}\]

\[ \mathrm{expose}: 1 \mapsto 1 \]

Resource sharers in AlgebraicJulia

using AlgebraicDynamics, DifferentialEquations

predator_prey = ContinuousResourceSharer{Float64}(
  2, # number of ports
  2, # number of states
  (u, p, t) -> p[1] * u[1] * u[2] * [1, -1], # vector field
  [1, 2] # exposed states
)

u0, params, tspan = [10.0, 100.0],  [0.015], (0.0, 5.0)

prob = ODEProblem(predator_prey, u0, tspan, params)
plot(solve(prob, Tsit5()), label=["predator" "prey"])

Undirected wiring diagrams (UWDs)

using Catlab

R = @relation (x, z) where (x, y, z) begin
  A(x, y)
  B(y, z)
  C(z, x)
end

to_graphviz(R, box_labels=:name)

The simplest combinatorial operad is undirected wiring diagrams. An undirected wiring diagram consists of boxes, ports, outer ports, and junctions.

UWDs as cospans of finite sets

Undirected wiring diagram style

Cospan style

Figure 1: Two styles of drawing an undirected wiring diagram

Undirected wiring diagrams as C-sets

An undirected wiring diagram is a functor into \(\mathsf{FinSet}\) from the category \(\mathsf{UWD}\) freely generated by:

So a UWD is a cospan with a grouping of the “ports” on one side into “boxes.”

Undirected wiring diagrams in Catlab

  • We have generic infrastructure for C-sets in Catlab (explored in exercises)
  • The easiest way to construct UWDs is via the @relation macro:
@relation (x,z) where (x,y,z) begin
  R(x,y)
  S(y,z)
end

This call constructs a UWD with

  • three junctions, named x,y,z
  • two outer ports, exposing x,z
  • two boxes (R,S), each of which has two ports, connected to x,y and y,z

Part 3: Operadic composition of dynamical systems

Composing systems using operads

In compositional systems theory, we want to…

  • put systems inside the bubbles of a composition pattern

  • “push a button” to get a larger, composite system

In the case study from Part 2:

  • the systems are dynamical systems as resource sharers
  • the composition patterns are undirected wiring diagrams

“Composition patterns” are formalized as the operations of an operad.

Operads and SMCs

(Typed) operads, aka symmetric multicategories

  • have objects (“types”)
  • have morphisms (“operations”) whose domains are finite lists of objects rather than single objects
  • formalize “hierarchical” composition
  • generalize symmetric monoidal categories

Any symmetric monoidal category \((\mathsf{C},\otimes)\) induces an operad \(\mathrm{Op}(\mathsf{C})\) with types given by the objects of \(\mathsf{C}\) and with operations \((A_1,\ldots,A_n) \to B\) given by the morphisms \(A_1 \otimes \cdots \otimes A_n \to B\) of \(\mathsf{C}\).

Why talk about both?

  • Operads are more practically convenient for specifying compositions
  • SMCs are more mathematically convenient

Category theory of UWDs

There is a symmetric monoidal category \(\mathsf{Csp}\) whose

  • objects are finite sets
  • morphisms are (equivalence classes of) cospans, composed by pushout

The symmetric monoidal product is \(+\) (coproduct in \(\mathsf{FinSet}\)).

So a UWD is an operation in the induced operad \(\mathsf{UWD} = \mathrm{Op}(\mathsf{Csp})\):

\[ A_1 + \cdots + A_n \longrightarrow X \longleftarrow B \]

Operad algebras

The thing that gives us the “button to push” to compose systems is an operad algebra:

An operad algebra \(F\) for an operad \(\mathcal{O}\) consists of

  • a set \(F(A)\) for every type \(A\)
  • a function \(F(f) \colon F(A_1) \times \cdots \times F(A_n) \to F(B)\) for every operation \(f \in \mathcal{O}(A_1,\ldots,A_n;B)\)

such that the assignment of functions to operations commutes with composition, identities, and symmetries.

The sets \(F(A)\) tell us what sort of things we are allowed to put in the bubbles, and then the functions are the buttons we push to compose.

Operad algebras from lax symmetric monoidal functors

A cospan-algebra is a lax symmetric monoidal functor \(F \colon (\mathsf{Csp}, +, 0) \to (\mathsf{Set}, \times, 1)\)

The laxators associated with the lax SMF are functions

\[ F_{A_1,\ldots,A_n}: F(A_1) \times \cdots \times F(A_n) \longrightarrow F(A_1 + \cdots + A_n) \]

for each \(A_1,\ldots,A_n \in \mathsf{FinSet}\).


We get an operad algebra of \(\mathsf{UWD}\): to each UWD

\[ A_1 + \cdots + A_n \xrightarrow{f} X \xleftarrow{g} B, \]

we associate the function

\[\begin{equation*} F(A_1) \times \cdots \times F(A_n) \xrightarrow{F_{A_1,\ldots,A_n}} F(A_1 + \cdots + A_n) \xrightarrow{F(f,g)} F(B) \end{equation*}\]

The operad algebra of resource sharers

Let’s define a lax symmetric monoidal functor \(\mathrm{RS} \colon (\mathsf{Csp}, +, 0) \to (\mathsf{Set}, \times, 1)\)

  • \(\mathrm{RS}(I)\) is the set of resource sharers with interface \(I\)

  • For the laxators, given resource sharers \((S_1,\mathrm{expose}_1, v_1), \ldots, (S_n,\mathrm{expose}_n, v_n)\) on interfaces \(I_1, \ldots, I_n\), we construct a resource sharer

    \[(S_1 + \cdots + S_n, \mathrm{expose}_1 + \cdots + \mathrm{expose}_n, v_1 \times \cdots \times v_n)\]

    on the interface \(I_1 + \cdots + I_n\).

  • Finally, for the action on morphisms, given a cospan \(I \rightarrow J \leftarrow I'\) and a resource sharer \((S, \mathrm{expose}, v \colon \mathbb{R}^S \to \mathbb{R}^S)\) on \(I\), we make a new resource sharer \((S', \mathrm{expose}', v')\) on \(I'\) by taking

    and \(v' := f_\ast \circ v \circ f^\ast \colon \mathbb{R}^{S'} \to \mathbb{R}^{S'}\)

Unpacking the composition of resource sharers

Via the pushout, any two state variables in \(S\) connected to the same junction in \(J\) by a variable in the interface \(I\) are merged:

Then, to make the vector field \(v'\), we precompose with \(f\) to go from \(\mathbb{R}^{S'}\) to \(\mathbb{R}^S\), apply the vector field \(v\), and finally pushforward along \(f\) to go from \(\mathbb{R}^S\) to \(\mathbb{R}^{S'}\), via

\[ f_\ast(s)_{x'} := \sum_{x \in f^{-1}(x')} s_x. \]

Intuitively, when state variables from multiple resource sharers are merged, we sum each resource sharer’s “pull” on that state variable to get the overall derivative.

Example setup

dotr(u,p,t) = p.α*u
dotrf(u,p,t) = [-p.β*u[1]*u[2], p.γ*u[1]*u[2]]
dotf(u,p,t) = -p.δ*u

rabbit_growth = ContinuousResourceSharer{Float64}(1, dotr)
rabbitfox_predation = ContinuousResourceSharer{Float64}(2, dotrf)
fox_decline = ContinuousResourceSharer{Float64}(1, dotf)

rf = @relation (rabbits,foxes) begin
  growth(rabbits)
  predation(rabbits,foxes)
  decline(foxes)
end

to_graphviz(rf, box_labels=:name, junction_labels=:variable)

Example composed

using DifferentialEquations, LabelledArrays

sys = oapply(rf, [rabbit_growth, rabbitfox_predation, fox_decline])

u0 = [10.0, 100.0]
params = LVector=.3, β=0.015, γ=0.015, δ=0.7)
tspan = (0.0, 100.0)

prob = ODEProblem(sys, u0, tspan, params)
plot(solve(prob, Tsit5()), label=["rabbits" "foxes"])

Bibliography

Baez, John C., and Kenny Courser. 2020. “Structured Cospans.” arXiv. https://doi.org/10.48550/arXiv.1911.04630.
Brown, Kristopher, Tyler Hanks, and James Fairbanks. 2022. “Compositional Exploration of Combinatorial Scientific Models.” arXiv. https://doi.org/10.48550/arXiv.2206.08755.
Halter, Micah, Evan Patterson, Andrew Baas, and James Fairbanks. 2020. “Compositional Scientific Computing with Catlab and SemanticModels.” arXiv. https://doi.org/10.48550/arXiv.2005.04831.
Libkind, Sophie, Andrew Baas, Micah Halter, Evan Patterson, and James Fairbanks. 2022. “An Algebraic Framework for Structured Epidemic Modeling.” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 380 (2233): 20210309. https://doi.org/10.1098/rsta.2021.0309.
Libkind, Sophie, Andrew Baas, Evan Patterson, and James Fairbanks. 2022. “Operadic Modeling of Dynamical Systems: Mathematics and Computation.” Electronic Proceedings in Theoretical Computer Science 372 (November): 192–206. https://doi.org/10.4204/EPTCS.372.14.
Patterson, Evan, Andrew Baas, Timothy Hosgood, and James Fairbanks. 2022. “A Diagrammatic View of Differential Equations in Physics.” Mathematics in Engineering 5 (2): 1–59. https://doi.org/10.3934/mine.2023036.
Patterson, Evan, Owen Lynch, and James Fairbanks. 2022. “Categorical Data Structures for Technical Computing.” Compositionality 4 (December): 5. https://doi.org/10.32408/compositionality-4-5.
Vagner, Dmitry, David I. Spivak, and Eugene Lerman. 2015. “Algebras of Open Dynamical Systems on the Operad of Wiring Diagrams.” arXiv. https://doi.org/10.48550/arXiv.1408.1598.