ProcessBasedModelling
— ModuleProcessBasedModelling.jl
ProcessBasedModelling.jl is an extension to ModelingToolkit.jl (MTK) for building a model of equations using symbolic expressions. It is an alternative framework to MTK's native component-based modelling, but, instead of components, there are "processes". This approach is useful in the modelling of physical/biological/whatever systems, where each variable corresponds to a particular physical concept or observable and there are few (or none) duplicate variables to make the definition of MTK "factories" worthwhile. On the other hand, there plenty of different physical representations, or processes to represent a given physical concept in equation form. In many scientific fields this approach parallels the modelling reasoning of the researcher more closely than the "components" approach.
Beyond this reasoning style, the biggest strength of ProcessBasedModelling.jl is the informative errors and automation it provides regarding incorrect/incomplete equations. When building the MTK model via ProcessBasedModelling.jl the user provides a vector of "processes": equations or custom types that have a well defined and single left-hand-side variable. This allows ProcessBasedModelling.jl to:
- Iterate over the processes and collect new variables that have been introduced by a provided process but do not themselves have a process assigned to them.
- For these collected "process-less" variables:
- If there is a default process defined, incorporate this one into the model
- If there is no default process but the variable has a default value, equate the variable to a parameter that has the same default value and throw an informative warning.
- Else, throw an informative error saying exactly which originally provided variable introduced this new "process-less" variable.
- Throw an informative error if a variable has two processes assigned to it (by mistake).
In our experience, and as we also highlight explicitly in the online documentation, this approach typically yields simpler, less ambiguous, and more targeted warning or error messages than the native MTK one's. This leads to faster identification and resolution of the problems with the composed equations.
ProcessBasedModelling.jl is particularly suited for developing a model about a physical/biological/whatever system and being able to try various physical "rules" (couplings, feedbacks, mechanisms, ...) for a given physical observable efficiently. This means switching arbitrarily between different processes that correspond to the same variable. Hence, the target application of ProcessBasedModelling.jl is to be a framework to develop field-specific libraries that offer predefined processes without themselves relying on the existence of context-specific predefined components. An example usage is in ConceptualClimateModels.jl.
Besides the informative errors, ProcessBasedModelling.jl also
- Provides a couple of common process subtypes out of the box to accelerate development of field-specific libraries.
- Makes named MTK variables and parameters automatically, corresponding to parameters introduced by the by-default provided processes. This typically leads to intuitive names without being explicitly coded, while being possible to opt-out.
- Provides some utility functions for further building field-specific libraries.
See the documentation online for details on how to use this package as well as examples highlighting its usefulness.
ProcessBasedModelling.jl development is funded by UKRI's Engineering and Physical Sciences Research Council, grant no. EP/Y01653X/1 (grant agreement for a EU Marie Sklodowska-Curie Postdoctoral Fellowship for George Datseris).
These docs assume that you have some basic familiarity with ModelingToolkit.jl. If you don't going through the introductory tutorial of ModelingToolkit.jl should be enough to get you started!
Like ModelingToolkit.jl, ProcessBasedModelling.jl also exports t
as the independent variable representing time. However, instead of the default t
of ModelingToolkit.jl, here t
is unitless. Do t = ModelingToolkit.t
to obtain the unitful version of t
.
Usage
In ProcessBasedModelling.jl, each variable is governed by a "process". Conceptually this is just an equation that defines the given variable. To couple the variable with the process it is governed by, a user either defines simple equations of the form variable ~ expression
, or creates an instance of Process
if the left-hand-side of the equation needs to be anything more complex (or, simply if you want to utilize the conveniences of predefined processes). In either case, the variable
and the expression
are both symbolic expressions created via ModellingToolkit.jl.
Once all the processes about the physical system are collected, they are given as a Vector
to the processes_to_mtkmodel
central function, similarly to how one gives a Vector
of Equation
s to e.g., ModelingToolkit.ODESystem
. processes_to_mtkmodel
also defines what quantifies as a "process" in more specificity. Then processes_to_mtkmodel
ensures that all variables in the relational graph of your equations have a defining equation, or throws informative errors/warnings otherwise. It also provides some useful automation, see the example below.
Example
Let's say we want to build the system of equations
\[\dot{z} = x^2 - z \\ \dot{x} = 0.1y \\ y = z - x\]
symbolically using ModelingToolkit.jl (MTK). We define
using ModelingToolkit
@variables t # independent variable _without_ units
@variables z(t) = 0.0
@variables x(t) # no default value
@variables y(t) = 0.0
\[ \begin{equation} \left[ \begin{array}{c} y\left( t \right) \\ \end{array} \right] \end{equation} \]
ProcessBasedModelling.jl (PBM) strongly recommends that all defined variables have a default value at definition point. Here we didn't do this for $x$ to illustrate what how such an "omission" will be treated by PBM.
ProcessBasedModelling.jl re-exports the whole ModelingToolkit
package, so you don't need to be using
both of them, just using ProcessBasedModelling
.
To make the equations we want, we can use MTK directly, and call
eqs = [
Differential(t)(z) ~ x^2 - z
Differential(t)(x) ~ 0.1y
y ~ z - x
]
model = ODESystem(eqs, t; name = :example)
equations(model)
\[ \begin{align} \frac{\mathrm{d} z\left( t \right)}{\mathrm{d}t} &= - z\left( t \right) + \left( x\left( t \right) \right)^{2} \\ \frac{\mathrm{d} x\left( t \right)}{\mathrm{d}t} &= 0.1 y\left( t \right) \\ y\left( t \right) &= - x\left( t \right) + z\left( t \right) \end{align} \]
All good. Now, if we missed the process for one variable (because of our own error/sloppyness/very-large-codebase), MTK will throw an error when we try to structurally simplify the model (a step necessary before solving the ODE problem):
# no errors:
model = ODESystem(eqs[1:2], t; name = :example)
\[ \begin{align} \frac{\mathrm{d} z\left( t \right)}{\mathrm{d}t} &= - z\left( t \right) + \left( x\left( t \right) \right)^{2} \\ \frac{\mathrm{d} x\left( t \right)}{\mathrm{d}t} &= 0.1 y\left( t \right) \end{align} \]
# here is the error
model = structural_simplify(model)
ERROR: ExtraVariablesSystemException: The system is unbalanced.
There are 3 highest order derivative variables and 2 equations.
More variables than equations, here are the potential extra variable(s):
z(t)
x(t)
y(t)
The error message is unhelpful as all variables are reported as "potentially missing". At least on the basis of our scientific reasoning however, both $x, z$ have an equation. It is $y$ that $x$ introduced that does not have an equation. Moreover, in our experience these error messages become increasingly less accurate or helpful when a model has many equations and/or variables. This makes it difficult to quickly find out where the "mistake" happened in the equations.
PBM resolves these problems and always gives accurate error messages when it comes to the construction of the system of equations. This is because on top of the variable map that MTK constructs automatically, PBM requires the user to implicitly provide a map of variables to processes that govern said variables. PBM creates the map automatically, the only thing the user has to do is to define the equations in terms of what processes_to_mtkmodel
wants (which are either Process
es or Equation
s as above).
For the majority of cases, PBM can infer the LHS variable a process "defines" automatically, just by passing in a vector of Equation
s, like in MTK. For cases where this is not possible a dedicated Process
type is provided, whose subtypes act as wrappers around equations providing some additional conveniences.
Here is what the user defines to make the same system of equations via PBM:
using ProcessBasedModelling
processes = [
ExpRelaxation(z, x^2), # defines z, introduces x; `Process` subtype
Differential(t)(x) ~ 0.1*y, # defines x, introduces y; normal `Equation`
y ~ z - x, # defines y; normal `Equation`
]
3-element Vector{Any}:
ExpRelaxation(z(t), x(t)^2, nothing)
Differential(t)(x(t)) ~ 0.1y(t)
y(t) ~ -x(t) + z(t)
which is then given to
model = processes_to_mtkmodel(processes; name = :example)
equations(model)
\[ \begin{align} \frac{\mathrm{d} z\left( t \right)}{\mathrm{d}t} &= - z\left( t \right) + \left( x\left( t \right) \right)^{2} \\ \frac{\mathrm{d} x\left( t \right)}{\mathrm{d}t} &= 0.1 y\left( t \right) \\ y\left( t \right) &= - x\left( t \right) + z\left( t \right) \end{align} \]
Notice that the resulting MTK model is not structural_simplify
-ed, to allow composing it with other models. By default t
is taken as the independent variable.
Now, in contrast to before, if we "forgot" a process, PBM will react accordingly. For example, if we forgot the process for $x$, then the construction will error informatively, telling us exactly which variable is missing, and because of which processes it is missing:
model = processes_to_mtkmodel(processes[[1, 3]])
ERROR: ArgumentError: Variable x(t) was introduced in process of variable z(t).
However, a process for x(t) was not provided,
there is no default process for x(t), and x(t) doesn't have a default value.
Please provide a process for variable x(t).
If instead we "forgot" the $y$ process, PBM will not error, but warn, and make $y$ equal to a named parameter, since $y$ has a default value. So, running:
model = processes_to_mtkmodel(processes[1:2])
equations(model)
\[ \begin{align} \frac{\mathrm{d} z\left( t \right)}{\mathrm{d}t} &= - z\left( t \right) + \left( x\left( t \right) \right)^{2} \\ \frac{\mathrm{d} x\left( t \right)}{\mathrm{d}t} &= 0.1 y\left( t \right) \\ y\left( t \right) &= y_{0} \end{align} \]
Makes the named parameter:
parameters(model)
1-element Vector{SymbolicUtils.BasicSymbolic{Real}}:
y_0
and throws the warning:
┌ Warning: Variable y(t) was introduced in process of variable x(t).
│ However, a process for y(t) was not provided,
│ and there is no default process for it either.
│ Since it has a default value, we make it a parameter by adding a process:
│ `ParameterProcess(y)`.
└ @ ProcessBasedModelling ...\ProcessBasedModelling\src\make.jl:65
Lastly, processes_to_mtkmodel
also allows the concept of "default" processes, that can be used for introduced "process-less" variables. Default processes are like processes
and given as a 2nd argument to processes_to_mtkmodel
. For example,
model = processes_to_mtkmodel(processes[1:2], processes[3:3])
equations(model)
\[ \begin{align} \frac{\mathrm{d} z\left( t \right)}{\mathrm{d}t} &= - z\left( t \right) + \left( x\left( t \right) \right)^{2} \\ \frac{\mathrm{d} x\left( t \right)}{\mathrm{d}t} &= 0.1 y\left( t \right) \\ y\left( t \right) &= - x\left( t \right) + z\left( t \right) \end{align} \]
does not throw any warnings as it obtained a process for $y$ from the given default processes.
The default process infrastructure of PBM is arguably its most powerful quality when it comes to building field-specific libraries. Its usefulness is illustrated in the derivative package ConceptualClimateModels.jl.
Special handling of timescales
In dynamical systems modelling the timescale associated with a process is a special parameter. That is why, if a timescale is given for either the TimeDerivative
or ExpRelaxation
processes, it is converted to a named @parameter
by default:
processes = [
ExpRelaxation(z, x^2, 2.0), # third argument is the timescale
TimeDerivative(x, 0.1*y, 0.5), # third argument is the timescale
y ~ z-x,
]
model = processes_to_mtkmodel(processes)
equations(model)
\[ \begin{align} \frac{\mathrm{d} z\left( t \right)}{\mathrm{d}t} \tau_{z} &= - z\left( t \right) + \left( x\left( t \right) \right)^{2} \\ \frac{\mathrm{d} x\left( t \right)}{\mathrm{d}t} \tau_{x} &= 0.1 y\left( t \right) \\ y\left( t \right) &= - x\left( t \right) + z\left( t \right) \end{align} \]
parameters(model)
2-element Vector{SymbolicUtils.BasicSymbolic{Real}}:
τ_z
τ_x
Note the automatically created parameters $\tau_x, \tau_z$. This special handling is also why each process can declare a timescale via the ProcessBasedModelling.timescale
function that one can optionally extend (although in our experience the default behaviour covers almost all cases).
If you do not want this automation, you can opt out in two ways:
- Provide your own created parameter as the third argument in e.g.,
ExpRelaxation
- Wrap the numeric value into
LiteralParameter
. This will insert the numeric literal into the equation.
See the section on automatic parameters for more related automation, such as the macro @convert_to_parameters
which can be particularly useful when developing a field-specific library.
Main API functions
ProcessBasedModelling.processes_to_mtkmodel
— Functionprocesses_to_mtkmodel(processes::Vector [, default]; kw...)
Construct a ModelingToolkit.jl model/system using the provided processes
and default
processes. The model/system is not structurally simplified. Use the function processes_to_mtkeqs
to obtain the raw Vector{Equation}
before it is passed to the MTK model/system like ODESystem
.
During construction, the following automations improve user experience:
- Variable(s) introduced in
processes
that does not itself have a process obtain a default process fromdefault
. - If no default exists, but the variable(s) itself has a default numerical value, a
ParameterProcess
is created for said variable and a warning is thrown. - Else, an informative error is thrown.
- An error is also thrown if any variable has two or more processes assigned to it.
processes
is a Vector
whose elements can be:
- Any instance of a subtype of
Process
.Process
is a wrapper aroundEquation
that provides some conveniences, e.g., handling of timescales or not having limitations on the left-hand-side (LHS) form. - An
Equation
. The LHS format of the equation is limited. Letx
be a@variable
andp
be a@parameter
. Then, the LHS can only be one of:x
,Differential(t)(x)
,Differential(t)(x)*p
,p*Differential(t)(x)
, however, the versions withp
may fail unexpectedly. Anything else will error. - A
Vector
of the above two, which is then expanded. This allows the convenience of functions representing a physical process that may require many equations to be defined (because e.g., they may introduce more variables). - A ModelingToolkit.jl
XDESystem
, in which case theequations
of the system are expanded as if they were given as a vector of equations like above. This allows the convenience of straightforwardly coupling with already existingXDESystem
s.
Default processes
processes_to_mtkmodel
allows for specifying default processes by giving default
. These default processes are assigned to variables introduced in the main input processes
, but themselves do not have an assigned process in the main input.
default
can be a Vector
of individual processes (Equation
or Process
). Alternatively, default
can be a Module
. The recommended way to build field-specific modelling libraries based on ProcessBasedModelling.jl is to define modules/submodules that offer a pool of pre-defined variables and processes. Modules may register their own default processes via the function register_default_process!
. These registered processes are used when default
is a Module
.
Keyword arguments
type = ODESystem
: the model type to make.name = nameof(type)
: the name of the model.independent = t
: the independent variable (default:@variables t
).t
is also exported by ProcessBasedModelling.jl for convenience.warn_default::Bool = true
: iftrue
, throw a warning when a variable does not have an assigned process but it has a default value so that it becomes a parameter instead.
ProcessBasedModelling.register_default_process!
— Functionregister_default_process!(process, m::Module; warn = true)
Register a process
(Equation
or Process
) as a default process for its LHS variable in the list of default processes tracked by the given module. If warn
, throw a warning if a default process with same LHS variable already exists and will be overwritten.
You can use default_processes
to obtain the list of tracked default processes.
If you are developing a new module/package that is based on ProcessBasedModelling.jl, and within it you also register default processes, then enclose your register_default_process!
calls within the module's __init__()
function. For example:
module MyProcesses
# ...
function __init__()
register_default_process!.(
[
process1,
process2,
# ...
],
Ref(MyProcesses)
)
end
end # module
ProcessBasedModelling.default_processes
— Functiondefault_processes(m::Module)
Return the dictionary of default processes tracked by the given module. See also default_processes_eqs
.
ProcessBasedModelling.default_processes_eqs
— Functiondefault_processes_eqs(m::Module)
Same as default_processes
, but return the equations of all processes in a vector format, which is rendered as LaTeX in Markdown to HTML processing by e.g., Documenter.jl.
Predefined Process
subtypes
ProcessBasedModelling.ParameterProcess
— TypeParameterProcess(variable, value = default_value(variable)) <: Process
The simplest process which equates a given variable
to a constant value that is encapsulated in a parameter. If value isa Real
, then a named parameter with the name of variable
and _0
appended is created. Else, if valua isa Num
then it is taken as the paremeter directly.
Example:
@variables T(t) = 0.5
proc = ParameterProcess(T)
will create the equation T ~ T_0
, where T_0
is a @parameter
with default value 0.5
.
ProcessBasedModelling.TimeDerivative
— TypeTimeDerivative(variable, expression [, τ])
The second simplest process that equates the time derivative of the variable
to the given expression
while providing some conveniences over manually constructing an Equation
.
It creates the equation τ_$(variable) Differential(t)(variable) ~ expression
by constructing a new @parameter
with default value τ
(if τ
is already a @parameter
, it is used as-is). If τ
is not given, then 1 is used at its place and no parameter is created.
Note that if iszero(τ)
, then the process variable ~ expression
is created.
ProcessBasedModelling.ExpRelaxation
— TypeExpRelaxation(variable, expression [, τ]) <: Process
A common process for creating an exponential relaxation of variable
towards the given expression
, with timescale τ
. It creates the equation:
τn*Differential(t)(variable) ~ expression - variable
Where τn
is a new named @parameter
with the value of τ
and name τ_($(variable))
. If instead τ
is nothing
, then 1 is used in its place (this is the default behavior). If iszero(τ)
, then the equation variable ~ expression
is created instead.
The convenience function
ExpRelaxation(process, τ)
allows converting an existing process (or equation) into an exponential relaxation by using the rhs(process)
as the expression
in the equation above.
ProcessBasedModelling.AdditionProcess
— TypeAdditionProcess(process, added...)
A convenience process for adding processes added
to the rhs
of the given process
. added
can be a single symbolic expression. Otherwise, added
can be a Process
or Equation
, or multitude of them, in which case it is checked that the lhs_variable
across all added components matches the process
.
Process
API
This API describes how you can implement your own Process
subtype, if the existing predefined subtypes don't fit your bill!
ProcessBasedModelling.Process
— TypeProcess
A new process must subtype Process
and can be used in processes_to_mtkmodel
. The type must extend the following functions from the module ProcessBasedModelling
:
lhs_variable(p)
which returns the variable the process describes (left-hand-side variable). There is a default implementationlhs_variable(p) = p.variable
if the field exists.rhs(p)
which is the right-hand-side expression, i.e., the "actual" process.- (optional)
timescale(p)
, which defaults toNoTimeDerivative
. - (optional)
lhs(p)
which returns the left-hand-side. Letτ = timescale(p)
. Then defaultlhs(p)
behaviour depends onτ
as follows:- Just
lhs_variable(p)
ifτ == NoTimeDerivative()
. Differential(t)(p)
ifτ == nothing
, or multiplied with a number ifτ isa LiteralParameter
.τ_var*Differential(t)(p)
ifτ isa Union{Real, Num}
. If real, a new named parameterτ_var
is created that has the prefix:τ_
and then the lhs-variable name and has default valueτ
. Else ifNum
,τ_var = τ
as given.- Explicitly extend
lhs_variable
if the above do not suit you.
- Just
ProcessBasedModelling.lhs_variable
— FunctionProcessBasedModelling.lhs_variable(p::Process)
Return the variable (a single symbolic variable) corresponding to p
.
ProcessBasedModelling.rhs
— FunctionProcessBasedModelling.rhs(process)
Return the right-hand-side of the equation governing the process.
ProcessBasedModelling.timescale
— FunctionProcessBasedModelling.timescale(p::Process)
Return the timescale associated with p
. See Process
for more.
ProcessBasedModelling.NoTimeDerivative
— TypeProcessBasedModelling.NoTimeDerivative()
Singleton value that is the default output of the timescale
function for variables that do not vary in time autonomously, i.e., they have no d/dt derivative and hence the concept of a "timescale" does not apply to them.
ProcessBasedModelling.lhs
— FunctionProcessBasedModelling.lhs(p::Process)
ProcessBasedModelling.lhs(eq::Equation)
Return the right-hand-side of the equation governing the process. If timescale
is implemented for p
, typically lhs
does not need to be as well. See Process
for more.
Automatic named parameters
ProcessBasedModelling.new_derived_named_parameter
— Functionnew_derived_named_parameter(variable, value, extra::String; kw...)
If value isa Num
return value
. If value isa LiteralParameter
, replace it with its literal value. Otherwise, create a new MTK @parameter
whose name is created from variable
(which could also be just a Symbol
) by adding the extra
string.
Keywords:
prefix = true
: whether theextra
is added at the start or the end, connecting with the with theconnector
.connector = "_"
: what to use to connectextra
with the name.
For example,
@variables x(t)
p = new_derived_named_parameter(x, 0.5, "τ")
Now p
will be a parameter with name :τ_x
and default value 0.5
.
ProcessBasedModelling.@convert_to_parameters
— Macro@convert_to_parameters vars...
Convert all variables vars
into @parameters
with name the same as vars
and default value the same as the value of vars
. The macro leaves unaltered inputs that are of type Num
, assumming they are already parameters. It also replaces LiteralParameter
inputs with its literal values. This macro is extremely useful to convert e.g., keyword arguments into named parameters, while also allowing the user to give custom parameter names, or to leave some keywords as numeric literals.
Example:
julia> A, B = 0.5, 0.5
(0.5, 0.5)
julia> C = first(@parameters X = 0.5)
julia> @convert_to_parameters A B C
3-element Vector{Num}:
A
B
X
julia> typeof(A) # `A` is not a number anymore!
Num
julia> default_value(A)
0.5
julia> C # the binding `C` still corresponds to parameter named `:X`!
X
ProcessBasedModelling.LiteralParameter
— TypeLiteralParameter(p)
A wrapper around a value p
to indicate to new_derived_named_parameter
or @convert_to_parameters
to not convert the given parameter p
into a named @parameters
instance, but rather keep it as a numeric literal in the generated equations.
Utility functions
ProcessBasedModelling.default_value
— Functiondefault_value(x)
Return the default value of a symbolic variable x
or nothing
if it doesn't have any. Return x
if x
is not a symbolic variable. The difference with ModelingToolkit.getdefault
is that this function will not error on the absence of a default value.
ProcessBasedModelling.has_symbolic_var
— Functionhas_symbolic_var(eqs, var)
Return true
if symbolic variable var
exists in the equation(s) eq
, false
otherwise. This works for either @parameters
or @variables
. If var
is a Symbol
isntead of a Num
, all variables are converted to their names and equality is checked on the basis of the name only.
has_symbolic_var(model, var)
When given a MTK model (such as ODESystem
) search in all the equations of the system, including observed variables.