Schooner (schooner v1.2.0)

Copy Markdown View Source

An embeddable, sandboxed Scheme interpreter for the BEAM, targeting the r7rs-small language minus its mutable operations.

The pipeline is Lexer → Reader → Expander → Eval. The Schooner.Value module provides the tagged-term value model and Schooner.Env the immutable-with-mutable-globals runtime environment. Macro bindings live in a separate expansion-time syntax env threaded through expansion only.

Choosing an entry point

Schooner's entry points come in pairs following the Elixir convention: a tagged-tuple form (run/1, eval/2,3) and a bang form (run!/1, eval!/2,3) that raises on failure. Choosing between run* and eval* is a trust decision, and the names can mislead: eval* is the sandbox-safe family, run* the permissive one.

FamilyAuto-importsTrust postureUse for
run/1/run!/1injects (import ...) of every shipped standard library when the script declares noneNot sandbox-safe. Every shipped primitive is in scope by default.tests, REPL-style use, your own scripts where you control the source
eval/2,3/eval!/2,3none — bindings come exclusively from the env argument and the script's own (import ...) declarationsSandbox-safe. The embedder controls the surface.embedding untrusted or semi-trusted scripts

run/run! evaluate against a fresh Schooner.Env with the implicit_imports: :all option of eval/3 and eval!/3. To get the same implicit imports against your own Schooner.Env, call eval/3 or eval!/3 with that option directly.

Within each family, the bang form raises one of:

The non-bang form returns {:ok, value} on success or {:error, exception} for any of the above. ArgumentError raised by malformed options is not caught — that is an embedder bug, not a script-level failure.

Embedding untrusted code

Use eval/2 (or eval!/2) and require the script to declare exactly which libraries it needs. A script that omits (import ...) cannot reach any primitive — even + is unbound:

iex> Schooner.eval!("(import (only (scheme base) +)) (+ 1 2)", Schooner.Env.new())
3

Pair this with BEAM-level resource limits — run eval!/2 inside a spawned process with :max_heap_size and a Task.shutdown/2 timeout so a runaway script cannot exhaust the host. Build the env inside that process: an env can only be used by the process that created it. See the Running Untrusted Scheme guide.

For richer sandbox composition (registering host libraries, pre-imports, etc.), construct a Schooner.Environment via Schooner.Environment.new/1 and pass it to eval/2.

Source locations

Script-level exceptions carry a :location field: a Schooner.Location with the file, line and column of the form that failed, or nil. Locations are recorded when you ask for them by passing file: (or locations: true) to eval/3 or compile/3; the message is then prefixed with file:line:col:. format_error/2 renders an error with a source excerpt.

Tracking positions makes reading and expanding a script about 15% slower; it costs nothing while the script runs. Without file: or locations: true, scripts are read without positions and errors have location: nil.

With locations on, these errors are always located:

  • lexer and reader errors;
  • malformed special forms and other syntax and expansion errors;
  • an (import ...) of a missing library;
  • unbound variables, and letrec bindings used before they are initialised.

Errors raised while applying a procedure are located only with debug: true: primitive type and domain errors, arity mismatches, applying a non-procedure, an uncaught raise or (error ...), and Schooner.Host.TypeErrors raised by host functions. Debug mode wraps each primitive call in a try and records every call for a backtrace (see "Backtraces" below), which together make call-heavy scripts two to three times slower. Without it these errors have location: nil.

An error inside a macro expansion is placed in the user's source: forms the macro introduces take the position of the macro use, and the user's own sub-forms keep theirs. An error in a procedure defined by a library loaded from a file is placed in that file (see Schooner.Library.Loader).

Backtraces

Proper tail calls leave no Scheme call stack behind, so with debug: true the evaluator keeps a history of the last :backtrace_depth procedure calls (32 by default) and attaches it to any error that escapes, as :scheme_backtrace: a list of Schooner.Frames, most recent first. Calls that have returned are dropped from it, so it lists the calls that were still running, each followed by the tail calls it made. Being bounded, the history never grows however long a loop runs. format_error/2 renders it below the excerpt.

Debug mode is chosen when the program is compiled to closures, so without it the closures contain no history code at all and :scheme_backtrace is nil. With it, every call is recorded; that is most of debug mode's cost.

Summary

Types

One macro use expanded by expand/3 with trace: true: the macro's name, the location of the use (nil when unknown), the use itself, and what the macro rewrote it to.

Functions

Invoke a Scheme procedure value from Elixir. Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use apply!/2 for the raising variant.

Bang form of apply/2 — raises on script-level failure. See apply/2 for arguments.

Check source against environment without running it, and return the problems found as a list of Schooner.Diagnostics. An empty list means nothing was found.

Compile source against a fresh Schooner.Environment.new/0, in which every shipped standard library is available to import. See compile/2.

Read, expand, and pre-resolve source against env_struct's registry. Returns {:ok, %Schooner.Compiled{}} on success, {:error, exception} on any source-level failure. Use compile!/1,2 for the raising variant.

Bang form of compile/1 — raises on source-level failure.

Bang form of compile/3 — raises on source-level failure.

Read and evaluate source against env, threading top-level definitions and any explicit (import ...) bindings into it.

Read and evaluate source against env with opts. Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use eval!/3 for the raising variant.

Bang form of eval/2 — raises on script-level failure.

Bang form of eval/3 — raises on script-level failure.

Expand the macros in source against environment without running it, and return the expanded top-level forms.

Render a script-level exception for people.

Read and evaluate source in a fresh empty env, with every shipped standard library implicitly imported when the script declares no imports of its own.

Bang form of run/1 — raises on script-level failure.

Evaluate a %Schooner.Compiled{} against env_struct. Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use run_compiled!/3 for the raising variant.

Bang form of run_compiled/3 — raises on script-level failure.

Types

expansion_step()

@type expansion_step() :: %{
  macro: binary(),
  location: Schooner.Location.t() | nil,
  before: Schooner.Value.t(),
  after: Schooner.Value.t()
}

One macro use expanded by expand/3 with trace: true: the macro's name, the location of the use (nil when unknown), the use itself, and what the macro rewrote it to.

Functions

apply(proc, args)

@spec apply(Schooner.Value.t(), [Schooner.Value.t()]) ::
  {:ok, Schooner.Value.t()} | {:error, Exception.t()}

Invoke a Scheme procedure value from Elixir. Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use apply!/2 for the raising variant.

proc may be any procedure value — a closure (returned by evaluating a lambda or a define form), a primitive, or a parameter. args is a list of Schooner.Value.t/0 arguments.

This is the host-side hook for callback patterns: pass a Scheme procedure into a host function, capture it, and invoke it later via apply/2.

apply!(proc, args)

Bang form of apply/2 — raises on script-level failure. See apply/2 for arguments.

check(source, environment, opts \\ [])

Check source against environment without running it, and return the problems found as a list of Schooner.Diagnostics. An empty list means nothing was found.

The script is read, its imports are resolved against environment's registry, and its macros are expanded, just as eval/3 would, but no part of it is evaluated and environment is left unchanged. That makes check/3 safe to call on untrusted scripts, for example when one is saved. Expansion still runs the script's own syntax-rules macros, and a macro that expands forever never returns, so bound the call with a timeout as you would eval/3.

The checker reports:

  • :read_error — the source doesn't parse. Nothing else is checked.
  • :syntax_error — a malformed special form, macro use or import set.
  • :unknown_library — an (import ...) of a library the environment's registry doesn't have.
  • :unbound — a reference to a name that isn't bound by the environment, an import or a definition anywhere in the script.
  • :arity — a call, with a fixed number of arguments, to a procedure that can't accept that many: a primitive or procedure the environment or an import binds, or one the script defines once with lambda (or (define (name ...) ...)) and nothing else binds.

Every form is checked, including branches that would never run, so a reference to an unbound name in dead code is still reported. When an import fails, the names it would have bound are unknown, so :unbound is not reported for that script.

iex> environment = Schooner.Environment.new(pre_imports: [["scheme", "base"]])
iex> Schooner.check("(define (f x) (+ x y))\n(f 1 2)", environment, file: "f.scm")
[
  %Schooner.Diagnostic{
    severity: :error,
    code: :unbound,
    message: "unbound variable: y",
    location: %Schooner.Location{file: "f.scm", line: 1, column: 20}
  },
  %Schooner.Diagnostic{
    severity: :error,
    code: :arity,
    message: "arity mismatch in `f`: expected 1, got 2",
    location: %Schooner.Location{file: "f.scm", line: 2, column: 1}
  }
]

Diagnostics are ordered by location. Every diagnostic reported today is an :error: something that fails the script if evaluation reaches it.

Options:

  • :file — the name of the script, recorded in the location of every diagnostic. Defaults to nil.

compile(source)

@spec compile(binary()) :: {:ok, Schooner.Compiled.t()} | {:error, Exception.t()}

Compile source against a fresh Schooner.Environment.new/0, in which every shipped standard library is available to import. See compile/2.

compile(source, env_struct, opts \\ [])

@spec compile(binary(), Schooner.Environment.t(), keyword()) ::
  {:ok, Schooner.Compiled.t()} | {:error, Exception.t()}

Read, expand, and pre-resolve source against env_struct's registry. Returns {:ok, %Schooner.Compiled{}} on success, {:error, exception} on any source-level failure. Use compile!/1,2 for the raising variant.

The compiled artifact can be passed to run_compiled/3 repeatedly against any compatible environment. Macros are expanded at compile time; variable bindings from (import ...) declarations are pre-resolved and baked into the artifact.

Options are :file, :locations, :debug and :backtrace_depth, as for eval!/3. They are kept in the artifact, so run_compiled/3 reports the same locations.

compile!(source)

@spec compile!(binary()) :: Schooner.Compiled.t()

Bang form of compile/1 — raises on source-level failure.

compile!(source, env_struct, opts \\ [])

Bang form of compile/3 — raises on source-level failure.

eval(source, env_or_environment)

@spec eval(binary(), Schooner.Env.t() | Schooner.Environment.t()) ::
  {:ok, Schooner.Value.t()} | {:error, Exception.t()}

Read and evaluate source against env, threading top-level definitions and any explicit (import ...) bindings into it.

Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use eval!/2 for the raising variant.

env may be either a Schooner.Env (low-level) or a Schooner.Environment (the embedding-friendly bundle of env + registry + syntax env produced by Schooner.Environment.new/1). When given an Env, no implicit imports are added — bindings come exclusively from env and the script's own (import ...) declarations.

This is the strict path and the right entry point for scripts whose source the host does not control. See "Choosing an entry point" in the moduledoc.

eval(source, env_or_environment, opts)

@spec eval(binary(), Schooner.Env.t() | Schooner.Environment.t(), keyword()) ::
  {:ok, Schooner.Value.t()} | {:error, Exception.t()}

Read and evaluate source against env with opts. Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use eval!/3 for the raising variant.

env may be a Schooner.Env or a Schooner.Environment. Options match eval!/3.

eval!(source, env_or_environment)

Bang form of eval/2 — raises on script-level failure.

eval!(source, env, opts)

Bang form of eval/3 — raises on script-level failure.

Options:

  • :file — the name of the script, recorded in the :location of any error and prefixed to its message. Passing it turns on :locations. Defaults to nil.
  • :locations — when true, record the source location of errors (see "Source locations" in the moduledoc). Defaults to true when :file or :debug is given and false otherwise.
  • :debug — when true, also locate errors raised while applying a procedure, at a cost on every primitive call, and attach a Scheme backtrace to errors (see "Backtraces" in the moduledoc). Implies locations: true. Defaults to false.
  • :backtrace_depth — with debug: true, the number of procedure calls the backtrace history keeps. A positive integer; defaults to 32.
  • :implicit_imports — controls implicit imports prepended to a script that declares none of its own. Only accepted with a Schooner.Env; a Schooner.Environment bakes its imports in at construction time.
    • :none (default) — no implicit imports. Bindings come exclusively from env and the script's own (import ...) declarations. Use this for untrusted input.
    • :all — implicitly import every shipped standard library. Skipped if the script declares any (import ...) of its own: an explicit import means the script has chosen a narrower surface.

expand(source, environment, opts \\ [])

@spec expand(binary(), Schooner.Environment.t(), keyword()) ::
  {:ok, [Schooner.Value.t()]}
  | {:ok, [Schooner.Value.t()], [expansion_step()]}
  | {:error, Exception.t()}

Expand the macros in source against environment without running it, and return the expanded top-level forms.

The script is read, its imports are resolved against environment's registry, and its macros are expanded, as eval/3 would, but no part of it is evaluated and environment is left unchanged. Many standard forms (cond, case, the let family, do, and, or, when, unless) are syntax-rules macros, so this shows what they, and the script's own macros, turn into. Render the forms with Schooner.Pretty.format/2:

iex> environment = Schooner.Environment.new(pre_imports: [["scheme", "base"]])
iex> {:ok, [form]} = Schooner.expand("(when (> n 0) (go n))", environment)
iex> Schooner.Pretty.format(form)
"(if (> n 0) (begin (go n)))"

The (import ...) forms are not returned, and nor are the define-syntax forms, whose macros are expanded where the script uses them. Expansion runs the script's syntax-rules macros, and a macro that expands forever never returns, so bound the call with a timeout as you would eval/3.

An identifier a macro introduces is renamed for hygiene, so that it can't capture or be captured by the script's own identifiers. Schooner.Pretty prints a renamed identifier with a number, such as tmp·1. Numbers are assigned from 1 in the order the identifiers appear in the result, so the output is the same on every run:

iex> environment = Schooner.Environment.new(pre_imports: [["scheme", "base"]])
iex> {:ok, [form]} = Schooner.expand("(or (f) tmp)", environment)
iex> Schooner.Pretty.format(form)
"((lambda (t·1) (if t·1 t·1 tmp)) (f))"

Returns {:ok, forms}, {:ok, forms, steps} with trace: true, or {:error, exception} when the script can't be read, imports a library the registry doesn't have, or uses a macro or special form wrongly. Errors are located as with eval/3 and locations: true.

Options:

  • :file — the name of the script, recorded in locations and error messages. Defaults to nil.

  • :step — :full (the default) expands until only core forms are left. :once expands each macro use that is not inside another macro use once, and leaves the macro uses in its output as they are:

    iex> environment = Schooner.Environment.new(pre_imports: [["scheme", "base"]])
    iex> {:ok, [form]} = Schooner.expand("(cond (a 1) (b 2))", environment, step: :once)
    iex> Schooner.Pretty.format(form)
    "(if a (begin 1) (cond·1 (b 2)))"
  • :trace — when true, also return every macro use expanded, in the order it was expanded, as a list of expansion_step/0 maps:

    iex> environment = Schooner.Environment.new(pre_imports: [["scheme", "base"]])
    iex> {:ok, _forms, [step]} = Schooner.expand("(unless ok (fail))", environment, trace: true)
    iex> {step.macro, step.location}
    {"unless", %Schooner.Location{file: nil, line: 1, column: 1}}
    iex> Schooner.Pretty.format(step.after)
    "(if ok (begin) (begin (fail)))"

    Defaults to false.

format_error(exception, opts \\ [])

@spec format_error(
  Exception.t(),
  keyword()
) :: binary()

Render a script-level exception for people.

The first line is the message, prefixed with the error's location (file:line:col: or line:col:) when it has one. With source: binary — the text of the script the location points into — a short excerpt follows, with a caret under the failing column:

iex> source = "(define x 1)\n(car y)"
iex> {:error, e} =
...>   Schooner.eval(source, Schooner.Env.new(), implicit_imports: :all, file: "demo.scm")
iex> Schooner.format_error(e, source: source)
"demo.scm:2:6: unbound variable: y\n  |\n2 | (car y)\n  |      ^"

An error raised with debug: true also has a Scheme backtrace (see "Backtraces" in the moduledoc), which is listed last, one call per line, most recent first. A call in tail position is marked (tail call):

iex> source = "(define (f x) (car x))\n(define (g x) (+ 1 (f x)))\n(g 1)"
iex> {:error, e} =
...>   Schooner.eval(source, Schooner.Env.new(),
...>     implicit_imports: :all, file: "demo.scm", debug: true)
iex> Schooner.format_error(e) |> String.split("\n")
[
  "demo.scm:1:15: type error in `car`: expected pair, got 1",
  "",
  "Scheme backtrace (most recent first):",
  "  car  demo.scm:1:15 (tail call)",
  "  f    demo.scm:2:20",
  "  g    demo.scm:3:1"
]

Options:

  • :source — the script's source text. Without it, or when the error has no location, only the message is returned.

run(source)

@spec run(binary()) :: {:ok, Schooner.Value.t()} | {:error, Exception.t()}

Read and evaluate source in a fresh empty env, with every shipped standard library implicitly imported when the script declares no imports of its own.

Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use run!/1 for the raising variant.

Not for untrusted input. A script with no (import ...) form reaches every primitive Schooner ships with. Use eval/2 for embedding scripts you do not control — see "Choosing an entry point" in the moduledoc.

run!(source)

@spec run!(binary()) :: Schooner.Value.t()

Bang form of run/1 — raises on script-level failure.

run_compiled(compiled, env_struct, opts \\ [])

@spec run_compiled(Schooner.Compiled.t(), Schooner.Environment.t(), keyword()) ::
  {:ok, Schooner.Value.t()} | {:error, Exception.t()}

Evaluate a %Schooner.Compiled{} against env_struct. Returns {:ok, value} on success, {:error, exception} for any script-level failure. Use run_compiled!/3 for the raising variant.

The compiled program's pre-resolved variable bindings are re-applied to env_struct.env on every call, so the (import ...) surface the program was compiled against is always in scope regardless of env_struct's registry. Macros are not re-expanded — the program's macro shape is frozen at compile time.

Options override those the program was compiled with:

  • :debug — as for eval!/3. The program is compiled to closures on each run, so one artifact can run with or without it. Errors are located only if the program was compiled with locations.
  • :backtrace_depth — as for eval!/3.

run_compiled!(compiled, env_struct, opts \\ [])

Bang form of run_compiled/3 — raises on script-level failure.