This page is the authority on what Jsonnet programs should do. It defines what input is / is not a syntactically valid Jsonnet program, and precisely how they are parsed. It describes which programs should be rejected statically (i.e. before execution). Finally, it specifies the manner in which the program is executed, i.e. the JSON that is output, or the dynamic error if there is one.
The specification is intended be terse but precise. The intention is to illuminate various subtleties and edge cases in order to allow fully-compatible reimplementations of the language, refactoring tools, etc. The specification employs some standard theoretical computer science techniques, namely type systems and big step operational semantics. If that's not your cup of tea, then see the more discussive description of Jsonnet behavior in tutorial.
In this notation, x★ defines a comma-separated possibly zero-length list of x (either a terminal or non-terminal) with an optional comma after the last element. The regular * and + have their usual meaning, i.e. repetition without commas.
expr ∈ Expr | ::= |
'null ' |
'true ' |
'false ' |
'self ' |
'super ' |
'$ ' |
string |
number
|
| |
'{ '
('local ' bind | field)★
'} '
| |
| |
'{ '
('local ' bind ',')*
'[ '
expr
'] '
': '
expr
'for '
id
'in '
expr
'} '
| |
| |
'[ '
expr★
'] '
| |
| |
'[ '
expr
'for '
id
'in '
expr
[
'if '
expr
]
'] '
| |
| |
expr
'. '
id
| |
| |
expr
'[ '
expr
'] '
| |
| |
expr
'( '
expr★
') '
| |
| | id | |
| |
'local '
bind
[
', '
bind
]*
'; '
expr
| |
| |
'if '
expr
'then '
expr
| |
| |
'if '
expr
'then '
expr
'else '
expr
| |
| | expr symbol expr | |
| | symbol expr | |
| |
expr
'{ '
field★
'} '
| |
| |
'function '
'( '
id★
') '
expr
| |
| |
'import '
string
| |
| |
'importstr '
string
| |
| |
'error '
expr
|
field | ::= |
fieldname
': '
[ ': ' ]
e
|
| |
fieldname
'( '
id★
') '
': '
[ ': ' ]
e
|
fieldname | ::= |
id |
string |
'[ '
e
'] '
|
bind | ::= |
id
'= '
e
|
| |
id
'( '
id★
') '
'= '
e
|
symbol | ::= ( |
'+ ' |
'- ' |
'* ' |
'/ ' |
'% ' |
'& ' |
'| ' |
'^ ' |
'= ' |
'< ' |
'> '
)+
|
Additionally, id is defined by regular expression: [a-zA-Z_][a-zA-Z0-9_]*. The
definition of string is equivalent to the JSON string, including escape characters. Finally,
number is equivalent to the JSON number, but without the leading -
.
The parsing of the concrete syntax into abstract syntax can be controlled by adding parentheses in order to resolve ambiguities explicitly. If parentheses are not given, the ambiguity is resolved according to the following rules:
Everything is left associative. In the case of local
, if
,
function
, and error
, ambiguity is resolved by consuming as many tokens as
possible on the right hand side. For example the parentheses are redundant in
local x = 1; (x + x)
.
All remaining ambiguities are resolved according to the following order of
precedence:
f()
arr[e]
obj[e]
obj.f
(application and indexing)+
-
!
~
(the unary operators)*
/
%
(these, and the remainder below, are binary operators)+
-
<<
>>
<
>
<=
>=
==
!=
&
^
|
&&
||
While the abstract syntax reserves arbitrary sequences of symbol characters for future use, only the operators in the above list are currently parsed.
In order to build a simpler model of Jsonnet's behavior, many of the language features are represented as syntax sugar. Below, the core syntax is defined, and the desugaring function. Both the static checking rules and the operational semantics are defined at the level of the core language, so it is reasonable to implement the desugaring in the parser.
e ∈ Core | ::= |
'null ' |
'true ' |
'false ' |
'self ' |
'super ' |
string |
number
|
| |
'{ '
(
'[ '
e
'] '
': '
[ ': ' ]
e
)★
'} '
| |
| |
'{ '
'[ '
e
'] '
': '
e
'for '
id
'in '
e
'} '
| |
| |
'[ '
e★
'] '
| |
| |
e
'[ '
e
'] '
| |
| |
e
'( '
e★
') '
| |
| | id | |
| |
'local '
id
'= '
e
[
', '
id
'= '
e
]*
'; '
e
| |
| |
'if '
e
'then '
e
'else '
e
| |
| | e symbol e | |
| | symbol e | |
| |
'function '
'( '
id★
') '
e
| |
| |
'error '
e
|
In the core language, the set of identifiers now includes $
, which is no-longer a
special keyword.
Additionally, the !=
operator is desugared to !(a == b)
and the unary
+
is stripped (it is a no-op).
Also removed in the core language are import
and importstr
. The
semantics of these constructs is that they are replaced with either the contents of the file, or an
error construct if importing failed (e.g. due to I/O errors). In the first case, the file is
parsed, desugared, and subject to static checking before it can be substituted. In the latter case,
the file is substituted in the form of a string, so it merely needs to contain valid UTF-8.
A given Jsonnet file can be recursively imported via import
. Thus, the
implementation loads files lazily (i.e. during execution) as opposed to via static desugaring. The
imported Jsonnet file is parsed and statically checked in isolation. Therefore, the behavior of the
import is not affected by the environment into which it is imported. The files are cached by
filename, so that even if the file changes on disk during Jsonnet execution, referential
transparency is maintained.
Desugaring is defined as the function desugar: (Expr × Boolean) → Core. The second parameter of the function tracks whether we are within an object.
desugar(
{
field₁, ... fieldₙ, binds
} , b)
| = |
where binds = local bind₁, ... local bindₙ let binds' = desugarbinds(binds, true )
{ desugarfield(field₁, binds'', b), ... desugarfield(fieldₙ, binds'', b)}
| |||||||
desugar({
binds,
[
expr
]
:
expr'
for
id
in
expr''
} , b)
| = |
where binds = local bind₁, ... local bindₙ let binds' = desugarbinds(binds, true )
local binds''; desugar(expr', true ) {
[
desugar(expr, b)
]
:
expr'''
for
id
in
desugar(expr'', b)
}
| |||||||
desugar([
expr
for
id
in
expr'
] , b)
| = |
std.map(function( id) desugar(expr', b), desugar(expr, b))
| |||||||
desugar([
expr
for
id
in
expr'
if
expr''
] , b)
| = |
where ffunc = function( id) desugar(expr'', b) where mfunc = function( id) desugar(expr, b) where arr = desugar(expr', b) std.filterMap( ffunc, mfunc, arr)
| |||||||
desugar(expr
.
id, b)
| = |
desugar(expr, b)["id"]
| |||||||
desugar(if e₁
then
e₂, b)
| = |
if desugar(e₁, b)
then
desugar(e₂, b) else null
| |||||||
desugar(expr { ... } , b)
| = |
desugar(expr + { ... } , b)
| |||||||
Other cases invoke structural recursion. |
desugarfield(fname :: expr, binds, b) | = |
desugarfname(fname, b) :: local binds ; desugar(expr, true )
|
desugarfield(fname : expr, binds, b) | = |
desugarfname(fname, b) : local binds ; desugar(expr, true )
|
desugarfield(fname ( params ) :: expr, b)
| = |
desugarfname(fname, b) :: local binds ; function ( params ) desugar(expr, true )
|
desugarfield(fname ( params ) : expr, b)
| = |
desugarfname(fname, b) : local binds ; function ( params ) desugar(expr, true )
|
desugarfname(id, b) | = |
["id"]
|
desugarfname(string, b) | = |
[string]
|
desugarfname([ expr] , b)
| = |
[ desugar(expr, b)]
|
desugarbinds(local id = expr, b)
| = |
local id = desugar(expr, b)
|
After the Jsonnet program is parsed and desugared, a syntax-directed algorithm is employed to
reject programs that contain certain classes of errors. This is presented like a static type
system, except that there are no static types. Programs are only rejected if they use undefined
variables, or if self
, super
or $
are used outside the bounds
of an object. In the core language, $
has been desugared to a variable, so its
checking is implicit in the checking of bound variables.
The static checking is described below as a judgement \(Γ ⊢ e\), where \(Γ\) is the set of
variables in scope of \(e\). The set \(Γ\) initially contains only std
, the implicit
standard library. In the case of imported files, each jsonnet file is checked independently of the
other files.
We present two sets of operational semantics rules. The first defines the judgement \(e ↓ v\) which represents the execution of Jsonnet expressions into Jsonnet values. The other defines the judgement \(v ⇓ j\) which represents manifestation, the process by which Jsonnet values are converted into JSON values.
We model both explicit runtime errors (raised by the error construct) and implicit runtime errors (e.g. array bounds errors) as stuck execution. Errors can occur both in the \(e ↓ v\) judgement and in the \(v ⇓ j\) judgement (because it is defined in terms of \(e ↓ v\)).
Jsonnet values are yielded by executing Jsonnet expressions. These are not the same as JSON values, although there is some overlap. In particular, they contain functions which are not representable in JSON, and both object fields and array elements have yet to be executed to yield values.
v | ∈ | Value | = |
{ null , true , false } ∪
String ∪
Double ∪
Object ∪
Function ∪
Array
|
o | ∈ | Object | = | String ⇀ (Hidden × Core) |
Hidden | = |
{ true , false }
| ||
Function | ::= |
function ( id* ) e
| ||
a | ∈ | Array | ::= |
[ e₀ ... eₙ ]
|
Objects are represented as finite partial functions, hence the use of the harpoon notation ⇀. If an object does not define a particular field, the function is undefined. Otherwise, the field is bound to a tuple containing a boolean indicating whether or not the field is hidden, and an expression to be evaluated when the field is accessed.
To construct a partial function, we use a bounded lambda notation, e.g. the object
{a: 5, b: 5}
would be represented by
λx∈{"a"
, "b"
}.
(false
, 5
). In order to allow substituting objects into
expressions e, we use the following function to convert:
The rules for capture-avoiding variable substitution e [ e' / id ] are an extension of those in the lambda calculus. In the following, assume that y ≠ x.
self [ e' / x ] = self
| |
super [ e' / x ] = super
| |
x [ e' / x ] = e' | |
y [ e' / x ] = y | |
{ ...[ e]: e''...}
[ e' / x ] =
{ ...[ e[e' / x]] : e''[e' /
x]...}
| |
{[ e]: e'' for x in
e'''}
[ e' / x ] =
{[ e]: e'' for x in
e''' [ e' / x ]}
| |
{[ e]: e'' for y in
e'''}
[ e' / x ] =
{[ e[ e' / x ]]: e''[ e' / x ] for y in
e''' [ e' / x ]}
| |
(let ... x= e... ; e'')
[ e' / x ] =
let ... x= e... ; e''
| If any variable matches. |
(let ... y= e... ; e'')
[ e' / x ] =
let ... y= e[ e' / x ]... ; e''[ e' / x ]
| If no variable matches. |
(function (... x...) e)
[ e' / x ] =
function (... x...) e
| If any variable matches. |
(function (... y...) e)
[ e' / x ] =
function (... x...) e[ e' / x ]
| If no variable matches. |
Otherwise, e [ e' / x ] proceeds via syntax-directed recursion into subterms of e. |
The rules for keyword substitution e ⟦ e' / kw ⟧ for kw ∈ {
self
, super
} avoid substituting keywords that are captured by nested
objects:
self
⟦ e' / self ⟧ = e'
|
super
⟦ e' / super ⟧ = e'
|
self
⟦ e' / super ⟧ = self
|
super
⟦ e' / self ⟧ = super
|
{ ...[ e]: e''...}
⟦ e' / kw ⟧ =
{ ...[ e⟦ e' / kw ⟧] :
e''...}
|
{ [e]: e'' for x in
e'''}
⟦ e' / kw ⟧ =
{ [e⟦ e' / kw ⟧]: e'' for x
in e'''⟦ e' / kw ⟧}
|
Otherwise, e ⟦ e' / kw ⟧ proceeds via syntax-directed recursion into subterms of e. |
The following big step operational semantics rules define the execution of Jsonnet programs, i.e. the reduction of a Jsonnet program e into its Jsonnet value v via the judgement \(e ↓ v\).
String concatenation will implicitly convert one of the values to a string if necessary. This is similar to Java. The referred function \(tostring\) returns its argument unchanged if it is a string. Otherwise it will manifest its argument as a JSON value \(j\) and unparse it as a single line of text.
The semantics of unary operators -
~
and !
are not given
above. Negation has its usual double precision floating point meaning. Unary ~
first
converts the value to a 64 bit signed integer, then does the unary negation, then casts back to
double precision floating point. The unary operator !
operates only on booleans and
has its usual meaning.
Binary operators can be divided into those that implement boolean arithmetic (short cut semantics, see above rules), floating point arithmetic (standard semantics, rules not given above), and the various special cases (rules given above): string concatenation, array concatenation, manifest equality, and object inheritance. The floating point arithmetic operations are standard, except in the case of bitwise operations, where the number is first cast to a 64 bit signed integer for the operation, then cast back to a double.
Equality has a subtle definition in Jsonnet, due to the desire for e.g.
{x:1, y:self.x}
to compare equal to {x:1, y: 1}
. To permit this, equality
is performed after manifesting the values being compared into JSON values, which binds self and
fully evaluates all the fields. One effect of this is that hidden fields are ignored when testing
for equality. There is another more fundamental reason why, even if two Jsonnet expressions
e₁ and e₂ compare equal with ==
, they cannot be substituted in an
arbitrary context. For example in the above case, e₁ + {x: 2}
will yield
{x: 2, y: 2}
, whereas e₂ + {x: 2}
will yield
{x: 2, y: 1}
. Note that such caveats are actually quite common in programming
languages, particularly those that perform automatic conversions on operands of the ==
operator.
The error
operator cannot be expressed in our rules, as we model errors as stuck
execution. The semantics of error
are that its subterm is evaluated to a Jsonnet
value. If this is a string, then that is the error that is raised. Otherwise, an error is raised
complaining that the error message was not a string. This specification does not specify how the
error is presented to the user, and whether or not there is a stack trace. Error messages are meant
for human inspection, and there is therefore no need to standardize them.
JSON values are unparsed to become the ultimate output of Jsonnet programs. Hidden fields in Jsonnet objects are lost on the conversion to JSON. Arrays elements and Object fields are fully evaluated. Functions cannot be represented.
j | ∈ | JValue | = |
{ null , true , false } ∪
String ∪
Double ∪
JObject ∪
JArray
|
JObject | = | String ⇀ JValue | ||
JArray | ::= |
[ j₀ ... jₙ ]
|
Note that JValue ⊂ Value.
JObjects are represented as partial functions, hence the use of the harpoon notation ⇀. If a JSON object does not define a particular field, the function is undefined. This partial function representation makes it clear that there can be no duplicate fields, and also the fields are unordered.
Partial functions are constructed with the bounded lambda notation described earlier.
Manifestation is the conversion of a Jsonnet value into a JSON value. It is represented with the judgement \(v⇓j\). The process requires executing arbitrary Jsonnet code fragments, so the two semantic judgements represented by \(↓\) and \(⇓\) are mutually recursive. Hidden fields are ignored during manifestation. Functions cannot be manifested, so an error is raised in that case.
Let D, E, F range over arbitrary expressions. Let ≡ mean contextual equivalence, i.e if D ≡ E then C[D] and C[E] will manifest to the same JSON, for any context C. If D yields an error, and E yields an error, then D ≡ E (regardless of what the exact text of the message is).
Associativity | (D + E) + F ≡ D + (E + F) | |
Idempotence | D + D ≡ D | (if D does not contain super ) |
Commutativity | D + E ≡ E + D | (if D, E do not contain super and have no common fields) |
Identity | D + { } ≡ D | |
{ } + D ≡ D |
The functions in the standard library that provide reflection capabilities are formalized below. Note the lexical sorting of the fields in the object-fields rule. This is because the fields have no order within the object value, but the resulting list does define an order. To avoid ambiguity, we therefore require the list of fields to be sorted alphabetically.
Except as noted, this content is licensed under Creative Commons Attribution 2.5.