Software Language Engineering (SLE): practice
1. Langium: a modern language engineering framework
Langium is an open-source language engineering framework developed by TypeFox and hosted by the Eclipse Foundation. It can be used to implement a software language in Typescript, which can then be deployed in a web environment or as an extension for the Visual Studio Code IDE.
| Langium is the "spiritual successor" of an older language engineering framework called Xtext, which can be used to implement a software language in Java and to generate a powerful integration of this language in the Eclipse IDE. |
2. Langium overview
The Langium framework is organized around three main components:
-
A meta-language, called the Langium grammar language, which can be used to create a Langium grammar. A Langium grammar defines both the abstract syntax and the concrete syntax of the software language being developed. The abstract syntax is written as a set of data types (with a notation similar to TypeScript), while the concrete syntax is defined in the form of a grammar rules (with a notation similar to EBNF).
-
A generator, which takes a Langium grammar as input and produces a set of Typescript language components as output (detailed below).
-
A Typescript library used by both the generator and the generated language components.
Using the Langium framework to create a new software language can be depicted as follows:
-
At the top, the software language engineer uses the Langium grammar language to define both the abstract and concrete syntaxes of the developed language.
-
The resulting Langium grammar is given to the Langium generator, which produces as output a wide array of artifacts:
-
A set of Typescript classes for all the concepts of the abstract syntax,
-
A parser, able to read a text file and to produce an AST as defined by the grammar rules,
-
A set of editing services which are essentially all the logic of a code editor for the language being developed (syntax highlighting, error highlighting, autocompletion, outline view, etc.).
-
A language API, which can be used as a single HTTP entrypoint to access all languages components. The services of this API follow the standardized Language Server Protocol (LSP).
-
A Visual Studio Code extension which can be used to easily deploy the software language implementation in any Visual Studio Code IDE. This extension is mostly an adapter that interacts with the language API to provide all services.
-
-
Finally, some parts of the language implementation must be developed manually by the language engineer. This includes the implementation of the semantics, either in the form of a code generator (if implemented as a compiler) or in the form of an executor (if implemented as an interpreter).
3. Preparing a Langium project
Langium provides a Yeoman generator which can be used to rapidly quickstart a Langium project.
Yeoman can be installed on your system with npm with the following command:
npm install -g yo generator-langium
The -g option requires that you first configure your system to install npm packages in your home directory, as written in this page of the npm documentation.
|
Then to generate a new folder that will contain your new Langium project, run this command:
yo langium
This will ask you different questions about your Langium project, such as the name of the software language being developed, the file extension for this language, and which development environments should be supported (CLI, Visual Studio Code, web, etc.). For more information, refer to this page of the Langium documentation.
| If you are unsure, you can always answer yes to all questions. |
After generation of your base Langium project, you obtain a Langium project structure.
This structure starts with the folder packages, which contains in the following subfolders:
-
language: Contains the main components of the software language, in particular:-
langium-config.json: Contains metadata about your language, including the language name, the file extension, or the path to the.langiumfile. -
src: Contains the.langiumfiles defining the Langium grammar of the language, which itself defines the abstract syntax and the concrete syntax of the language (will be presented in the next section), with one subfolder:-
generated: Contains files generated by the Langium generator from the.langiumfiles, including theast.tsfile that contains the data types for all the language concepts.
-
-
test: Contains test cases for the language.
-
-
cli: contains a base implementation of a Command Line Interface (CLI) for the language, which by default only includes a base code generator that implements the semantics of the language (will be presented in a later section). -
extension: Contains the code of the Visual Studio Code extension of the language.
The generated project also comes with a package.json file that defines the following important actions that can be performed using npm:
-
npm run langium:generate: Call the Langium Generator. -
npm run build: Compiles all the Typescript code (_must be executed after any code change and after any call to the Langium Generator). -
npm run test: Run all test cases (only works if you have generated the test folder, and if you have written test cases).
4. The Langium grammar language
A Langium grammar is written in the packages/language/src folder in the form of one or multiple .langium files written using the Langium grammar language.
This language can be divided in two parts:
-
A subset of the grammar language called "Declared Types" can be used to define the abstract syntax of the developed language, in the form of simple object-oriented data types.
-
The rest of the grammar language can be used to define the concrete syntax of the developed language, in the form of grammar rules.
| Quite confusingly, the Langium documentation uses the term Semantic Model to refer to the abstract syntax of the developed language. Be careful: while the word "semantic" is used, this has nothing to do with what we called earlier the semantics of the language! |
The abstract and concrete syntaxes can either be defined in separate .langium files, or in the same .langium file.
|
| This chapter does not cover all the many possibilities offered by the Langium grammar language. For more information, refer to the corresponding page in the official documentation. |
4.1. Defining the abstract syntax using Declared Types
Defining a new concept of the abstract syntax is achieved using the interface keyword:
interface MyConcept {
…
}
| This is similar to a class in an object-oriented model or program, but without operations/methods. |
A concept can contain properties, each with a name and a mandatory type using the : separator:
interface MyConcept {
aPrimitiveProperty: string
aContainmentProperty: OtherType
aCrossReferenceProperty: @AnotherType
}
There are three main types of properties:
-
A primitive property is typed with one of the Langium primitive types:
boolean,number,string,bigintorDate. -
A containment property is typed with another existing concept of the abstract syntax. In an AST, when a node A is linked to a node B using a containment, the node B is said to be contained (or owned) by the node A. In an AST, a node can only be contained by a single container node at a time.
-
A cross-reference property is typed with another existing concept of the abstract syntax, using the character
@as prefix. In an AST, when a node A is linked to a node B using a cross-reference, the node B is said to be referenced (or known) by the node A.
|
A cross-reference from A to B is only possible if:
|
| Apart from the root node, each node of an AST must be contained in another node. |
A property can also define a cardinality using either the optional suffix ? on the property identifier, or the array suffix [] on the type name.
interface MyConcept {
anOptionalProperty?: string
aArrayProperty: number[]
}
-
An optional property means that there can be zero or one value.
-
A array property means that there can be zero or multiple values.
| It is of course possible to combine the different types of properties with the cardinality suffixes to define "optional containments", "arrays of primitives", "arrays of cross-references", etc. |
Finally, a concept can be defined as an subtype of another existing concept using the extends keyword:
interface MyConcept extends OtherConcept {
…
}
This gives the possibility to generalize different concepts by introducing a supertype, which is both useful to define a reference to multiple types, and to avoid redundant properties.
We consider once more the abstract syntax we designed earlier for the State Machines DSL:
We would like to implement this DSL using Langium. We therefore start by implementing this abstract syntax using Langium Declared Types :
interface StateMachine {
name: string
commands: Command[]
states: State[]
events: Event[]
initialState: @State
}
interface Command {
name: string
}
interface State {
name: string
transitions: Transition[]
on_entry: @Command[]
}
interface Event {
name: string
}
interface Transition {
trigger: @Event
target: @State
}
4.2. Defining the concrete syntax
A Langium concrete syntax is defined in the form of a grammar and grammar rules. When a grammar is "executed", it transforms the text of an input program into a structured AST of the same program.
First, a grammar is declared with the keyword grammar:
grammar MyDSL
Another .langium file can be imported using the import keyword, without writing the .langium extension of said file:
import './otherfile' // this imports the file 'otherfile.langium'
| This can for instance be used to import the abstract syntax defined in a separate file. |
A grammar rule can be understood as a function that transforms a specific piece of text of the input program into a node of the output AST.
This node is an instance of a concept of the abstract syntax (specified using the returns keyword).
interface MyConcept { … }
// Parser rule that produces an instance of 'MyConcept'
MyConceptRule returns MyConcept:
…
;
// Terminal rule that recognizes a valid identifier, and outputs it as a string
terminal ID returns string:
/[_a-zA-Z][\w_]*/
;
There are three sorts of grammar rules:
-
A parser rule defines a sequence of tokens that is possible at this location of the input program. From an abstract syntax point of view, executing a parser rule generally produces an instance of a concept.
-
A terminal rule is declared with the
terminalkeyword, and contains a regular expression. It defines a possible kind of lexical token that can be parsed using the grammar, such as an identifier, an arbitrary string, an integer, etc. From an abstract syntax point of view, executing a terminal rule produces the value of a primitive property. -
A hidden terminal rule is declared with the is declared with the
hidden terminalkeywords, and also contains a regular expression. However, instead of defining lexical tokens, it defines which sequences of characters that must be be ignored during parsing. This is typically used to ignore whitespaces and comments.
| Most of the time, we can simply reuse the default terminal rules generated in a new Langium project. |
A token declared in a parser rule can be defined in two ways:
interface MyConcept {
name: string
}
// Parser rule that produces an instance of 'MyConcept'
MyConceptRule returns MyConcept:
'somekeyword' name=ID ';'
;
-
A keyword, written between single quotes, which means that this exact string should be found at this specific location in the program. For example, here both
somekeywordand;are keywords, and mean thatMyConceptRulealways starts withsomekeywordand ends with;. -
A call to an existing grammar rule, which means that this other grammar rule should apply at this location of the program. In addition, calling another grammar rule requires assigning (
=) the output of the called rule into a containment property of the output instance. For example here the ruleIDis expected aftersomekeyword, and the output of the ruleIDis stored in the propertynameof the outputMyConceptinstance.
| By default Langium considers that there can be an arbitrary amount of whites spaces between two tokens. |
It is possible to define repeated tokens in a parser rule:
interface MyConcept {
name: string
children: OtherConcept[]
}
interface OtherConcept {}
MyConceptRule returns MyConcept:
'somekeyword' name=ID (children+=OtherConceptRule)* ';'
;
OtherConceptRule returns OtherConcept: … ;
-
A repetition of a group of tokens is declared with parentheses
()followed by either a star\*or a plus`. This means that, at this location of the program, the enclosed tokens must be found _zero, one or multiple times_ for a `*`, and _one or multiple times_ for a `. -
An assignment in a repetition must be written as
+=, and must mandatorily target an containment array property of the returned concept. For instance herechildrenis a containment array property, therefore we can have a repetition that adds multipleOtherConceptinstances in the property using theOtherConceptRule.
It is possible to define optional tokens in a parser rule:
interface MyConcept {
name: string
isActive: boolean
}
MyConceptRule returns MyConcept:
'somekeyword' (isActive?='active')? name=ID ';'
;
-
A optional group of tokens is declared with parentheses
()followed by a question mark?. This means that the enclosed tokens can be found either zero or one time at this location. -
The presence of an optional token can be stored in a boolean property of the concept using the
=?operator. For instance hereisActiveis a boolean property which istrueif the keywordactiveis found after the keywordsomekeyword, and false otherwise.
It is possible to define alternatives in a group of tokens:
interface MyConcept {
name: string
isActive: boolean
others: OtherConcept[]
anothers: AnotherConcept[]
}
interface OtherConcept {}
interface AnotherConcept {}
MyConceptRule returns MyConcept:
'somekeyword' ( others+=OtherConceptRule | anothers+=AnotherConceptRule )* ';'
;
OtherConceptRule returns OtherConcept: … ;
AnotherConceptRule returns AnotherConcept: … ;
-
Inside any group declared with parenthese
(), the pipe operator|can be used to declare mutually exclusive sequences of tokens. This can be combined with a repetition, meaning that each iteration of the repetition can have multiple possibilities. For example here the repetition will use eitherOtherConceptRuleorAnotherConceptRuleat each iteration, storing the result inothersoranothersrespectively.
It is possible to define a specific kind of token corresponding to a cross-reference of the abstract syntax:
interface MyConcept {
name: string
existingOther: @OtherConcept
}
interface OtherConcept {
name: string
}
MyConceptRule returns MyConcept:
'somekeyword' name=ID existingOther=[OtherConcept:ID] ';'
;
OtherConceptRule returns OtherConcept: …
'otherkeyword' name=ID
;
-
A special cross-reference token can be declared with brackets with the following syntax
<cross-reference>=[<concept>:<terminal rule>]:-
<cross-reference>is the name of a cross-reference property -
<concept>is the name of the concept (and not the parser rule!) referenced by the cross-reference property -
<terminal rule>is the name of the terminal rule that is used to obtain thenameattribute of the referenced concept.
-
For instance here the name of an OtherConcept is obtained using the ID terminal rule.
Accordingly, the value of the existingOther cross-reference is an instance of OtherConcept whose name is written as an ID.
| Writing parser rules can be a difficult task when writing a Langium grammar − and we only cover here a small subset of possibilities! If you want to learn more, the corresponding part in the Langium documentation is very complete. |
Finally, each Langium grammar must have a unique entry parser rule, which is a grammar rule declared with the keyword entry:
entry MyConceptRule returns MyConcept: … ;
An entry rule is the first rule executed when parsing a file with the grammar.
It is comparable to the main function of a program.
We continue the State Machines DSL started previously, and for which we stored the abstract syntax in a file named statemachines-dsl.abstractsyntax.langium, with the following contents:
interface StateMachine {
name: string
commands: Command[]
states: State[]
events: Event[]
initialState: @State
}
interface Command {
name: string
}
interface State {
name: string
transitions: Transition[]
on_entry: @Command[]
}
interface Event {
name: string
}
interface Transition {
trigger: @Event
target: @State
}
We would like to be able to write a state machine program in the following way:
// Declares a StateMachine
statemachine threestates
// Declares the 'events' of the StateMachine
events
e1 e2 e3
// Declares the 'commands' of the StateMachine
commands
c1 c2 c3
// Declares the 'initialState' of the StateMachine
initial S1
// Declares a State
state S1
// Declares the 'onEntry' of the State
on_entry { c1 }
// Declares the 'transitions' of the State
e1 => S2
end
state S2
on_entry { c2 }
e2 => S3
end
state S3
on_entry { c3 }
e3 => S1
end
We therefore write the following grammar (which, as a reminder, relies on the concepts of the previously defined abstract syntax):
grammar StateMachinesDsl
import './statemachines-dsl.abstractsyntax'
entry StateMachineRule returns StateMachine:
'statemachine' name=ID
('events' events+=EventRule+)?
('commands' commands+=CommandRule+)?
'initial' initialState=[State:ID]
states+=StateRule*;
EventRule returns Event:
name=ID;
CommandRule returns Command:
name=ID;
StateRule returns State:
'state' name=ID
('on_entry' '{' on_entry+=[Command:ID]+ '}')?
transitions+=TransitionRule*
'end';
TransitionRule returns Transition:
trigger=[Event:ID] '=>' target=[State:ID];
hidden terminal WS: /\s+/;
terminal ID returns string: /[_a-zA-Z][\w_]*/;
terminal INT returns number: /[0-9]+/;
terminal STRING: /"(\\.|[^"\\])*"|'(\\.|[^'\\])*'/;
hidden terminal ML_COMMENT: /\/\*[\s\S]*?\*\//;
hidden terminal SL_COMMENT: /\/\/[^\n\r]*/;
Using the parser generated from this grammar, we can parse the example State Machines DSL program shown just before, which results in the following AST:
5. The Langium generator
Once a complete Langium grammar has been defined in the packages/language/src folder in the form of one or multiple .langium files, it is time to call the Langium generator to automatically produce data types, an editor and a parser for our software language.
To run the Langium generator, and to compile the generated Typescript code:
npm run langium:generate && npm run build
This mainly automatically produces the following artifacts:
-
packages/language/src/generated/ast.ts: Typescript classes for all concepts of your abstract syntax, along with many utility functions and boilerplate code for an easy manipulation of your AST. -
packages/language/src/generated/grammar.ts: a JSON representation of your grammar, which is then automatically used to configure the Langium parser. -
packages/language/src/generated/module.ts: a Typescript module with metadata (such as the file extension) and pointers to the other language components.
| You will never have to manually edit these files, as they are entirely derived from your Langium code. |
6. Testing a generated editor of a Langium language
If the call to the Langium generator was successful, and if all the Typescript code compiles successfully, it is time to test the language!
Open the "Run and Debug" pane:
Observe that two run configurations have been prepared by Yeoman when you initialized your project:
-
"Run Extension" allows you to test your language by starting a second Visual Studio Code window in which your language in developpment will be installed as a Visual Studio Code extension.
-
"Attach to Language Server " allows you to debug your language implementation by attaching a TypeScript debugger to an already running extension of your language (requires doing Run Extension first).
When you use "Run Extension", a second Visual Studio Code window appears in an empty workspace. In this empty workspace, create a new file using the file extension of your Langium language (important). This will open the customized editor for your language:
7. Implementing a code generator for a Langium language
Using the Langium grammar language and the Langium generator, we can efficiently obtain software components that implement both the abstract syntax and the concrete syntax of a software language. But how can we implement the semantics of a Langium-based software language?
We have seen earlier that the semantics of a software language can be implemented either as a code generator or as an AST interpreter. In this section we focus on the case of code generators. More specifically, we consider the case of a code generator whose target language is a GPL such as Java or C, or a markup language such as HTML or Markdown.
A code generator must accomplish two main types of tasks:
-
Visiting the AST of the program being compiled, which means going over all nodes of the AST. Programming this visit is achieved by the abstract syntax data types that are part of the language implementation (the
ast.tsfile in Langium). -
Producing output code by constructing a string containing the code to be generated. The contents of this output string depends on the contents discovered during the visit of the AST. Constructing such string is commonly achieved using string templates.
7.1. Visiting the AST with Langium
Visiting an AST using Langium is achieved using the TypeScript types generated from the abstract syntax.
These types are found in the generated ast.ts file, an can be used in an object oriented fashion.
For example, given the following abstract syntax in a Langium grammar:
interface Formular {
name: string
fields: MultipleChoiceField[]
}
interface MultipleChoiceField {
name: string
choices: Choice[]
default: @Choice
}
interface Choice {
name: string[]
}
Then we call the Langium generator to produce an ast.ts file containing all the abstract syntax data types.
The ast.ts file is generated in the language package of the Langium project, and can therefore be imported like so:
import type { Formular, MultipleChoiceField, Choice }
from 'formular-language';
where:
-
Formular,MultipleChoiceField,Choiceare concepts of the abtract syntax that we need to import, -
formular-languageis the name of the package for the DSL calledformular.
Going back to our visitor, this ast.ts file can therefore be used in the following way:
// Import the abstract syntax data types
import type { Formular, MultipleChoiceField, Choice }
from 'formular-language';
// Define a function to visit a 'Formular' node
function visitFormular(formular: Formular) {
console.log("Visiting a Formular with name: " + formularASTRoot.name)
for (let multipleChoiceField of formular.fields) {
visitMultipleChoiceField(multipleChoiceField)
}
}
// Define a function to visit a 'MultipleChoiceField' node
function visitMultipleChoiceField(multipleChoiceField: MultipleChoiceField) {
console.log("Visiting a MultipleChoiceField with name: "
+ multipleChoiceField.name)
console.log("and with default choice: "
+ multipleChoiceField.default.ref!.name)
for (let choice of multipleChoiceField.choices) {
visitChoice(choice)
}
}
// Define a function to visit a 'Choice' node
function visitChoice(choice : Choice) {
console.log("Visiting a Choice with name: " + choice.name)
console.log("and contained in MultipleChoiceField: " + choice.$container.name)
}
// Obtain the root of the AST produced by the parser, a 'Formular' node
const formularASTRoot : Formular = …
// Call visit function
visitFormular(formularASTRoot)
-
visitFormularis a function that visits aFormularnode, and that then trigger the visit of all enclosedTextFieldnodes using thevisitTextFieldfunction, -
visitMultipleChoiceFieldis a function that visits aMultipleChoiceFieldnode, that then trigger the visit of all enclosedChoicenodes using thevisitChoicefunction,-
in the expression
multipleChoiceField.defaultChoice.ref!.name, therefattribute is required to resolve a cross-reference property (here thedefaultChoicecross-reference),
-
-
visitChoiceis a function that visits aChoicenode,-
in the expression
choice.$container.namethe$containerattribute is used to access the parent node (here the enclosingMultipleChoiceField)
-
| While we achieve a similar goal, we are not strictly speaking following the visitor design pattern, because the code here is not object-oriented, and because it mixes the algorithm that perform the navigation between node, and the code that performs actions when visiting a node. |
Then if we call this piece of code on the following AST:
We obtain the following output:
Visiting a Formular with name: Fancy formular
Visiting a MultipleChoiceField with name: Choose color
and with default choice: red
Visiting a Choice with name: red
and contained in MultipleChoiceField: Choose color
Visiting a Choice with name: blue
and contained in MultipleChoiceField: Choose color
Visiting a MultipleChoiceField with name: Choose animal
and with default choice: cat
Visiting a Choice with name: cat
and contained in MultipleChoiceField: Choose animal
Visiting a Choice with name: dog
and contained in MultipleChoiceField: Choose animal
We write the following sample visit code for the State Machines DSL:
import type { StateMachine, Command, Event, State, Transition }
from 'state-machines-language';
function visitStateMachine(stateMachine: StateMachine) {
console.log("Visiting state machine: " + stateMachine.name)
console.log("with initial state: " + stateMachine.initialState.ref!.name)
for (let command of stateMachine.commands) {
visitCommand(command)
}
for (let event of stateMachine.events) {
visitEvent(event)
}
for (let state of stateMachine.states) {
visitState(state)
}
}
function visitCommand(command: Command) {
console.log("Visiting command: " + command.name)
}
function visitEvent(event: Event) {
console.log("Visiting event: " + event.name)
}
function visitState(state: State) {
console.log("Visiting state: " + state.name)
for (let transition of state.transitions) {
visitTransition(transition)
}
}
function visitTransition(transition: Transition) {
console.log("Visiting transition contained in state: "
+ transition.$container.name)
}
If we call visitStateMachine on the following AST (shown before):
Then we obtain the following output:
Visiting state machine: threestates
with initial state: S1
Visiting command: c1
Visiting command: c2
Visiting command: c3
Visiting event: e1
Visiting event: e2
Visiting event: e3
Visiting state: s1
Visiting transition contained in state: s1
Visiting state: s2
Visiting transition contained in state: s2
Visiting state: s3
Visiting transition contained in state: s3
7.2. Producing output code using string templates
The goal when visiting an AST is to generate some output code.
This requires producing a very large string using some form of string concatenation.
Because string concatenation can be quite verbose and annoying to write (with lots of expressions like "first" + "second"), code generation is better achieved using string templates.
A string template is a string expression where it is possible to directly embed variables or function calls within a string. For example in TypeScript:
let name: string = "Georges"
let output: string = `Hello ${name}!`
console.log(output)
produces:
Hello Georges!
Here:
-
nameis a variable, -
outputis a string we construct using a string template, which is declared using backticks. In the template we use the placeholder syntax${…}to write a valid string expression − here we simply use the availablenamevariable.
Another benefit of string templates is that they can be written on multiple lines:
let name: string = "Georges"
let city: string = "Nantes"
let output: string = `
Hello ${name}
from ${city}!
`
console.log(output)
produces:
Hello Georges
from Nantes!
We can put any sort of expression in a placeholder ${…}, including function calls, which enables code like this:
function generateHello(name: string): string {
return `Hello ${name}`
}
function generateFrom(city: string): string {
return `from ${city}`
}
let name: string = "Georges"
let city: string = "Nantes"
let output: string = `
${generateHello(name)}
${generateFrom(city)}!
`
console.log(output)
which produces:
Hello Georges
from Nantes!
Unfortunately, it is not possible to declare a for loop within a template in order to display a collection of values.
However a good alternative is to combine on the map and join operators available in TypeScript.
For example:
type Person = {
name: string
city: string
}
function generateHelloFromPerson(person: Person): string {
return `
Hello ${person.name}
from ${person.city}!
`
}
let manyPersons: Person[] = [
{ name: "Georges", city: "Nantes" },
{ name: "Julia", city: "Bordeaux" }
]
let output: string = `
Let's welcome everyone:
${manyPersons.map(person => generateHelloFromPerson(person)).join("\n\n")}
`
console.log(output)
which produces:
Let's welcome everyone:
Hello Georges
from Nantes!
Hello Julia
from Bordeaux!
Using string templates and visit functions, we would like to implement a code generator for the State Machines DSL. The idea is to be able to generate, from a valid state machine, a Java class which roughly follows the state design pattern.
For example the following state machine:
// Declares a StateMachine
statemachine threestates
// Declares the 'events' of the StateMachine
events
e1 e2 e3
// Declares the 'commands' of the StateMachine
commands
c1 c2 c3
// Declares the 'initialState' of the StateMachine
initial S1
// Declares a State
state S1
// Declares the 'onEntry' of the State
on_entry { c1 }
// Declares the 'transitions' of the State
e1 => S2
end
state S2
on_entry { c2 }
e2 => S3
end
state S3
on_entry { c3 }
e3 => S1
end
Which can be parsed in the following AST:
Should result in the following Java code:
package threestates;
public class threestates {
State currentState = new S1();
public static void c1() {
System.out.println("Executing command c1");
// TODO command provide implementation
}
public static void c2() {
System.out.println("Executing command c2");
// TODO command provide implementation
}
public static void c3() {
System.out.println("Executing command c3");
// TODO command provide implementation
}
public void processEvent(Event event) {
System.out.println("Processing event " + event);
switch (event) {
case e1:
this.currentState = this.currentState.e1();
break;
case e2:
this.currentState = this.currentState.e2();
break;
case e3:
this.currentState = this.currentState.e3();
break;
}
System.out.println("Current state: " +
this.currentState.getClass().getSimpleName());
if (currentState != null) {
currentState.entryCommands();
}
}
public enum Event {
e1, e2, e3
}
interface State {
State e1();
State e2();
State e3();
void entryCommands();
}
class S1 implements State {
@Override
public State e1() {
return new S2();
}
@Override
public State e2() {
return this;
}
@Override
public State e3() {
return this;
}
@Override
public void entryCommands() {
threestates.c1();
}
}
class S2 implements State {
@Override
public State e1() {
return this;
}
@Override
public State e2() {
return new S3();
}
@Override
public State e3() {
return this;
}
@Override
public void entryCommands() {
threestates.c2();
}
}
class S3 implements State {
@Override
public State e1() {
return this;
}
@Override
public State e2() {
return this;
}
@Override
public State e3() {
return new S1();
}
@Override
public void entryCommands() {
threestates.c3();
}
}
}
We can achieve this result with the following code generator:
import type { Command, Event, State, Transition, StateMachine } from 'state-machines-language';
function generateStateMachineCode(stateMachine: StateMachine): string {
return `
package ${stateMachine.name};
public class ${stateMachine.name} {
State currentState = new ${stateMachine.initialState.ref?.name}();
${stateMachine.commands.map(command =>
generateCommandDeclarationCode(command)).join("\n")}
public void processEvent(Event event) {
System.out.println("Processing event " + event);
switch (event) {
${stateMachine.events.map(event =>
generateStateMachineEventHandler(event)).join("\n")}
}
System.out.println("Current state: " +
this.currentState.getClass().getSimpleName());
if (currentState != null) {
currentState.entryCommands();
}
}
public enum Event {
${stateMachine.events.map(event => event.name).join(",")}
}
interface State {
${stateMachine.events.map(event =>
generateEventDeclarationCode(event)).join("\n")}
void entryCommands();
}
${stateMachine.states.map(state => generateStateCode(state)).join("\n")}
}
`
}
function generateStateCode(state: State): string {
return `
class ${state.name} implements State {
${state.$container.events.map(event =>
generateStateEventHandlerCode(state, event)).join("\n")}
@Override
public void entryCommands() {
${state.on_entry.map(commandRef =>
generateCommandCall(commandRef.ref!)).join("\n")}
}
}
`
}
function generateCommandDeclarationCode(command: Command): string {
return `
public static void ${command.name}() {
System.out.println("Executing command ${command.name}");
//TODO command provide implementation
}
`
}
function generateEventDeclarationCode(event: Event): string {
return `
State ${event.name}();
`
}
function generateStateEventHandlerCode(state: State, event: Event): string {
return `
@Override
public State ${event.name}() {
return ${generateEventHandlerReturn(state, event)};
}
`
}
function generateEventHandlerReturn(state: State, event: Event) {
let targetState : State = findTargetState(state, event);
if (targetState === state){
return "this"
} else {
return `new ${state.name}()`
}
}
function findTargetState(state: State, event: Event): State {
const candidateTransitions: Transition[] =
state.transitions.filter(transition => transition.trigger.ref! === event)
if (candidateTransitions.length > 1) {
throw "Generation error: cannot process a non deterministic state machine"
} else if (candidateTransitions.length == 0) {
return state
} else {
return candidateTransitions.pop()?.target.ref!
}
}
function generateStateMachineEventHandler(event: Event): string {
return `
case ${event.name}:
this.currentState = this.currentState.${event.name}();
break;
`
}
function generateCommandCall(command: Command): string {
return `
${command.$container.name}.${command.name}();
`
}
7.3. Integrating the code generator in the CLI
| In this section we assume that, when your Langium project was generated using Yeoman, you chose to not generate an example project. |
Once the code generator code has been implemented for a given Langium language, the last step is to provide a user interface to make the code generator available to language users. In Langium by default this is achieved by implementing a generate command in the Command Line Interface (CLI) of the language.
A Langium project freshly created using Yeoman should have the following files:
-
packages/cli/src/main.tscontains the entry point of the CLI, and should include agenerateActionfunction. This function is called when thegeneratekeyword is given to the CLI, and it accomplishes two important tasks:-
extractAstNodeis the function that will call your parser in order to produce an AST, that is then stored in themodelvarible, -
generateOutputis the main function of your code generator provided in thepackages/cli/src/generator.tsfile.
-
-
packages/cli/src/generator.tscontains the code generator itself, and in particular a main function calledgenerateOutput.
In order to integrate your code generator in this architecture, put your code generator in the packages/cli/src/generator.ts file, and call your code generation functions from the generateOutput function.
Change the generateOutput as much as required to create folders as needed by your code generation logic.
When npm run build is executed, this produces the ./packages/cli/bin/cli.js containing the CLI, which can be tested by simply calling:
node ./packages/cli/bin/cli.js generate <path to program to compile>
You can also make your CLI available globally in your system by running the npm link command in your project folder.
This should create a command <language name>-cli available in your console, which is a symbolic link to the ./packages/cli/bin/cli.js file of your language.
|
The generate command, by default, will put produced files in the generated folder.
For the State Machines DSL, we start in the packages/cli/src/generator.ts, and we look at the generateOutput function.
Then we modify this function to properly create the Java package folder required for the generated Java class, and to prepare the path to the generated Java file, before calling the code generator:
export function generateOutput(model: StateMachine, source: string, destination: string): string {
// Prepare generation paths
const data = extractDestinationAndName(destination);
const generatedJavaPackageName = model.name
const generatedJavaFileName = `${model.name}.java`
const generatedFolderPath = path.join(data.destination,
generatedJavaPackageName);
const generatedFilePath = path.join(generatedFolderPath,
generatedJavaFileName)
// Call the code generator
const fileContents = generateStateMachineCode(model)
// Write the generated code in a file
fs.mkdirSync(generatedFolderPath, { recursive: true });
fs.writeFileSync(generatedFilePath, fileContents);
return generatedFilePath;
}
We then run npm run build and run our code generator with the CLI :
node ./packages/cli/bin/cli.js generate examples/threestates.sm
which produces the file generated/threestates/threestates.java.
8. References
-
🔗 Langium official documentation, by TypeFox (accessed on 11/2025)