📘 Ch5. Semantics with code generation

The semantics of a software language defines what happens when a program/model of the language is compiled and/or executed. Without a semantics, a language is purely syntactic and cannot be used to perform anything tangible.

In this chapter we focus on code generation semantics. We will first learn how to design a semantics using annotated mockups, then we will learn to implement an abstract syntax in a Langium DSL.

🚧 This chapter is still under construction.

1. Foreword

1.1. Types of semantics

There is a large variety of semantics for software languages, depending on its purpose and context of use. We can distinguish different types of semantics, such as:

Execution semantics are for languages whose programs/models can be executed to produce some form of computation or effect. An execution is defined by some runtime state in memory that changes over time (for example the current statement of a program), and optionnaly by side effects that occur during the execution (for example creating/modifying a file or sending messages over the network).
- Examples: any GPL such as Java, UML activity diagrams, Logo.
Structural semantics are for languages that are used to describe a structure or organisation of a system. Models of such languages cannot be "executed" and can instead be used as a structural dependency for another model (for example a UML activity diagram can use the data types of a UML class diagram), or can be purely be used as a blueprint for the engineering of a system (for example a UML component diagam can be used as a blueprint for manually implementing a software system).
- Examples: UML class/package/component diagrams
Visualization semantics are for languages that are used to display some graphical or textual information, typically a document or an interactive media. Such semantics describe how should some information visually appear and how it should be distributed in space.
- Examples: HTML, CSS, Markdown

In this chapter we only see examples of execution semantics, but we will later see an example of structural semantics in an exercise.

1.2. Interpretation VS code generation

We have seen in 📘 Ch2. Principles of SLE that there are two main approaches to implement a semantics: either in the form of an AST interpreter (the language implementation is then called an interpreter), or in the form of of a code generator (the language implementation is then called a compiler).

While building an interpreter is usually considered the simplest and cheapest option to implement the semantics of a DSL, it very often fails to address many important concerns, in particular:

It is technically difficult to obtain good performance such as speed and low memory consumption.
The programming language chosen to implement an AST interpreter will constrain what software librairies and tools can be reused in the interepter’s code.
If the DSL is used in a context where it must be used jointly with more traditionnal GPL code, an integration problem can appear to allow a GPL program to call a DSL program (or the opposite).

For this reason we focus solely in this chapter on how to design and engineer a semantics in the form of a code generator, which allows the execution to be performed by lower-level code, and thus to rely on the performance of the target software language. Code generation also allows generating code in a convenient target language, for example to be more easily integrated with manually written code.

2. Designing a semantics

2.1. Natural language description

Before diving into code generation, an important first step is to clarify as much as possible the meaning of a program/model of the DSL we are making. This can be simply achieved with natural language (such as french or english).

A pragmatic way to describe a semantics is to use an algorithmic style − which can even include pseudo-code − that explains how the elements of an AST are iterated over, and what effects occurs during this iteration.

TODO .Natural language description of the semantics of a tiny programming DSL

Example 1. Natural language description of the semantics of a Petri nets DSL

TODO

2.2. Reference example implementation

TODO

Example 2. Reference example implementation of a tiny programming DSL program

TODO

Example 3. Reference example implementation of a Petri nets DSL program

TODO

2.3. Mapping a reference example with an abtract syntax

TODO

Example 4. Abstract syntax mapping of a reference example implementation of a tiny programming DSL program

TODO

Example 5. Abstract syntax mapping of a reference example implementation of a Petri nets DSL program

TODO

2.4. Good practices

TODO

3. Implementation of a code generation semantics

TODO

Using the Langium Grammar Language and the Langium generator, we can efficiently obtain software components that implement both the abstract syntax and the concrete syntax of a software language. But how can we implement the semantics of a Langium-based software language?

3.1. Visiting an AST

Visiting an AST using Langium is achieved using the TypeScript types generated from the abstract syntax. These types are found in the generated ast.ts file, an can be used in an object oriented fashion.

For example, given the following abstract syntax in a Langium grammar:

interface Formular {
    name: string
    fields: MultipleChoiceField[]
}

interface MultipleChoiceField {
    name: string
    choices: Choice[]
    default: @Choice
}

interface Choice {
    name: string[]
}

Then we call the Langium generator to produce an ast.ts file containing all the abstract syntax data types.

The ast.ts file is generated in the language package of the Langium project, and can therefore be imported like so:

import type { Formular, MultipleChoiceField, Choice }
                            from 'formular-language';

where:

Formular, MultipleChoiceField, Choice are concepts of the abtract syntax that we need to import,
formular-language is the name of the package for the DSL called formular.

Going back to our visitor, this ast.ts file can therefore be used in the following way:

// Import the abstract syntax data types
import type { Formular, MultipleChoiceField, Choice }
                            from 'formular-language';

// Define a function to visit a 'Formular' node
function visitFormular(formular: Formular) {
    console.log("Visiting a Formular with name: " + formularASTRoot.name)

    for (let multipleChoiceField of formular.fields) {
        visitMultipleChoiceField(multipleChoiceField)
    }
}

// Define a function to visit a 'MultipleChoiceField' node
function visitMultipleChoiceField(multipleChoiceField: MultipleChoiceField) {
    console.log("Visiting a MultipleChoiceField with name: "
                                            + multipleChoiceField.name)
    console.log("and with default choice: "
                                + multipleChoiceField.default.ref!.name)

    for (let choice of multipleChoiceField.choices) {
        visitChoice(choice)
    }
}

// Define a function to visit a 'Choice' node
function visitChoice(choice : Choice) {
    console.log("Visiting a Choice with name: " + choice.name)
    console.log("and contained in MultipleChoiceField: " + choice.$container.name)
}

// Obtain the root of the AST produced by the parser, a 'Formular' node
const formularASTRoot : Formular = …

// Call visit function
visitFormular(formularASTRoot)

visitFormular is a function that visits a Formular node, and that then trigger the visit of all enclosed TextField nodes using the visitTextField function,
visitMultipleChoiceField is a function that visits a MultipleChoiceField node, that then trigger the visit of all enclosed Choice nodes using the visitChoice function,
- in the expression multipleChoiceField.defaultChoice.ref!.name, the ref attribute is required to resolve a cross-reference property (here the defaultChoice cross-reference),
visitChoice is a function that visits a Choice node,
- in the expression choice.$container.name the $container attribute is used to access the parent node (here the enclosing MultipleChoiceField)

While we achieve a similar goal, we are not strictly speaking following the visitor design pattern, because the code here is not object-oriented, and because it mixes the algorithm that perform the navigation between node, and the code that performs actions when visiting a node.

Then if we call this piece of code on the following AST:

We obtain the following output:

Visiting a Formular with name: Fancy formular
Visiting a MultipleChoiceField with name: Choose color
and with default choice: red
Visiting a Choice with name: red
and contained in MultipleChoiceField: Choose color
Visiting a Choice with name: blue
and contained in MultipleChoiceField: Choose color
Visiting a MultipleChoiceField with name: Choose animal
and with default choice: cat
Visiting a Choice with name: cat
and contained in MultipleChoiceField: Choose animal
Visiting a Choice with name: dog
and contained in MultipleChoiceField: Choose animal

Example 6. AST visit for the State Machines DSL

We write the following sample visit code for the State Machines DSL:

import type { StateMachine, Command, Event, State, Transition }
                                from 'state-machines-language';

function visitStateMachine(stateMachine: StateMachine) {
    console.log("Visiting state machine: " + stateMachine.name)
    console.log("with initial state: " + stateMachine.initialState.ref!.name)

    for (let command of stateMachine.commands) {
        visitCommand(command)
    }

    for (let event of stateMachine.events) {
        visitEvent(event)
    }

    for (let state of stateMachine.states) {
        visitState(state)
    }

}
function visitCommand(command: Command) {
    console.log("Visiting command: " + command.name)
}

function visitEvent(event: Event) {
    console.log("Visiting event: " + event.name)
}

function visitState(state: State) {
    console.log("Visiting state: " + state.name)

    for (let transition of state.transitions) {
        visitTransition(transition)
    }
}

function visitTransition(transition: Transition) {
    console.log("Visiting transition contained in state: "
                                + transition.$container.name)
}

If we call visitStateMachine on the following AST (shown before):

Then we obtain the following output:

Visiting state machine: threestates
with initial state: S1
Visiting command: c1
Visiting command: c2
Visiting command: c3
Visiting event: e1
Visiting event: e2
Visiting event: e3
Visiting state: s1
Visiting transition contained in state: s1
Visiting state: s2
Visiting transition contained in state: s2
Visiting state: s3
Visiting transition contained in state: s3

3.2. Implementing a code generator

We have seen earlier that the semantics of a software language can be implemented either as a code generator or as an AST interpreter. In this section we focus on the case of code generators. More specifically, we consider the case of a code generator whose target language is a GPL such as Java or C, or a markup language such as HTML or Markdown.

A code generator must accomplish two main types of tasks:

Visiting the AST of the program being compiled, which means going over all nodes of the AST. Programming this visit is achieved by the abstract syntax data types that are part of the language implementation (the ast.ts file in Langium).
Producing output code by constructing a string containing the code to be generated. The contents of this output string depends on the contents discovered during the visit of the AST. Constructing such string is commonly achieved using string templates.

3.3. Implementing an AST interpreter

TODO

3.3.1. Producing output code using string templates

The goal when visiting an AST is to generate some output code. This requires producing a very large string using some form of string concatenation. Because string concatenation can be quite verbose and annoying to write (with lots of expressions like "first" + "second"), code generation is better achieved using string templates.

A string template is a string expression where it is possible to directly embed variables or function calls within a string. For example in TypeScript:

let name: string = "Georges"
let output: string = `Hello ${name}!`

console.log(output)

produces:

Hello Georges!

Here:

name is a variable,
output is a string we construct using a string template, which is declared using backticks. In the template we use the placeholder syntax ${…} to write a valid string expression − here we simply use the available name variable.

Another benefit of string templates is that they can be written on multiple lines:

let name: string = "Georges"
let city: string = "Nantes"

let output: string = `
    Hello ${name}
    from ${city}!
`

console.log(output)

produces:

Hello Georges
from Nantes!

We can put any sort of expression in a placeholder ${…}, including function calls, which enables code like this:

function generateHello(name: string): string {
    return `Hello ${name}`
}

function generateFrom(city: string): string {
    return `from ${city}`
}

let name: string = "Georges"
let city: string = "Nantes"

let output: string = `
    ${generateHello(name)}
    ${generateFrom(city)}!
`

console.log(output)

which produces:

Hello Georges
from Nantes!

Unfortunately, it is not possible to declare a for loop within a template in order to display a collection of values. However a good alternative is to combine on the map and join operators available in TypeScript. For example:

type Person = {
    name: string
    city: string
}

function generateHelloFromPerson(person: Person): string {
    return `
        Hello ${person.name}
        from ${person.city}!
    `
}

let manyPersons: Person[] = [
    { name: "Georges", city: "Nantes" },
    { name: "Julia", city: "Bordeaux" }
]

let output: string = `
    Let's welcome everyone:

    ${manyPersons.map(person => generateHelloFromPerson(person)).join("\n\n")}
    `

console.log(output)

which produces:

    Let's welcome everyone:

        Hello Georges
        from Nantes!

        Hello Julia
        from Bordeaux!

Example 7. Code generator using string templates and visit functions for the State Machines DSL

Using string templates and visit functions, we would like to implement a code generator for the State Machines DSL. The idea is to be able to generate, from a valid state machine, a Java class which roughly follows the state design pattern.

For example the following state machine:

statemachine threestates

events:
    e1, e2, e3

commands:
    c1, c2, c3

initial: S1

state S1:
    on_entry: c1
    e1 => S2
end

state S2:
    on_entry: c2
    e2 => S3
end

state S3:
    on_entry: c3
    e3 => S1
end

Which can be parsed in the following AST:

Should result in the following Java code:

package threestates;

public class threestates {

    State currentState = new S1();

    public static void c1() {
        System.out.println("Executing command c1");
        // TODO command provide implementation
    }

    public static void c2() {
        System.out.println("Executing command c2");
        // TODO command provide implementation
    }

    public static void c3() {
        System.out.println("Executing command c3");
        // TODO command provide implementation
    }

    public void processEvent(Event event) {
        System.out.println("Processing event " + event);
        switch (event) {

            case e1:
                this.currentState = this.currentState.e1();
                break;
            case e2:
                this.currentState = this.currentState.e2();
                break;
            case e3:
                this.currentState = this.currentState.e3();
                break;
        }
        System.out.println("Current state: " + 
                        this.currentState.getClass().getSimpleName());

        if (currentState != null) {
            currentState.entryCommands();
        }
    }

    public enum Event {
        e1, e2, e3
    }

    interface State {
        State e1();
        State e2();
        State e3();
        void entryCommands();
    }

    class S1 implements State {

        @Override
        public State e1() {
            return new S2();
        }

        @Override
        public State e2() {
            return this;
        }

        @Override
        public State e3() {
            return this;
        }

        @Override
        public void entryCommands() {
            threestates.c1();
        }

    }

    class S2 implements State {

        @Override
        public State e1() {
            return this;
        }

        @Override
        public State e2() {
            return new S3();
        }

        @Override
        public State e3() {
            return this;
        }

        @Override
        public void entryCommands() {
            threestates.c2();
        }

    }

    class S3 implements State {

        @Override
        public State e1() {
            return this;
        }

        @Override
        public State e2() {
            return this;
        }

        @Override
        public State e3() {
            return new S1();
        }

        @Override
        public void entryCommands() {
            threestates.c3();
        }
    }
}

We can achieve this result with the following code generator:

import { StateMachine, Event, Command, State, Transition } from "./generated/ast.js"

export function generateStateMachineCode(stateMachine: StateMachine): string {
    return `
    package ${stateMachine.name};

    public class ${stateMachine.name} {
    
        State currentState = new ${stateMachine.initialState.ref?.name}();

        ${stateMachine.commands.map(command => generateCommandDeclarationCode(command)).join("\n")}   

        public void processEvent(Event event) {
            System.out.println("Processing event " + event);
            switch (event) {
                ${stateMachine.events.map(event => generateStateMachineEventHandler(event)).join("\n")}
            }
            System.out.println("Current state: " + this.currentState.getClass().getSimpleName());

            if (currentState != null) {
                currentState.entryCommands();
            }
        }

        public enum Event {
            ${stateMachine.events.map(event => event.name).join(",")}
        }

       
        interface State {
            ${stateMachine.events.map(event => generateEventDeclarationCode(event)).join("\n")}
            void entryCommands();
        }

        ${stateMachine.states.map(state => generateStateCode(state)).join("\n")}

    }
    `

}

function generateStateCode(state: State): string {
    return `

        class ${state.name} implements State {

            ${state.$container.events.map(event => generateStateEventHandlerCode(state, event)).join("\n")}

            @Override
            public void entryCommands() {
                ${state.on_entry.map(commandRef => generateCommandCall(commandRef.ref!)).join("\n")}
            }
            
        }
    `
}

function generateCommandDeclarationCode(command: Command): string {
    return `
        public static void ${command.name}() {
            System.out.println("Executing command ${command.name}");
            //TODO command provide implementation
        }
    `
}

function generateEventDeclarationCode(event: Event): string {
    return `
        State ${event.name}();
    `
}

function generateStateEventHandlerCode(state: State, event: Event): string {
    return `
        @Override
        public State ${event.name}() {
            return ${generateEventHandlerReturn(state, event)};
        }
    `
}

function generateEventHandlerReturn(state: State, event: Event) {
    let targetState : State = findTargetState(state, event);
    if (targetState === state){
        return "this"
    } else {
        return `new ${targetState.name}()`
    }
}

function findTargetState(state: State, event: Event): State {
    const candidateTransitions: Transition[] = state.transitions.filter(transition => transition.trigger.ref! === event)
    if (candidateTransitions.length > 1) {
        throw "Generation error: cannot process a non deterministic state machine"
    } else if (candidateTransitions.length == 0) {
        return state
    } else {
        return candidateTransitions.pop()?.target.ref!
    }
}


function generateStateMachineEventHandler(event: Event): string {
    return `
        case ${event.name}:
            this.currentState = this.currentState.${event.name}();
            break;
    `
}

function generateCommandCall(command: Command): string {
    return `
        ${command.$container.name}.${command.name}();
    `
}

3.3.2. Integrating the code generator in the CLI

In this section we assume that, when your Langium project was generated using Yeoman, you chose to not generate an example project.

Once the code generator code has been implemented for a given Langium language, the last step is to provide a user interface to make the code generator available to language users. In Langium by default this is achieved by implementing a generate command in the Command Line Interface (CLI) of the language.

A Langium project freshly created using Yeoman should have the following files:

packages/cli/src/main.ts contains the entry point of the CLI, and should include a generateAction function. This function is called when the generate keyword is given to the CLI, and it accomplishes two important tasks:
- extractAstNode is the function that will call your parser in order to produce an AST, that is then stored in the model varible,
- generateOutput is the main function of your code generator provided in the packages/cli/src/generator.ts file.
packages/cli/src/generator.ts contains the code generator itself, and in particular a main function called generateOutput.

In order to integrate your code generator in this architecture, put your code generator in the packages/cli/src/generator.ts file, and call your code generation functions from the generateOutput function. Change the generateOutput as much as required to create folders as needed by your code generation logic.

Example 8. Integration of the State Machines DSL code generator in the CLI

For the State Machines DSL, we start in the packages/cli/src/generator.ts, and we look at the generateOutput function. Then we modify this function to properly create the Java package folder required for the generated Java class, and to prepare the path to the generated Java file, before calling the code generator:

export function generateOutput(model: StateMachine, source: string, destination: string): string {
    // Prepare generation paths
    const data = extractDestinationAndName(destination);
    const generatedJavaPackageName = model.name
    const generatedJavaFileName = `${model.name}.java`
    const generatedFolderPath = path.join(data.destination,
                                            generatedJavaPackageName);
    const generatedFilePath = path.join(generatedFolderPath,
                                            generatedJavaFileName)

    // Call the code generator
    const fileContents = generateStateMachineCode(model)

    // Write the generated code in a file
    fs.mkdirSync(generatedFolderPath, { recursive: true });
    fs.writeFileSync(generatedFilePath, fileContents);
    return generatedFilePath;
}

We then run npm run build and run our code generator with the CLI :

node ./packages/cli/bin/cli.js generate examples/threestates.sm

which produces the file generated/threestates/threestates.java.

4. References

🔗 Langium official documentation, by TypeFox (accessed on 11/2025)