📘 Ch1. Introduction and motivation
1. Abstractions: the heart and soul of software engineering
While initially software was written in the form of binary code or assembly code that computers could directly understood, it became rapidly necessary to write software using reusable abstractions (such as loops, conditionals, functions) that generalized common redundant patterns found in code. Not only these abstractions were a way to avoid redundancies, but they also started providing higher-level concepts and ideas adapted to the mind of software engineers, and to the problems they needed to solve. This eventually led to the birth of general-purpose programming languages (GPLs) in their modern form, such as Pascal, Fortran, C, Java, and many others.
In this chapter, we understand the word abstraction as the act of identifying common characteristics of different entities, and defining these characteristics as proper self-reliant concepts. Or, as written on the Wiktionary:
The act of comparing commonality between distinct objects and organizing using those similarities; the act of generalizing characteristics; the product of said generalization.
Wiktionary (accessed on 16/11/2024)
While GPLs are key abstractions used to build software, the whole field of software engineering nowadays continuously relies on the definition of suitable and efficient new abstractions to solve always more complex problems. This abstraction-centered engineering process can be summarized as follows:
-
On the right, a given set of problems is depicted. The size of a set of problems can greatly vary, ranging from "writing data structures and algorithms to be executed on computers" (the very large set of problems addressed by GPLs), to "writing Java test cases" (the smaller set of problems addressed by the JUnit framework), or to something even more domain-specific such as "exchanging instant messages over the network" (problems addressed by the Matrix communication protocol).
-
On the left, the abstraction engineer is a software engineer that analyzes a given set of problems, and produces a set of reusable abstractions as a result (such as a software language, library, framework, or protocol). These abstractions are made of meticulously identified concepts fitting to solve a given set of problems. Consequently, engineering abstractions requires a very good understanding of the domain of expertise related to the set of problems.
-
At the bottom, the abstraction user is a software engineer that take advantage of existing abstractions in order to build a solution (typically a software model or a program) to an specific problem. For the abstractions to be useful, the problem at stake must be part of the set of problems targeted by the abstractions.
| In 99% of cases, defining new abstractions requires pre-existing abstractions! For example, implementing a new software library is achieved using an existing software language and using other existing libraries (called dependencies). Consequently, the abstraction engineer is always also an abstraction user. |
Abstractions can take many forms in the great wild of software engineering, such as:
-
Operating systems (OSs) and device drivers provide abstractions to manage hardware resources. For instance, a software application can use the file system abstraction provided by an OS to interact with a storage device.
-
Communication protocols provide abstractions allowing two or more entities to exchange information. For instance, the HTTP protocol provide abstractions (
GET,POST,PUT, etc.). to interact with a web server. -
An Application programming interface (API) provides a set of abstractions to programmatically interact with an existing piece of software (a server, a library, an application, etc.). This allows such piece of software to be used by other code without exposing any of its internals.
-
A software library is a collection of abstractions in the form of reusable data structures and functions targeting a particular computing problem, and that can serve to create new software. A software framework is a specific sort of library where parts of the library are extensible and can be redefined for a specific use.
-
Last but not least, software languages (which include both programming languages and modeling languages) are fundamental abstractions used to design and build software. We focus on this unique category of abstractions in the following section.
2. Software languages as powerful (but costly) abstractions
Software languages can come in many different shapes and forms:
-
General-purpose programming languages (GPLs) (Python, Java, C++, etc.) are the best known software languages. They provide all commonly expected abstractions to write algorithms and data structures. They are used to write programs for the vast majority of software today. Being general-purpose, these languages are often quite large and complex, and therefore require a lot of engineering effort for their creation and maintenance. Their lifespan is usually very long, and their user base very large.
-
A modeling language (UML, SysML, AADL, etc.) is a software language commonly used to design software and systems. They often provide high-level abstractions specialized for specific concerns or processes (for instance AADL focuses on architecture design). While some modeling languages have precise execution semantics (for example Petri nets and Statecharts) and can be executed as programming languages, other modeling languages do not have semantics precise enough to be executable (for example UML). Their notation is often based on graphical shapes, arrows, and diagrams, making them easy to use on drawing boards.
-
A domain-specific languages (DSLs) is a programming or modeling language of smaller size designed for a much smaller set of problems. The problems that are covered by a given DSL − also called the domain covered by the DSL − can be related both to business concerns (salary calculation, robotics, hospital management, etc.) and more cross-cutting technical concerns (security policies, concurrent behaviors, user interfaces). A DSL is typically created for a limited set of users of for a specific engineering needs (for example the needs of a specific branch of a specific company), with a shorter lifespan and maintenance effort than a GPL.
| The line between a software language of "normal size" and a DSL is not strict at all, and it can be sometimes difficult to decide whether a given software language qualifies as a DSL or not. For example, some considers HTML to be a DSL because it focuses on a very specific and limited technical concern (which is defining the contents of a graphical user interface), while some consider it is too broadly used and has too many applications to qualify as a DSL. |
| Low-code development relies on simplified development environments called Low-code development platforms (LCDP) that can be used by non-software experts to write software components. An LCDP is usually built around a set of well-defined DSLs easy to use and understand. |
Compared with other forms of abstractions (such as software librairies), software languages have a characteristic making them unique: a software language defines a fully dedicated syntax for all the abstractions it provides. This syntax can be either textual (such as Java) or graphical (such as UML), and can be extremely powerful in how it allows an engineer to directly manipulate the provided abstractions.
| On the opposite, using the abstractions provided by a software library requires being skilled with the programming language the library was made for. |
We presented in the previous section the always growing need to define new abstractions to tackle the always growing complexity of problems to solve. While it remains more common today to create new abstractions in the form of software libraries, there is a rising trend to rely on the engineering of new software languages − especially DSLs − to answer the need for abstractions. This is especially true when non-software experts are involved in the engineering process, and can therefore remarkably benefit from a dedicated DSL syntax both tailored for their domain of expertise, and simpler to learn than GPLs. For example, a DSL to configure the behavior of specific physical machines can be designed to be used directly by the experts of said machines, without the contant help of a software engineer.
3. Software Language Engineering (SLE)
It can be observed with little surprise that implementing a new software language, with its own interpreter/compiler and tools, can be a costly endeavor. The field of Software Language Engineering (SLE) emerged for this purpose: providing dedicated methods, principles, approaches and tools to efficient engineer new software languages with limited cost.
3.1. SLE as an engineering problem and discipline
SLE is a specific case of abstraction engineering that can be summarized as follows:
-
On the left, the software language engineer is the role responsible for designing, building and maintaining a software language. This person is a specific kind of software engineer that is specialized in the making of languages. Their goal is to provide a software language that is made to solve a specific set of problems called the domain. To achieve this goal, the software language engineer relies on a diverse toolbox that includes:
-
Metalanguages, which are specific kinds of software languages dedicated to the engineering of new software languages. This includes languages to implement grammars (eg. ANTLR, Langium), languages to implement graphical syntaxes (eg. Sirius, Langium), languages to implement abstract syntaxes (eg. Langium, Ecore), or languages to traverse and transform abstract-syntax trees (eg. Stratego, Kermeta, ATL).
-
Programming languages, which are well-known traditionnal software languages (eg. Java, Rust, TypeScript). Since a sofware language is made of software, programming languages can be used to implement all parts of a software language (eg. OpenJDK is implemented in Java, C++, C), either in place of metalanguages or in addition to metalanguages.
-
A language workbench (eg. Langium, MPS), which is a set of tools dedicated to the engineering of software languages. A language workbench may be provided in the form of an Integrated Development Environment (IDE), and typically includes a set of metalanguages to implement the different parts of a language. Most language workbenches are dedicated to the engineering of domain-specific languages (DSLs), instead of bigger general-purpose programming languages (GPLs).
-
-
In the middle, the domain expert is the user of the software language. This person is a specialist of the considered domain, and hence should easily use the language engineered specifically for this domain. Using the provided software language, the domain expert produces a set of software artifacts (programs, or models) which are used in the concrete realization of a system (eg. a vending machine, a website, a robot…).
Depending on the context and domain, the domain expert may or may not be a software engineer! In particular, a DSL can be made for experts of a domain that is not directly related to computer science, such as a DSL for electrical engineers, or for hospital management, or railway design… -
Finally, on the right, the final user is the person that uses the resulting system.
In this process, each role provides a service to the next: the software language engineer provides a language, used by the domain expert to provide a system to the final user. This means that, from the software language engineer perspective, the "user" is the domain expert, while from the domain expert perspective it is the "final user" shown in the figure.
3.2. SLE for engineering complex systems
We have seen the importance of abstractions to be able to build systems, and we have seen that software languages are a specific and powerful kind of abstractions to achieve this goal. We have also seen that many sorts of software languages exist, such as general-purpose programming languages (GPLs), modeling languages, and domain-specific languages (DSLs).
However, if we focus on the engineering of complex systems (eg. cyberphysical systems such as cars or airplanes), it is often impossible to design and implement a single whole system using a single software language. Each facet of the system (security, structure, behavior, …) requires specific abstractions, which are each best provided by a corresponding software language. We can therefore end-up with a situation where heterogeneous software languages are used together to build the system:
As shown above, it is possible to rely on specialized DSLs to engineer specific facets of a complex system. While some DSLs are available off-the-shelf, it is often required to engineer custom DSLs adapted to the modeling/programming needs of the target system. In this case, SLE become a central part of the system engineering: we build the languages the best adapted to the problems we aim to solve, ie. the languages that best match the domains covered by the system.