Software Language Engineering: Creating Domain-Specific Languages Using Metamodels

by Anneke Kleppe

Overview

Software practitioners are rapidly discovering the immense value of Domain-Specific Languages (DSLs) in solving problems within clearly definable problem domains. Developers are applying DSLs to improve productivity and quality in a wide range of areas, such as finance, combat simulation, macro scripting, image generation, and more. But until now, there have been few practical resources that explain how DSLs work and how to construct them for optimal use.

 

Software Language Engineering fills that need. Written by expert DSL consultant Anneke Kleppe, this is the first comprehensive guide to successful DSL design. Kleppe systematically introduces and explains every ingredient of an effective

language specification, including its description of concepts, how those concepts are denoted, and what those concepts mean in relation to the problem domain. Kleppe carefully illuminates good design strategy, showing how to maximize the flexibility of the languages you create. She also demonstrates powerful techniques for creating new DSLs that cooperate well with general-purpose languages and leverage their power.

 

Completely tool-independent, this book can serve as the primary resource for readers using Microsoft DSL tools, the Eclipse Modeling Framework, openArchitectureWare, or any other DSL toolset. It contains multiple examples, an illustrative running case study, and insights and background information drawn from Kleppe’s leading-edge work as a DSL researcher.

 

Specific topics covered include

 

• Discovering the types of problems that DSLs can solve, and when to use them
• Comparing DSLs with general-purpose languages, frameworks, APIs, and other approaches
• Understanding the roles and tools available to language users and engineers
• Creating each component of a DSL specification
• Modeling both concrete and abstract syntax
• Understanding and describing language semantics
• Defining textual and visual languages based on object-oriented metamodeling and graph transformations
• Using metamodels and associated tools to generate grammars
• Integrating object-oriented modeling with graph theory
• Building code generators for new languages
• Supporting multilanguage models and programs

 

This book provides software engineers with all the guidance they need to create DSLs that solve real problems more rapidly, and with higher-quality code.

Product Details

ISBN-13:	9780321606464
Publisher:	Pearson Education
Publication date:	12/09/2008
Sold by:	Barnes & Noble
Format:	eBook
Pages:	240
File size:	2 MB
Age Range:	18 Years

About the Author

Anneke Kleppe has over twenty years of experience in IT. She started her career in telecommunications and then worked as an independent consultant with her own company, Klasse Objecten. She has coached and trained employees of companies working with MDA, OCL, and UML. Currently, she is a consultant at Capgemini and is responsible for the introduction of domain-specific languages for various clients.

Table of Contents

 

Background Information  xvii

Preface    xix

Foreword  xxvii

 

Chapter 1: Why Software Language Engineering?   1

 

1.1 An Increasing Number of Languages 1

1.2 Software Languages  3

1.3 The Changing Nature of Software Languages   4

1.3.1 Graphical versus Textual Languages   5

1.3.2 Multiple Syntaxes 6

1.4 The Complexity Crisis    7

1.5 What We Can Learn From ...    8

1.5.1 Natural-Language Studies    9

1.5.2 Traditional Language Theory 10

1.5.3 Graph Theory 10

1.6 Summary   12

 

Chapter 2: Roles in Language Engineering   15

 

2.1 Different Processes, Different Actors  15

2.2 The Language User   16

2.2.1 Tool Set of the Language User   17

2.3 The Language Engineer    19

2.3.1 Tool Set for the Language Engineer   19

2.3.2 Tool Generators   20

2.4 Summary   21

 

Chapter 3: Languages and Mograms  23

 

3.1 What Is a Language? 23

3.1.1 Mogram, or Linguistic Utterance 24

3.1.2 Primitive Language Elements and Libraries 26

3.2 Abstraction Levels and Expressiveness  27

3.2.1 Abstract versus Incomplete  29

3.2.2 Raising the Level of Abstraction 29

3.2.3 Growing Business Expectations   31

3.2.4 Languages and Abstraction Levels 32

3.3 Domain-Specific Languages     33

3.3.1 Domain-Specific versus General Languages  33

3.3.2 Domain Experts versus Computer Experts    33

3.3.3 Large User Group versus Small User Group  34

3.3.4 Horizontal DSLs versus Vertical DSLs 35

3.3.5 DSLs versus Frameworks and APIs 37

3.3.6 DSLs as Software Languages  37

3.4 Summary   38

 

Chapter 4: Elements of a Language Specification 39

 

4.1 Language Specification   39

4.1.1 Forms of a Mogram 40

4.1.2 Parts   41

4.1.3 Creation Process  42

4.1.4 An Example   43

4.2 Formalisms to Specify Languages   47

4.2.1 Context-Free Grammars  47

4.2.2 Attributed Grammars    49

4.2.3 Graph Grammars    51

4.2.4 UML Profiling 52

4.2.5 Metamodeling 53

4.2.6 Formalism of Choice    53

4.3 Summary   54

 

Chapter 5: Metamodeling      57

 

5.1 Foundations of Metamodeling   57

5.1.1 Graphs  58

5.1.2 Models and Instances   61

5.1.3 Constraint Types  62

5.1.4 UML Diagrams as Notation    66

5.1.5 Metamodels   68

5.2 Relation with Model-Driven Approaches  69

5.2.1 How to Understand the Term Platform  69

5.2.1 Transformations and Language Specifications    70

5.3 Summary   73

 

Chapter 6: Abstract Syntax   75

 

6.1 The Pivotal Role of Abstract Syntax    75

6.1.1 The Hidden, Underlying, Unifying Structure     75

6.1.2 The Gateway to Semantics    76

6.2 Mogram/Language Relationship  77

6.2.1 Abstract Form of a Mogram   78

6.2.2 Concrete form of a Mogram   78

6.2.3 Syntactically, Not Semantically, Correct  79

6.2.4 Syntactically Incorrect Mograms 80

6.3 How to Create an Abstract Syntax Model 80

6.3.1 Domain Modeling for Vertical DSLs    81

6.3.2 Pattern Finding for Horizontal DSLs  82

6.4 Alan: An Example Language     83

6.4.1 Introducing Alan  83

6.4.2 The Observer Pattern in Alan    84

6.4.3 Generic Types in Alan  85

6.4.4 Alan’s Standard Library     86

6.5 Alan’s Abstract Syntax Model  86

6.5.1 Types   86

6.5.2 Expressions  87

6.5.3 The Observer Pattern   89

6.5.4 The Library  90

6.6 Summary   90

 

Chapter 7: Concrete Syntax    93

 

7.1 Concrete Syntax and Tool Support  93

7.1.1 Phases in the Recognition Process    94

7.1.2 Two Types of Editors   100

7.1.3 The Textual-Graphical Mismatch, or Blind Spots 101

7.2 Concrete Syntax Model    104

7.2.1 Guidelines for Creating a Concrete Syntax Model     105

7.2.2 Alan’s Graphical Syntax Model   106

7.3 Summary   110

 

Chapter 8: Generating Textual Concrete Syntax   113

 

8.1 The Grasland Generator   113

8.2 The Abstract-to-Concrete Transformation 117

8.2.1 Handling References    117

8.2.2 Alan’s Textual Concrete Syntax Model 118

8.2.3 Designer Input to the asm2tcsm  Transformation 120

8.3 The Model-to-BNF Grammar Algorithm 121

8.3.1 Generation of the BNFset    121

8.3.2 Designer Input to the tcsm2bnf   Transformation     122

8.4 The Static Semantic Analyzer  126

8.4.1 Binding 126

8.4.2 Checking 128

8.5 Summary   129

 

Chapter 9: Semantics: The Meaning of Language   131

 

9.1 Semantics Defined   131

9.1.1 Understanding Is Personal   131

9.1.2 The Nature of a Semantics Description 133

9.2 Semantics of Software Languages   134

9.2.1 Ways to Describe Semantics  135

9.2.2 The “Best” Semantics   138

9.3 Operational Semantics Using Graphs 138

9.3.1 Modeling Semantics     139

9.3.2 Consequences of the Von Neumann Architecture   140

9.3.3 States in the Abstract Machine  142

9.3.4 The Rules of Execution: Transitions  143

9.3.5 Example: Simple Semantics   144

9.4 Summary   148

 

Chapter 10: Translational Semantics: Code Generation 149

 

10.1 Code Generation and Language Design   149

10.1.1 Abstraction Levels    150

10.1.2 Multiple Semantics    150

10.2 Building a Code Generator    151

10.2.1 Hard-Coded or Model-Transformation Rules 151

10.2.2 Source-Driven or Target-Driven Translation    153

10.2.3 Concrete or Abstract Form Target    154

10.2.4 A Template Language Targeting Abstract Form   157

10.3 Code-Generation Patterns     158

10.3.1 Treewalkers, or the Visitor Pattern 158

10.3.2 CreateOrFind 159

10.3.3 Mappers 160

10.4 Extension Points in the Generated Code 161

10.5 Other Issues in Code Generation  163

10.5.1 Bidirectionality 163

10.5.2 Target Platform with or without Framework     164

10.5.3 Modularization of the Generation    167

10.5.4 Built-in Flexibility  167

10.6 Summary  169

 

Chapter 11: Combining Multiple Languages   171

 

11.1 Multiple Mograms for One Application  171

11.1.1 Advantages of Multiple Mograms 172

11.1.2 Multiple Mograms: One or More Languages  172

11.2 Intermogram References  173

11.2.1 Hard and Soft References   173

11.2.2 Passive and Active Languages   174

11.2.3 Information Hiding    175

11.3 Language Interfaces     176

11.3/1 Offered or Required Language Interface   177

11.4 Consequences for Language User Tool Set    179

11.4.1 Resolving and Checking References   179

11.4.2 Handling Changes 180

11.4.3 Code Generation  181

11.5 Support for Language Evolution   182

11.6 Conclusion    183

11.7 Summary  183

 

Appendix A Glossary     185

Appendix B References   189

Index     197

Preface

Preface

This book started out as a PhD thesis, a work I started rather late in my career. In it you can find a condensed report of the knowledge that I have gained over the years, on the topic of designing languages for the software industry. I hope this book will be helpful for you in learning about languages, what their constituting elements are and how they can be defined. However, before you start reading, I would like to explain why this book is not a thesis.

Not a Thesis

As it turned out, for me it was a wrong move to go from industry to academia. There are certain requirements to a PhD thesis (or maybe it is the way these requirements are upheld?) that prohibited a truthful expression of my ideas and insights in the subject matter. First, the work should be new. Second, the work should be judged by a jury of peers. Third, the work should be based on a sound theory. So, you might ask, what’s wrong with that? These requirements certainly appear to be very good ones. Let me elaborate on these for a bit.

For a scientist to be sure that the work he/she is doing is new, requires a large investment in studying what other people have been doing. With the amount of scientists ever growing, as well as the amount of publications per scientist, this requirement can in practice not be verified, only falsified. The limited life span of the average scientist is simply too short in view of the large body of scientific publications. This is the reason that the work of most scientists resembles the work being done by a worm: dissecting a single square yard (or even worse using metrics: a single square meter) of the garden of knowledge completely and ever sothoroughly. Only on this small area the scientist can be sure he/she knows all related work. Unfortunately, the work I wanted to do is better resembled by the work of a bee: visiting the many wonderful flowers in the garden of knowledge and bringing ideas, concepts, and techniques from the one to the other.

Furthermore, one can question this requirement even more fundamentally: what is newness? How do we decide what is new and what is not? The following quote¹ expresses beautifully how deceptive the difference between old and new is.

"... (to) make something that was not there before, ... is deceptive, because the separate elements already existed and floated through history, but they were never before assembled in this manner. Joining, assembling, the new will always consist of that." (Connie Palmen, Lucifer, page 261)

As an example, when Jos Warmer and I created the Object Constraint Language, did we create something new? In one way we did not, because every element of the language was already there: set theory, object-orientation, the notation used in Smalltalk, pre- and postconditions, you name it. Yet somehow we managed to put all these things together like ingredients in a soup, stir them, and end up with something that most people consider to be a new language.

The second scientific requirement is that the work should be judged by a jury of peers. In practice this means that the life of a scientist is all about judging (doing reviews) and being judged (getting reviewed). I have found that this is not beneficial for qualities like open-mindedness and creativity which every scientist is assumed to possess and cherish. Nor does it encourage cooperation, for who takes the credits? This life full of judgement, combined with the pressure to publish regularly, simply does not provide the right breeding ground for innovative research.

Furthermore, who are these peers that should do the judging? When you are a worm-like scientist, most of the times it is possible to find a few others working on the same square yard of the garden; people that are able to understand the work presented. Still many scientists complain about the quality of the reviewers assigned to judge their work. Even worse is the case for a bee-like scientist. Suppose the bee takes an idea from patch A and applies it to patch B. The people in A will not be interested, whereas the people in B will have trouble understanding this strange idea. In a world full of worms the bee has no peers. Please note that I do not object to having worms in the garden of knowledge. The work of the worms is necessary to keep the soil fertile. I just wish there where more bees, because they are just as necessary; without their work the garden cannot bear fruit.

Let’s proceed to the last requirement, which is that the work should be based on a sound theory. Actually, I have found that this requirement collides with the first. If you want to do innovative research, which I do, it appears that in computer science, the advances in theory no longer precede the advances in practice. Most of the major programming languages (Java, C#), modeling languages (UML), combined with model driven software development, application architectures (SOA), network architectures (.Net, J2EE) of the last two decades were conceived in industry. Academic computer science is in danger of slowly becoming an empirical field which studies the advances being made in industry.

Furthermore, most theory in academic computer science is mathematics based. Because only the very basic elements of mathematics are taught to students, most of whom end up working in industry, the gap between theory and practice is growing larger and larger. That leaves us with innovative research on the one side of the gap, in industry, and sound theory on the other, in academia.

A thing that widens the gap even further, is the existence of a certain disdain for applied research in academia. When faced with making their theories applicable to industry most scientists react as the mathematician in a well-known joke. This mathematician wakes up to see a starting fire in his bedroom. He looks at the fire and thinks: "Well, I have got a bucket in kitchen, water in the bathroom, and I know how to apply these to put out the fire. So, all’s well." And he goes back to sleep. In my opinion, progress in computer science is not only about knowing how to do things, but also about doing them. Specially because doing things always poses new challenges, thus providing you with extra knowledge about how to do (or not to do) these things. On the other hand, a different type of disdain exists in industry. Scientist are regarded as being strange creatures that speak gibberish and are extremely unpractical. Because of this, people from industry are rather reluctant to use new ideas from academia, even when they are good ones.

As is, this book is not a thesis. It presents some things that are new but which are based on existing concepts, and some things which are old but still relevant for software language design. It is not in detail judged by peers, although many parts have been published in reviewed conferences. Finally, I am certain that the theory on which it is based, leaves lots of room for worm-like scientists to work on and improve what is presented here.

What is in this Book

If this book is not a thesis, then what can you expect to find in it? First, there are many areas of expertise that can contribute to your knowledge of software languages. In this book the following areas of expertise will be visited (randomly ordered):

•Fundamental Mathematics
•Graphs and Graph Transformations
•Philosophy
•Grammars and Compiler Technology
•Computational linguistics
•Visual Language Design
•Natural languages research
•Modeling and Metamodeling
•Model Driven Software Development

Not all of these areas are as standalone as they appear to be. Studying the wealth of knowledge that these areas comprise, you can find many connections between them and similarities in the concepts used. Examples are grammar versus metamodel, model versus graph, and textual versus visual (graphical) language. Building on these connections and similarities, this book offers you:

•A way to define languages (textual and visual), including their semantics, based on object-oriented (meta)modeling and graph transformations.
•An insight in the differences and similarities of the theories of textual and visual languages.
•Additions to the integration of object-oriented modeling with graph theory.
•A method for generating (1) a grammar from a metamodel and (2) the tools for supporting this.
•Insights in how to built a code generator for a new language.
•A way to create support for multi-language models/programs.

What you will not find in this book is a step-by-step plan of how to create a new software language. Instead it shows you what the ingredients of a language specification are and how each of these can be best expressed.

Acknowledgements

During the years many people have helped me gain the knowledge that I am sharing with you in this book, for which I am very grateful to them. However, there are a number of people that on this occasion deserve special attention. First and foremost, I want to thank Arend Rensink for given me the opportunity to experience life in academia. We cooperated on the work for Chapter 5, and together with Harmen Kastenberg we worked on the topic of operational semantics for software languages, which you can find in Chapter 9. In all this he proved to be much more than a worm-like scientist: open and broad-minded, while remaining critical on details and underlying theory.

I would also like to thank another former colleague from the University of Twente: Theo Ruys. Our discussions on the grammar generator where very helpful. My thanks go to David Akehurst for working with me on the comparison of textual and graphical syntaxes, to Jos Warmer for his cooperation on multi-language support, and to my students for bringing some fun into my research. Certainly the reviewers of early versions of this book deserve to be mentioned: Tony Clark, Ron Kersic, Markus Vöter, Anders Hessellund, Dave Hendricksen, and Barney Boisvert. I am grateful for their remarks, which hopefully made this book a better read for all of you.

Finally I want to thank the Netherlands Organization for Scientific Research (NWO) which funded the GRASLAND project at the University of Twente of which I have been part.

Finally

As I wrote in the first paragraph of this preface, I hope this book will be helpful for you in learning about software languages. Like the author of a thesis I too feel that I have gained some good insights in this topic. I have written this book simply to offer my knowledge to you, its reader. Whether or not you agree with what I write, I hope it will improve your knowledge of the subject.

Soest, Netherlands, June 2008

1Translated from Dutch by AK. The original quote is "... wanneer je zelf iets maakt wat er niet eerder was, ... is dat bedrieglijk, want de afzonderlijke elementen bestonden wel en zweefden in de geschiedenis, maar ze werden nooit eerder op deze wijze samengebracht. Samenbrengen, bijeenvoegen, het nieuwe zal altijd daaruit bestaan."

© Copyright Pearson Education. All rights reserved.

Editorial Reviews

“Kleppe provides a rewarding reading experience. She masterfully blends the valuable insights of both academics and practitioners, and her holistic approach to the subject will benefit software engineers with helpful insights into their craft and tools.”

–Patrick Bailey, Computer Science Department, Calvin College

“With the advent of domain-specific languages in modern software development, an understanding of software language engineering becomes essential for developers. Kleppe offers an erudite and easy-going introduction to the key concepts of the subject.”

–Anders Hessellund, IT University of Copenhagen, Denmark

“Ms. Kleppe has risen to the challenge by successfully combining the traditional theory of text-based programming languages with the brave new world of computer modeling languages into a single comprehensive and consistent conceptual framework. Furthermore, she has done so with wit and style. The text is eminently readable without being condescending, and should please both pragmatically inclined software developers as well as theoreticians who seek abstractions. This is a rare example of a technical book that fully realizes the adage that ‘things should be made as simple as possible, but no simpler.’”

–Bran Selic, Malina Software Corp.

“By finally gathering under one roof the philosophy, concepts, and terminology of language design–and by breaking down many of the needless barriers between graphical and textual languages–this book provides a good introduction for the novice and is a useful refresher for the expert.”

–Dr. Laurence Tratt, Bournemouth University, UK

“Software Language Engineering is an excellent resource for teaching you how to create software languages that are specific to your domain by utilizing a domain modeling approach.”

–Dave Hendricksen, Software Architect

“Languages to define procedures, objects, and intent have proliferated unchecked since Ada Byron. Lady Lovelace first realized in 1830 that some day machines would be programmed. The vast majority of these languages have failed to feature much in the way of design, despite pioneering work by computer scientists such as Niklaus Wirth. But if, as Wirth says, “... precision and perfection are not a dispensable luxury, but a simple necessity,” then language design, in fact precise language design, becomes a critical skill. The advent of general language-modeling languages (like Meta Object Facility) and languages designed to be made domain-specific (like Unified Modeling Language) make this skill even more pressing. No one better understands these issues, or better presents a unified path to precise language design, than Anneke Kleppe. Using clear and concise examples and detailed explanations, this book gives the language designer (and in fact any architect or developer) the background, tools, and skills necessary to design general and domain-specific languages right, the first time. It belongs on the bookshelf of every systems architect and software developer.”

–Richard Mark Soley, Ph.D., Chairman and CEO, Object Management Group, Inc.

“In this book Anneke brings together the “old” knowledge of programming languages with the “new” field of modeling languages. She makes it crystal clear how they complement each other instead of how they compete with each other. I especially like the new term “mogram,” which is the generic term for both model and program. Anyone who is designing or implementing programming languages, modeling languages, or domain specific languages should read this book. It provides the reader with the fundamental background on software language engineering.”

–Jos Warmer, Partner at Odina

From the Publisher