The DSL Forge is now open-source!

First of all, happy new year! I’m writing this post to give some fresh news about the current state of DSL Forge.

Getting the tool to its current state wasn’t an easy task. At the time I was working for itemis, the prototype attracted customers attention, as it was a precursor in the area of domain-specific language web editors. Things were going in the right direction, however, for political reasons, the Xtext team at that time wanted to provide a new tool built from scratch.

That sounds weird right? Anyway, that was the reason why I left the company in July 2015 and started PlugBee, a software company specialized in Model-Driven Engineering. DSL Forge has evolved since the original prototype released in January 2014, the tool is now  open sourced under the EPL licence. You can download it from here, you will also find initial documentation to get started. The code is also available on GitHub.

A reminder of what’s the tool for:

The DSL Forge Framework


dslforgeDSL Forge framework allows you to get web-based textual editors from ANTLR grammars. From the grammar specification, a JavaScript parser and lexer are automatically generated and embedded into Cloud9’s ACE editor.

How it Works

The core of the generator is a model to text transformation between Xtext and ANTLR. Indeed, the Xtext grammar specification language uses a subset of ANTLR, it also adds extra sugar to make parsing EMF models more accurate. The generator extracts the core ANTLR spec from the Xtext spec and outputs an ANTLR grammar file. From the ANTLR file, JavaScript parser and lexer are instantly generated. This way, the client parser can be used to mirror the Xtext parser on the client side. It’s up to the developer to customize the way the parsers are used, the editors can be adapted for this purpose.


ACE is one of the most robust JavaScript editors, it has a clean JavaScript API, it comes with a lot of features already bound by default. The fact is, Cloud9 has specialized the editor for dozens of languages such as (JavaScript, Phyton, etc) but there isn’t any methodology nor tool to specialize ACE for custom languages. Thanks to our experience in model-driven engineering, we were able to automatize such a task, making it possible to get the editor just from the grammar specification.

To Backend or Not to Backend

It doesn’t matter in fact, both scenarios are handled by the tool. The tool is able to generate a static web project containing the ACE editor already customized for the language. This is a HTML/JavaScript project which runs entirely on the client side. Moreover, it can generate a JEE application based on Eclipse RAP with an Xtext backend. In the last scenario, the developer is free to customize where the features are handled. More info about the use cases in the online documentation.

Online Coding/Modeling Taken to Another Level

It’s quite impressive to see how fast the number of cloud IDEs is growing these days, in particular online language editors as master components in all the available offers. A lot of online language editors are shipped with e-learning tools and cloud IDEs, most of them are based on well-established text editors such as ACE or CodeMirror. There are online language editors for almost all programming languages including: Java, JavaScript, C++, C# , PHP, HTML, CSS, XML, SQL, Python, Ruby, Scala, Sass, Objective-C, etc.

What about online DSL editors ?

Before attempting to answer the question, we should make clear why should someone need a DSL. There is a must-read paper available on the web, it gives you the big picture, so I won’t argue on top of this.  But to be more concrete, languages like Matlab, Modelica, or R can be seen as DSLs for mathematical modeling, HTML can be seen as a DSL for web applications, SQL is a DSL for manipulating relational databases… These are well-known DSLs, and there is a plenty of experts’ communities where DSLs are used. Not all of these people know today’s technologies can provide tools which could exploit DSLs to bring better understanding, productivity, and maintenability. Roughly speaking, it’s like telling the computer what to do, in the language You understand. The computer should understand your language (whatever it is), and not the opposite! 

Today, the only answer to the question is DSLFORGE. We continue growing, and as already mentioned in a previous post, we focused the last few weeks on the enhancement of the editors’ generator, and the results are promising. We are about to achieve a complete online modeling experience that goes beyond what we were expecting!

A quick review of the new features.


Client scripts have been cleaned-up and some tricky bugs have been fixed mainly in the theming. I was conscious to keep the Eclipse look-and-feel (keyword coloration, error/warning/info annotations, etc.) as if the editors were running on top of the usual Eclipse RCP. The theme is polished and clean now. Because the client is lifted by ACE, it gives a wide range of customization possibilities, and customizing the theme is not an exception: simply replace the theme script by another script (a multitude of themes are available for free in ACE demo site) and you are done.

Content Assist

We also added more functional features to the generated editors. A lot of effort has been spent on improving the content assist popup. Customizable templates (or snippets) are now proposed apart from the keywords and references. Different icons are now displayed next to each proposal (snippet, keyword, or reference) which make it more comfortable to distinguish between the available alternatives. In addition, documentation popup is also displayed next to each selected proposal.


Text Hover

It is now possible to display a documentation popup when hovering over a reference. HTML formatting is planned. The content of the popup is customized on the client side. To avoid unnecessary ping pongs between the client and the server for such a non critical feature, we recommend sending the documentation content right from the beginning, then manage the text hover purely on the client. In the back-end scenario, mapping documentation to EMF annotations is also supported.


Server-side Validation

Server-side validation is used when the editors need to interact with a Java-based back-end. The synchronization between the client and the server is more flexible now: DSL developers can decide whether they want to parse files content systematically on the server, or to delegate to the client parser (and synchronize on-demand). Both strategies are supported.

In the back-end scenario, legacy EMF validators, such as the Xtext validators, work out-of-the-box since the runtime plugin is packaged as-is in the web application (this is also the case for interpreters and code generators). In the example below, changing entity  name from “Adress” to “address” raises both error and info annotations. The info is reported by a custom @Check method in the Xtext validator, the error is reported by the linker as the reference named “adress” is no longer resolved.



Features Summary

Below the complete list of features already handled by default in the generated editors.

  • Syntax Highlighting (keywords, strings, comments, annotations, etc.),
  • Syntax validation (using generated parser and lexer),
  • Content assist (keywords and references),
  • Sever-side validation (aka semantic validation),
  • Scoping (global and local)
  • Template proposals (snippets),
  • Text hovering (display documentation),
  • Default Key bindings,
  • Undo/Redo Support,
  • Code Folding.

We’ve added some videos showcasing these features, but you can also get hands on the new features by your own here.

What’s new in DSL Forge?

The browser-based cool stuff soon available to play with!


When it comes to manage multiple users accessing the same model resource online, the most straight forward way is to consider using a repository, such as EMFStore or CDO. The main reason why these technologies are not well suited for textual resources is the serialization format. Classical EMF resources are persisted in XMI, while resources with textual content are often persisted as-is, and this is also the case of Xtext resources.

Moreover, I don’t like the idea of persisting model content into databases… such a decision will make migrating models complex afterwards, compared to traditional file storage systems.

Anyway, I have at least two reasons to consider the problem right from the beginning. What we need is a component acting like a workspace in a web context. With the first prototype presented at MoDELS, the workspace was simply mapped to a root folder in the server’s file storage system.

To be able to manage concurrent users access, we need to keep track of information about the users accessing the workbench, such as their ID, name, company, projects, etc. We need to maintain the state of files currently locked, and by who the files are locked.

A database is a good candidate to store and manage such metadata, then we need to map the files in the file system to that metadata in the database, and provide a convenient API unifying the access to the couple (file storage system, database) as a unique singleton providing basic functions like creating a projet, creating a file, locking a file, etc. Ok, this sounds like an ECM repository right? After all, DSL files can be seen as “enterprise content”, and they are first-class entities in our scenario.

The Workbench has now an embedded Derby database, persistency is managed using Gemini and EclipseLink.

We also added an extension point that allows to integrate DSL editors easily. These are the DSLs currently contributed to the workbench:

  • Fowler’s DSL: a language for describing state machines,
  • Arithmetics: a language for describing arithmetic expressions,
  • Dimain Model: classical entities and relationships language
  • Humming Bird: a language for describing embedded system devices

Bringing domain-specific languages to the Web

It seems inevitable that everything we do in software engineering is going Web. This is true also for Eclipse-based modeling environments. Whether or not online IDEs will make developer’s life easier that is another question. At least in the area of Domain-Specific Languages where automated processes are heavily used for model transformations, code generation, etc., one can think about web-based approaches. DSL users are supposed to be non-developers, say “domain experts”. Their focus is mainly on business added value, not the implementation details. Examples of domains making extensive use of model-driven engineering are automotive, energy, and defense.

Sharing tools and resources across multiple users has always been used in traditional developments environments. This is the case for versioning systems, workspace provisioning systems, shared repositories, and even content management systems. However, with the increasing of end users mobility and for some reasons of remote configuration, e-learning, webminars, wikis, etc., there is still a need for more lightweight applications shifted on web browsers.

In this screencast, I’m going to show you how you can derive a Web editor from an existing Xtext grammar, with the following features already given for free:

–        Syntax highlighting

–        Syntax validation

–        Content assist

If this prototype interests you, you can contact me at: amlajmi[at]gmail[dot]com