The DSL Forge is now open-source!

First of all, happy new year! I’m writing this post to give some fresh news about the current state of DSL Forge.

Getting the tool to its current state wasn’t an easy task. At the time I was working for itemis, the prototype attracted customers attention, as it was a precursor in the area of domain-specific language web editors. Things were going in the right direction, however, for political reasons, the Xtext team at that time wanted to provide a new tool built from scratch.

That sounds weird right? Anyway, that was the reason why I left the company in July 2015 and started PlugBee, a software company specialized in Model-Driven Engineering. DSL Forge has evolved since the original prototype released in January 2014, the tool is now  open sourced under the EPL licence. You can download it from here, you will also find initial documentation to get started. The code is also available on GitHub.

A reminder of what’s the tool for:

The DSL Forge Framework

 

dslforgeDSL Forge framework allows you to get web-based textual editors from ANTLR grammars. From the grammar specification, a JavaScript parser and lexer are automatically generated and embedded into Cloud9’s ACE editor.

How it Works

The core of the generator is a model to text transformation between Xtext and ANTLR. Indeed, the Xtext grammar specification language uses a subset of ANTLR, it also adds extra sugar to make parsing EMF models more accurate. The generator extracts the core ANTLR spec from the Xtext spec and outputs an ANTLR grammar file. From the ANTLR file, JavaScript parser and lexer are instantly generated. This way, the client parser can be used to mirror the Xtext parser on the client side. It’s up to the developer to customize the way the parsers are used, the editors can be adapted for this purpose.

Why ACE

ACE is one of the most robust JavaScript editors, it has a clean JavaScript API, it comes with a lot of features already bound by default. The fact is, Cloud9 has specialized the editor for dozens of languages such as (JavaScript, Phyton, etc) but there isn’t any methodology nor tool to specialize ACE for custom languages. Thanks to our experience in model-driven engineering, we were able to automatize such a task, making it possible to get the editor just from the grammar specification.

To Backend or Not to Backend

It doesn’t matter in fact, both scenarios are handled by the tool. The tool is able to generate a static web project containing the ACE editor already customized for the language. This is a HTML/JavaScript project which runs entirely on the client side. Moreover, it can generate a JEE application based on Eclipse RAP with an Xtext backend. In the last scenario, the developer is free to customize where the features are handled. More info about the use cases in the online documentation.

Online Coding/Modeling Taken to Another Level

It’s quite impressive to see how fast the number of cloud IDEs is growing these days, in particular online language editors as master components in all the available offers. A lot of online language editors are shipped with e-learning tools and cloud IDEs, most of them are based on well-established text editors such as ACE or CodeMirror. There are online language editors for almost all programming languages including: Java, JavaScript, C++, C# , PHP, HTML, CSS, XML, SQL, Python, Ruby, Scala, Sass, Objective-C, etc.

What about online DSL editors ?

Before attempting to answer the question, we should make clear why should someone need a DSL. There is a must-read paper available on the web, it gives you the big picture, so I won’t argue on top of this.  But to be more concrete, languages like Matlab, Modelica, or R can be seen as DSLs for mathematical modeling, HTML can be seen as a DSL for web applications, SQL is a DSL for manipulating relational databases… These are well-known DSLs, and there is a plenty of experts’ communities where DSLs are used. Not all of these people know today’s technologies can provide tools which could exploit DSLs to bring better understanding, productivity, and maintenability. Roughly speaking, it’s like telling the computer what to do, in the language You understand. The computer should understand your language (whatever it is), and not the opposite! 

Today, the only answer to the question is DSLFORGE. We continue growing, and as already mentioned in a previous post, we focused the last few weeks on the enhancement of the editors’ generator, and the results are promising. We are about to achieve a complete online modeling experience that goes beyond what we were expecting!

A quick review of the new features.

Theming

Client scripts have been cleaned-up and some tricky bugs have been fixed mainly in the theming. I was conscious to keep the Eclipse look-and-feel (keyword coloration, error/warning/info annotations, etc.) as if the editors were running on top of the usual Eclipse RCP. The theme is polished and clean now. Because the client is lifted by ACE, it gives a wide range of customization possibilities, and customizing the theme is not an exception: simply replace the theme script by another script (a multitude of themes are available for free in ACE demo site) and you are done.

Content Assist

We also added more functional features to the generated editors. A lot of effort has been spent on improving the content assist popup. Customizable templates (or snippets) are now proposed apart from the keywords and references. Different icons are now displayed next to each proposal (snippet, keyword, or reference) which make it more comfortable to distinguish between the available alternatives. In addition, documentation popup is also displayed next to each selected proposal.

 

Text Hover

It is now possible to display a documentation popup when hovering over a reference. HTML formatting is planned. The content of the popup is customized on the client side. To avoid unnecessary ping pongs between the client and the server for such a non critical feature, we recommend sending the documentation content right from the beginning, then manage the text hover purely on the client. In the back-end scenario, mapping documentation to EMF annotations is also supported.

 

Server-side Validation

Server-side validation is used when the editors need to interact with a Java-based back-end. The synchronization between the client and the server is more flexible now: DSL developers can decide whether they want to parse files content systematically on the server, or to delegate to the client parser (and synchronize on-demand). Both strategies are supported.

In the back-end scenario, legacy EMF validators, such as the Xtext validators, work out-of-the-box since the runtime plugin is packaged as-is in the web application (this is also the case for interpreters and code generators). In the example below, changing entity  name from “Adress” to “address” raises both error and info annotations. The info is reported by a custom @Check method in the Xtext validator, the error is reported by the linker as the reference named “adress” is no longer resolved.

 

 

Features Summary

Below the complete list of features already handled by default in the generated editors.

  • Syntax Highlighting (keywords, strings, comments, annotations, etc.),
  • Syntax validation (using generated parser and lexer),
  • Content assist (keywords and references),
  • Sever-side validation (aka semantic validation),
  • Scoping (global and local)
  • Template proposals (snippets),
  • Text hovering (display documentation),
  • Default Key bindings,
  • Undo/Redo Support,
  • Code Folding.

We’ve added some videos showcasing these features, but you can also get hands on the new features by your own here.