DSL Forge 0.9.1 is out!

We had a successful year setting the ground for the DSL Forge to be more robust and useful for language developers. The tool is now more mature, and we are proud to announce the 0.9.1 release, with a better support for Xtext, an improved interaction between client and server parsers, and a more robust workspace management.

On the generated editors side, features like content assist and global scoping are now fully integrated and customizable. We’ve had interesting case studies, typical requests are about the integration with forms, views, and other third party widgets, the integration with Docker, the implementation of the Language Server Protocol, or the integration with Cloud IDEs such as Eclipse Dirigible. Unfortunately, by lack of resources, all feature requests have not been adressed in v0.9.1. However, users’ satisfaction being of most importance to us, we will continue next year to cover the requests the best we can.

The styled text widget on top of which the editors are built, have passed the Eclipse IP log review, and will be incubated under Eclipse RAP. Three languages are contributed as for now: Java, JavaScript, and JSON. The integration is fully based on ACE, the code is available on GitHub.

Another good news is the release of ANTLR 4.6 a couple of days ago, we did not evaluate yet the JavaScript target API but we do have such an integration in mind for 2017.

Coding Park’s IDE for Kids is also evolving, new sneaky levels are available now. It can take for your kid one hour of code easy, we’re working on how to evaluate the so-called coding style with quantified metrics, something funny and very instructive. Try it on CodingPark!

Monster's Pyramid

The Entrepreneur Challenge, one of the new levels in Coding Park!

 Update sites

v0.9.1 is available on the regular update sites:

  • https://www.dslforge.org/downloads/tooling/repository
  • https://www.dslforge.org/downloads/runtime/repository.

Older versions are available under:

  • https://www.dslforge.org/downloads/tooling/releases/{version}
  • https://www.dslforge.org/downloads/runtime/releases/{version}

DSL Forge 0.9.0 is Out!

dslforge-header

DSL Forge v0.9.0 is based on Eclipse Neon packages, with RAP 3.1.0 and Xtext 2.10. ANTLR v3.3 with JavaScript target has been patched for Java 8. This version comes with core refactorings on the generators. Mainly, the ACE/ANTLR Generator has been extracted from the Xtext/RAP Generator, which has been enhanced with new features. The Xtext content assist feature has been ported to RAP with very localized changes. The difficulty is not in porting the Xtext interfaces, but rather to manage the dependencies to JFace (no web support) and the adherence to the Eclipse JDT. Well, the Xtext team have refactored their features since version 2.9.x and the dependencies are now managed much better.

The Xtext/RAP Generator produces now 3 projects. For example, if you have the following RCP plugins:

  • /org.eclipse.xtext.example.statemachine
  • /org.eclipse.xtext.example.statemachine.ui

The generator outputs 3 projects:

  • /org.eclipse.xtext.example.statemachine.web
  • /org.eclipse.xtext.example.statemachine.web.build
  • /org.eclipse.xtext.example.statemachine.web.target

The web plugin to package with RAP, a Maven/Tycho project to actually build and package the editor as a web application archive (war), and the target platform on top of which to compile and build the web application.

4 Steps to get your DSL deployed on the web!

Step 1: Generate a web editor from an existing Xtext grammar.

Step 2: Set up the debug configuration and launch the web application from Eclipse.

Step 3: run the Maven build to get a web application archive ready to be deployed.

Step 4: Deploy the web archive on Tomcat

Limitations

Xbase is out of scope for now, as its adherence to the JDT cannot be satisfied in the current scenario. Moreover, Xtext grammar inheritence is not yet translated into ANTLR grammar inheritence, so if your grammar imports other grammars, generate the editor from the main grammar then copy the rules from the original Xtext files to the ANTLR file. Contact the professional support for additional details.

Download P2 Sites

Tooling: http://dslforge.org/downloads/tooling/repository/

Runtime: https://dslforge.org/downloads/runtime/repository/

Get the Source Code

https://github.com/plugbee/dslforge

Report issues

GitHub: https://github.com/plugbee/dslforge/issues

mailto: team@dslforge.org

dslforge

Pirate Robot: A Cool Application of Domain-Specific Languages!

As a case study, we used DSL Forge to build an e-learning platform for teaching kids the basics of computer programming. There is a blog post on Modeling-Languages.com which describes the use case in details. The Adventures of the Pirate Robot is an online and publicly available coding game which aims to teach kids computational thinking by coding the progression of a pirate robot in a game canvas. The technologies used to build the game (RAP, EMF, Xtext, Xtend) are traditionally used in the industry to build advanced RCP platforms for software and systems engineering. What we have done is to apply these technologies in the unusual context of online education and gamification.

Pirate Robot

Pirate Robot

This use case will be demonstrated during the next EclipseCon France 2016 as an application of DSLs for gamification, do *not* miss the talk!

Aknowledgement

Special thanks to Jabier Martinez for his contribution to the design of the game canvas, something I wouldn’t be able to do better than him given my lack of experience with sprite sheets and the short time we had.

The DSL Forge is now open-source!

First of all, happy new year! I’m writing this post to give some fresh news about the current state of DSL Forge.

Getting the tool to its current state wasn’t an easy task. At the time I was working for itemis, the prototype attracted customers attention, as it was a precursor in the area of domain-specific language web editors. Things were going in the right direction, however, for political reasons, the Xtext team at that time wanted to provide a new tool built from scratch.

That sounds weird right? Anyway, that was the reason why I left the company in July 2015 and started PlugBee, a software company specialized in Model-Driven Engineering. DSL Forge has evolved since the original prototype released in January 2014, the tool is now  open sourced under the EPL licence. You can download it from here, you will also find initial documentation to get started. The code is also available on GitHub.

A reminder of what’s the tool for:

The DSL Forge Framework

 

dslforgeDSL Forge framework allows you to get web-based textual editors from ANTLR grammars. From the grammar specification, a JavaScript parser and lexer are automatically generated and embedded into Cloud9’s ACE editor.

How it Works

The core of the generator is a model to text transformation between Xtext and ANTLR. Indeed, the Xtext grammar specification language uses a subset of ANTLR, it also adds extra sugar to make parsing EMF models more accurate. The generator extracts the core ANTLR spec from the Xtext spec and outputs an ANTLR grammar file. From the ANTLR file, JavaScript parser and lexer are instantly generated. This way, the client parser can be used to mirror the Xtext parser on the client side. It’s up to the developer to customize the way the parsers are used, the editors can be adapted for this purpose.

Why ACE

ACE is one of the most robust JavaScript editors, it has a clean JavaScript API, it comes with a lot of features already bound by default. The fact is, Cloud9 has specialized the editor for dozens of languages such as (JavaScript, Phyton, etc) but there isn’t any methodology nor tool to specialize ACE for custom languages. Thanks to our experience in model-driven engineering, we were able to automatize such a task, making it possible to get the editor just from the grammar specification.

To Backend or Not to Backend

It doesn’t matter in fact, both scenarios are handled by the tool. The tool is able to generate a static web project containing the ACE editor already customized for the language. This is a HTML/JavaScript project which runs entirely on the client side. Moreover, it can generate a JEE application based on Eclipse RAP with an Xtext backend. In the last scenario, the developer is free to customize where the features are handled. More info about the use cases in the online documentation.

Getting Hands on Codenvy and Eclipse Che, a Quick Feedback

Let’s have a very quick tour inside Eclipse Che which is the open-sourced Codenvy cloud IDE available under the umbrella of Eclipse Cloud Development (ECD). Well, the stuff is promising as it offers a platform for developing Cloud IDEs based on services, and comes with a methodology to build extensions – like we do in Eclipse RCP – but fully on the web.

After watching the videos on the official website here,  I started with the instructions on GitHub and built the Codenvy app with Maven. First potential pitfall: make sure you don’t have both Maven command line tool and the embedded Maven plugin in your Eclipse IDE. You may experience compilation problems, in that case, it’s not a bad idea to purge your .m2 folder.

All right, after five minutes, you got the beast built and ready to use (you may be asked to move to JDK 1.8 when compiling GWT). Let’s have a look into the checkouted git repo and see what actually happened.

Like Oh my God! You have a dedicated Tomcat IDE lying on your file system! The corollary of this: if you have already a Tomcat running on your machine, expect having troubles when running the startup script.

Once all the services running on port 80 have been stopped (by default Che runs on localhost:8080), let’s move to the next step. Under Windows, open a command prompt, navigate to the root of the repository then type che.bat start. Once the webapp starts, you are prompted to select a workspace location in your file system.

Now let’s see what’s already packaged with the default application, and start by creating a project… The good news is: the catalog of technologies already available is quite large. You can create projects targeting GWT, JSP, Spring, and a lot of other stuff I’ve never experienced before (well I’m not a reference)… Let’s create a Maven project then!

The project is created, you can check your file system, it’s already there. Then, create a Java class. The editor is opened and here we are! Let’s enter some code… And what a surprise! No syntax validation, no content assist… I’ve missed something or what?

Okay, let’s see what’s happening under the hood: open the JavaScript Console (if you are using Chrome), and guess what ? Already 24 errors out there….

Hmmm, let’s try to change the editor, go to Preferences > IDE Settings > Editors and change from Orion to CodeMirror (by the way, there is no other theme than the shitty black theme, I don’t know why most cloud IDEs are black by default…). Have a look to the sources, both editors source code are already available on the client side… Changing the editor does not fix the problems… Come on guys!!

I like the idea behind Eclipse Che, as it offers a methodology fully based on the web, i.e. a cloud IDE for developing cloud IDEs (of course, if you dislike the recursion, you can still develop cloud IDEs from an Eclipse IDE, ndlr Eclipse RAP) but it seems like it’s still in its early phase… I’m curious to see how fast it’s going to be adopted by developers. Let’s wait and see…

Online Coding/Modeling Taken to Another Level

It’s quite impressive to see how fast the number of cloud IDEs is growing these days, in particular online language editors as master components in all the available offers. A lot of online language editors are shipped with e-learning tools and cloud IDEs, most of them are based on well-established text editors such as ACE or CodeMirror. There are online language editors for almost all programming languages including: Java, JavaScript, C++, C# , PHP, HTML, CSS, XML, SQL, Python, Ruby, Scala, Sass, Objective-C, etc.

What about online DSL editors ?

Before attempting to answer the question, we should make clear why should someone need a DSL. There is a must-read paper available on the web, it gives you the big picture, so I won’t argue on top of this.  But to be more concrete, languages like Matlab, Modelica, or R can be seen as DSLs for mathematical modeling, HTML can be seen as a DSL for web applications, SQL is a DSL for manipulating relational databases… These are well-known DSLs, and there is a plenty of experts’ communities where DSLs are used. Not all of these people know today’s technologies can provide tools which could exploit DSLs to bring better understanding, productivity, and maintenability. Roughly speaking, it’s like telling the computer what to do, in the language You understand. The computer should understand your language (whatever it is), and not the opposite! 

Today, the only answer to the question is DSLFORGE. We continue growing, and as already mentioned in a previous post, we focused the last few weeks on the enhancement of the editors’ generator, and the results are promising. We are about to achieve a complete online modeling experience that goes beyond what we were expecting!

A quick review of the new features.

Theming

Client scripts have been cleaned-up and some tricky bugs have been fixed mainly in the theming. I was conscious to keep the Eclipse look-and-feel (keyword coloration, error/warning/info annotations, etc.) as if the editors were running on top of the usual Eclipse RCP. The theme is polished and clean now. Because the client is lifted by ACE, it gives a wide range of customization possibilities, and customizing the theme is not an exception: simply replace the theme script by another script (a multitude of themes are available for free in ACE demo site) and you are done.

Content Assist

We also added more functional features to the generated editors. A lot of effort has been spent on improving the content assist popup. Customizable templates (or snippets) are now proposed apart from the keywords and references. Different icons are now displayed next to each proposal (snippet, keyword, or reference) which make it more comfortable to distinguish between the available alternatives. In addition, documentation popup is also displayed next to each selected proposal.

 

Text Hover

It is now possible to display a documentation popup when hovering over a reference. HTML formatting is planned. The content of the popup is customized on the client side. To avoid unnecessary ping pongs between the client and the server for such a non critical feature, we recommend sending the documentation content right from the beginning, then manage the text hover purely on the client. In the back-end scenario, mapping documentation to EMF annotations is also supported.

 

Server-side Validation

Server-side validation is used when the editors need to interact with a Java-based back-end. The synchronization between the client and the server is more flexible now: DSL developers can decide whether they want to parse files content systematically on the server, or to delegate to the client parser (and synchronize on-demand). Both strategies are supported.

In the back-end scenario, legacy EMF validators, such as the Xtext validators, work out-of-the-box since the runtime plugin is packaged as-is in the web application (this is also the case for interpreters and code generators). In the example below, changing entity  name from “Adress” to “address” raises both error and info annotations. The info is reported by a custom @Check method in the Xtext validator, the error is reported by the linker as the reference named “adress” is no longer resolved.

 

 

Features Summary

Below the complete list of features already handled by default in the generated editors.

  • Syntax Highlighting (keywords, strings, comments, annotations, etc.),
  • Syntax validation (using generated parser and lexer),
  • Content assist (keywords and references),
  • Sever-side validation (aka semantic validation),
  • Scoping (global and local)
  • Template proposals (snippets),
  • Text hovering (display documentation),
  • Default Key bindings,
  • Undo/Redo Support,
  • Code Folding.

We’ve added some videos showcasing these features, but you can also get hands on the new features by your own here.

The Workbench is now available

Finally, the beast is online! For this very first version, I worked more on the surrounding infrastructure than the editor itself. But now that the ground work is done, my focus will be on enhancing the generated editor features for the upcoming releases. Again, if you are interested in integrating the workbench to your private network, publish your own language, provide use cases, or simply report a bug, please contact team@dslforge.org. Today, only users who have subscribed to the mailing list have thier credentials, so subscribe and get yours.

What’s new in DSL Forge?

The browser-based cool stuff soon available to play with!

dslforge-workbench

When it comes to manage multiple users accessing the same model resource online, the most straight forward way is to consider using a repository, such as EMFStore or CDO. The main reason why these technologies are not well suited for textual resources is the serialization format. Classical EMF resources are persisted in XMI, while resources with textual content are often persisted as-is, and this is also the case of Xtext resources.

Moreover, I don’t like the idea of persisting model content into databases… such a decision will make migrating models complex afterwards, compared to traditional file storage systems.

Anyway, I have at least two reasons to consider the problem right from the beginning. What we need is a component acting like a workspace in a web context. With the first prototype presented at MoDELS, the workspace was simply mapped to a root folder in the server’s file storage system.

To be able to manage concurrent users access, we need to keep track of information about the users accessing the workbench, such as their ID, name, company, projects, etc. We need to maintain the state of files currently locked, and by who the files are locked.

A database is a good candidate to store and manage such metadata, then we need to map the files in the file system to that metadata in the database, and provide a convenient API unifying the access to the couple (file storage system, database) as a unique singleton providing basic functions like creating a projet, creating a file, locking a file, etc. Ok, this sounds like an ECM repository right? After all, DSL files can be seen as “enterprise content”, and they are first-class entities in our scenario.

The Workbench has now an embedded Derby database, persistency is managed using Gemini and EclipseLink.

We also added an extension point that allows to integrate DSL editors easily. These are the DSLs currently contributed to the workbench:

  • Fowler’s DSL: a language for describing state machines,
  • Arithmetics: a language for describing arithmetic expressions,
  • Dimain Model: classical entities and relationships language
  • Humming Bird: a language for describing embedded system devices

Modeling and The Cloud

MoDELS 2014 Conference, in Valencia-Spain, was a very interesting opportunity to meet with researchers from academia and from the industry, and by the way discover the latest model-driven approaches and tools. “Modeling” technologies and the “Cloud” were quite recurrent terms. The buzz word was used in posters, demonstrations, and some talks as well. A satellite event called CloudMDE was organized in parallel, the proceedings are available here. Cool, so this is good news! The Cloud is gaining more attention in the Modeling community, and that’s normal; a lot of areas of computer science are seeking the Cloud, and Modeling is a part of it. The event was also an opportinuty to demonstrate DSLFORGE in front of a big audience (more than 400 conference attendees)… that was a great moment!

There is a plenty of reasons why people seek the browser approach. One interesting discussion was on how to use DSLFORGE as a front-end for demonstration purposes. Indeed, many people work remotely with customers, and they don’t necessarily want to impose to decision makers a full fledged IDE to see their latest achievements.

My focus this time was more on tools naturally, looking for some advanced techniques to do modeling online. At a first glance, modeling online seems today possible, there were some interesting demos indeed such as Umple and Txture. The former allows to update textual representations from UML diagrams, the latter allows to update textual representations using forms. None of the demonstrated tools allows to edit directly textual editors. There is a lack of technology support to get the working online wokbench you dreamt of, still a lot of challenges to come across before being able to see such next generation tools. I’m more convinced now that what we’ve done in DSLFORGE is a big step ahead.

I’ve been asked whether the tool is available for download… well the code is not open-sourced yet, however there is a mailing list if you want to get notified about the latest news. Please fill in your contact information below, you can unsubscribe at any time simply by sending an email to team@dslforge.org