Generate Web Editor from Xtext

Prerequisites

Follow this tutorial to get an Eclipse RAP web application with ACE/ANTLR running in the front-end, and Xtext in the backend.

To run the web editor on Eclipse RAP, you need to change the target platform. A target platform is a file which contains a set of URLs pointing to different update sites. When you open a target platform file (.target), the Eclipse provisioning system, P2, takes care of downloading the dependencies. You should however wait for a while, go take a coffee! Wait until the list of sites expand like this.

Target Resolved

Target Resolved

Target resolving...

Target resolving…

This process is done the first time, then P2 loads the plugins from its cache. You can export the plugins to your file system and use it offline.

To swich between targets, use Preferences>Plugin Development>Target Platform, the current target is marked as “Active”. Select Running Platform to get back to the default target.

Eclipse Preferences - Switching between targets

Eclipse Preferences – Switching between targets

 

The Statemachine Example

From the Package Explorer, select “New Project…” then expand “Xtext”, expand “Examples” then select the “State Machine Example”. Two projects are created:

  • eclipse.xtext.example.fowlerdsl
  • eclipse.xtext.example.fowlerdsl.ui

Here are the steps:

  1. Expand org.eclipse.xtext.example.fowlerdsl,
  2. Navigate to the workflow file “GenerateStatemachine.mwe2” which is located in the org.eclipse.xtext.example.fowlerdsl package,
  3. Right click on the file then click on “Run as>Mwe2 Workflow”.

You should now have the Xtext artefacts generated in both projects; you can already create a runtime Eclipse configuration, create a general project, then create some DSL files (right click from the newly create project, select New>File, enter a file name with “statemachine” as extension, and actually play with the Statemachine RCP editor.

Generate Xtext RAP Editor

Here are the steps to get the web artefacts: In Eclipse, select New > Project, select “Generate Web Editor from Xtext”, click next, load the Xtext grammar, and click finish.

DSL Forge Xtext/RAP Generator

DSL Forge Xtext/RAP Generator

Load Xtext grammar

Load Xtext grammar

Generator Options

Generator Options

Switch the Target Platform

In the generated .target project, under the folder named “statemachine”, you have a statemachine.target file,

  1. double click on the target definition file, and wait until the target is resolved.
  2. click on “Set as Target Platform” and you’re done.

When you switch to the Runtime target, the Xtext UI plugin does not compile, which is normal: the plugin has dependencies to Eclipse UI no longer resolved (you may also remove the dependency to org.eclipse.emf.mwe2.launch plugin in eclipse.xtext.example.fowlerdsl to fix possible compilation issues against the RAP target).

Exploring the Web Plugin content

The web plugin contains three source folders:

  • src and src-gen contain Java classes
  • src-js contains JavaScript files
 The Java Packages

The Java packages contain mainly the web editor, the language-specific text widget, the runtime Guice binding module, and the standalone setup. Report to Customize the editor for a detailed description of the src and the src-gen packages.

Web Plugin Packages

RAP Web Plugin – Generated Java Packages

The JavaScript package (src-js)

In the src-js/*.web package, you find the JavaScript RAP widget which has the same name as the grammar short name. Additionally there is a file called global-index which is a shared worker between the web editors. This shared worker makes it possible to cross reference editors between each other.

Then you have the src-js/*.web.ace package which contains the ACE files that are specific to the language, mainly:

  • mode-language: ACE uses the so called “modes” to embed language-specific attributes, such as syntax highlighting, code folding, brace matching, etc.
  • worker-language : a JavaScript worker which manages the interaction between the generated parser and lexer, and the language mode.
  • theme : contains the CSS theme of the editor.

In the src-js/*.web.ace.snippets, there is the templates file loaded by ACE Language Tools module, so that the templates appear in the content assist popup.

In the src-js/*.web.parser package, gibt es the generated ANTLR grammar file (.g), the JavaScript parser and lexer, and the standard library necessary to run the parser/lexer in a standalone mode.

Web Plugin - JavaScript packages

RAP Web Plugin – Generated JavaScript packages

Build the DSL Grammar

Once the grammar file (.g) is generated in the web plugin, the ANTLR builder generates automatically the JavaScript parser and lexer. If the generated grammar is correct, you should have something like the following in the console output.

ANTLR Parser Generator  Version 3.3 Nov 30, 2010 12:50:56.

Grammar:D:\dev\www\Tests\eclipse-rcp-mars-1-win32-x86_64\workspace\org.eclipse.xtext.example.fowlerdsl.web\src-js\org\eclipse\xtext\example\fowlerdsl\web\parser\InternalStatemachine.g

BUILD SUCCESS

Total time: 604 milliseconds

ANTLR Editor and Parser Generator

ANTLR Editor and Parser Generator

Launch the Web editor

A launch configuration is already generated inside the web plugin, simply right click on the launch file and select Run as > StatemachineWebEditor.

Launch Web Editor

Launch Web Editor

 

Customize the workspace root path

The workspace is mapped by default to D:/www/dslforge/workspace. You can customize this path using the org.dslforge.workspace.config.configuration extension point. Here is an example:

Workspace Configuration

Workspace Configuration

 

In Eclipse, under Debug Configuration, run as a RAP Application, then Open in External Web browser. In the Text Editor Demo, the servlet path is /texteditor.

In the Bundles tab, make sure the imported plugins satisfy the following constraints:

  • Your language plugins are selected (the original Xtext plugin and the generated web plugin),
  • The text editor demo plugin is slected
  • Equinox plugins are selected,
  • Jetty plugins are selected,
  • Felix plugins are selected,

Adjust the plugins start levels. The workspace plugin start level should be higher than the persistency unit start level, which is in turn higher than gemini.jpa start level. For example, if the default level is 4, than make sure the following order is respected.

  • org.osgi.enterprise startLevel = 1
  • org.eclipse.gemini.jpa startLevel = 2
  • org.dslforge.workspace.jpa startLevel = 3
  • org.dslforge.workspace startLevel = default
Debug Configuration

Debug Configuration

Click “Debug” now, you should have the workbench demo running with your DSL and its Xtext features (validation, code generation, content assist) already integrated with the workspace, menus, wizards, etc. Welcome on board!

DSL Forge Text Editor Demo

DSL Forge Text Editor Demo

Supported Browsers

Chrome is preferred, as the Eclipse default browser doesn’t implement the Shared Worker specification; multithreading will work only between each editor and its language worker, however the global index will not be initialized. In this situation, one can compute and push to the client the index references from the server instead of using the shared worker.

Deployment

Jetty is preferred in Eclipse/OSGI environments. For J2EE environments, the editors can be deployed using Maven, Tycho and Jenkins on classical servlet containers (Apache Tomcat); the Equinox Servlet Bridge component being packaged with the war bundle,  allows translating network requests between OSGI and the Servlet Container. Contact the support for more information about “real world” servers deployment.

Useful links

[1] Installing RAP target: https://eclipse.org/rap/developers-guide/devguide.php?topic=target-install.html&version=2.0

[2] Setting up ANTLR with Eclipse: https://vimeo.com/groups/29150/videos/8001326

[3] ACE on GitHub: https://github.com/ajaxorg/ace/wiki/Embedding—API

[4] Convert range to offset: http://avalonedit.net/documentation/html/5b1854b4-884c-4713-b921-b28e96a1b43e.htm

[5] Qooxdoo: http://qooxdoo.readthedocs.org/en/latest/pages/core/oo_feature_summary.html#inheritance

[6] RAP incubator: https://eclipse.org/rap/incubator/

[7] Xtext: https://eclipse.org/Xtext/

[8] ACE: https://ace.c9.io

[9] ANTLR Works https://github.com/antlr/antlrworks

[10] ACE API https://ace.c9.io/api/selection.html

[11] Five minutes introduction to ANTLR 3: https://theantlrguy.atlassian.net/wiki/display/ANTLR3/Five+minute+introduction+to+ANTLR+3

[12] ANTLR: http://www.antlr.org/

4 Steps to get your DSL online!

Get the web plugin to package with RAP, a Maven/Tycho project to actually build and package the editor as a web application archive (war), and the target platform on top of which to compile and build the web application. Build the web archive and deploy on Tomcat.

Installation

The current DSL Forge version is based on Eclipse Mars packages. The packages are available under http://www.eclipse.org/downloads/packages/ To know which package is the most appropriate for you, select one of the following alternatives.

Overview

DSL Forge is a framework for the development of domain-specific language (DSL) editors for web applications and cloud IDEs. The framework makes it possible to create and edit models, launch automated tool chains on the server, including model-to-model transformations, code generation, live validation, and execution of the generated artefacts.

Generate Web Editor from Xtext

Generate a RAP editor with both client-side ANTLR Parser, and Xtext runtime features in the back-end. The Xtext/RAP Generator option produces an Eclipse plugin with the suffix “.web”. To compile the plugin, don’t forget to switch the target platform to RAP.

Generate Web Editor from ANTLR

The Generator takes as input an Xtext grammar, generates JavaScript parser and lexer, and does the wiring with an ACE Editor. As Xtext takes a subset of ANTLR while adding the integration layer for EMF, the generator ignores the extra info, and extracts the ANTLR grammar to generate the “.g” file.

Customize the Web editor

The generated code contains Java classes and JavaScript sripts; both artefacts can be customized according the developer's needs. Below some hints on how to customize the generated editors, for more advanced customizations, feel free contact the support.