Generate Web Editor from ANTLR

Use this tutorial to get a standalone (in the sense not wired to any Java back-end) Web Editor written in HTML, CSS, and JavaScript. The editor is packaged in the form of an Eclipse plugin. The language is specified with ANTLR, a builder is provided to update the infrastructure on-the-fly. Simply edit the grammar with ANTLR v3 and save the specification, the infrastructure is kept up-to-date with the grammar.

The editor is based on Cloud9’s ACE editor (ajax.org).

Prerequisites

Before using the generator, make sure you have the following environment:

  • Eclipse Web Tools Platform (WTP),
  • DSL Forge Tooling installed.

The Pirate Robot example

We take a simplified example of the PirateRobot language. In this tutorial, we generate an editor for a very simple language for instructing actions to a robot: it can go up, down, jump, fight, etc. More information about the example can be found in modeling-languages.com

Here are the steps to get a text editor for the language:

  1. In Eclipse, click File > New > Project …
  2. Expand DSL Forge category,
  3. Select Generate Web Editor from ANTLR,
  4. Enter the project name, for example org.codingpark.piraterobot,
  5. Enter the name of the language grammar, for example PirateRobot,
  6. Finish!
ACE/ANTLR Generator

New ACE/ANTLR Generator

Enter the project and the grammar names

Enter the project and the grammar names

The Generator outputs a parser and lexer, and does the wiring with ACE (language mode and a worker responsible for syntax validation). The grammar is localized in the “PirateRobot.g” file.

/*Generated by DSLFORGE*/

grammar Piraterobot;

options {
language=JavaScript;
output=AST;
ASTLabelType=CommonTree;
}

@lexer::header {
}

@parser::header {
}

rule_PlayModel:
actions+= rule_Action*
;

rule_Action:
(kind =’up’ ‘(‘ (parameter = INT)? ‘)’) |
(kind =’down’ ‘(‘ (parameter = INT)? ‘)’) |
(kind=’left’ ‘(‘ (parameter = INT)? ‘)’) |
(kind =’right’ ‘(‘ (parameter = INT)? ‘)’) |
(kind=’jump’ ‘(‘ (parameter = INT)? ‘)’) |
(kind=’fight’ ‘(‘ (parameter = INT)? ‘)’) |
(kind=’dig’ ‘(‘ (parameter = INT)? ‘)’) |
;

ID : (‘a’..’z’ | ‘A’..’Z’ | ‘_’) (‘a’..’z’ | ‘A’..’Z’ | ‘_’ | ‘0’..’9′)* ;

STRING : (‘”‘ (‘\\’ (‘b’|’t’|’n’|’f’|’r’|’u’|'”‘|’\”|’\\’)|~((‘\\’|'”‘)))* ‘”‘|’\” (‘\\’ (‘b’|’t’|’n’|’f’|’r’|’u’|'”‘|’\”|’\\’)|~((‘\\’|’\”)))* ‘\”);

COMMENT : (‘/*’ .* ‘*/’ | ‘//’ ~(‘\r’ | ‘\n’)*) { $channel = HIDDEN; } ;

WS: (‘ ‘|’\r’|’\t’|’\u000C’|’\n’) {$channel=HIDDEN;} ;

INT: (‘0′..’9′)+;

 

This grammar is a starting point, when you change the grammar and save the file, (1) the builder regenerates the parser on-the-fly under WebContent/ace/parser. Then you have to (2) synchronize the ACE modules with the grammar using the contextual menu action.

Both steps are illustrated below.

1. Customize the Grammar

Whenever the grammar file is modified, the ANTLR Builder generates automatically the parser and lexer for the laguage. If the grammar is correct, you should have something like this in the console output.

ANTLR Parser Generator Version 3.3 Nov 30, 2010 11:52:12.
Grammar: /org.codingpark.piraterobot/WebContent/parser/Piraterobot.g
BUILD SUCCESS
Total time: 196 ms

Otherwise, you’ll have the ANTLR errors:

line 27:37 mismatched input ‘)’ expecting ‘:’
line 28:9 mismatched input ‘?’ expecting ‘:’
line 28:34 mismatched input ‘=’ expecting ‘:’
line 28:39 mismatched input ‘)’ expecting ‘:’

ANTLR v3 Builder

ANTLR v3 Builder

2. Regenerate ACE modules

Once the parser/lexer regenerated by the incremental builder, the ACE mode and worker can be synchronized by right clicking on the grammar file and selecting Generate artefacts from ANTLR action. The action is accessible from the package explorer, and the ANTLR editor as well.

Generate ACE Artefacts

Generate ACE Artefacts

 

Run the Web Editor

Under Eclipse WTP, use the HTTP Preview server as  a runtime server, for viewing the generated web project. The Preview server is embedded as test serer integrated with the workbench; a convenient way to check for errors before deploying on real servers.

To create the server, here are the steps:

In the Servers View, right-click then select New > Server.

In New Server wizard, under the Select the server type list, Under Basic, select HTTP Preview. Click Next.

 

Server Configuration

Define New Server

In Add and Remove. Under Available projects, select the Web project and click Add button. Click Finish. The HTTP Preview server should be visible in the Servers view.

test-editor-httppreview2

In the Project Explorer view, navigate to the Web project, right-click on the index file, Select Run As > Run on Server.

test-editor-httppreview3

In the Server wizard, verify Choose an existing server is enabled. Under Select the server that you want to use, select your sever. Click Next.

test-editor-httppreview4

The Add and Remove page opens. Verify the project is listed under Configured projects. Click Finish.

 

test-editor-httppreview5

 

Supported Browsers

Chrome is preferred, as the Eclipse default browser doesn’t implement the Shared Worker specification; multithreading will work only between each editor and its language worker, however the global index will not be initialized.

test-editor-httppreview6

Eclipse Internal Browser

Exploring the Web Editor project

The WebContent/ace folder contains the ACE files, mainly:

  • ace.js : this is the base script that manages all ACE modules. This script is kept as-is from the ACE repository, no need to tweak it.
  • ext-language_tools : this is the modules which manages the content assist popup. The original script has been adapted to the DSL editors
  • ext-searchbox : manages the search function (CRTL+F)
  • ext-tooltip : display the text hover
  • mode-language: ACE uses the so called “modes” to embed language-specific attributes, such as syntax highlighting, code folding, brace matching, etc.
  • worker-language : a JavaScript worker which manages the interaction between the generated parser and lexer, and the language mode.
  • eclipse-theme : contains the CSS theme of the editor.

In WebContent/ace/snippets, there is the templates file loaded by ACE Language Tools module, so that the templates appear in the content assist popup.

In WebContent/ace/parser package, there is the generated parser and lexer, and the ANTLR v3.3 library necessary to run the parser and lexer with JavaScript as target.

Useful links

[1] Current WTP state: https://jaxenter.com/eclipse-mars-javascript-development-tools-118280.html

[2] WTP tutorial: http://www.vogella.com/tutorials/EclipseWTP/article.html

[4] Setting up ANTLR with Eclipse: https://vimeo.com/groups/29150/videos/8001326

[5] ACE on GitHub: https://github.com/ajaxorg/ace/wiki/Embedding—API

[6] http://modeling-languages.com/javascript-drawing-libraries-diagrams/

[7] Xtext: https://eclipse.org/Xtext/

[8] ACE: https://ace.c9.io

[9] ANTLR Works https://github.com/antlr/antlrworks

[10] Testing HTTP Preview http://help.eclipse.org/juno/index.jsp?topic=%2Forg.eclipse.wst.server.ui.doc.user%2Ftopics%2Fthttppreview.html

[11] ACE API https://ace.c9.io/api/selection.html

[12] Five minutes introduction to ANTLR 3: https://theantlrguy.atlassian.net/wiki/display/ANTLR3/Five+minute+introduction+to+ANTLR+3

[13] ANTLR: http://www.antlr.org/

4 Steps to get your DSL online!

Get the web plugin to package with RAP, a Maven/Tycho project to actually build and package the editor as a web application archive (war), and the target platform on top of which to compile and build the web application. Build the web archive and deploy on Tomcat.

Installation

The current DSL Forge version is based on Eclipse Mars packages. The packages are available under http://www.eclipse.org/downloads/packages/ To know which package is the most appropriate for you, select one of the following alternatives.

Overview

DSL Forge is a framework for the development of domain-specific language (DSL) editors for web applications and cloud IDEs. The framework makes it possible to create and edit models, launch automated tool chains on the server, including model-to-model transformations, code generation, live validation, and execution of the generated artefacts.

Generate Web Editor from Xtext

Generate a RAP editor with both client-side ANTLR Parser, and Xtext runtime features in the back-end. The Xtext/RAP Generator option produces an Eclipse plugin with the suffix “.web”. To compile the plugin, don’t forget to switch the target platform to RAP.

Generate Web Editor from ANTLR

The Generator takes as input an Xtext grammar, generates JavaScript parser and lexer, and does the wiring with an ACE Editor. As Xtext takes a subset of ANTLR while adding the integration layer for EMF, the generator ignores the extra info, and extracts the ANTLR grammar to generate the “.g” file.

Customize the Web editor

The generated code contains Java classes and JavaScript sripts; both artefacts can be customized according the developer's needs. Below some hints on how to customize the generated editors, for more advanced customizations, feel free contact the support.