{"id":3091,"date":"2018-11-21T12:10:26","date_gmt":"2018-11-21T12:10:26","guid":{"rendered":"https:\/\/sandbox.weareadaptive.com\/?p=3091\/"},"modified":"2020-08-13T15:32:07","modified_gmt":"2020-08-13T14:32:07","slug":"using-antlr-parse-calculate-expressions-part","status":"publish","type":"post","link":"https:\/\/sandbox.weareadaptive.com\/fr\/2018\/11\/21\/using-antlr-parse-calculate-expressions-part\/","title":{"rendered":"Using ANTLR to parse and calculate expressions (Part I)"},"content":{"rendered":"

In an upcoming series of blog posts, we are going to talk about how we have developed and integrated a simple Domain Specific Language using ANTLR4 with some of our Visual Studio projects on .NET and C#. We will also show how we\u2019ve used the generated code to evaluate expressions at runtime for various mathematical calculations.<\/p>\n

In this first entry, we\u2019ll discuss the initial steps, such as creating a grammar, visualizing the parsed expression tree and moving on to code generation as well as its inclusion in the .NET projects. We\u2019ll take advantage of some of the features in the new C# project structure (csproj) that comes with Visual Studio 2017 to ensure that the latest version of our grammar is always parsed, the code generated and included in our project structure.<\/p>\n

In later posts, we\u2019ll see how we can stream updates to a client, whenever a component\u2019s value is updated. Using EventStore and Reactive Extensions we can make it a push based model. But more on that later.<\/p>\n

For the uninitiated ANTLR or \u201cANother Tool for Language Recognition\u201d can be used (among other things) to build languages. More info on the ANTLR4 official website<\/a>.<\/p>\n

The Grammar<\/h2>\n

The first step is understanding the problem and writing a simple grammar to solve it. We need a way to parse custom expressions or \u2018formulas\u2019 that are allowed, parsed and evaluated at runtime once we have all the necessary information. Some of the variables in our grammar can be constant values, some are fed into our application by an external source, and at \u2018evaluation\u2019 time, we substitute them and calculate a result.
\nAn example of this could be the following:<\/p>\n

FXRate(\u2018EURUSD\u2019) * UomConvert(\u2018ST\u2019,\u2019MT\u2019) * 100<\/strong>
\nIn this formula, we need the EURUSD foreign exchange rate, the conversion factor between Short Tons and Metric Tons and finally, we multiply that by 100.<\/em><\/p><\/blockquote>\n

It is a simple example, but it represents the basic operations that we need to support.
\nParsing can be done with Regex, however, the resulting pattern to accommodate the requirements would be complex and difficult to understand, not to mention that it could also be more error-prone. A better solution should be something testable and extensible, as well as easy to understand to a newcomer. This is where ANTLR can be beneficial. Having a grammar that defines all our supported operations makes the code more readable and more maintainable than a very complex regular expression.
\nWe start with a grammar file that then gets fed into the ANTLR binary and the necessary classes are generated, to successfully work with the operations we want, in the form of C# classes.<\/p>\n

The grammar file defines every single element of our language. Starting from what is considered the building blocks (a digit, number, alphanumerical characters etc.) to the functions we want to support and finally the full set of allowed operations.
\nFor the purposes of this post, we will define a simple set of rules and some basic expressions that our language will allow to parse the above expression:<\/p>\n

grammar MyGrammar;\r\n\/* * Parser Rules *\/\r\nnumber: INT | FLOAT;\r\nfromUomCode: NAME | IDENTIFIER;\r\ntoUomCode: NAME | IDENTIFIER;\r\nfxRateFunc: \u2018FXRate\u2019 \u2018(\u2018currencyPair\u2019)\u2019;\r\ncurrencyPair: NAME;\r\nuomConvertFunc: \u2018UomConvert\u2019 \u2018(\u2018fromUomCode \u2018,\u2019 toUomCode \u2018)\u2019;\r\nexpr: expr op=(MUL | DIV) expr #mulDiv\r\n| expr op=(ADD | SUB) expr #addSub\r\n| number | \u2018(\u2018expr\u2019)\u2019 #num\r\n| fxRateFunc#fxRate\r\n| uomConvertFunc #uomFactor\r\n;\r\n\/* * Lexer Rules *\/\r\nfragment DIGIT: [0 \u2013 9];\r\nfragment LETTER: [a \u2013 zA \u2013 Z];\r\nINT: DIGIT +;\r\nFLOAT: DIGIT + \u2018.\u2019\r\nDIGIT +;\r\nSTRING_LITERAL: \u2018\\\u201d.* ? \u2018\\\u201d;\r\nNAME: LETTER(LETTER | DIGIT) * ;\r\nIDENTIFIER: [a-zA-Z0-9]+;\r\nMUL: \u2018*\u2019;\r\nDIV: \u2018\/\u2019;\r\nADD: \u2018+\u2019;\r\nSUB: \u2018-\u2018;\r\nWS: [\\t\\r\\n]+ -> skip;\r\n<\/pre>\n
Without needing to go into too much detail, we can see the basic components of our language. Lexer Rules define what a Digit, Letter, and Integer are. Other structures like Float, Name, String literal etc. are composed by combining primitive types, and basic arithmetic operators are also specified. As we can see, Lexer rules defining digits, letters and even string literals are very similar to Regex (in fact, it is all regex underneath). Meanwhile, ANTLR is keeping us away from the more complex regular expressions that are happening under the hood.<\/p>\n
The more interesting part of the grammar comes from the parser rules. These are defining the structure of the operations that we want to support in our parser. In our case, we want to support the following:<\/p>\n