Skip to main content

B3.Syntax Summary

trydoforOriginal7/4/22MeepoTemplateAbout 11 min

B3.Syntax Summary

In scenarios where templates are used, it is particularly desirable that the template syntax does not destroy the syntax of the target file and that the two coexist without interference.

For example, when generating java with velocity, I hope the template can be checked and highlighted in both velocity and java syntax. when generating html, the template should be able to preview directly in the browser without breaking the html syntax.

  • template syntax - syntax of template engine, such as FreeMarker, Velocity.
  • target syntax - aka. native syntax, syntax of the target file, such as java, html.
  • meepo syntax - simple markup directives that use the comments of the target syntax to translate the template.
  • All text order is left-to-right, right-to-left language is not supported.

The directive is made by comment of the native syntax and the replacement is processed line by line to output the underlay template (backend).

The processing of the native syntax is divided into two processes, parse and merge, and the lookup in parsing depends on regular expressions. When merging, except for RNA, it is output directly, and the effectiveness is equal to StringBuilder.append.

There is no complex flow control and execute function in RNA, so parsing once and the subsequent merging is very efficient.

  • Without RNA, equivalent to a static string for idempotent operations, merge only once, used directly afterwards
  • RNA relies on an execution engine, equivalent to Map + StringBuilder except for dynamic language
  • java execution engine, dynamically compiled to bytecode, equal to native java class after first compilation
  • StringBuilder pre-calculates the length to avoid expansion copying in the process

B3.1.Template Features

There are some basic concepts and conventions in Meepo syntax, as follows,

  • blank - one 0x20 or 0x09, i.e. space or \t
  • alphanum - [a-zA-z0-9], letters and numbers, case-sensitive
  • native comment - comments of native language, e.g., //, /* and */
  • directive - special annotation in Meepo, such as a prefix
  • DNA - Defined Native Annotation
  • RNA - Runnable Native Annotation
  • ? +* - [0,1], [1,∞), [0,∞), respectively
  • directive line - directive line is only parsed by Meepo and invisible in merge
  • line - refers to [^\n]\n or [^\n]$ format

For simplicity, the native comment + blank before the directive is omitted in the following,

  • HI-MEEPO Hi! Meepo, define native comment for subsequent parsing
  • DNA:SET Set replacement, define a substitution and its scope
  • DNA:END End effect, stop acting when or where
  • DNA:BKB Effect Immunity, the region before END is not parsed
  • DNA:RAW Native raw string, after execution, output the raw string behind it
  • DNA:SON Sub-template, used as a component to import other templates during parsing
  • RNA:PUT Put variable, execute the execute body and store the result in the environment
  • RNA:USE Use variable, use the built-in or PUTed variable in environment
  • RNA:RUN Execute every time, executes function body every time, such as counter
  • RNA:WHEN Conditional execution, combined into if-elseif-else logic blocks
  • RNA:EACH Loop execution, loop in an array or collection
  • RNA:ELSE Else condition, the else branch for WHEN and EACH
  • RNA:DONE Stop executing, ends the scope of WHEN and EACH

Always use \n as line terminators to break line, even if \r\n in window is processed as \n. In the single line comment style that ends with\n, the\n is considered as a syntax terminator, and will not be output in the merge.

Some RegRxp background knowledge is required, as Java's RegExp is used to parse the template.

  • In searching, .^$?+*{}()[]\| is always misused, no need to escape some chars in []
  • In replacement, \ is mainly used as a back reference, and some of the () combinations have special meanings.

The regular compile options are UNIX_LINES and MULTILINE, set by the embedded flag,

  • (?idmsuxU-idmsuxU) Nothing, but turns match flags on/off
  • (?idmsux-idmsux:X) X, as a non-capturing group with the given flags on/off
  • i Pattern.CASE_INSENSITIVE Not case-sensitive, default off
  • d Pattern.UNIX_LINES Only \n as line terminators, default on
  • m Pattern.MULTILINE ^ and $ will match line terminators, default on
  • s Pattern.DOTTAL . match line terminators, default off
  • u Pattern.UNICODE_CASE Unicode-aware case folding, default off
  • x Pattern.COMMENTS whitespace and comments in pattern, default off
  • U Pattern.UNICODE_CHARACTER_CLASS, default off

B3.2.HI-MEEPO Hi! Meepo

Syntax: comment head blank+ HI-MEEPO !? blank+ comment tail?

Define comment and identify this file as the native file that Meepo will parse.

  • comment head - The first part of a single or multi comment, the previous non-blank string
  • HI-MEEPO - The strange name (hi meepo) is to avoid repetition and escape
  • ! - The ! preserves the blank before and after the directive, explained later
  • comment tail - The end of multi-line comment, the latter non-blank string

Hi! Meepo must be on a separate line, and using blank can improve the readability. Similar to using DELIMITER in sql to define the terminator, for example,

  • java - // HI-MEEPO, // is comment
  • java - /* HI-MEEPO */, /* and */ are comment
  • sql - -- HI-MEEPO, -- is comment
  • bash - # HI-MEEPO, # is comment
  • html - <!-- HI-MEEPO -->, <!-- and --> are comment

Note: that the comment head can have repeated letters, and Meepo only handles the repetition of the same character, for example,

  • /* - /***** DNA:RAW, invalid, nothing to do
  • // - ////// DNA:RAW, valid, handle the repeats
  • # - ##### DNA:RAW, valid, handle the repeats

For the following text (DNA and RNA) parsing, there are 2 types of parsing, line parsing and block parsing, rules as follows,

  • HI-MEEPO is always a line parser and must occupy a separate line.
  • single line comment style Meepo is parsed line by line.
  • Multi-line comment style Meepo is read across lines and parsed by block.
  • Parsing is not always done line by line, so^ and $ are not always at the beginning and end of a line in the regular matching.

HI-MEEPO! and HI-MEEPO handle the first blank of the directive line according to the following rules,

  • Without ! and the whole line directive will ignore its line in the output, i.e. the first \n after the directive line.
  • With !, only the directive between the head and tail of the Meepo is processed, and the blanks before and after are preserved.
  • DNA:RAW is special, the ! has no effect on it, it keeps its outside and removes the first blank inside it.
  • In @<!--_DNA:RAW_SUPER_-->@, @ and _ denote reserved and removed blanks, respectively.

In subsequent examples, // HI-MEEPO is used as an example, but omitted.

B3.3.Factory DNA, Static Replacement

DNA is like a factory director that defines replacement directives for efficient static text replacement at parse time.

B3.4.DNA:SET Setting Replacement

Syntax: DNA:SET blank+ delim find delim repl delim effect?

Replaces the pattern matching string within a specified scope or times.

  • delim - delimiter, the 1st non-(blank, !, alphanum) 1-2byte char, such as /
  • find - RexExp pattern (without delim), refer to backreference if grouping exists
  • find is empty, this SET is ignored.
  • repl - replace, string (without delim), refer to backreference if grouping exists
  • repl is empty, it means delete, i.e. replace with empty
  • effect - The number of times to take effect, i.e. when/where to stop, not blank

The backreference refers to the case where there is () in find or the \1 inrepl. This affects the delimiter of the find/replace string, and also avoid writing complex expressions with the following convention rules.

  • If there is no group in find, use group(0), i.e., match all.

  • If find has group, take the first (, i.e. group(1) content.

  • If ((A)(B(C)), count ( in the order from left to right.

    • group(1) - ((A)(B(C)))
    • group(2) - (A)
    • group(3) - (B(C))
    • group(4) - (C)

  • times, values separated by , or closed range with -, e.g. 1-3,15.

  • named, infinite times that can be terminated by END.

// DNA:SET /false/{{user.male}}/
var isMale = false;
/* Replace only false in above line with the variable user.male. the output is :
var isMale = {{user.male}};
*/

B3.5.DNA:END End Effect

Syntax: DNA:END (blank+ effect)+

End the scope of effects created by directives, such as the named scope of SET.

// DNA:SET /1010100/{{id}}/id
// DNA:SET /"(VAR)"/{{desc}}/1
// DNA:SET /VAR/{{info}}/2
SUPER(1010100, "ConstantEnumTemplate", "VAR", "VAR")
// DNA:END id
/* Defines named id, desc's group(1), info's 2 matches. the output is :
SUPER({{id}}, "ConstantEnumTemplate", "{{desc}}", "{{info}}")
*/

B3.6.DNA:BKB Effect Immunity

Syntax: DNA:BKB blank+ effect

Define a named global effect immunity scope that can be ended by END, and the text and directive within it will not be processed.

  • text - any text that is not Meepo directive
  • directive - treated as text except for the END which corresponds to the current BKB.
  • There can only be one BKB currently at a time.
// DNA:BKB KING
// DNA:SET /"(VAR)"/desc/1
SUPER(1010100, "ConstantEnumTemplate", "VAR", "VAR")
// DNA:END KING
/* ignore the SET directive, the output is :
// DNA:SET /"(VAR)"/desc/1
SUPER(1010100, "ConstantEnumTemplate", "VAR", "VAR")
*/

B3.7.DNA:RAW Native Raw String

Syntax: DNA:RAW blank+ raw string

Define a template with comment syntax to compensate for cases where the native syntax is not supported. The use of single-line comment is simple and clear. and the use of multi-line comment to keep only the content between head and tail.

The effect is to remove the comment head, DNA:RAW, comment tail and the blank in them.

/* The following 2 lines have the same output i.e. `// DNA:RAW ` */
SUPER(1010100, "ConstantEnumTemplate", "VAR", "VAR")
// DNA:RAW SUPER(1010100, "ConstantEnumTemplate", "VAR", "VAR")

B3.8.DNA:SON Sub-Template

Syntax: DNA:SON blank+ path

Reads the resource in UTF8 from path and expands its contents at the current position, and then parse them all. The path can contain a protocol, classpath by default, only the following protocols are supported.

  • http://,https://, read by GET request
  • file://,/, read from file system
  • classpath:, read from the classloader, Note no // here
  • Starting with / or . means the relative path to the host template, but is not recommended.

Note: the son templates must include HI-MEEPO separately, which is a static parsing in the separate context and later merged into the current parent template.

B3.9.Workshop RNA, Dynamic Execution

RNA is like a workshop manager, defining execution directives, calling the execution engine at merge time, and using its result as a replacement.

  • An execution engine can execute func (function body) of multiple types, where one type is referred to as an engine.
  • The name of engines must be alphanum, case sensitive, e.g. js.
  • Engine whose name endwith ! such as js!, will ignore errors ,continue execution, returns null.
  • If the execution returns null, empty string is used to merge the template.

The default engine in RNA is map. Users can register other engines with RnaManager, described later.

  • map - session level, use func as key, go value from environment, if not, return the key.
  • raw - nothing level, return the func directly as a string without expanding the escapes.

Meepo uses multi-line block parsing for multi-line comments, so the func naturally supports multi-line to improve readability.

B3.A.RNA:PUT Put Variable

Syntax: RNA:PUT blank+ engine? delim var delim func delim

Specify the engine to execute the func and store the function or execution result in the environment (refers map engine) for other RNA to use.

  • environment is the Meepo context and some scripting engine's context
  • engine see engine description for details
  • delim is the same as SET.
  • var is the variable stored into the context, not the native literal.
  • func is executed by a specific engine, such as spring, then it can be executed a SpEL.
  • if var or func is empty, nothing is executed.
// DNA:PUT os/who/basename $(pwd)/
/* Take the output of `basename $(pwd)` and store it in context with `who` as the key */

B3.B.RNA:USE Use Variable

Syntax: RNA:USE blank+ delim find delim var delim effect?

RNA version of SET, the difference is the var value is get from the map engine, not the literal replacement of the underlying template. The rules for getting variable(e.g., navigation class objects, pipeline handling functions) are described in the map engine section.

Automatic multi-paragraph indentation support is performed when merging vars, depending on the type of var value satisfying the follows,

  • find string starts with indented whitespace.
  • var value is an Array or Collection and its size is greater than 1.

Indenting more than 2 elements will align with the 1st element. sometimes lines may be unintelligent and look bad. eg.

  • Indented object, no \n ending, no line break, like zebra stripes
  • Unindented object, contains \n, with line break, look interlaced
// DNA:USE /meepo/user.home/
var userHome = "meepo";
/* get System.getProperty("user.home"). the output is:
var userHome = "/home/trydofor";
*/

B3.C.RNA:RUN Execute Every Time

Syntax: RNA:RUN blank+ engine? delim find delim func delim effect?

combine PUT and USE, support indentation, the difference are,

  • find, empty means execute only, no replace
  • func, use execution result directly, no store to var
  • execute every time, eg. counter function, increment on each call, no cache.
// DNA:RUN os/rand/echo $RANDOM/1-3
var userName = "meepo-rand";
var userPass = "rand-rand";
/* get random number each time and print. the output is:
var userName = "meepo-12599";
var userPass = "16345-31415";
*/

B3.D.RNA:WHEN Conditional Execution

Syntax: RNA:WHEN blank+ engine? delim bool delim func delim group

Multiple WHEN can be combined into if-else if-else logic blocks.

  • bool - Must be y|yes|n|no|not, indicating true or false
  • func - Evaluate the execution result to bool
  • group - Must be alphanum, group ELSE and DONE in the logic scope

Evaluate the following cases get false, evaluate not to false get true

  • boolean false
  • object null
  • number NaN or between positive/negative 0.000000001 (9 digits)
  • empty string, zero size of Array/Collection/Map
<!-- RNA:WHEN /yes/it.rem0/bg -->
<li value="code">rem0-name</li>
<!-- RNA:WHEN /not/it.rem1/bg -->
<li value="code">rem2-name</li>
<!-- RNA:ELSE bg -->
<li value="code">rem1-name</li>
<!-- RNA:DONE bg -->

Equivalent to the if(a){}else if(!b){}else{} in following js pseudocode

if (it.rem0){
    console.log('<li value="code">rem0-name</li>')
} else if (!it.rem1){
    console.log('<li value="code">rem2-name</li>')
} else {
    console.log('<li value="code">rem1-name</li>')
}

B3.E.RNA:EACH Loop Execution

Syntax: RNA:EACH blank+ engine? delim step delim func delim group

Using group to refer to the current element in the loop. if the group is named it, then it.x represents the x attribute of the element.

  • step - must be -?[0-9]+, the order and step, negative is reverse looping
  • func - engine execution result, must be array/collection, otherwise equal to RNA:PUT
  • group - Must be alphanum, group ELSE and DONE in the logic scope, and refer to current element

Depending on the data type, different loop processing is performed, null or empty is skipped and can be executed within ELSE.

  • Array - Class.isArray()
  • Collection<E> - instance of Collection
  • Other types, do not loop
  • Reverse-order loop, not (RandomAccess or ReverseIterator) are converted to toArray

In the loop, the following built-in properties indicate the current status, given the group named it, then

  • it - The current element reference, should avoid the name conflict to pollute the context
  • to reference the x attribute of the current element useit.x
  • it._count - Built-in variable, current loop counter, 1-base, 0 means not looped
  • it._total - Built-in variable, total number of elements in the group
  • it._first - Built-in variable, whether the first element
  • it._last - Built-in variable, whether the last element
  • These built-in variables are not removed at the end of the loop and can be used outside the loop.

Since Meepo is a professional non-professional template engine, this for-each is very low-level,

  • Limited support for object navigation, separated by ., see the map engine for details.
  • The element in the loop supports only Map<String,? > and JavaBean's Getter.
  • No scope isolation, the group name can override the variables within the context.
<!-- RNA:EACH map/2/items/it -->
<!-- RNA:USE /name/it.name/* -->
<li value="code">rem0-name</li>
<!-- RNA:ELSE it -->
<li>no item</li>
<!-- RNA:DONE it -->
<!-- RNA:USE /total/it._total/ -->
<!-- RNA:USE /count/it._count/ -->
<div>result=count/total</div>

Equivalent to the for(;;) or for-in loop logic of the following js pseudo-code, depending on the collection type and the step positive or negative.

let step=2 // Loop steps, negative numbers in reverse order, must not 0
let index=0 // Temporary variables in the process
let it = null, count=0, total=items.length; // built-in variable
for(it in items){
    if(index++ % step !== 0) continue // Control step
    count++
    console.log('<li value="code">rem0-'+it.name+'</li>')
}
if(count === 0){
    console.log('<li>no item</li>')
}
console.log('<div>result='+count+'/'+total+'</div>')

B3.F.RNA:ELSE Else Condition

Syntax: RNA:ELSE blank+ group

group by group and handle else branch on WHEN or EACH with the same group.

  • If WHEN, it means none of the WHEN is executed
  • If EACH, it means the loop body is never executed (e.g. no elements)
  • EACH-ELSE has the same semantics as pebble, is different from python's for-else

B3.G.RNA:DONE Stop Executing

Syntax: RNA:DONE (blank+ group)+

group by group to stop executing of one or more WHEN and EACH with the same group.

B3.H.Placeholder Template

Simplified template to perform only expression-level replacement or function handling, rather than the full Meepo template syntax. For example, placeholders in configuration files usually require simple replacement or string conversion.

To use it, just customize the boundary around the variable, {{ and }} by default.

Define escaping char can escape the boundary, default is empty, no escaping. The escaping has the following characteristics, using \ for example,

  • Available only for boundary, e.g. \{{ and \}} to {{ and }}
  • Escape itself before the boundary, such as \\{{var}} to \ + var value
  • Placeholder, paired from left to right, use the closest pairs, non-matching boundary is treated as plain text.
  • Other cases are invalid, such as \n
  • Nested Placeholders is not supported
  • Variable names cannot contain spaces, or they will be resolved as a function

B3.I.Limitation of Using RNA

Since the template generates a syntax tree during static parsing, the conditionals and loop execution in RNA do not recognize the runtime structure.

The following template intent is, to use the first truthy value between GitHash and ModTime by pattern hash. Intuitively, this format is more consistent with procedure-oriented conditional assignment, but causes a crossover scope error during syntax checking.

// HI-MEEPO
// RNA:WHEN /yes/GitHash/bg
//   RNA:USE /hash/GitHash/
// RNA:ELSE bg
//   RNA:USE /hash/ModTime/
// RNA:DONE bg
project hash

The crossover error occurs because the pattern hash has are two RNA:USE using it, In static parsing, the runtime value is not available, so it can not determine the correct the syntax tree to belong to, change it as follows

// HI-MEEPO
// RNA:USE /hash/GitHash/
// RNA:USE /time/ModTime/

// RNA:WHEN /yes/GitHash/bg
project hash
// RNA:ELSE bg
project time
// RNA:DONE bg

Or Use the fun:see function to help

// HI-MEEPO
// RNA:USE /hash/fun:see GitHash ModTime/
project hash