• View
• Edit
• History
• Attach
• Print
• Backlinks

PmSyntax

Syntax highlighting for the PmWiki documentation and edit form

How to add your custom markup to PmSyntax, see Cookbook:CustomSyntax.

Description

This is an experimental recipe that enables syntax highlighting for the PmWiki documentation, and optionally for the basic PmWiki edit form.

It has a standalone renderer function, no longer dependent on Highlight.js.

The recipe is enabled on pmwiki.org, you can try it on any page. Quick demo of the styling.

The reasons for writing a new recipe from scratch, rather than reusing the CodeMirror / Highlight.js integration, are:

• Simple and lightweight.
• Familiar/consistent highlighting styles, easily adaptable.
• Easy to install and update.
• It works well with Cookbook:EditHelp and the PmWiki core features remembering the edit position, not saved warning.
• On GNU/Linux, the standard selection-copy and middleclick-paste functions are preserved, as well as the standard keyboard shortcuts for plain text editing.
• The browser's built-in spell-checker works fine.
• Like for the core documentation, it highlights for easy noticing most of the PmWiki-specific punctuation, variables, WikiStyles, and directives from the core and from many recipes. It doesn't change the font sizes, variants and decorations.
• Core (:markup:) blocks are highlighted -- can be used with (:markup class=norender:) to highlight blocks without processing the markup.
• Adds specific wikistyles %hlt pmwiki% and %pmhlt% which can be applied to preformatted blocks.
• Nested markups should work well in most real-world cases.
• It really is a joy to use!

Installation

This recipe requires a recent PmWiki version (2.3.1 or more recent), or the subversion pre-release of PmWiki which you can get from PmWiki:ChangeLog.

To enable it, add to config.php:

The recipe is enabled on pmwiki.org, you can test it on any page.

Usage

Reading

You can see the PmWiki syntax highlighted in the core documentation, for example in BasicEditing, Tables, Forms, WikiStyles, or see many markup rules on the same page.

1. The (:markup:) blocks are automatically highlighted (see left column):

* List [[Link]]
## Item %item value=4%

2. Preformatted blocks with the wikistyle %pmhlt% or %hlt pmwiki% are not rendered by PmWiki but are highlighted (see right column):

%pmhlt%[@
* List
## Item [[Link]]

(:pagelist group=Main \
 name="WikiSandbox,HomePage" \
 order=-$Title $:Summary=?*:)
@]

* List
## Item [[Link]]

(:pagelist group=Main \
 name="WikiSandbox,HomePage" \
 order=-$Title $:Summary=?*:)

Editing

If you have enabled the highlighting in the text editor ($EnablePmSyntax = 2;) you can also highlight the wiki markup while you edit pages.

Initially the syntax highlighting is disabled. When you open a page for editing, there is a new label [[Highlight]] near the top right corner of the edit text area. Click it to toggle the highlighting. It will remember your choice in the browser's local storage and will reuse it when you edit another page.

The edit highlighting has only been tested with the latest Firefox and Chromium browsers, and on a recent computer running Ubuntu. If you experience bugs or difficulties, you can click the [[Highlight]] label and use the plain text editor. In this case, please report any details to us so we can review it.

Configuration

Enabling the function in config.php with either one among the following lines:

• $EnablePmSyntax = 1; # enable the syntax highlighting of documentation blocks.
• $EnablePmSyntax = 2; # also enable the experimental syntax highlighting in the basic edit form.
• if($action=='edit') $EnablePmSyntax = 2; # only enable the edit form highlighting

Colors, fonts and spacing

The colors are suitable for websites with white or light backgrounds. They are somewhat similar to the default color theme from highlight.js but with mostly safer web colors, and less faded out.

#wikiedit #hwrap textarea#text, #wikiedit #htext, .pmhlt {
  /* base text color, background and size */
  --pmsyntax-color: inherit;
  --pmsyntax-bgcolor: inherit;
  --pmsyntax-fontsize: 0.9333em;

  /* [=Escaped text=], not processed by PmWiki */
  --pmsyntax-escaped: rgba(0,0,0,.7);
  --pmsyntax-escapedbg: rgba(0,0,0,.07);

  /* (:comment text:)*/
  --pmsyntax-comment: #777;

  /* core meta directives and wikistyles */
  --pmsyntax-meta: #279;
  --pmsyntax-metabg: #eff;

  /* other core and custom directives */
  --pmsyntax-directive: black;
  --pmsyntax-directivebg: #eee;

  /* markup expressions */
  --pmsyntax-mx: #470;
  --pmsyntax-mxbg: #efe;

  /* !! Headings */
  --pmsyntax-heading: #800;
  --pmsyntax-headingbg: #ffe7e1;

  /* list items, line breaks, indents */
  --pmsyntax-bullet: #470;
  --pmsyntax-bulletbg: #dfd;

  /* simple tables */
  --pmsyntax-table: #470;
  --pmsyntax-tablebg: #dfd;

  /* inline punctuation like '''bold''' */
  --pmsyntax-punct: #a00;

  /* i18n strings like $[Edit] or entities like   */
  --pmsyntax-string: #800;

  /* attributes and values in directives and wikistyles */
  --pmsyntax-attr: #f70;
  --pmsyntax-value: #a00;

  /* page (text) variables, template variables, like {*$:Summary} */
  --pmsyntax-var: #f40;

  /* keyword in forms, templates, conditionals */
  --pmsyntax-keyword: #800;


  /* Link URLs, InterMap links */
  --pmsyntax-url: #37a;

  /* textarea of the edit form, and highlighted block under it */
  --pmsyntax-border: 1px inset #ccc;
  --pmsyntax-radpad: 3px;
  --pmsyntax-fontfamily: monospace;

  --pmsyntax-fontsize-editform: 14px;
  --pmsyntax-lineheight: 1.3;

  /* color of the blinking cursor in the textarea */
  --pmsyntax-caretcolor: red;
}

See this thread on the mailing list with suggestions for reversing the colors on the [[Highlight]] label and making it more prominent.

Internationalization

The label [[Highlight]] can be translated in an XLPage, for example in French:

Design (edit form)

The edit text area ("textarea") can only have a single plain-text font in a single color. The colored block is under a transparent edit form.

Toggle section about the design choices and drawbacks

Instead of creating a ContentEditable block for editing with tremendous complexity, we use the basic edit textarea, and under it, we precisely and absolutely position a preformatted block that will contain the cloned text, highlighted. The highlighted block has the same monospaced font, the same font and white-space metrics, box dimensions, padding, and overflow as the textarea, and they are both resized and scrolled at the same time.

The textarea has transparent text and transparent background, so only the highlighted rich text below it is seen, and you actually type transparent characters. On every change of the textarea, adding or removing text, scrolling or resizing, the highlighted block is updated. This happens instantly, for modern browsers on recent computers.

You can normally select text, which then becomes visible, and undecorated. The in-browser spellcheck also underlines the transparent characters but we see the words below them. The "edit toolbar" buttons work as usual, as well as the PmWiki core features (not saved warning, remember edit position).

Because of this design, and the need for precise positioning, highlighting the editor on older browsers or old computers, especially editing very long pages, might be difficult. In that case, it may be better to use the plain text editor (uncheck the checkbox). I suspect on the same hardware with the same software, any syntax-highlighting editor would have these difficulties, or possibly more.

Design (markup engine)

An outline about how the text is processed and marked up, and how to add new markup rules from Cookbook recipes can be found at Cookbook:CustomSyntax.

Notes

The recipe works with the default edit form or one reasonably close to it; custom edit forms may not be supported.

• Works with Cookbook:SectionEdit.

Some limitations outlined at Cookbook:CodeMirror#limitations apply to this recipe as well, notably: "As the PmWiki language is a very convenient markup set for the user, writing a syntaxic highlighter for it is rather hard. The one provided here is far from perfect, but should be able to render most of the language features." --Dfaure

PmSyntax-specific limitations for its early releases are:

• Vertical bar "|" and dash-greater arrow "->" may be highlighted even outside links or image captions.
• Some unbalanced punctuation ignored by PmWiki like ]] or ''' will be highlighted.
• Text that looks like a generic directive as in (:something attr=value:) will be highlighted, even if PmWiki may not know how to process it. This is to make it work out of the box for many current and future Cookbook recipes.
• Start-of-line space+bullet or space+hash are highlighted as these may be indented list items. Unlike the core markup engine, PmSyntax doesn't check whether the previous line is a list item, and whether the bullet is aligned with the text-start on the previous line. In some cases this may be a false-positive.

For a pragmatic, real-world, everyday usage, I hope these limitations would not be a deal-breaker.

To do / some day / maybe

• Add recipes that already use standard markup which is mostly correctly highlighted, to the PmSyntax category. Let us know if assistance is needed for configuring other recipes.
• Highlight correctly locally enabled Markup Expressions.

Change log / Release notes

For changes after 2.3.0 - see PmWiki:ChangeLog.

• 2022-01-11 : Allow for mode "external>html".
• 2022-01-07 : Add "external" semantic type, calls Highlight.js, mark \ joinline when first character in text.
• 2022-01-05 : Highlight more unconsumed special characters; refactor/streamline Keep3->Keep5 (for CustomMarkup#fivematches), PTVs will highlight the first colon as in (:Var:Value:).
• 2022-01-04 : Fix recently broken template keywords; group generic directives and move after forms, change semantic type "tab" with "table".
• 2022-01-03 : $CustomSyntax now requires 2 or more white-space characters as separators, to allow a regular expression to contain a space.
• 2022-01-02 : Mark hidden parts of link texts, like [[PmWiki (.) documentation( index)]] and tooltip titles like [[Link "Tooltip title"|text]]. Adaptation of the edit form layout for more skins
• 2021-12-31 : Fix for blank text when dragging. CSS variables with smaller scope.
• 2021-12-30 : Markup expressions will now also have nested expressions in bold, as in
  {(tolower (substr (asspaced "{*$Name}Talk") 2))}.
• 2021-12-29 : Add "keyword" type used in forms, conditionals and templates, unconsumed multiple dots as special, markup expressions - highlight nested parentheses.
• 2021-12-28 : Fix scrolling mismatch after adding/removing linebreaks. Replace toggle checkbox with highlighted label (better compatibility with skins). Prevent directives in headings to become dark red.
• 2021-12-27 : CSS variable prefix changed to --pmsyntax-; much faster updating of #htext noticeable for long pages.
• 2021-12-24 : Move PTV before other directives, add function toggleStyles(), highlight unconsumed [#!*?&+|,]+ in directives and meta, highlight "!" in "||!" table headers, nested page (text) variables inside other variables like {{*$Name}$:Summary}; tentative to fix a sub-pixel sizing mismatch between #text and #htext, a possible reason for intermittent line breaks.
• 2021-12-23 : Add (:nl:).
• 2021-12-19 : Swap optional container before main_rx (like the a>b>c case).
• 2021-12-16 : Cleanup; add classname .pmnobg for transparent background; allow for recipes to define syntax rules in a space-delimited string; allow for parent>child1>child2 nested processing.
• 2021-12-15 : Heading background slightly to warmer red/salmon nuance; remove empty anchors except "_url".
• 2021-12-14 : Allow for $when to be after rule with ">rule"; use actual InterMaps for URL highlighting; fix recursion, replace array.pop() with .slice(); refactor, change order in a rule: name, cname|fn, rx, [container_rx]; minor fixes; add $CustomSyntax for recipes to define syntax rules.
• 2021-12-13 : Refactor code, remove custom functions, recursion; combine most render functions in one, window.PmSyntaxCustomMarkup, script.dataset.custom; fix (:else3:); optimize simple directives; more distinctive color for variables; move (:redirect...:) with (:template abc:), both can have arguments.
• 2021-12-12 : More distinctive color for attributes; remove textarea background images; replace pre#htext with div#htext because some skins set !important to pre size; improve (:comment ...:) highlighting.
• 2021-12-11 : Color values in starting definitions; minor fixes; CSS slightly lighter background for escaped text; bare URLs can contain % so before wikistyles; update license block; recognize {<$:Name} variables; order and document CSS variables, use mostly "safe" colors; optimize hattr(), add Keep3(), sortable rules, add .pmtag; edit form push up 1px; unshift HTML snippet before other styles to allow easy changes; "inert" block (future browsers may disable find-in-page); i18n checkbox label.
• 2021-12-11 - New syntax renderer function written from scratch, independent from Highlight.js (suggested by Dfaure). Add to core for 2.2.146, disabled by default.

Earlier versions based on Highlight.js:

• 20211123 - Force same rules on the textarea and the colored block for line-breaks on long strings without white space. Correct (c) start year.
• (Nov 18-20, 2021: Grammar only escape special characters in "escaped" text, detect InterMap links before start-of-line properties/PTVs, add relevance weight for language auto-detection.)
• 20211117a - Improve handling of trailing backslash (join-lines) and nested multiline escaped blocks (core, PmWiki grammar).
• 20211116b - Fix trailing newline in code was ignored.
• 20211116a - Optimize API call to Highlight.js (no need to guess the language, we know it is PmWiki).
• 20211116 - Fixes for the find-in-page search field of the browser, could break the alignment the text. Optimization of the enable-disable highlighting functions.
• 20211115 - More precise positioning of the highlighted block; download as a ZIP archive, updated install instructions.
  • Grammar updates: Fix directives split by line breaks could be highlighted as valid, Fix trailing backslash inside valid directives was not recognized.
• 20211114 - First public release, ready to be tested.

See also

Contributors

• Written and maintained by Petko.
• Dfaure was very enthusiastic about the earliest prototype, suggested dropping the original Highlight.js requirement and writing an independent function coloring PmWiki markup, encouraged me to do more and to consider more usage cases, including those for CustomSyntax.
• Simon did early testing and made suggestions.

Comments

See discussion at PmSyntax-Talk