November 2021

Looking at the wider ecosystem, it certainly appears that there is a growing appetite for Org markup outside org-mode. More projects like Hugo and Logseq seem to be interested in supporting Org markup, and there has been a recent growth in editor extensions like Neovim’s orgmode.nvim (started in March this year) and Sublime Text’s OrgExtended (started in June this year).

Interest in Org as a general-usage markup format can also be seen within the Org project. Primarily lead by Nicolas Goaziou, there is an ongoing attempt to codify the Org syntax in a formal specification in the Worg document Org Syntax (draft). Other members of the Org mailing list have directed their effort to creating non-elisp parsers for Org, both to help Org tools be created in other languages, and as put in the README for Tom Gillespie’s laundry parser

The long term goal of this work is to provide a reference that can be used to standardize Org syntax and behavior and to specify various levels of compliance for an implementation of Org mode.

Earlier this week Karl Voit, the author of the rather well-known document Org Mode Is One of the Most Reasonable Markup Languages to Use for Text, surprised the mailing list by announcing his independent creation of a multi-leveled standard for Org syntax subsets called “Orgdown” (the name is a blend of “Org-mode” and “markdown”, but the standard is only a subset of Org). Each level defines a compliance score given by a mix of parsing and editing support, with example compliance scores for the first (and currently only) level of the standard given for common tools.

At this stage, it isn’t clear exactly how the Org-outside-Emacs landscape will evolve, but the swelling interest is very encouraging.

Speaking of parsers, I may be somewhat biased but I’m quite happy that a Org parser for Julia now exists 🎉.

github.com

GitHub - tecosaur/OrgMode.jl: A Julia library for working with Org, WIP.
A Julia library for working with Org, WIP. Contribute to tecosaur/OrgMode.jl development by creating an account on GitHub.

OrgMode.jl is a parser, but also intended as a general-purpose Org library for Julia. It’s only been a week since development started, but it currently supports most of the Org Syntax draft specification, along with the rendering of a parsed Org AST to a TTY or back to Org text. A few utility functions are also included, such as filtermap which operates similarly to org-element-map.

One small but impactful change is autoloading of citation backends. Until recently before say using the csl backend, one needed to (require 'oc-csl) or face error messages.

Now, if you have a line like:

#+cite_export: FORMAT ...

org-mode will try to load the file oc-FORMAT before trying to process citations.

This should make getting started with citations in Org just a bit easier.

The standard way of setting a :tangle-mode has typically been by providing a closure that makes use of Elisp’s octal syntax, such as (identity #o755). This is unnecessarily verbose, and certainly doesn’t feel natural.

With the addition of a small mode-interpreting function (org-babel-interpret-file-mode) It is now possible to specify :tangle-mode using three different forms of shorthand

octal

o755 is equivalent to (identity #o755)

chmod

chmod-style inputs like u+x are now parsed to a file mode¹ with the the base/default mode set by org-babel-tangle-default-file-mode.

ls -l

strings of the form given by ls -l like rwxr-xr-x are also accepted

This means the following forms are now all equivalent:

:tangle-mode (identity #o755)
:tangle-mode o755
:tangle-mode a=rx,u+w
:tangle-mode rwxr-xr-x

It has also been noted on the mailing list that the :tangle-mode (identity #o755) form works by being transformed to :tangle-mode 493 during parsing. Similarly :tangle-mode 755 is equivalent to :tangle-mode (identity #o1363). For some values the decimal and octal interpretation are both valid file modes. Due to the clear potential for confusion, and since file permissions are an important security consideration, it has been suggested on the mailing list that these forms should be depreciated with a warning in future. No decision has been made yet though.

Ihor Radchenko has done some fantastic work over the past few months by overhauling parts of org-element.el to introduce extensive caching. org-element is the Org markup parser inside org-mode. This allows for a huge jump in speed, and also provides a few functions which fetch information without updating the cache — allowing for particularly speedy lookups with a small sacrifice to correctness guarantees on one or two properties in particular cases.

Several org-mode APIs now make use of the cache to dramatically improve speed. Aside from improvements to typically slow operations, this is ideal for situations involving frequent buffer edits. It’s no understatement to say that this work is transformative.

One potential beneficiary from this work is actually fontification. It has become increasingly apparent that the current regex-based method for buffer fontification is imperfect, and can actually differ from the true structure of the document as parsed (authoritatively) by org-element. This has lead to the well-received suggestion on the mailing list to rewrite the fontification code to be built on org-element instead.

I think inline source code blocks are an underappreciated feature of Org. I don’t think it’s helped that they have not been visually treated at all differently from plain text. Now though, they have a new dedicated face (org-inline-src-block) and in the same manner as source blocks, based on org-src-fontify-natively can be fontified using the language’s major mode.

If you aren’t familiar with inline source blocks, you’re missing out. They are very much the inline cousin of source blocks, and so support all your favourite Babel features like code execution and header arguments. This provides a fantastic capacity to inline dynamically computed expressions, and optionally show the code that produces them.

Matt Huszagh has contributed a patch that allows functions to be used as values for default header arguments. This is great for arguments where a sensible default can be provided by evaluating a function on-the-fly.

Consider for example the arguments required to produce a simple image using R with Babel:

#+begin_src R :results graphics file :file myimage.svg
library(ggplot2)
ggplot(mpg, aes(displ, hwy, colour = class)) + geom_point()
#+end_src

In a Jupyter-style (.ipynb) or throwaway document, we likely don’t care about the file name at all. With these new capabilities, we can provide a file name dynamically as a default argument!

First we must write a function that when run at the source block will give us a suitable file name, like so

(defun my/org-src-sha-to-image ()
  (concat "generated-"
          (substring
           (sha1 (org-element-property :value (org-element-at-point)))
           0 8)
          ".svg"))

Let’s also write a function to guess whether the source block produces a plot by checking if there’s a plot command on the last line.

(defun my/org-src-guess-results-type ()
  (if (string-match-p "^ *\\(?:plot\\|ggplot\\)([^\n]+\n?\\'"
                      (org-element-property :value (org-element-at-point)))
      "graphics file" "replace"))

Then we can just use these function in place of a static value in the default header arguments variable — that’s all it takes.

(setq org-babel-default-header-args:R
      '((:results . my/org-src-guess-results-type)
        (:file . my/org-src-sha-to-image)))

This means for most cases we can now get away without any header arguments at all.

#+begin_src R
library(ggplot2)
ggplot(mpg, aes(displ, hwy, colour = class)) + geom_point()
#+end_src

It’s always lovely to see more ways of reducing boilerplate.

Previously, as long as org-image-actual-width was nil or a list of the form (default-value), org-mode would display images according to a :width attribute (e.g. #+attr_html: :width 400px) by simply looking for the first #+attr_ affiliated keyword and reading the numeric component of the :width as the number of pixels wide the image should be.

This has now become somewhat fancier. The image-width determining logic has been extracted to a new function (org-display-inline-image--width) which will now extract floating-point values like 0.7 and interpret them as that portion of the accessible text width in the buffer.

This means that a width parameter like #+attr_latex: :width 0.7\linewidth the image will displayed as 70% of the buffer text width. This also supports percentage value, like #+attr_html: :width 80% by dividing the number before the % by 100 as a floating-point value. As always, if you don’t like the way display width is inferred here you can override it by putting a #+attr_org: :width X statement first.

Support for proportional image widths extends to the (default-value) form of org-image-actual-width, as now if you set it to say (0.9) which will cause images without any width specification to be displayed at 90% of the buffer text width.

If you want to have some images displayed as their actual width you can use the new special width parameter t to set this on a per-image basis with #+attr_org: :width t. Now all you need to do is remember to put this first. Based on current discussions on the mailing list though, soon #+attr_org will be prioritised when determining display image width, no matter which order you put the attributes in. I do like having one less thing to remember 🙂.

• Allow citations immediately following an item bullet TEC
• Allow citations immediately following a footnote definition Nicolas Goaziou
• Update some obsolete function references Marco Wahl
• ob-gnuplot is now maintained by Ihor Radchenko
• Improve makescript support for ORGVERSION in tag-less mirrors Nicholas Vollmer
• New ob-julia, now maintained by Pedro Bruel
• Allow for no indentation, but preserving current indentation by setting org-indent-indentation-per-level to 0 David Lukes
• Eliminate some byte-compile warnings Nicholas Vollmer Bastien
• Support Greek smart quotes Juan Manuel Macías
• org-mouse support for intermediate-state checkboxes Jim Porter
• Allow nested parenthesis in org-compile-prefix-format %(sexp) expressions Ihor Radchenko
• oc-csl / citeproc improvements András Simonyi
• Move more unmaintained/overly niche ob-* files to the contrib repo, reducing the maintainer burden Bastien
• Allow use of a function for org-agenda-overriding-header for dynamic headers Christopher League
• Improve org-protocol URI decoding Max Nikulin
• Remove some obsolete LaTeX packages from the default packages list TEC
• Add support for text and year citation styles to oc-csl András Simonyi
• Produce lower-case keywords in ox-org TEC
• Improve ob-gnuplot argument processing Ihor Radchenko
• A collection of oc-* improvements Nicholas Goaziou
• Support bare author citations in oc-csl TEC
• Add :options LaTeX attribute to tables Juan Manuel Macías
• Fix display error with ob-plantuml and html export Su Lin
• More tests! Ihor Radchenko
• Documentation improvements! Marco Wahl Stefan Kangas Daniel Fleischer Wiliam Denton Thomas Dye Bastien Bruce D’Arcus Kyle Meyer Nicolas Goaziou

• Fix heading insertion in a case where point is before any heading Marco Wahl
• Prevent stringp error when tangling Org from an org-src edit buffer Mark Dawson
• Prevent indent-tabs-mode from messing with justification in ASCII exports Morgan Willcock
• Fix form of default Babel haskell header args Ihor Radchenko
• No more duplicated logbook entries for repeated tasks Ihor Radchenko
• A headline fontification edge case Sébastien Miquel
• Refactor code that needed Emacs 28 Kyle Meyer
• Make sure a terminating emphasis marker can’t be used as a beginning emphasis marker in fontification Ihor Radchenko
• Allow footnotes at footnote definition start Nicholas Goaziou