experiences about indentation-based syntax for Lisp from Calcit Lang

[tiye]

This post is about the areas I explored in Cirru(http://cirru.org/) and Calcit(http://calcit-lang.org/) to write S-Expressions without all parentheses. It's totally another language from Racket, but I found we share very similar goals since:

In the case of Lisp languages like Racket, Matthew Flatt is right: among a certain audi­ence of poten­tial users, paren­theses are a deter­rent. Who knows why? But in the last 60 years, simply grab­bing these dissenters by the lapels and fumi­gating them with the stinky garlic breath of paren­the­sized S-expres­sions has not been a winning strategy. If Lisp parti­sans want different results in the next 60 years, we need to try some­thing new.

I've tried a lot in the past years for Lisp without parentheses syntax of S-Expressions but using tools from ClojureScript. Here are several major projects from mine:

Cirru Text Syntax http://text.cirru.org/

which might looks just like Lisp without top-level parentheses,

set a (add (number 1) (nubmer 2))

but it's also identical to:

set a $ add
  number 1
  number 2

and

set a
  add
    number 1
    number 2

so it's basically replacing some of parentheses with 2-spaces based indentions and a notation of $. This looks similar to sweet-expression, but being different in many aspects,

• in Cirru "a" and a are identical since double quotes are only used for supporting escaping,
• in Cirru (, a) is identical to a since , means "unfolding", to add flexibility to indentations
  ...and maybe more since I'm not familiar with sweet.

You my try this online parser for Cirru syntax https://repo.cirru.org/parser.coffee/ to get details.

Also there's a real-world example in https://github.com/calcit-lang/calcit_runner.rs/blob/main/src/cirru/calcit-core.cirru which uses Cirru text syntax to describe core functions in Calcit.

Also something to note that when you write data in Cirru EDN format, it's all prefixed notations like this:

{} (:a 1.0)
  :b $ [] 2.0 3.0 4.0
  :c $ {} (:d 4.0)
    :e true
    :f :g
    :h $ {} (|a 1.0)
      |b true

Cirru Editor https://github.com/Cirru/calcit-editor

which looks like:

hard to explain by words, but try this:

It's a tree editor based on DOM nodes, with a bunch of keyboard shortcuts around DOM structures, providing an experience of editing the structure of S-Expressions directly.

Could be mind-blowing for a Lisp, or a little, but Cirru text syntax was designed after the idea of this DOM tree editor and strange aspects in Cirru text syntax originated from Cirru Editor. And the initial goal was to create a code editor that layouts automatically, now based on CSS layout engine. Parser for Cirru text syntax was implemented earlier though.

The most benefit I found is in Cirru Editor, the form of the code is a piece of data, or mostly you can think it of nested lists that represents code, rather than some text buffer. That means, if I could support extensions for Cirru Editor in the future, an extension would rewrite selected list of code just in data, no parsing required.

Calcit Interpreter https://github.com/calcit-lang/calcit_runner.rs

This is more like a proof of the power of Cirru for a real language. Previously I use Cirru to generate code to JavaScript and Clojure to actually run it. Now I use Cirru in a standalone scripting language, after many work in the past year(although it's still mostly used in compiling to js).

Macros in Calcit is too naive for you Racket users(I don't have much experience on Racket...), but I just want to demonstrate that macros can be described naturally inside Cirru text syntax like this:

defmacro &case (item default pattern & others)
  assert "|`case` expects pattern in a pair"
    if (list? pattern) (&= 2 (&list:count pattern)) false
  let
      x $ &list:first pattern
      branch $ last pattern
    quasiquote
      if (&= ~item ~x) ~branch
        ~ $ if (&list:empty? others) default
          quasiquote
            &case ~item ~default ~@others

this already begins to look like indentation-based languages like Python, and it's easy to write with a plain text editor, with no support from extensions like Paredit. Meanwhile it's still holding most of the power of S-Expressions.

---

The goals for Rhombus might be far beyond what I've explored in Cirru but I hope the experiences in Cirru could contribute to the efforts of "S-Expressions with fewer parentheses".

Most treasured concept when I write code in both Cirru and Clojure,

Homoiconicity - the text syntax for Cirru, the expression in Cirru Editor, they both share the same data form. If I make another parentheses based syntax for compatibility for Cirru, it will still be consistent with Cirru by sharing the same data form. And at its core, Cirru is just like ["def", "f", ["x", "y"], ["+", "x", "y"]], which is totally plain JSON data. That was originated from Lisp, while Cirru is only sugars.

To me, this is the most shinny idea from Lisp, which means I can manipulate syntax tree in a very cheap way, writing in text, render to DOM nodes, rewriting by functions, sharing among various editors/logs/github... always cheaper to extend and explore. Seeing from this view, even Clojure's sugars for [1,2,3,4] {:a 1 :b 2} would break, they are not easily manipulated. And in the bigger picture of Cirru, I could still explored really "tree-based" editing experience in VR 3D space...

...maybe I should just stop here. These are the key points of what I've explored Cirru. Hope this would help in you big picture for Rhombus in reducing parentheses.

[tiye]

Plus some early attempts:

• Cirru Parser in Racket https://github.com/Cirru/cirruparser-rkt
• Tiny demo converting Cirru to Racket code https://github.com/Cirru/cirrusepal-rkt

probably outdated...

[buster-blue]

I really like the way that looks! I always felt that utilizing significant indentation could be an easy way to make s-expressions more readable. Glad to see another approach to this.

[mflatt]

@jiyinyiyong Thanks for your post! I've added Cirru and Calcit to prior-art.md.

[rogerkeays]

sweet-exp does a pretty good job of this in racket. See also https://dwheeler.com/readable/index.html

[tiye]

I know sweet-expression, haven't looked really into details, but read about some of its examples. I suggest that Cirru Text Syntax is more familiar with S-Expressions than with sweet-expression or Python.

taking an example from https://sourceforge.net/p/readable/code/ci/develop/tree/examples/basic-sweet.sscm

define fibup(max count n-1 n-2)
  if {max = count}
    {n-1 + n-2}
    fibup max {count + 1} {n-1 + n-2} n-1

the syntax contains infix operators, curry brackets, which makes it more like Python.

Cirru is more S-Expressions notated with indentations. Taking an example from my recent code, https://github.com/calcit-lang/calcit_wasmtime/blob/b43a82aaa8fbbe5751eb751c3461954cba7949ac/tests/syntax.rs#L111-L134

The snippet in Cirru can be transformed into S-Expressions almost like just adding parentheses(except for comments and string notations).

func $sum_struct_create
  param $sum_struct_ptr i32
  param $var$a i32
  param $var$b i32
  ;; "c// sum_struct_ptr->a = a;"
  i32.store
    get_local $sum_struct_ptr
    get_local $var$a
  ;; "c// sum_struct_ptr->b = b;"
  i32.store offset=4
    get_local $sum_struct_ptr
    get_local $var$b

(func $sum_struct_create (param $sum_struct_ptr i32) (param $var$a i32) (param $var$b i32)
  ;; "c// sum_struct_ptr->a = a;"
  (i32.store (get_local $sum_struct_ptr) (get_local $var$a))
  ;; "c// sum_struct_ptr->b = b;"
  (i32.store offset=4 (get_local $sum_struct_ptr) (get_local $var$b)))

it might be more readable with infix operators in some cases, but "homoiconicity" is essential feature in Cirru and that makes it different from Python or Sweet.

[rogerkeays]

`sweet` is still homoiconic as far as I am aware. At least, I use it to mix code and data. FWIW, I don't use the infix syntax of `sweet` because mixing curly parens and smooth parens was annoying (even with indentation). Instead, I use `fluent`, which I mentioned in another thread. But I'm sure there is going to be plenty said about infix though, and I don't want to hijack your thread here. I'm all for indentation syntax and haven't had any problems using `sweet`. Most developers indent their code anyway so it is easier to read. Extra parens is just pollution.

…

On Tue, Sep 21, 2021 at 08:38:41AM -0700, Jon wrote: I know sweet-expression, haven't looked really into details, but read about some of its examples. I suggest that Cirru Text Syntax is more familiar with S-Expressions than with sweet-expression or Python. taking an example from https://sourceforge.net/p/readable/code/ci/develop/tree/ examples/basic-sweet.sscm define fibup(max count n-1 n-2) if {max = count} {n-1 + n-2} fibup max {count + 1} {n-1 + n-2} n-1 the syntax contains infix operators, curry brackets, which makes it more like Python. Cirru is more S-Expressions notated with indentations. Taking an example from my recent code, https://github.com/calcit-lang/calcit_wasmtime/blob/ b43a82aaa8fbbe5751eb751c3461954cba7949ac/tests/syntax.rs#L111-L134 The snippet in Cirru can be transformed into S-Expressions almost like just adding parentheses(except for comments and string notations). func $sum_struct_create param $sum_struct_ptr i32 param $var$a i32 param $var$b i32 ;; "c// sum_struct_ptr->a = a;" i32.store get_local $sum_struct_ptr get_local $var$a ;; "c// sum_struct_ptr->b = b;" i32.store offset=4 get_local $sum_struct_ptr get_local $var$b (func $sum_struct_create (param $sum_struct_ptr i32) (param $var$a i32) (param $var$b i32) ;; "c// sum_struct_ptr->a = a;" (i32.store (get_local $sum_struct_ptr) (get_local $var$a)) ;; "c// sum_struct_ptr->b = b;" (i32.store offset=4 (get_local $sum_struct_ptr) (get_local $var$b))) it might be more readable with infix operators in some cases, but "homoiconicity" is essential feature in Cirru and that makes it different from Python or Sweet. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. *

[tiye]

I use fluent

which means, Sweet also has another mode other than the snippet I found?

Extra parens is just pollution.

I agree. But also I have to admit that parentheses offer more information to help tools to understand the code. Indentations might confuse the code in some cases since it's so concise.

[rogerkeays]

On Tue, Sep 21, 2021 at 09:38:14AM -0700, Jon wrote: I use fluent which means, Sweet also has another mode other than the snippet I found?

no, this is independent of sweet: https://github.com/rogerkeays/racket-fluent they make a pretty good combination though

Extra parens is just pollution. I agree. But also I have to admit that parentheses offer more information to help tools to understand the code. Indentations might confuse the code in some cases since it's so concise.

Tools can deal with it. I meant to add that I really don't like the $ syntax as a sort of 'inline indentation'. I think WASP does the same thing using : but it is way to cryptic for my tastes. I'd rather just put in the parens if I don't want a new line.

[tiye]

I really don't like the $ syntax as a sort of 'inline indentation'

I was trying hard to learn Haskell for many years, $ is a simple syntax from Haskell. Different tastes though.

[tiye]

by "homoiconicity" I mostly stick to some meanings from Wiki:

A language is homoiconic if a program written in it can be manipulated as data using the language, and thus the program's internal representation can be inferred just by reading the program itself. This property is often summarized by saying that the language treats "code as data".

Plus some rules form https://youtu.be/o7zyGMcav3c?t=1402

in this sense, I suppose syntax of curly brackets are not represented with data literals directly, so it's deviated a bit from the direction of "homoiconicity". Infix operator may break or may not, I'm not sure.

[tiye]

oh, I don't know much about Racket macros https://github.com/rogerkeays/racket-fluent/blob/main/main.rkt#L13 , but it always looks cooler than Clojure's.

[rogerkeays]

Yeh, Racket is the only language I know of that completely separates the syntax from the computation model. It would be great to see more languages do this. I think Java were trying to do it with their Truffle project, but that looked a lot like writing compilers like people have done for decades. Implementing a language using macros/pattern matching/transformations is like having superpowers.

…

On Tue, Sep 21, 2021 at 12:43:27PM -0700, Jon wrote: oh, I don't know much about Racket macros https://github.com/rogerkeays/ racket-fluent/blob/main/main.rkt#L13 , but it always looks cooler than Clojure's. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. *