Refactor nbJs (#148)

* basics are working

* fix tests

* update counters.nim

* exclude interactivity.nim from docs until rewritten

* bye bye old gensym

* remove old code. So little code left :o

* update interactivity.nim

* update lots of small things

* update nbJs tests

* update text of counters.nim

* refactor code accoridng to review

* update tests

* update counters

* make interactivty compile. Still have to update it

* draft updated interactivity

* review updates

* make nimibCode official

* update changelog

* Update docsrc/interactivity.nim

Co-authored-by: Pietro Peterlongo <[email protected]>

* Update src/nimib.nim

Co-authored-by: Pietro Peterlongo <[email protected]>

* Update src/nimib/renders.nim

Co-authored-by: Pietro Peterlongo <[email protected]>

* rename nbCodeToJs partial to nbJsFromCode

* add link to changelog

* add ceasar link to interactivity

* update index and readme

* fix counters not showing code

* bump nimble version

* bump changelog version

* add nimibCode to changelog

* upload thumbnail nimconf

Co-authored-by: Pietro Peterlongo <[email protected]>

README.md

@@ -110,7 +110,7 @@ in this repo:
 * [mostaccio](https://pietroppeter.github.io/nimib/mostaccio.html): examples of usage of nim-mustache and of dark mode.
 * [interactivity](https://pietroppeter.github.io/nimib/interactivity.html): shows the basic API of creating interactive elements using `nbJsFromCode`.
 * [counter](https://pietroppeter.github.io/nimib/counters.html): shows how to create reusable interactive widgets by creating a counter button.
-* [caesar](https://pietroppeter.github.io/nimib/caesar.html): a Caesar cipher implemented using `nbJsFromCode` and `karax`.
+* [caesar](https://pietroppeter.github.io/nimib/caesar.html): a Caesar cipher implemented using `nbKaraxCode` and `karax`.
 
 
 elsewhere:
@@ -153,7 +153,7 @@ Currently most of the documentation on customization is given by the examples.
 
 * `nbImage`: image command to show images (see `penguins.nim` example linked above)
 * `nbFile`: content (string or untyped) is saved to file (see example document [files](https://pietroppeter.github.io/nimib/files.html))
-* `nbRawOutput`: called with string content, it will add the raw content to document (html backend)
+* `nbRawHtml`: called with string content, it will add the raw content to document (html backend)
 * `nbTextWithCode`: a variant of `nbText` that also reads nim source. See example of usage
   at the end of the source in `numerical.nim` linked above.
 * `nbPython`:  can be used after calling `nbInitPython()` and it runs and capture output of python code;

changelog.md

@@ -8,9 +8,17 @@ made this development possible.
 When contributing a fix, feature or example please add a new line to briefly explain the changes. It will be used as release documentation here: https://github.com/pietroppeter/nimib/releases
 
 ## 0.3.x
-
 * _add next change here_
 
+## 0.3.3
+
+* Refactored nbJs (#148)
+  * **Breaking**: All `nbJsFromCode` blocks are now inserted into the same file (Compared to previously when each block was compiled as its own file).
+  So this will break any reusable components as you will get `redefinition of variable` errors. The solution is to use `nbJsFromCodeInBlock` which puts the code inside a `block`. Imports can't be done in there though so you must do them in a separate `nbJsFromCode` or `nbJsFromCodeGlobal` before. 
+  * See [https://pietroppeter.github.io/nimib/interactivity.html](https://pietroppeter.github.io/nimib/interactivity.html) for a more detailed guide on how to use the new API.
+* Added `nimibCode` template. One problem with using `nbCode` is that you can't show *nimib* code using it because it nests blocks and wrecks havoc.
+So `nimibCode` allows you to show *nimib* code but at the cost of not capturing output of the code.
+
 ## 0.3.2
 
 * Add `hlHtml` and `hlHtml` to nimiBoost

docsrc/counters.nim

@@ -6,25 +6,27 @@ nbInit
 nbText: hlMd"""
 # Counters - Creating reusable widgets
 
-This document will show you how to create reusable widgets using `nbJsFromCode`. Specifically we will make a counter:
+This document will show you how to create reusable widgets using `nbJsFromCodeInBlock`. Specifically we will make a counter:
 A button which increases a counter each time you click it. We will do this in two different ways, using `std/dom` and `karax`.
 ## std/dom
 
 The first method is to use Nim like you would have used Javascript using `getElementById` and `addEventListener`: 
 """
-nbCode:
+nimibCode:
+  ## 0:
+  nbJsFromCodeGlobal:
+    import std/dom
   ## 1:
   template counterButton(id: string) =
     let labelId = "label-" & id
     let buttonId = "button-" & id
     ## 2:
-    nbRawOutput: """
+    nbRawHtml: """
 <label id="$1">0</label>
 <button id="$2">Click me</button>
 """ % [labelId, buttonId]
     ## 3:
-    nbJsFromCode(labelId, buttonId):
-      import std/dom
+    nbJsFromCodeInBlock(labelId, buttonId):
       ## 4:
       var label = getElementById(labelId.cstring)
       var button = getElementById(buttonId.cstring)
@@ -38,10 +40,13 @@ nbCode:
 
 nbText: hlMd"""
 Let's explain each part of the code:
+
+0. We import `std/dom` in a `nbJsFromCodeGlobal` block. `std/dom` is where many dom-manipulation functions are located.
 1. We define a template called `counterButton` which will create a new counter button. So if you call it somewhere it will
 place the widget there, that's the reusable part done. But it also takes an input `id: string`. This is to solve the problem of each widget needing unique ids. It can also be done with `nb.newId` as will be used in the Karax example.
 2. Here we emit the `<label>` and `<button>` tags and insert their ids.
-3. `nbJsFromCode` is the template that will turn our Nim code into Javascript and we are capturing `labelId` and `buttonId` (Important that you capture all used variables defined outside the code block). `std/dom` is where many dom-manipulation functions are located.
+3. `nbJsFromCodeInBlock` is the template that will turn our Nim code into Javascript and we are capturing `labelId` and `buttonId` (Important that you capture all used variables defined outside the code block).
+The reason we are using `nbJsFromCodeInBlock` instead of `nbJsFromCode` is that we need to put the code of the different components in different blocks to avoid errors like `redefinition of label`.
 4. We fetch the elements we emitted above by their ids. Remember that most javascript functions want `cstring`s!
 5. We create a variable `counter` to keep track of the counter and add the eventlistener to the `button` element. There we increase the counter and update the `innerHtml` of the `label`.
 

docsrc/index.nim

@@ -88,7 +88,7 @@ in this repo:
 * [mostaccio]({docs}/mostaccio.html): examples of usage of nim-mustache and of dark mode.
 * [interactivity]({docs}/interactivity.html): shows the basic API of creating interactive elements using `nbJsFromCode`.
 * [counter]({docs}/counters.html): shows how to create reusable interactive widgets by creating a counter button.
-* [caesar]({docs}/caesar.html): a Caesar cipher implemented using `nbJsFromCode` and `karax`.
+* [caesar]({docs}/caesar.html): a Caesar cipher implemented using `nbKaraxCode` and `karax`.
 
 
 elsewhere:
@@ -131,7 +131,7 @@ Currently most of the documentation on customization is given by the examples.
 
 * `nbImage`: image command to show images (see `penguins.nim` example linked above)
 * `nbFile`: content (string or untyped) is saved to file (see example document [files]({docs}/files.html))
-* `nbRawOutput`: called with string content, it will add the raw content to document (html backend)
+* `nbRawHtml`: called with string content, it will add the raw content to document (html backend)
 * `nbTextWithCode`: a variant of `nbText` that also reads nim source. See example of usage
   at the end of the source in `numerical.nim` linked above.
 * `nbPython`:  can be used after calling `nbInitPython()` and it runs and capture output of python code;

docsrc/interactivity.nim

@@ -3,56 +3,126 @@ import nimib
 nbInit
 
 nbText: hlMd"""
-# Creating interactive components in Nimib
+# Creating interactive components in nimib
 
 Nimib can easily be used to create static content with `nbText` and `nbCode`, but did you know that you can create interactive
 content as well? And that you can do it all in Nim even! This can be achieved using either the `nbJsFromCode`-API or `nbKaraxCode`.
 They work by compiling Nim code into javascript and adding it to the resulting HTML file.
-This means that arbitrary Javascript can be written but also that Karax, which compiles to javascript, also can be used.
+This means that arbitrary Javascript can be written but also that Karax, which compiles to javascript, can be used.
 
-## nbJsFromCodeInit
-This is the fundamental API used for compiling Nim-snippets to javascript. It consists of three templates:
-- `nbJsFromCodeInit` - Creates a new code script that further code can be added to later.
-- `addCodeToJs` - Adds to an existing code script
-- `addToDocAsJs` - Takes the Nim code in a script and compiles it to javascript. 
+In the same way that code from nbCode blocks are all compiled into a single file,
+all code to be compiled in javascript will be put in a single file. 
+This has the advantage that a single compilation is performed
+and code from a previous block can be used in subsequent blocks.
+The api looks like this:
+
+- `nbJsFromCode`: nim code will be appended to the file and compiled during `nbSave`.
+- `nbJsFromCodeInBlock`: same as `nbJsFromCode` but the code is put inside a `block`.
+- `nbJsFromCodeGlobal`: the code here will be put at the top of the file.
+
+If you wish to compile to a separate file you can do that.
+Indeed this is what is done for a special block that allows you to use karax without boilerplate:
+
+- `nbJsFromCodeOwnFile`: compile to js as its own file.
+- `nbKaraxCode`: Sugar on top of `nbJsFromCodeOwnFile` for writing Karax components.
+
+## nbJsFromCode
+This is the fundamental API used for compiling Nim-snippets to javascript.
 Here is a basic example:
 """
 
-nbCode:
-  let script = nbJsFromCodeInit:
-    echo "Hello world!"
-  let x = 3.14
-  script.addCodeToJs(x):
-    echo "Pi is roughly ", x
-  ## Uncomment this line:
-  ##script.addToDocAsJs()
-script.addToDocAsJs()
-nbJsShowSource("This is the complete script:")
+nimibCode:
+  nbJsFromCode:
+    let x = "Hello world!"
+    echo x
 
+nbText: hlMd"""
+If you now go to your browser's javascript console you should see `Hello world` printed there.
+So the code we passed to `nbJsFromCode` has been compiled to Javascript and is run by your browser!
 
+### Capturing variables
+If you have a variable in your code that you want to access inside a
+nbJs-block, you have to capture it. This can be done by passing it to the block like this:
+"""
+nimibCode:
+  # This variable is defined in C-land
+  let captureVariable = 3.14
+  nbJsFromCode(captureVariable): # capture it
+    # use it in JS-land
+    echo "Pi is roughly ", captureVariable
 nbText: hlMd"""
-The reason `script.addToDocAsJs()` is commented out is just a limitation of nimib not handling nested blocks well.
-If you now go to your browser's javascript console you should see `Hello world` and `Pi is roughly 3.14` printed there.
-What is up with `script.addCodeToJs(x)` though? Why is `(x)` needed? It is because we have to capture the value of `x`
-to be able to use it in the javascript. The code block will basically be copy-pasted into a separate file and
-compiled into javascript. And `x` isn't defined there so we have to capture it. This is true for any variable that
-we want to use that is defined outside the script blocks.
+If you look at the console you should see that it prints out `Pi is roughly 3.14`.
+The capturing is done by serializing the variable to JSON, so the captured type has to support it.
 
-## nbJsFromCode
-This is basically a shorthand for running `nbJsFromCodeInit` and `addToDocAsJs` in a single call:
-```nim
-let x = 3.14
-nbJsCode(x):
-  echo "Pi is roughly ", x
-```
+Capturing variables is especially important when creating reusable components as they allow you to
+generate the HTML using `nbRawHtml` and then pass in the ids of the elements by capturing them.
+Examples of this can be seen in the [counters tutorial](counters.html).
+
+
+## nbJsFromCodeInBlock
+`nbJsFromCodeInBlock` works the same as `nbJsFromCode`, except that it puts the code inside a block.
+This is a feature which is important if you are making a reusable piece of code, like a component.
+This is because it allows you to reuse the same variable name in multiple blocks. 
+Using `nbJsFromCode` would yield a `redefinition of variable` error. 
+Here is an example showing how the same variable name can be used:
+"""
+nimibCode:
+  nbJsFromCodeInBlock:
+    let sameVariable = "First block"
+    echo sameVariable
+  nbJsFromCodeInBlock:
+    let sameVariable = "Second block"
+    echo sameVariable
+
+nbText: hlMd"""
+The case when this is really needed is when you have a `nbJsFromCodeInBlock` inside a template like this:
+"""
+nimibCode:
+  template jsGoodbyeWorld() =
+    nbJsFromCodeInBlock:
+      let s = "Good bye world"
+      echo s
+
+  jsGoodbyeWorld()
+  # Without block the second call would give `redefinition of 's'`
+  jsGoodbyeWorld()
+
+
+nbText: hlMd"""
+If you look in the console you should see that it prints out `Good bye world` once for each call to `jsGoodbyeWorld` call.
+
+Because the code is put inside of a block, any code needing to be put at the top-level (like imports)
+must be done in a separate `nbJsFromCode` or `nbJsFromCodeGlobal` before it.
+
+## nbJsFromCodeGlobal
+`nbJsFromCodeGlobal` works similarly to `nbJsFromCode`, except that it places the code at the top of the generated js file.
+So it is well suited for `import`s and defining global variables you want to be able to access in multiple blocks. 
+Code defined here is available in all `nbJsFromCode` and `nbJsFromCodeInBlock` blocks.
+"""
+
+nimibCode:
+  nbJsFromCodeGlobal:
+    import std / dom # this will be imported for all your nbJs blocks
+    var globalVar = 1
+  nbJsFromCode:
+    echo "First block: ", globalVar
+    globalVar += 1
+  nbJsFromCode:
+    echo "Second block: ", globalVar
+
+nbText: hlMd"""
+## nbJsFromCodeOwnFile
+The above-mentioned nbJs blocks are all compiled in the same file. But if you want to compile a code block
+in its own file you can use `nbJsFromCodeOwnFile`. This also means you can't access any variables defined
+in for example `nbJsFromCodeGlobal`.
 
 ## nbKaraxCode
 
 If you want to write a component using karax this is the template for you!
 A normal karax program has the following structure:
 ```nim
-nbJsFromCode(rootId):
-  import karax / [kbase, karax, karaxdsl, vdom, compact, jstrutils, kdom]
+nbJsFromCodeOwnFile(rootId):
+  include karax / prelude
 
   karaxCode  # some code, set up global variables for example
 
@@ -86,8 +156,71 @@ nbCode:
           proc onClick() =
             message = "Poof! Gone!"
 
-nbText: "This is the output this code produces:"
+nbText: "This is the output this code produces when called:"
 
 karaxExample()
 
+nbText: "Another example on how to use `nbKaraxCode` can be found in the [caesar document](./caesar.html) by clicking the `Show Source` button at the bottom."
+
+nbText: hlMd"""
+## Internal workings
+### nbJsFromCode
+Any code defined in `nbJsFromCode`, `nbJsFromCodeInBlock` and `nbJsFromCodeGlobal` will be pasted into a common file.
+- Any code passed to `nbJsFromCodeGlobal` will be put at the top of the file without any blocks. 
+- Any code passed to `nbJsFromCode` will be placed in the order they are called without any blocks.
+- Any code passed to `nbJsFromCodeInBlock` will be placed in the order they are called inside blocks.
+
+Here is an example of how the code will be ordered:
+```nim
+nbJsFromCode:
+  echo 1
+nbJsFromCodeInBlock:
+  echo 2
+nbJsFromCodeGlobal:
+  echo 3
+nbJsFromCode:
+  echo 4
+nbJsFromCodeGlobal:
+  echo 5
+```
+This will be transformed into something like this:
+```nim
+echo 3 # Global is placed at the top
+
+echo 5 # the other Global
+
+echo 1 # no block for nbJsFromCode
+
+block:
+  echo 2 # placed inside block
+
+echo 4 # no block
+```
+
+### nbKaraxCode
+`nbKaraxCode` works a bit differently, there each code block will be compiled in its own file so there is no global scope.
+So (`nbJsFromCode` + `nbJsFromCodeGlobal`) and `nbKaraxCode` are totally isolated from each other. 
+
+### Caveats
+Because of the way Nim gensym's variable names in the generated Javascript code, compiling two identical `nbKaraxCode` would
+cause Nim to generate the same variable names for the variables defined in them. An example is `varName_123456`. This is really bad as changing the variable in
+one component would change it in the other one as well! The solution we are using for this is to bump gensym by 1 each time we compile a
+`nbKaraxCode`. So a variable being generated as `varName_123456` the first time will be generated as `varName_123457` the second time. 
+
+This works well for most scenarios, but there is still a small risk that it will generate variable names that collide **if**
+you are defining multiple different variables with the same name in your code. For example:
+"""
+nimibCode:
+  nbKaraxCode:
+    var counter: int
+    block:
+      var counter: int
+
+nbText: hlMd"""
+The two variables `counter` are different variables but have the same name. Lets say the generated names for them the first time we compile this block are
+`counter_1` and `counter_2` for simplicity. The next time the generated names have been incremented with one and is instead `counter_2` and `counter_3`.
+And here the problem lies: `counter_2` is generated both times we compile the block! So this could lead to unwanted interactions between the two codes!
+The solution is stated above: don't name multiple separate variables the same in a `nbKaraxCode` or `nbJsFromCodeOwnFile` block! 
+This isn't a problem for the other nbJs blocks luckily. 
+"""
 nbSave

nimib.nimble

@@ -1,6 +1,6 @@
 # Package
 
-version       = "0.3.2"
+version       = "0.3.3"
 author        = "Pietro Peterlongo"
 description   = "nimib 🐳 - nim 👑 driven ⛵ publishing ✍"
 license       = "MIT"