import syntax

[mflatt]

The current syntax for import is

import:
  module_path:
    import_modifier
    ...
  ...

This is simple and regular, but it means that no_prefix (or prefix) is relegated to the import_modifier part, which feels verbose and backwards. A for_meta or for_label similarly feels too buried in the import_modifier part.

Here's a real example from "rhombus.rhm" as part of the current implementation of scribble/rhombus:

import:
  rhombus/macro: no_prefix
  "typeset-rhombus.rkt": no_prefix
  "typeset_meta.rhm":
    for_meta
  scribble/base:
    prefix scribble
    expose: elem

The rhombus/macro: no_prefix part is especially common. Also, a typical ".scrbl" file implementing the current documentation has import: "common.rhm": no_prefix; using a prefix would not work right for "common.rhm", because it's meant to provide for_label bindings of rhombus and other exports for typesetting. In other cases, I find that using a prefix for my own local modules (like "typeset_meta.rhm") feels like overkill, but maybe that's just my Racket habits misleading me.

I'm tempted to go back to the old =-based syntax for import prefixes, especially if we avoid using = as a comparison or assignment operator and reserve as a kind of delimiter within a form like fun or import. Also, I'm tempted to have for_meta and for_label work to group a set of module_paths. With those changes, the example above would look like this:

import:
  = rhombus/macro
  = "typeset-rhombus.rkt"
  scribble = scribble/base
    expose: elem
  for_meta:
    "typeset_meta.rhm"

The leading = does a better job of highlighting namespace dumping, and typing import: = rhombus/maco is nicer than import: rhombus/macro: no_prefix (which I have to type all the time).

Here's the current "common.rhm":

import:
  rhombus/macro:
    no_prefix
  rhombus:
    no_prefix
    for_label
  rhombus/macro:
    no_prefix
    for_label

Possible revision:

import:
  = rhombus/macro
  for_label:
    = rhombus
    = rhombus/macro

Any opinions or other ideas?

[samth]

An alternative would be a postfix operation as seen in some other languages. Eg

import:
  rhombus/list
  rhombus/macro as mac
  rhombus/common as _ // following Scala

[tail-reversion]

The postfix idea could be extended, taking some inspiration from Haskell and Agda:

import:
  rhombus/math
  rhombus/macro as mac
  rhombus/list only:
    map
    filter
    foldr
  rhombus/port except:
    open_input_port
    close_input_port

[plane]

This is really nice! How would you work rename: into this sort of scheme? Something like this?

import:
  rhombus/list only:
    map = renamed_map

[tail-reversion]

Yes, renaming inline like that would be ideal. Python uses old_name as new_name, but I don’t know if it’d be possible to overload as to work on both modules and bindings.

[mflatt]

We still want modifiers to compose, but I think these ideas would work out with a syntax for import that lets modifiers be placed in various places as a kind of shorthand for a canonical that's general and compositional:

import:
  _import_clause_

_import_clause_ = _module_path_:             (a)
                    _modifier_
                    ...
                | _module_path_ _modifier_    (b)
                | _module_path_ _modifier_:   (c)
                    _modifier_
                    ...
                | _modifier_:                (d)
                    _import_clause_
                    ...

(a) is the canonical/general form

(b) is equivalent to

       _module_path_:
         _modifier_

where the _modifier_ might include an argument block

(c) is equivalent to

       _module_path_:
         _modifier_
         _modifier_
         ...

where the first _modifier_ cannot include an argument block

(d) is equivalent to pushing _modifier_ down as the first modifier of each nested _import_clause_,
where the _modifier_ cannot include an argument block.

Core modifiers:

• as _identifier_

  Sets the import prefix to _identifier_.

• open

  Omits the import prefix (instead of no_prefix or as _).

• open: _identifier_ ...; ...

  Exposes each specific _identifier_ for access without a prefix (instead of expose).

• only: _identifier_ ...; ...

  Constrains imports to the _identifier_s.

• except: _identifier_ ...; ...

  Constrains imports to not the _identifier_s.

• rename: _identifier_ as _identifier_; ...

  Renames individual imports (which may or may not be opened).

• for_meta [_n_]

  Phase shift by _n_, defaulting to 1.

• for_label

  Label-phase shift.

[mflatt]

It doesn't work to collapse no_prefix and expose into open, because then there's ambiguity for open between cases (b) and (c). But keeping expose instead of replacing it with with open works well.

In practice, this change makes most of my Scribble code only a little simpler by having open in place of : no_prefix. Still, it looks better that way, and the reordered for_meta and for_label are much better.

[tail-reversion]

Regarding phase levels, I strongly prefer the approach of grouping imports by phase because it makes it much easier to know what bindings are available at which phase. Therefore, I really like having for_meta and for_label groups sets of module paths.

[rocketnia]

How about this? The following could all be equivalent ways to import racket/math so that we could use the prefixed form math.pi in our code:

import:
  racket/math
  racket / math
  racket/"math.rkt"
  racket/math ~nicknamed math
  racket/"math.rkt" ~nicknamed math
  ~from racket/_:
    math
    "math.rkt"
    math ~nicknamed math
    "math.rkt" ~nicknamed math

While verbose, the term ~nicknamed can be just as clear about which name is which in both import lists and export lists. The externally visible name, which has more qualifiers attached to it to specify what collection or parent module it's in, goes on the left-hand (or leading) margin, where those qualifiers can be lined up with other similar qualifiers. The internally visible name, which is likely to resemble the external one but with the qualifiers stripped off, is on the right-hand (or trailing) side, where it's in close proximity to the part of the external name it corresponds to. Even if we used something more terse like =, it should probably be in this order.

The path is built up out of operators. In addition to the module-or-collection-forming operator /, there could be an operator :: to form submodule paths and an operator . to form paths to particular module exports. Altogether, a path could look like data/#{red-black}/augmented::uncontracted.nil.

The ~from clause can change how the following module paths are understood by making it so they're concatenated into the part marked by _. This would work by fairly rudimentary concatenation, and some of the fully concatenated paths built this way could be malformed in the end. (This choice of rudimentary concatenation serves to embrace the ambiguity between collection names and module names, an ambiguity which is often exploited as a convention in analogous names like racket/contract and racket/contract/base. Perhaps alternatively, there could be an explicit operation to convert a collection path like racket/contract into the module path that resembles it, like racket/contract&, but that extra punctuation would have to be written for every single import.)

Since the local alias for an import would oftentimes resemble an unqualified version of the external name, the default local alias would be inferred based on the last component of the imported path. So the above examples all import racket/math under the local alias math.

Depending on the path, the last component could be a top-level collection (in which case the collection's name is used, not "main.rkt"), a subcollection, a module, a filename-named module (in which case the file extension is dropped), a submodule, or an export.

The following would all be equivalent ways to import the pi export of racket/math under the local alias pi:

import: racket/math.pi
import: racket/math.pi ~nicknamed pi
import: ~from racket/math._: pi
import: ~from racket/math._: pi ~nicknamed pi

Imports of all the exports of a module could be specified using ._ in the import path itself, Scala-style:

import: racket/math._

As discussed here, code that uses this kind of import can be unstable.

However, I believe it can be justified in cases where the author of this client code is in control of the library's exports or in cases where the library promises not to add arbitrary exports in future versions.

In particular, I think

using a prefix for my own local modules (like "typeset_meta.rhm") feels like overkill, but maybe that's just my Racket habits misleading me

is correct. If the person importing the modules is the same person who's maintaining them, there's no need for them to worry that there'll be unanticipated exports in the future. (Hopefully they're testing their code well enough to catch import conflicts they create in their own files.)

It's only imports of other packages' modules that are in particular need of explicit prefixes or import lists, and even many of those cases can be factored out to be uncommon:

• Imports from external packages can be gathered into a single file, which can then be imported in a namespace-dumping way by just about every other file in the codebase. My proposal for project manifests is based on this train of thought.
• Libraries that export particularly useful bundles of exports, like rhombus/macro here, could define their own version-frozen bundles of imports. If you specifically import everything from the v1 bundle of rhombus/macro, you can be confident that if and when future versions of rhombus/macro add new exports, the v1 bundle import still won't let those in to clobber anything else.

I'm going to take a detour back to to the idea of importing prefixes. If we're using the exports of a module according to a prefix, what about using that module's submodules? If the module is racket/contract, what about using that prefix to access racket/contract/base?

I propose adding another path operator, ^, that allows us to use our local aliases within an import path. For instance, we might pick out a certain variant of a module and use that to import several other things:

import:
  data/#{red-black}/augmented::uncontracted ~nicknamed rb

... do things with rb.nil and rb.#{new-tree} ...

import:
  ^rb.nil
  ^rb.#{new-tree} ~nicknamed new_tree

... do things with nil and new_tree ...

In this case, by defining an alias rb that stands for a whole module path, we can easily remove the ::uncontracted part of the path when we want to diagnose bugs, and it'll affect our uses of rb both within the module body and within the other import paths.

Perhaps the more obvious design would be to allow this kind of prefixing even without the ^ operator:

import:
  rb.nil
  rb.#{new-tree} ~nicknamed new_tree

However, I think that would lead to surprises in common situations:

import:
  racket/syntax
  syntax/parse   ; tries to import `racket/syntax/parse`

Besides sidestepping these surprises, an explicit ^ to signal the use of a local alias means when the language encounters this ^ operator, it can actually expect the operand to be locally bound as an alias, and it can use that local binding to determine what to do to import this thing. This gives us an in for user-defined module path composition operators. :)

And that brings me back to the idea of importing bundles of imports all at once. I mentioned the potential of importing just the v1 exports of rhombus/macro. Another situation where shorthands for bundles are useful in Racket is when exporting all the fields of a struct. And just like exporting the fields of a struct, importing them all at once is justifiable for any struct whose set of exports is guaranteed not to change. In some cases, generic interfaces are like structs in this respect, with all their methods exported and a decisive plan not to add any more methods in the future.

Now we can consider using operations version, struct, and generics within a module path to modify it.

import:
  rhombus/macro/^version("1.0")._
  racket/base/^struct/date._
  racket/dict/^generics/#{gen:dict} ~nicknamed dict

These would be in some ways similar to the aliases we get from importing a module with a prefix, except that these could be used somewhere other than the beginning of a module path. As shown in one case (version), these could take additional macro-like arguments as well. They would consume their given arguments and all preceding parts of the path and perform their own macroexpansion logic based on that input. Their expanded form would contain information that let them serve as a virtual module path. The virtual module path racket/base/^struct could be used as the collection path for the virtual module path racket/base/^struct/date, which could be used as a module path to access the exports date, #{date?}, #{date-second}, #{date-minute}, etc.

In the case of racket/base/^struct/date, the example above uses ._ to import all its exports. Perhaps the date struct is stable enough for this.

In the case of racket/dict/^generics/#{gen:dict}, the set of primitive methods seems to cover a lot of ground. Perhaps it's subject to change in the future if future dict implementations admit more optimal implementations of certain operations. So in the example above, I've imported it with ~nicknamed dict to remove all doubt that it'll coexist properly with everything else I'm importing and defining.

All right. One last aspect to account for is phase-shifting (for-syntax, for-template, for-label). I would probably consider that notable enough to make it its own subsection of imports. However, I'd write it directly in the import line rather than putting it on a line of its own, so that the modules imported that way can have the same indentation they have when they're imported at phase 0:

import ~for_meta:
  rhombus/macro
import:
  rhombus/macro

If you prefer putting everything in one import, I guess I'd go with:

import:
  ~for_meta:
    rhombus/macro
  ~for_meta 0:
    rhombus/macro

With all these ideas in place, the example at the start of the thread would become:

import ~for_meta:
  "typeset_meta.rhm"
import:
  rhombus/macro/^version("1.0")._
  "typeset-rhombus.rkt"._
  scribble/base ~nicknamed scribble
  scribble/base.elem

[mflatt]

We'll need some way like this to access submodules as an extension of the core _module_path_ syntax. I appreciate that you haven't tried to make submodule access use the hierarchical-name ., because I've thought about that direction, and it looks like submodules and hierarchical names end up having significantly different properties.

A ~from-like form seems like it could be good as a derived (i.e., macro-implemented) import form.

[rocketnia]

I appreciate that you haven't tried to make submodule access use the hierarchical-name ., because I've thought about that direction, and it looks like submodules and hierarchical names end up having significantly different properties.

A module can have both a test submodule and a test export, and I don't see a benefit to referring to them with identical notation. That said, I think the virtual module paths I'm talking about would address the high-level purposes of submodules, leaving the :: notation to describe the low-level, non-virtual paths.

For reference, this is what I was suggesting for importing a single export from a submodule:

import:
  data/#{red-black}/augmented::uncontracted.nil

If the data/red-black/augmented module exported a variable called uncontracted, and if that variable's transformer binding worked as a virtual module path macro, then it would work with the ^ operator so that we could write....

import:
  data/#{red-black}/augmented.uncontracted ~nicknamed rb
  ^rb.nil

And I think if a module path has a . partway through, rather than at the end, it'll make sense for it to use the specified export's virtual module path behavior to interpret the rest of the path. This would let us abbreviate the above to:

import:
  data/#{red-black}/augmented.uncontracted.nil

This brings us back to something resembling the ::-based import, but potentially more flexible, since virtual module paths could have their own user-defined kinds of "smart" path logic.

A ~from-like form seems like it could be good as a derived (i.e., macro-implemented) import form.

What do you mean? On the one hand, I don't think there's a single keyword or grammar nonterminal that I wouldn't eventually like users to be able to write their own macros to alias or abstract over, so I'd prefer for ~from to be a macro like anything else. On the other hand, ~from is serving the purpose of only-in here, and I think relegating it to a library would make people more likely to reach for namespace-dumping whenever they didn't want to use a prefix for something.

[mflatt]

I now see that you're using ~from for individual imports.I was thinking more of the original example where (if I understand) it groups module with a common prefix. My initial reaction to ~from for only-in is that seems verbose, but I haven't actually tried it.

[rocketnia]

In terms of this approach, multiple exports from a single module would have paths with a common prefix. This means it's not necessary for as and rename to be separate notations. Thanks to that conflation, certain exported bindings could be allowed to work like module aliases in their own right. This in turn provides a going-with-the-grain way to add export bundling concepts like struct and version to the design, which facilitate the particular scenarios where namespace-dumping can be used reliably. And programs that use this namespace-dumping can save on verbosity.

Regardless of lexical syntax, I like how a lot of these concepts dovetail together.

With fresh eyes on the syntax, and with more attention to curbing verbosity, I'm thinking of revising my syntax choices to this:

import ~for_meta:
  "typeset_meta.rhm"
import:
  rhombus/macro.^version("1.0"): *
  "typeset-rhombus.rkt": *
  scribble/base: . ~dubbed scribble; elem

The changes here (and a few more changes not exemplified here) are:

• No more writing out the ~from keyword; every import clause with a block after it is implicitly a ~from clause.
• No more _ in ~from path templates; all the import paths in the ~from body extend the path template at the end, and if one of them needs to extend the template using / or ::, it can now write ./ or .:: at the beginning of its path. Extending the template anywhere other than the end is no longer permitted, but it could be simulated with a custom module path component macro.
• No more ._ at the end of namespace-dumping imports. Instead, inside a ~from body, the import * can be used as a namespace-dumping import.
• Custom path components like ^version("1.0") are now preceded by . instead of / so that their appearance in a ~from body doesn't have to have a leading ./.
• Multiple non-~from imports can now be separated by semicolons.
• The ~nicknamed keyword is now ~dubbed.
• The empty import path can be written as ., which allows us to specify a nickname for a whole module at the same level as the other bindings we're creating based on that module.