nu.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neural TTR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This module is defined in the Python file nu.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pprint import pprint\n",
    "from nu import Type, BType, PType, DepType, Pred, MeetType_n, FunType, InhibitType_n, StringType_n, Ty, iota, gensym_n, nu, and_n, MeetType, or_n, JoinType, labels, Rec, RecType\n",
    "from neurons import Network, Neuron, Synapse, ActivityPattern\n",
    "from utils import show, example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1:  `nu` maps types to types of neural activation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Types in general correspond to a pattern of activation on a given network. We define a type `T` called `MyType`.  Its corresponding type of neural activation is `nu(T)` which by default has the name `MyType_n`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MyType_n\n"
     ]
    }
   ],
   "source": [
    "T = Type('MyType')\n",
    "Tn = nu(T)\n",
    "print(show(Tn))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now create a network `N` and add a single neuron `n1` to it.  A network is a one dimensional array of neurons. `n1` will be indexed as `[0]` in the network `N`.  We define an activity pattern `h1` which is a two dimensional array indicating which neurons should be activated at each step.  `h1` has only a single step which activates the 0th neuron in a network. We add this activity pattern to the type `Tn` we defined above.  Note that when we add an activity pattern for a type we mention the network for which it holds.  This type may be implemented as different patterns of activation for different networks.  That is, we should not expect to find identical implementations of types in different brains.\n",
    "\n",
    "In order to see what happens in `N` when we it represents this type we use the method `ntrace` to turn on neural tracing.  We then create an instance of the neural type `Tn` on `N` and display `N`'s history.  The neuron has been given a default name of `neuron1`.  First it was off and then in was on.  That is, first there was no signal on its axon and then there was."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------  -  -\n",
      "neuron0  0  1\n",
      "-------  -  -\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "n1 = N.add_neuron()\n",
    "h1 = ActivityPattern([[0]])\n",
    "Tn.add_apat(N,h1)\n",
    "N.ntrace()\n",
    "Tn.create_n(N)\n",
    "N.display_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Neural types have a method `query_n` which takes a network as argument and tells you whether the history you have recorded using `ntrace` contains an instantiation of the neural type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "print(Tn.query_n(N))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2:  Activity patterns can involve more than one neuron"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a neural type for a basic type. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MyBasicType_n\n"
     ]
    }
   ],
   "source": [
    "T = BType('MyBasicType')\n",
    "Tn = nu(T)\n",
    "print(show(Tn))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example `T` corresponds to the activation of two neurons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------  -  -  -\n",
      "neuron1  0  1  1\n",
      "neuron2  0  0  1\n",
      "-------  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "n1 = N.add_neuron()\n",
    "n2 = N.add_neuron()\n",
    "h1 = ActivityPattern([[0],[1]])\n",
    "Tn.add_apat(N,h1)\n",
    "N.ntrace()\n",
    "Tn.create_n(N)\n",
    "N.display_history()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And, as before, we can check whether the history instantiates the neural type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "print(Tn.query_n(N))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3:  neural types for recursive ptypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In TTR 'hug($a$,$b$)' is known a ptype (a type constructed from a predicate together with its arguments).  Intuitively it is a type of situation in which $a$ hugs $b$.\n",
    "\n",
    "Here we use straight pyttr to create a type `Ind`, with name `'Ind'`, to serve as the type of individuals and we use the `judge`-method to assert that `'a'` and `'b'` are of this type.  (We are allowing individuals to be represented by Python strings.)  We declare `hug` to be a predicate, with name `'hug'` which takes two arguments of type `Ind`.  We can now create a ptype with `hug` as predicate and `'a'` and `'b'` as arguments.  Here we set the Python variable `hug_a_b` to this ptype.  We can use `show` to give a not unexpected printout of this ptype."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hug(a, b)\n"
     ]
    }
   ],
   "source": [
    "Ind = Type('Ind')\n",
    "Ind.judge('a')\n",
    "Ind.judge('b')\n",
    "hug = Pred('hug',[Ind,Ind])\n",
    "hug_a_b = PType(hug,['a','b'])\n",
    "print(show(hug_a_b))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now connect what we have done to neural TTR, starting by creating a network. `iota` (think of $\\iota$ as related to individuals) is a repository of activation patterns corresponding to individuals represented as Python strings.  `iota` has a method `add_grandmother` (as in  \"grandmother neuron\") which will add a single neuron to a network and associate activation of that neuron with the individual in its first argument. \n",
    "\n",
    "We create a type of neural activation, `hug_n`, corresponding to the predicate `hug` and call the `add_grandmother`-method for that type.  This will create a new neuron for the network and make activation of that neuron be the activity pattern associated with the type `hug_n`.  We now have what we need to create a neural type for `hug_a_b` and we set the Python variable `hug_a_b_n` to the neural type.\n",
    "\n",
    "We now turn on neural tracing, create an event of type `hug_a_b_n` on the network and display the history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------  -  -  -  -  -\n",
      "a       0  0  1  0  0\n",
      "b       0  0  0  1  0\n",
      "hug_n   0  1  0  0  0\n",
      "ptype2  *  1  1  1  0\n",
      "rel     *  1  0  0  0\n",
      "arg0    *  0  1  0  0\n",
      "arg1    *  0  0  1  0\n",
      "------  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "iota.add_grandmother('a',N)\n",
    "iota.add_grandmother('b',N)\n",
    "hug_n = nu(hug)\n",
    "hug_n.add_grandmother(N)\n",
    "hug_a_b_n = nu(hug_a_b)\n",
    "\n",
    "N.ntrace()\n",
    "hug_a_b_n.create_n(N)\n",
    "N.display_history()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This trace of the history of the network is possibly more complicated than you had been expecting.  The first three rows correspond to the neurons that we added by our calls to the `add_grandmother` methods.  In the first column these three neurons are off (the network is at rest). Then in three successive steps the predicate, the first argument and the second argument are activated.  Finally, in the fifth column, the network is at rest again.  \n",
    "\n",
    "Why did we choose to activate the neurons in order rather than, say, activate all three at the same time?  The reason is that the network needs to be able to distinguish between 'hug($a$,$b$)' and 'hug($b$,$a$)'.  If all three neurons were activated at the same time both of these would have the same representation.  In the literature on neural representation this is often referred to as $\\textit{the binding problem}$.\n",
    "\n",
    "But what is going on in the lower four rows of the history display?  In creating a neural event of the required type, the network discovered that it needed to create more neurons for book-keeping purposes.  In the first column these neurons were not present in the network and this is represented by the occurrences of `2`.  By the time we get to the second column the additional four neurons have been created.  Creating new neurons like this may seem implausible as a model of what happens in an animal brain.  However, it does not seem so implausible that many neurons in an animal brain are available to be designated as having a certain function.  Thus the creation of new neurons in our computational model could be seen as corresponding to the use of previously unused neurons in an animal brain.\n",
    "\n",
    "The new neurons are used for bookkeeping purposes.  How do we know that an event on the network of the type represented in the top three rows is in fact the representation of a ptype rather than just a sequence of three representations: of a predicate, followed by a representation of `'a'` and a separate representation of `'b'`.  The three events are tied together by the activation of the neuron labelled `ptype2` (\"ptype with two arguments\") throughout the duration of the three events.  The role of the individual events in the ptype are indicated by the remaining three neurons, labelled `rel`, `arg0` and `arg1` which are active simultaneously with `hug_n`, `a` and `b` respectively.  This means that the order of the three individual events is of itself not important, although the current implementation will always produce them in the order given here.  They could, however, in principle occur in any order preserving the simultaneity of activation of the role-labelling neurons and the neurons `hug_n`, `a` and `b`.  This, then, represents our approach to the binding problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another central problem for representing the kind of content you need for natural language on a neural architecture is known as $\\textit{the recursion problem}$.  We illustrate this here by allowing predicates to take ptypes as arguments, thus allowing ptypes within ptypes to an arbitrary depth of embedding.\n",
    "\n",
    "First we turn off neural tracing in the network `N`.  This will remove the previous history of the network and no events on the network will be stored until we restart neural tracing.\n",
    "\n",
    "We now extend our environment using pyttr.  We declare a new individual `'c'` and a new predicate `believe` whose first argument is required to be an individual and whose second argument is required to be a type.  (The type `Ty` is provided by pyttr as the type of types, that is, anything defined as a type in pyttr will be of the type `Ty`.)  We then define the ptype `believe_c_hug_a_b` corresponding to believe($c$, hug($a$, $b$)). \n",
    "\n",
    "We now use neural TTR to create a type of neural activation `believe_c_hug_a_b_n` corresponding to this ptype. We add a grandmother neuron for `'c'` to `N`, generate the type of neurological activation `believe_n` for the predicate `believe` and add a grandmother neuron for it to the network.\n",
    "\n",
    "Finally, we turn on neural tracing, create an event of type `believe_c_hug_a_b_n` on the network and display the history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------  -  -  -  -  -  -  -  -\n",
      "a          0  0  0  0  1  0  0  0\n",
      "b          0  0  0  0  0  1  0  0\n",
      "hug_n      0  0  0  1  0  0  0  0\n",
      "ptype2     0  1  1  1  1  1  1  0\n",
      "rel        0  1  0  0  0  0  0  0\n",
      "arg0       0  0  1  0  0  0  0  0\n",
      "arg1       0  0  0  1  1  1  1  0\n",
      "c          0  0  1  0  0  0  0  0\n",
      "believe_n  0  1  0  0  0  0  0  0\n",
      "ptype2     *  0  0  1  1  1  0  0\n",
      "rel        *  0  0  1  0  0  0  0\n",
      "arg0       *  0  0  0  1  0  0  0\n",
      "arg1       *  0  0  0  0  1  0  0\n",
      "---------  -  -  -  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N.nontrace()\n",
    "\n",
    "\n",
    "Ind.judge('c')\n",
    "believe = Pred('believe',[Ind,Ty])\n",
    "believe_c_hug_a_b = PType(believe,['c',hug_a_b])\n",
    "\n",
    "believe_c_hug_a_b_n = nu(believe_c_hug_a_b)\n",
    "iota.add_grandmother('c',N)\n",
    "believe_n = nu(believe)\n",
    "believe_n.add_grandmother(N)\n",
    "\n",
    "\n",
    "N.ntrace()\n",
    "believe_c_hug_a_b_n.create_n(N)\n",
    "N.display_history()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we now have a second collection of neurons `ptype2`, `rel`, `arg0` and `arg1` to deal with the fact that we have represented a ptype which has a ptype within it.  Notice that the activation of `arg1` for the `ptype2` whose relation is represented by `believe_n` coincides with the activation of the second `ptype2` whose relation is represented by `hug_n`.\n",
    "\n",
    "We can increase the embedding to an arbitrary depth, 'know($d$, believe($c$, hug($a$,$b$)))' and so on, and with each increase in depth we will generate a new collection of book-keeping neurons.  While for any snapshot of the neural architecture at a point in time it will have what is needed to process a certain finite depth of embedding, there is in principle no limit to the depth of embedding that it can process since it can always expand to account for an extra level of embedding.  In practice, of course, the depth of embedding is limited by the computational resources available (the size of the memory in the computer or the number of neurons in the brain).  This simple technique might provide us with one route into understanding how the plasticity of a brain allows it to devote resources to tasks on an as needed basis.\n",
    "\n",
    "This, then, is our proposal for dealing with the recursion problem. Note that it relies on two aspects of our proposal:  firstly, that informational content is represented as $\\textit{events}$ on a network rather than as architectural structure and secondly, that the network has $\\textit{plasticity}$ in that it can expand or dedicate unused neurons to a particular task. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 4: dependent types as functions with arbitrary depth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another place where you need recursion is when you characterize functions, since a function can return another function as result.  In this implementation we only treat functions which are dependent types, that is, they return a type or a function which is a dependent type.  The examples we will deal with here are $\\lambda v\\!:\\!\\textit{Ind}\\ .\\ \\textrm{hug}(v,b)$ and $\\lambda x\\!:\\!\\textit{Ind}\\ .\\ \\lambda y\\!:\\!\\textit{Ind}\\ .\\ \\textrm{hug}(x,y)$.\n",
    "\n",
    "First we use pyttr to create a dependent type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lambda v:Ind . hug(v, b)\n"
     ]
    }
   ],
   "source": [
    "T = DepType('v',Ind,PType(hug,['v','b']))\n",
    "print(show(T))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we use neural TTR to turn off neural tracing on the network `N`.  We create a type of neural activity corresponding to `Ind` and add a grandmother for it on `N`.  We now create a type of neural activity `Tn` corresponding to the dependent type.\n",
    "\n",
    "We now turn on neural tracing and create an event of the neural type on `N` and display `N`'s history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------  -  -  -  -  -  -  -\n",
      "a          0  0  0  0  0  0  0\n",
      "b          0  0  0  0  1  0  0\n",
      "hug_n      0  0  1  0  0  0  0\n",
      "ptype2     0  0  1  1  1  0  0\n",
      "rel        0  0  1  0  0  0  0\n",
      "arg0       0  0  0  1  0  0  0\n",
      "arg1       0  0  0  0  1  0  0\n",
      "c          0  0  0  0  0  0  0\n",
      "believe_n  0  0  0  0  0  0  0\n",
      "ptype2     0  0  0  0  0  0  0\n",
      "rel        0  0  0  0  0  0  0\n",
      "arg0       0  0  0  0  0  0  0\n",
      "arg1       0  0  0  0  0  0  0\n",
      "Ind_n      0  1  0  0  0  0  0\n",
      "lambda     *  1  1  1  1  1  0\n",
      "dom        *  1  0  0  0  0  0\n",
      "var        *  1  0  1  0  0  0\n",
      "rng        *  0  1  1  1  1  0\n",
      "---------  -  -  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N.nontrace()\n",
    "Ind_n = nu(Ind)\n",
    "Ind_n.add_grandmother(N)\n",
    "Tn = nu(T)\n",
    "\n",
    "N.ntrace()\n",
    "Tn.create_n(N)\n",
    "N.display_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we now have a collection of book-keeping neurons corresponding to the structure of the function which is the dependent type.  When the neuron `lambda` is active we are in the midst of an event representing a function.  In the first part of the function we represent its domain (`dom`) and its variable (`var`).  These neurons are active simultaneously with the neuron `Ind_n`, encoding the type of the domain of the function. The body of the function is coded by the neuron `rng` (\"range\") which is in fact a two place ptype, as coded by the onset of the first `ptype2` neuron being simultaneous with that of `rng`.  The ptype is coded as before except that the first argument is now indicated to be the variable which was introduced in the function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to show the kind of activity associated with a neural type on a network is to create an event on the network in the way that we have been doing so far.  Sometimes, however, it is more convenient to inspect the activity pattern which is encoded on the neural type for that particular network.  This is what we do below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(14, 'lambda', 1), (15, 'dom', 1), (16, 'var', 1), (13, 'Ind_n', 1)],\n",
      " [(15, 'dom', 0),\n",
      "  (16, 'var', 0),\n",
      "  (13, 'Ind_n', 0),\n",
      "  (17, 'rng', 1),\n",
      "  (3, 'ptype2', 1),\n",
      "  (4, 'rel', 1),\n",
      "  (2, 'hug_n', 1)],\n",
      " [(4, 'rel', 0), (2, 'hug_n', 0), (5, 'arg0', 1), (16, 'var', 1)],\n",
      " [(5, 'arg0', 0), (16, 'var', 0), (6, 'arg1', 1), (1, 'b', 1)],\n",
      " [(3, 'ptype2', 0), (6, 'arg1', 0), (1, 'b', 0)],\n",
      " [(14, 'lambda', 0), (17, 'rng', 0)]]\n"
     ]
    }
   ],
   "source": [
    "pprint(Tn.show_apat(N))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The activity pattern is displayed as a list of lists of triples.  The first element in each triple is the unique identifier of a neuron.  The second element is the convenient name we have assigned the neuron to make our displays human readable.  The third element is either `1` for activate the neuron or `0` for deactivate.  A list of triple thus represents the activations and deactivations which are to be carried out at one particular time step.  The list of lists represents the activations/deactivations to be carried out over a series of time steps.  Note that if a neuron is activated at one time step it will remain active until it is deactivated at a subsequent time step.  Thus in this pattern neuron `14` (`lambda`) is activated in the first time step and is not deactivated until the final time step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------  -  -  -  -  -  -  -  -  -\n",
      "a          0  0  0  0  0  0  0  0  0\n",
      "b          0  0  0  0  0  0  0  0  0\n",
      "hug_n      0  0  0  1  0  0  0  0  0\n",
      "ptype2     0  0  0  1  1  1  0  0  0\n",
      "rel        0  0  0  1  0  0  0  0  0\n",
      "arg0       0  0  0  0  1  0  0  0  0\n",
      "arg1       0  0  0  0  0  1  0  0  0\n",
      "c          0  0  0  0  0  0  0  0  0\n",
      "believe_n  0  0  0  0  0  0  0  0  0\n",
      "ptype2     0  0  0  0  0  0  0  0  0\n",
      "rel        0  0  0  0  0  0  0  0  0\n",
      "arg0       0  0  0  0  0  0  0  0  0\n",
      "arg1       0  0  0  0  0  0  0  0  0\n",
      "Ind_n      0  1  1  0  0  0  0  0  0\n",
      "lambda     0  1  1  1  1  1  1  1  0\n",
      "dom        0  1  0  0  0  0  0  0  0\n",
      "var        0  1  0  0  1  0  0  0  0\n",
      "rng        0  0  1  1  1  1  1  1  0\n",
      "lambda     *  0  1  1  1  1  1  0  0\n",
      "dom        *  0  1  0  0  0  0  0  0\n",
      "var        *  0  1  0  0  1  0  0  0\n",
      "rng        *  0  0  1  1  1  1  0  0\n",
      "---------  -  -  -  -  -  -  -  -  -\n",
      "[[(14, 'lambda', 1), (15, 'dom', 1), (16, 'var', 1), (13, 'Ind_n', 1)],\n",
      " [(15, 'dom', 0),\n",
      "  (16, 'var', 0),\n",
      "  (13, 'Ind_n', 0),\n",
      "  (17, 'rng', 1),\n",
      "  (18, 'lambda', 1),\n",
      "  (19, 'dom', 1),\n",
      "  (20, 'var', 1),\n",
      "  (13, 'Ind_n', 1)],\n",
      " [(19, 'dom', 0),\n",
      "  (20, 'var', 0),\n",
      "  (13, 'Ind_n', 0),\n",
      "  (21, 'rng', 1),\n",
      "  (3, 'ptype2', 1),\n",
      "  (4, 'rel', 1),\n",
      "  (2, 'hug_n', 1)],\n",
      " [(4, 'rel', 0), (2, 'hug_n', 0), (5, 'arg0', 1), (16, 'var', 1)],\n",
      " [(5, 'arg0', 0), (16, 'var', 0), (6, 'arg1', 1), (20, 'var', 1)],\n",
      " [(3, 'ptype2', 0), (6, 'arg1', 0), (20, 'var', 0)],\n",
      " [(18, 'lambda', 0), (21, 'rng', 0)],\n",
      " [(14, 'lambda', 0), (17, 'rng', 0)]]\n"
     ]
    }
   ],
   "source": [
    "N.nontrace()\n",
    "N.ntrace()\n",
    "T= DepType('x',Ind,DepType('y',Ind,PType(hug,['x','y'])))\n",
    "Tn = nu(T)\n",
    "Tn.create_n(N)\n",
    "N.display_history()\n",
    "pprint(Tn.show_apat(N))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "every(lambda x:Ind . dog(x), lambda x:Ind . run(x))\n",
      "-------  -  -  -  -  -  -  -  -  -  -  -  -  -\n",
      "every_n  0  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "dog_n    0  0  0  1  0  0  0  0  0  0  0  0  0\n",
      "run_n    0  0  0  0  0  0  0  0  1  0  0  0  0\n",
      "Ind_n    0  0  1  0  0  0  0  1  0  0  0  0  0\n",
      "ptype2   *  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "rel      *  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "arg0     *  0  1  1  1  1  1  0  0  0  0  0  0\n",
      "arg1     *  0  0  0  0  0  0  1  1  1  1  1  0\n",
      "lambda   *  0  1  1  1  1  0  1  1  1  1  0  0\n",
      "dom      *  0  1  0  0  0  0  1  0  0  0  0  0\n",
      "var      *  0  1  0  1  0  0  1  0  1  0  0  0\n",
      "rng      *  0  0  1  1  1  0  0  1  1  1  0  0\n",
      "ptype1   *  0  0  1  1  0  0  0  1  1  0  0  0\n",
      "rel      *  0  0  1  0  0  0  0  1  0  0  0  0\n",
      "arg0     *  0  0  0  1  0  0  0  0  1  0  0  0\n",
      "-------  -  -  -  -  -  -  -  -  -  -  -  -  -\n",
      "[[(4, 'ptype2', 1), (5, 'rel', 1), (0, 'every_n', 1)],\n",
      " [(5, 'rel', 0),\n",
      "  (0, 'every_n', 0),\n",
      "  (6, 'arg0', 1),\n",
      "  (8, 'lambda', 1),\n",
      "  (9, 'dom', 1),\n",
      "  (10, 'var', 1),\n",
      "  (3, 'Ind_n', 1)],\n",
      " [(9, 'dom', 0),\n",
      "  (10, 'var', 0),\n",
      "  (3, 'Ind_n', 0),\n",
      "  (11, 'rng', 1),\n",
      "  (12, 'ptype1', 1),\n",
      "  (13, 'rel', 1),\n",
      "  (1, 'dog_n', 1)],\n",
      " [(13, 'rel', 0), (1, 'dog_n', 0), (14, 'arg0', 1), (10, 'var', 1)],\n",
      " [(12, 'ptype1', 0), (14, 'arg0', 0), (10, 'var', 0)],\n",
      " [(8, 'lambda', 0), (11, 'rng', 0)],\n",
      " [(6, 'arg0', 0),\n",
      "  (7, 'arg1', 1),\n",
      "  (8, 'lambda', 1),\n",
      "  (9, 'dom', 1),\n",
      "  (10, 'var', 1),\n",
      "  (3, 'Ind_n', 1)],\n",
      " [(9, 'dom', 0),\n",
      "  (10, 'var', 0),\n",
      "  (3, 'Ind_n', 0),\n",
      "  (11, 'rng', 1),\n",
      "  (12, 'ptype1', 1),\n",
      "  (13, 'rel', 1),\n",
      "  (2, 'run_n', 1)],\n",
      " [(13, 'rel', 0), (2, 'run_n', 0), (14, 'arg0', 1), (10, 'var', 1)],\n",
      " [(12, 'ptype1', 0), (14, 'arg0', 0), (10, 'var', 0)],\n",
      " [(8, 'lambda', 0), (11, 'rng', 0)],\n",
      " [(4, 'ptype2', 0), (7, 'arg1', 0)]]\n",
      "--------------  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n",
      "every_n         0  0  0  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "dog_n           0  0  0  0  0  1  0  0  0  0  0  0  0  0  0\n",
      "run_n           0  0  0  0  0  0  0  0  0  0  1  0  0  0  0\n",
      "Ind_n           0  0  0  0  1  0  0  0  0  1  0  0  0  0  0\n",
      "ptype2          0  0  0  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "rel             0  0  0  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "arg0            0  0  0  0  1  1  1  1  1  0  0  0  0  0  0\n",
      "arg1            0  0  0  0  0  0  0  0  0  1  1  1  1  1  0\n",
      "lambda          0  0  0  0  1  1  1  1  0  1  1  1  1  0  0\n",
      "dom             0  0  0  0  1  0  0  0  0  1  0  0  0  0  0\n",
      "var             0  0  0  0  1  0  1  0  0  1  0  1  0  0  0\n",
      "rng             0  0  0  0  0  1  1  1  0  0  1  1  1  0  0\n",
      "ptype1          0  0  0  0  0  1  1  0  0  0  1  1  0  0  0\n",
      "rel             0  0  0  0  0  1  0  0  0  0  1  0  0  0  0\n",
      "arg0            0  0  0  0  0  0  1  0  0  0  0  1  0  0  0\n",
      "every dog runs  0  1  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay           0  0  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay           0  0  0  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay           0  0  0  0  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay           0  0  0  0  0  1  1  1  1  1  1  1  1  1  0\n",
      "Delay           0  0  0  0  0  0  1  1  1  1  1  1  1  1  0\n",
      "Delay           0  0  0  0  0  0  0  1  1  1  1  1  1  1  0\n",
      "Delay           0  0  0  0  0  0  0  0  1  1  1  1  1  1  0\n",
      "Delay           0  0  0  0  0  0  0  0  0  1  1  1  1  1  0\n",
      "Delay           0  0  0  0  0  0  0  0  0  0  1  1  1  1  0\n",
      "Delay           0  0  0  0  0  0  0  0  0  0  0  1  1  1  0\n",
      "Delay           0  0  0  0  0  0  0  0  0  0  0  0  1  1  0\n",
      "Delay           0  0  0  0  0  0  0  0  0  0  0  0  0  1  0\n",
      "--------------  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "Ppty = FunType(Ind,Ty)\n",
    "every = Pred('every',[Ppty,Ppty])\n",
    "every_n = nu(every)\n",
    "N = Network()\n",
    "every_n.add_grandmother(N)\n",
    "\n",
    "dog = Pred('dog',[Ind])\n",
    "dog_n = nu(dog)\n",
    "dog_n.add_grandmother(N)\n",
    "\n",
    "run = Pred('run',[Ind])\n",
    "run_n = nu(run)\n",
    "run_n.add_grandmother(N)\n",
    "\n",
    "Ind_n.add_grandmother(N)\n",
    "dog_ppty = DepType('x',Ind,PType(dog,['x']))\n",
    "run_ppty = DepType('x',Ind,PType(run,['x']))\n",
    "\n",
    "Tedr = PType(every,[dog_ppty,run_ppty])\n",
    "print(show(Tedr))\n",
    "\n",
    "Tedr_n = nu(Tedr)\n",
    "N.ntrace()\n",
    "Tedr_n.create_n(N)\n",
    "N.display_history()\n",
    "pprint(Tedr_n.show_apat(N))\n",
    "\n",
    "\n",
    "\n",
    "N.nontrace()\n",
    "m = N.memorize_type(Tedr_n,'every dog runs')\n",
    "N.ntrace()\n",
    "m.excite()\n",
    "N.run()\n",
    "N.display_history()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(0, 'Ind_n', 1)]]\n",
      "[[(1, 'a', 1)]]\n",
      "[[(0, 'Ind_n', 1), (1, 'a', 1)], [(0, 'Ind_n', 0), (1, 'a', 0)]]\n",
      "-----  -  -  -  -  -\n",
      "Ind_n  0  0  0  1  0\n",
      "a      0  0  0  1  0\n",
      "b      0  0  0  0  0\n",
      "a:Ind  0  1  1  1  0\n",
      "Delay  0  0  1  1  0\n",
      "Delay  0  0  0  1  0\n",
      "-----  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "Ind_n.add_grandmother(N)\n",
    "iota.add_grandmother('a',N)\n",
    "iota.add_grandmother('b',N)\n",
    "a_n = nu('a')\n",
    "pprint(Ind_n.show_apat(N))\n",
    "pprint(a_n.show_apat(N))\n",
    "pprint(Ind_n.judgmnt_type_n(a_n).show_apat(N))\n",
    "\n",
    "m = N.memorize_judgmnt(Ind_n,a_n,'a:Ind')\n",
    "N.ntrace()\n",
    "m.excite()\n",
    "N.run()\n",
    "N.display_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n",
      "every_n           0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "dog_n             0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0\n",
      "run_n             0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0\n",
      "Ind_n             0  0  0  0  1  0  0  0  0  1  0  0  0  0  0  0\n",
      "e                 0  0  0  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "ptype2            0  0  0  1  1  1  1  1  1  1  1  1  1  1  0  0\n",
      "rel               0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "arg0              0  0  0  0  1  1  1  1  1  0  0  0  0  0  0  0\n",
      "arg1              0  0  0  0  0  0  0  0  0  1  1  1  1  1  0  0\n",
      "lambda            0  0  0  0  1  1  1  1  0  1  1  1  1  0  0  0\n",
      "dom               0  0  0  0  1  0  0  0  0  1  0  0  0  0  0  0\n",
      "var               0  0  0  0  1  0  1  0  0  1  0  1  0  0  0  0\n",
      "rng               0  0  0  0  0  1  1  1  0  0  1  1  1  0  0  0\n",
      "ptype1            0  0  0  0  0  1  1  0  0  0  1  1  0  0  0  0\n",
      "rel               0  0  0  0  0  1  0  0  0  0  1  0  0  0  0  0\n",
      "arg0              0  0  0  0  0  0  1  0  0  0  0  1  0  0  0  0\n",
      "e:every(dog,run)  0  1  1  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  1  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  1  1  1  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  1  1  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  1  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  0  1  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  0  0  1  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  0  0  0  1  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  0  0  0  0  1  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  0  0  0  0  0  1  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  0  0  0  0  0  0  1  1  0\n",
      "Delay             0  0  0  0  0  0  0  0  0  0  0  0  0  0  1  0\n",
      "----------------  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "# uses variables from example 5\n",
    "N = Network()\n",
    "every_n.add_grandmother(N)\n",
    "dog_n.add_grandmother(N)\n",
    "run_n.add_grandmother(N)\n",
    "Ind_n.add_grandmother(N)\n",
    "T = PType(every,[dog_ppty,run_ppty])\n",
    "T_n = nu(T)\n",
    "iota.add_grandmother('e',N)\n",
    "e_n = nu('e')\n",
    "m = N.memorize_judgmnt(T_n,e_n, 'e:every(dog,run)')\n",
    "N.ntrace()\n",
    "m.excite()\n",
    "N.run()\n",
    "N.display_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------  -  -  -  -  -  -  -  -\n",
      "T1_n     0  0  0  0  1  0  0  0\n",
      "T2_n     0  0  0  0  0  1  0  0\n",
      "&_n      0  0  0  1  0  0  0  0\n",
      "a        0  0  0  1  1  1  1  0\n",
      "ptype2   0  0  0  1  1  1  0  0\n",
      "rel      0  0  0  1  0  0  0  0\n",
      "arg0     0  0  0  0  1  0  0  0\n",
      "arg1     0  0  0  0  0  1  0  0\n",
      "a:T1&T2  0  1  1  1  1  1  1  0\n",
      "Delay    0  0  1  1  1  1  1  0\n",
      "Delay    0  0  0  1  1  1  1  0\n",
      "Delay    0  0  0  0  1  1  1  0\n",
      "Delay    0  0  0  0  0  1  1  0\n",
      "Delay    0  0  0  0  0  0  1  0\n",
      "-------  -  -  -  -  -  -  -  -\n",
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "T1 = Type('T1')\n",
    "T2 = Type('T2')\n",
    "T1_n = nu(T1)\n",
    "T2_n = nu(T2)\n",
    "T1_n.add_grandmother(N)\n",
    "T2_n.add_grandmother(N)\n",
    "and_n.add_grandmother(N)\n",
    "iota.add_grandmother('a',N)\n",
    "T3 = MeetType(T1,T2)\n",
    "T3_n = nu(T3)\n",
    "m = N.memorize_judgmnt(T3_n,a_n,'a:T1&T2')\n",
    "N.ntrace()\n",
    "m.excite()\n",
    "N.run()\n",
    "N.display_history()\n",
    "\n",
    "print(N.match_apat(T1_n.judgmnt_type_n(a_n).getapat(N)))\n",
    "print(N.match_apat(T2_n.judgmnt_type_n(a_n).getapat(N)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------  -  -  -  -  -  -  -\n",
      "T1_n     0  0  0  0  1  0  0\n",
      "T2_n     0  0  0  0  0  1  0\n",
      "v_n      0  0  0  1  0  0  0\n",
      "a        0  0  0  1  0  0  0\n",
      "ptype2   0  0  0  1  1  1  0\n",
      "rel      0  0  0  1  0  0  0\n",
      "arg0     0  0  0  0  1  0  0\n",
      "arg1     0  0  0  0  0  1  0\n",
      "a:T1vT2  0  1  1  1  1  1  0\n",
      "Delay    0  0  1  1  1  1  0\n",
      "Delay    0  0  0  1  1  1  0\n",
      "Delay    0  0  0  0  1  1  0\n",
      "Delay    0  0  0  0  0  1  0\n",
      "-------  -  -  -  -  -  -  -\n",
      "False\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "T1 = Type('T1')\n",
    "T2 = Type('T2')\n",
    "T1_n = nu(T1)\n",
    "T2_n = nu(T2)\n",
    "T1_n.add_grandmother(N)\n",
    "T2_n.add_grandmother(N)\n",
    "or_n.add_grandmother(N)\n",
    "iota.add_grandmother('a',N)\n",
    "T3 = JoinType(T1,T2)\n",
    "T3_n = nu(T3)\n",
    "m = N.memorize_judgmnt(T3_n,a_n,'a:T1vT2')\n",
    "N.ntrace()\n",
    "m.excite()\n",
    "N.run()\n",
    "N.display_history()\n",
    "\n",
    "print(N.match_apat(T1_n.judgmnt_type_n(a_n).getapat(N)))\n",
    "print(N.match_apat(T2_n.judgmnt_type_n(a_n).getapat(N)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "#Subtyping for neural types in terms of a relation on apats on a given network.  Works for these examples...\n",
    "print(T1_n.judgmnt_type_n(a_n).subtype_of_n(T3_n.judgmnt_type_n(a_n),N))\n",
    "and_n.add_grandmother(N)\n",
    "T4 = MeetType(T1,T2)\n",
    "T4_n = nu(T4)\n",
    "print(T1_n.judgmnt_type_n(a_n).subtype_of_n(T4_n.judgmnt_type_n(a_n),N))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 11"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(10, 'rec', 1), (4, 'field', 1), (5, 'label', 1), (1, 'l_e', 1)],\n",
      " [(5, 'label', 0), (1, 'l_e', 0), (6, 'value', 1), (3, 's', 1)],\n",
      " [(6, 'value', 0), (3, 's', 0), (4, 'field', 0)],\n",
      " [(7, 'field', 1), (8, 'label', 1), (0, 'l_x', 1)],\n",
      " [(8, 'label', 0), (0, 'l_x', 0), (9, 'value', 1), (2, 'a', 1)],\n",
      " [(9, 'value', 0), (2, 'a', 0), (7, 'field', 0)],\n",
      " [(10, 'rec', 0)]]\n",
      "-----  -  -  -  -  -  -  -  -\n",
      "l_x    0  0  0  0  1  0  0  0\n",
      "l_e    0  1  0  0  0  0  0  0\n",
      "a      0  0  0  0  0  1  0  0\n",
      "s      0  0  1  0  0  0  0  0\n",
      "field  0  1  1  0  0  0  0  0\n",
      "label  0  1  0  0  0  0  0  0\n",
      "value  0  0  1  0  0  0  0  0\n",
      "field  0  0  0  0  1  1  0  0\n",
      "label  0  0  0  0  1  0  0  0\n",
      "value  0  0  0  0  0  1  0  0\n",
      "rec    0  1  1  1  1  1  1  0\n",
      "-----  -  -  -  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "labels.add_grandmother('l_x',N)\n",
    "labels.add_grandmother('l_e',N) \n",
    "iota.add_grandmother('a',N)\n",
    "iota.add_grandmother('s',N)\n",
    "r = Rec({'l_x':'a','l_e':'s'})\n",
    "r_n = nu(r)\n",
    "pprint(r_n.show_apat(N))\n",
    "\n",
    "N.ntrace()\n",
    "r_n.create_n(N)\n",
    "N.display_history()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(10, 'rec', 1), (4, 'field', 1), (5, 'label', 1), (1, 'l_e', 1)],\n",
      " [(5, 'label', 0),\n",
      "  (1, 'l_e', 0),\n",
      "  (6, 'value', 1),\n",
      "  (11, 'lambda', 1),\n",
      "  (12, 'dom', 1),\n",
      "  (13, 'var', 1),\n",
      "  (2, 'Ind_n', 1)],\n",
      " [(12, 'dom', 0),\n",
      "  (13, 'var', 0),\n",
      "  (2, 'Ind_n', 0),\n",
      "  (14, 'rng', 1),\n",
      "  (15, 'ptype1', 1),\n",
      "  (16, 'rel', 1),\n",
      "  (3, 'dog_n', 1)],\n",
      " [(16, 'rel', 0), (3, 'dog_n', 0), (17, 'arg0', 1), (13, 'var', 1)],\n",
      " [(15, 'ptype1', 0), (17, 'arg0', 0), (13, 'var', 0)],\n",
      " [(11, 'lambda', 0), (14, 'rng', 0)],\n",
      " [(0, 'l_x', 1)],\n",
      " [(0, 'l_x', 0)],\n",
      " [(6, 'value', 0), (4, 'field', 0)],\n",
      " [(7, 'field', 1), (8, 'label', 1), (0, 'l_x', 1)],\n",
      " [(8, 'label', 0), (0, 'l_x', 0), (9, 'value', 1), (2, 'Ind_n', 1)],\n",
      " [(9, 'value', 0), (2, 'Ind_n', 0), (7, 'field', 0)],\n",
      " [(10, 'rec', 0)]]\n",
      "------  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n",
      "l_x     0  0  0  0  0  0  0  1  0  0  1  0  0  0\n",
      "l_e     0  1  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "Ind_n   0  0  1  0  0  0  0  0  0  0  0  1  0  0\n",
      "dog_n   0  0  0  1  0  0  0  0  0  0  0  0  0  0\n",
      "field   0  1  1  1  1  1  1  1  1  0  0  0  0  0\n",
      "label   0  1  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "value   0  0  1  1  1  1  1  1  1  0  0  0  0  0\n",
      "field   0  0  0  0  0  0  0  0  0  0  1  1  0  0\n",
      "label   0  0  0  0  0  0  0  0  0  0  1  0  0  0\n",
      "value   0  0  0  0  0  0  0  0  0  0  0  1  0  0\n",
      "rec     0  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "lambda  0  0  1  1  1  1  0  0  0  0  0  0  0  0\n",
      "dom     0  0  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "var     0  0  1  0  1  0  0  0  0  0  0  0  0  0\n",
      "rng     0  0  0  1  1  1  0  0  0  0  0  0  0  0\n",
      "ptype1  0  0  0  1  1  0  0  0  0  0  0  0  0  0\n",
      "rel     0  0  0  1  0  0  0  0  0  0  0  0  0  0\n",
      "arg0    0  0  0  0  1  0  0  0  0  0  0  0  0  0\n",
      "------  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "labels.add_grandmother('l_x',N)\n",
    "labels.add_grandmother('l_e',N)\n",
    "Ind_n.add_grandmother(N)\n",
    "dog_n.add_grandmother(N)\n",
    "Dog = DepType('v',Ind,PType(dog,['v']))\n",
    "T_dog = RecType({'l_x':Ind,\n",
    "                 'l_e':(Dog,['l_x'])})\n",
    "T_dog_n = nu(T_dog)\n",
    "pprint(T_dog_n.show_apat(N))\n",
    "\n",
    "N.ntrace()\n",
    "T_dog_n.create_n(N)\n",
    "N.display_history()\n",
    "#Problem with two labels at same time in dependent fields\n",
    "#Now solved: a label neuron is marked as either a label or part of a value\n",
    "\n",
    "#Random order? np.random.shuffle()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 13"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dog(a)\n",
      "\n",
      "\n",
      "[[(3, 'lambda', 1), (4, 'dom', 1), (5, 'var', 1), (1, 'Ind_n', 1)],\n",
      " [(4, 'dom', 0),\n",
      "  (5, 'var', 0),\n",
      "  (1, 'Ind_n', 0),\n",
      "  (6, 'rng', 1),\n",
      "  (7, 'ptype1', 1),\n",
      "  (8, 'rel', 1),\n",
      "  (0, 'dog_n', 1)],\n",
      " [(8, 'rel', 0), (0, 'dog_n', 0), (9, 'arg0', 1), (5, 'var', 1)],\n",
      " [(7, 'ptype1', 0), (9, 'arg0', 0), (5, 'var', 0)],\n",
      " [(3, 'lambda', 0), (6, 'rng', 0)]]\n",
      "\n",
      "\n",
      "[[(2, 'a', 1)]]\n",
      "\n",
      "\n",
      "[[(7, 'ptype1', 1), (8, 'rel', 1), (0, 'dog_n', 1)],\n",
      " [(8, 'rel', 0), (0, 'dog_n', 0), (9, 'arg0', 1), (2, 'a', 1)],\n",
      " [(7, 'ptype1', 0), (9, 'arg0', 0), (2, 'a', 0)]]\n"
     ]
    }
   ],
   "source": [
    "#Function application\n",
    "N = Network()\n",
    "dog_n.add_grandmother(N)\n",
    "Ind_n.add_grandmother(N)\n",
    "Dog = DepType('v',Ind,PType(dog,['v']))\n",
    "iota.add_grandmother('a',N)\n",
    "print(show(Dog.app('a')))\n",
    "print('\\n')\n",
    "Dog_n = nu(Dog)\n",
    "a_n = nu('a')\n",
    "Dog_a_n = nu(Dog.app('a'))\n",
    "pprint(Dog_n.show_apat(N))\n",
    "print('\\n')\n",
    "pprint(a_n.show_apat(N))\n",
    "print('\\n')\n",
    "pprint(Dog_a_n.show_apat(N))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 14"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(11, 'rec', 1), (5, 'field', 1), (6, 'label', 1), (1, 'l_e', 1)],\n",
      " [(6, 'label', 0), (1, 'l_e', 0), (7, 'value', 1), (4, 's1', 1)],\n",
      " [(7, 'value', 0), (4, 's1', 0), (5, 'field', 0)],\n",
      " [(8, 'field', 1), (9, 'label', 1), (0, 'l_x', 1)],\n",
      " [(9, 'label', 0), (0, 'l_x', 0), (10, 'value', 1), (2, 'a', 1)],\n",
      " [(10, 'value', 0), (2, 'a', 0), (8, 'field', 0)],\n",
      " [(11, 'rec', 0)]]\n"
     ]
    }
   ],
   "source": [
    "#Substitution in records\n",
    "N = Network()\n",
    "labels.add_grandmother('l_x',N)\n",
    "labels.add_grandmother('l_e',N) \n",
    "iota.add_grandmother('a',N)\n",
    "iota.add_grandmother('s',N)\n",
    "r = Rec({'l_x':'a','l_e':'s'})\n",
    "iota.add_grandmother('s1',N)\n",
    "r1 = r.subst('s','s1')\n",
    "r1_n = nu(r1)\n",
    "pprint(r1_n.show_apat(N))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{l_e : (lambda v:Ind . dog(v), [l_x]), l_x : Ind}\n",
      "{l_e : (lambda v:Ind . cat(v), [l_x]), l_x : Ind}\n",
      "[[(11, 'rec', 1), (5, 'field', 1), (6, 'label', 1), (1, 'l_e', 1)],\n",
      " [(6, 'label', 0),\n",
      "  (1, 'l_e', 0),\n",
      "  (7, 'value', 1),\n",
      "  (12, 'lambda', 1),\n",
      "  (13, 'dom', 1),\n",
      "  (14, 'var', 1),\n",
      "  (2, 'Ind_n', 1)],\n",
      " [(13, 'dom', 0),\n",
      "  (14, 'var', 0),\n",
      "  (2, 'Ind_n', 0),\n",
      "  (15, 'rng', 1),\n",
      "  (16, 'ptype1', 1),\n",
      "  (17, 'rel', 1),\n",
      "  (3, 'dog_n', 1)],\n",
      " [(17, 'rel', 0), (3, 'dog_n', 0), (18, 'arg0', 1), (14, 'var', 1)],\n",
      " [(16, 'ptype1', 0), (18, 'arg0', 0), (14, 'var', 0)],\n",
      " [(12, 'lambda', 0), (15, 'rng', 0)],\n",
      " [(0, 'l_x', 1)],\n",
      " [(0, 'l_x', 0)],\n",
      " [(7, 'value', 0), (5, 'field', 0)],\n",
      " [(8, 'field', 1), (9, 'label', 1), (0, 'l_x', 1)],\n",
      " [(9, 'label', 0), (0, 'l_x', 0), (10, 'value', 1), (2, 'Ind_n', 1)],\n",
      " [(10, 'value', 0), (2, 'Ind_n', 0), (8, 'field', 0)],\n",
      " [(11, 'rec', 0)]]\n",
      "\n",
      "\n",
      "[[(11, 'rec', 1), (5, 'field', 1), (6, 'label', 1), (1, 'l_e', 1)],\n",
      " [(6, 'label', 0),\n",
      "  (1, 'l_e', 0),\n",
      "  (7, 'value', 1),\n",
      "  (12, 'lambda', 1),\n",
      "  (13, 'dom', 1),\n",
      "  (14, 'var', 1),\n",
      "  (2, 'Ind_n', 1)],\n",
      " [(13, 'dom', 0),\n",
      "  (14, 'var', 0),\n",
      "  (2, 'Ind_n', 0),\n",
      "  (15, 'rng', 1),\n",
      "  (16, 'ptype1', 1),\n",
      "  (17, 'rel', 1),\n",
      "  (4, 'cat_n', 1)],\n",
      " [(17, 'rel', 0), (4, 'cat_n', 0), (18, 'arg0', 1), (14, 'var', 1)],\n",
      " [(16, 'ptype1', 0), (18, 'arg0', 0), (14, 'var', 0)],\n",
      " [(12, 'lambda', 0), (15, 'rng', 0)],\n",
      " [(0, 'l_x', 1)],\n",
      " [(0, 'l_x', 0)],\n",
      " [(7, 'value', 0), (5, 'field', 0)],\n",
      " [(8, 'field', 1), (9, 'label', 1), (0, 'l_x', 1)],\n",
      " [(9, 'label', 0), (0, 'l_x', 0), (10, 'value', 1), (2, 'Ind_n', 1)],\n",
      " [(10, 'value', 0), (2, 'Ind_n', 0), (8, 'field', 0)],\n",
      " [(11, 'rec', 0)]]\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "labels.add_grandmother('l_x',N)\n",
    "labels.add_grandmother('l_e',N)\n",
    "Ind_n.add_grandmother(N)\n",
    "dog_n.add_grandmother(N)\n",
    "cat = Pred('cat',[Ind])\n",
    "cat_n = nu(cat)\n",
    "cat_n.add_grandmother(N)\n",
    "Dog = DepType('v',Ind,PType(dog,['v']))\n",
    "Cat = DepType('v',Ind,PType(cat,['v']))\n",
    "T_dog = RecType({'l_x':Ind,\n",
    "                 'l_e':(Dog,['l_x'])})\n",
    "T_cat = T_dog.subst(Dog,Cat)\n",
    "print(show(T_dog))\n",
    "print(show(T_cat))\n",
    "T_dog_n = nu(T_dog)\n",
    "T_cat_n = nu(T_cat)\n",
    "pprint(T_dog_n.show_apat(N))\n",
    "print('\\n')\n",
    "pprint(T_cat_n.show_apat(N))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(12, 'rec', 1), (6, 'field', 1), (7, 'label', 1), (1, 'l_e', 1)],\n",
      " [(7, 'label', 0),\n",
      "  (1, 'l_e', 0),\n",
      "  (8, 'value', 1),\n",
      "  (13, 'ptype1', 1),\n",
      "  (14, 'rel', 1),\n",
      "  (5, 'dog_n', 1)],\n",
      " [(14, 'rel', 0), (5, 'dog_n', 0), (15, 'arg0', 1), (2, 'a', 1)],\n",
      " [(13, 'ptype1', 0), (15, 'arg0', 0), (2, 'a', 0)],\n",
      " [(8, 'value', 0), (6, 'field', 0)],\n",
      " [(9, 'field', 1), (10, 'label', 1), (0, 'l_x', 1)],\n",
      " [(10, 'label', 0), (0, 'l_x', 0), (11, 'value', 1), (4, 'Ind_n', 1)],\n",
      " [(11, 'value', 0), (4, 'Ind_n', 0), (9, 'field', 0)],\n",
      " [(12, 'rec', 0)]]\n",
      "\n",
      "\n",
      "[[(13, 'ptype1', 1), (14, 'rel', 1), (5, 'dog_n', 1), (3, 's', 1)],\n",
      " [(14, 'rel', 0), (5, 'dog_n', 0), (15, 'arg0', 1), (2, 'a', 1)],\n",
      " [(13, 'ptype1', 0), (15, 'arg0', 0), (2, 'a', 0)],\n",
      " [(3, 's', 0)],\n",
      " [(4, 'Ind_n', 1), (2, 'a', 1)],\n",
      " [(4, 'Ind_n', 0), (2, 'a', 0)]]\n",
      "\n",
      "\n",
      "------  -  -  -  -  -  -  -\n",
      "l_x     0  0  0  0  0  0  0\n",
      "l_e     0  0  0  0  0  0  0\n",
      "a       0  0  1  0  0  1  0\n",
      "s       0  1  1  1  0  0  0\n",
      "Ind_n   0  0  0  0  0  1  0\n",
      "dog_n   0  1  0  0  0  0  0\n",
      "field   0  0  0  0  0  0  0\n",
      "label   0  0  0  0  0  0  0\n",
      "value   0  0  0  0  0  0  0\n",
      "field   0  0  0  0  0  0  0\n",
      "label   0  0  0  0  0  0  0\n",
      "value   0  0  0  0  0  0  0\n",
      "rec     0  0  0  0  0  0  0\n",
      "ptype1  0  1  1  0  0  0  0\n",
      "rel     0  1  0  0  0  0  0\n",
      "arg0    0  0  1  0  0  0  0\n",
      "------  -  -  -  -  -  -  -\n",
      "[[(12, 'rec', 1), (6, 'field', 1), (7, 'label', 1), (17, 'l_obj', 1)],\n",
      " [(7, 'label', 0),\n",
      "  (17, 'l_obj', 0),\n",
      "  (8, 'value', 1),\n",
      "  (24, 'rec', 1),\n",
      "  (18, 'field', 1),\n",
      "  (19, 'label', 1),\n",
      "  (1, 'l_e', 1)],\n",
      " [(19, 'label', 0), (1, 'l_e', 0), (20, 'value', 1), (3, 's', 1)],\n",
      " [(20, 'value', 0), (3, 's', 0), (18, 'field', 0)],\n",
      " [(21, 'field', 1), (22, 'label', 1), (0, 'l_x', 1)],\n",
      " [(22, 'label', 0), (0, 'l_x', 0), (23, 'value', 1), (2, 'a', 1)],\n",
      " [(23, 'value', 0), (2, 'a', 0), (21, 'field', 0)],\n",
      " [(24, 'rec', 0)],\n",
      " [(8, 'value', 0), (6, 'field', 0)],\n",
      " [(9, 'field', 1), (10, 'label', 1), (16, 'l_type', 1)],\n",
      " [(10, 'label', 0),\n",
      "  (16, 'l_type', 0),\n",
      "  (11, 'value', 1),\n",
      "  (24, 'rec', 1),\n",
      "  (18, 'field', 1),\n",
      "  (19, 'label', 1),\n",
      "  (1, 'l_e', 1)],\n",
      " [(19, 'label', 0),\n",
      "  (1, 'l_e', 0),\n",
      "  (20, 'value', 1),\n",
      "  (25, 'lambda', 1),\n",
      "  (26, 'dom', 1),\n",
      "  (27, 'var', 1),\n",
      "  (4, 'Ind_n', 1)],\n",
      " [(26, 'dom', 0),\n",
      "  (27, 'var', 0),\n",
      "  (4, 'Ind_n', 0),\n",
      "  (28, 'rng', 1),\n",
      "  (13, 'ptype1', 1),\n",
      "  (14, 'rel', 1),\n",
      "  (5, 'dog_n', 1)],\n",
      " [(14, 'rel', 0), (5, 'dog_n', 0), (15, 'arg0', 1), (27, 'var', 1)],\n",
      " [(13, 'ptype1', 0), (15, 'arg0', 0), (27, 'var', 0)],\n",
      " [(25, 'lambda', 0), (28, 'rng', 0)],\n",
      " [(0, 'l_x', 1)],\n",
      " [(0, 'l_x', 0)],\n",
      " [(20, 'value', 0), (18, 'field', 0)],\n",
      " [(21, 'field', 1), (22, 'label', 1), (0, 'l_x', 1)],\n",
      " [(22, 'label', 0), (0, 'l_x', 0), (23, 'value', 1), (4, 'Ind_n', 1)],\n",
      " [(23, 'value', 0), (4, 'Ind_n', 0), (21, 'field', 0)],\n",
      " [(24, 'rec', 0)],\n",
      " [(11, 'value', 0), (9, 'field', 0)],\n",
      " [(12, 'rec', 0)]]\n",
      "------  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n",
      "l_x     0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  1  0  0  1  0  0  0  0  0\n",
      "l_e     0  0  1  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "a       0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "s       0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "Ind_n   0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  1  0  0  0  0\n",
      "dog_n   0  0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "field   0  1  1  1  1  1  1  1  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "label   0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "value   0  0  1  1  1  1  1  1  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "field   0  0  0  0  0  0  0  0  0  0  1  1  1  1  1  1  1  1  1  1  1  1  1  1  0  0\n",
      "label   0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "value   0  0  0  0  0  0  0  0  0  0  0  1  1  1  1  1  1  1  1  1  1  1  1  1  0  0\n",
      "rec     0  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  0\n",
      "ptype1  0  0  0  0  0  0  0  0  0  0  0  0  0  1  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "rel     0  0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "arg0    0  0  0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "l_type  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "l_obj   0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "field   0  0  1  1  0  0  0  0  0  0  0  1  1  1  1  1  1  1  1  0  0  0  0  0  0  0\n",
      "label   0  0  1  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "value   0  0  0  1  0  0  0  0  0  0  0  0  1  1  1  1  1  1  1  0  0  0  0  0  0  0\n",
      "field   0  0  0  0  0  1  1  0  0  0  0  0  0  0  0  0  0  0  0  0  1  1  0  0  0  0\n",
      "label   0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0\n",
      "value   0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0\n",
      "rec     0  0  1  1  1  1  1  1  0  0  0  1  1  1  1  1  1  1  1  1  1  1  1  0  0  0\n",
      "lambda  0  0  0  0  0  0  0  0  0  0  0  0  1  1  1  1  0  0  0  0  0  0  0  0  0  0\n",
      "dom     0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "var     0  0  0  0  0  0  0  0  0  0  0  0  1  0  1  0  0  0  0  0  0  0  0  0  0  0\n",
      "rng     0  0  0  0  0  0  0  0  0  0  0  0  0  1  1  1  0  0  0  0  0  0  0  0  0  0\n",
      "------  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -\n"
     ]
    }
   ],
   "source": [
    "N = Network()\n",
    "labels.add_grandmother('l_x',N)\n",
    "labels.add_grandmother('l_e',N) \n",
    "iota.add_grandmother('a',N)\n",
    "iota.add_grandmother('s',N)\n",
    "Ind_n.add_grandmother(N)\n",
    "r = Rec({'l_x':'a','l_e':'s'})\n",
    "r_n = nu(r)\n",
    "dog_n.add_grandmother(N)\n",
    "Dog = DepType('v',Ind,PType(dog,['v']))\n",
    "T_dog = RecType({'l_x':Ind,\n",
    "                 'l_e':(Dog,['l_x'])})\n",
    "T_dog_n = nu(T_dog)\n",
    "pprint(T_dog_n.resolve(r_n).show_apat(N))\n",
    "print('\\n')\n",
    "j_n = T_dog_n.judgmnt_type_n(r_n)\n",
    "pprint(j_n.show_apat(N))\n",
    "print('\\n')\n",
    "\n",
    "\n",
    "\n",
    "N.ntrace()\n",
    "j_n.create_n(N)\n",
    "N.display_history()\n",
    "\n",
    "\n",
    "\n",
    "labels.add_grandmother('l_type',N)\n",
    "labels.add_grandmother('l_obj',N)\n",
    "j_n = nu(T_dog.aus_prop(r))\n",
    "pprint(j_n.show_apat(N))\n",
    "N.nontrace()\n",
    "N.ntrace()\n",
    "j_n.create_n(N)\n",
    "N.display_history()\n",
    "\n",
    "# See ausprop.pdf for an annotated version of the last example\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}