probttr.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from probttrtypes import Type, PConstraint, BType, PType, Pred,\\\n",
    "Possibility, MeetType, JoinType, RecType, Fun, DisjProb, PDiv, PTimes, PExtreme, NegType\n",
    "from ttrtypes import showmodel\n",
    "from utils import show, show_latex, ttrace, nottrace, to_latex\n",
    "from records import Rec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Judging probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The witness cache in `probttr` is a pair whose first member is a list of objects and whose second member is a list of probabilities (actually probability constraints)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "([], [])\n"
     ]
    }
   ],
   "source": [
    "T = Type()\n",
    "print(T.witness_cache)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.5'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.judge('a',.5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'([a], [0.5])'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.witness_cache)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.6'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.judge('a',.6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'([a], [0.6])'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.witness_cache)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding an additional probability argument to `judge()` gives you a minimum and maximum probability constraint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.6'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.judge('a',.6,1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\geq0.6\\&\\leq0.8\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_latex(T.judge('a',.6,.8))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\leq0.6\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_latex(T.judge('a',0,.6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`judge(a,n,n)` is the same as `judge(a,n)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}0.6\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_latex(T.judge('a',.6,.6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Probabilities must be between 0 and 1.  This can be checked by using the method `validate()` on a probability constraint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-1.0 is less than 0.\n",
      "False\n",
      "0.6 is greater than 0.1\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "print(T.judge('a',-1).validate())\n",
    "print(T.judge('a',.6,.1).validate())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In contrast to non-probabilistic `pyttr` we can store a negative judgement in witness cache."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\langle [ \\text{a}], [ 0.0]\\rangle\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T.judge('a',0)\n",
    "show_latex(T.witness_cache)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For compatibility with non-probabilistic TTR:  `judge(a)` is the same as `judge(a,1)` (which is the same as `judge(a,1,1)`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.judge('a'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Non-specific judging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also make judgements about the probability that something belongs to a type using the method `judge_nonspec`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.3&<=0.4'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T_new = Type()\n",
    "T_new.judge_nonspec(.3,.4)\n",
    "show(T_new.prob_nonspec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Querying probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying unconditional probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If `a` is in the witness cache, `query(a)` returns the probability stored in the witness cache for `a`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.query('a'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If `a` is not in the witness cache and we have no other way of computing a probability that `a` is a witness for `T`, then the probability range is `[0,1]`, i.e. $\\leq 1$.  This corresponds to returning an answer `Don't know` or `Undecided`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'<=1.0'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.query('b'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Don't know` results are not added to the witness cache.  (This may or may not be a good idea.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'([a], [1.0])'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T.witness_cache)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Witness conditions are functions which return probability constraints. Here is type `Real` for real numbers, implemented as floating point decimals where the witness condition gives a categorical judgement for any object, that is it returns probability 1 or 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def RealClassifier(n):\n",
    "    if isinstance(n,float):\n",
    "        return PConstraint(1)\n",
    "    else:\n",
    "        return PConstraint(0)\n",
    "Real = Type('Real')\n",
    "Real.learn_witness_condition(RealClassifier)\n",
    "show(Real.query(.6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.0'"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(Real.query('a'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'([0.6, a], [1.0, 0.0])'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(Real.witness_cache)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an example with a couple of witness conditions which return probability constraints.  `query(a)` returns the maximum obtained by the witness conditions for `a`.  Note that this need not be identical with either of the constraints returned by the individual witness conditions since the maximum of a collection of probability constraints is defined has as minimum the maximum of all the minima and as maximum the maximum of all the maxima.  (It could, of course, be done differently...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.8&<=0.9'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T_at = Type('T_at')\n",
    "def Classifier_a(s):\n",
    "    if 'a' in s:\n",
    "        return PConstraint(.8,.9)\n",
    "    else:\n",
    "        return PConstraint(.1,.3)\n",
    "def Classifier_t(s):\n",
    "    if 't' in s:\n",
    "        return PConstraint(.2,.95)\n",
    "    else:\n",
    "        return PConstraint(.15,.7)\n",
    "T_at.learn_witness_condition(Classifier_a)\n",
    "T_at.learn_witness_condition(Classifier_t)\n",
    "show(T_at.query('a'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.2&<=0.95'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T_at.query('t'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.8&<=0.95'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T_at.query('at'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.15&<=0.7'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T_at.query('b'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying conditional probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conditions are provided as a second argument to the `query()` method as a list each of whose members is _either_ a tuple, `(a,T)`, where `a` is an object and `T` is a type, _or_ a type.  The idea is that, for example, `T.query(a,[(b,T1),T2])` queries the probability that `a` is of type `T` given that `b` is of type `T1` and there is some witness for `T2`.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider a query `T.query(a,Conds)`.  If the probability that `a` is of type `T` does not depend on any of the conditions in `Conds` then what is returned is the same as `T.query(a)`, that is the unconditional probability. The default assumption is that probabilities are independent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.6'"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T1 = Type()\n",
    "T2 = Type()\n",
    "T1.judge('a',.6)\n",
    "show(T1.query('a',[('b',T2)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One kind of dependency between probabilities relates to subtyping in the type theory.  Suppose that $T_2\\sqsubseteq T_1$. Then $p(a:T_1|a:T_2)=1$. A limit case of this is where we have the same type judgement in the conditions as the one we are querying (types are subtypes of themselves): $p(a:T|a:T)$, that is, the probability that $a$ is of type $T$ given that $a$ is of type $T$ has to be $1$, no matter what the unconditional probability is that $a$ is of type $T$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T1.query('a',[('a',T1)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now create a type `T3` which is a subtype of `T1`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T3 = Type()\n",
    "T1.learn_witness_condition(lambda x: T3.query(x))\n",
    "T3.subtype_of(T1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the probability that something is of type `T1` given that it is of type `T3`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T1.query('b',[('b',T3)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dependent probabilities that are not related to subtyping have to be provided by an oracle which is given as a third argument to `query()`.  An oracle is a python function which takes three arguments:  an object, a type and a list of conditions of the kind  which is used as an argument to `query()`. For any such argument it returns either a probability constraint (e.g. `PConstraint(.6)`) or `None`.  As a python function the oracle may call on resources external to `pyttr`, for example, conditional probability tables or Bayesian networks.  A call `T.query(a,c,o)` where `c` does not contain `(a,TT)` where `TT` is a subtype of `T` will return `o(a,T,c)` if this is not `None`. Otherwise `T.query(a,c,o)` will return `T.query(a)` (the unconditional probability).  If `c` _does_ contain `(a,TT)` where `TT` is a subtype of `T`, then the oracle will be ignored and `T.query(a,c,o)` will return `PConstraint(1)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an example we define a trivial oracle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def SillyOracle(a,T,c):\n",
    "    if a == 'a'and T is T1 and ('b',T2) in c:\n",
    "        return PConstraint(.7,.8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the oracle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.7&<=0.8'"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T1.query('a',[('b',T2)],SillyOracle))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The oracle is not defined (returns `None`) and the result is the unconditional probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.6'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T1.query('a',[T2],SillyOracle))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The oracle is defined but is ignored because of the subtyping condition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T1.query('a',[('b',T2),('a',T3)],SillyOracle))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The probability that $a$ is to the left of $b$ is the same as the probability the $b$ is to the right of $a$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6)]\n",
      "left(a, b): [(s2, 0.7)]\n",
      "_____________________________________________\n",
      "\n",
      "0.6\n",
      "0.7\n",
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.7)]\n",
      "left(a, b): [(s2, 0.7), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n"
     ]
    }
   ],
   "source": [
    "Ind = BType('Ind')\n",
    "Ind.judge('a')\n",
    "Ind.judge('b')\n",
    "left = Pred('left',[Ind,Ind])\n",
    "right = Pred('right',[Ind,Ind])\n",
    "left.learn_witness_fun(lambda args: PType(right,[args[1],args[0]]))\n",
    "right.learn_witness_fun(lambda args: PType(left,[args[1],args[0]]))\n",
    "left_a_b = PType(left,['a','b'])\n",
    "right_b_a = PType(right,['b','a'])\n",
    "M = Possibility('M')\n",
    "right_b_a.in_poss(M).judge('s1',.6)\n",
    "left_a_b.in_poss(M).judge('s2',.7)\n",
    "print(show(M))\n",
    "print(show(left_a_b.in_poss(M).query('s1')))\n",
    "print(show(right_b_a.in_poss(M).query('s2')))\n",
    "print(show(M))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happens if we revise our original judgement?  This shows the downside of caching the result of an inference in a dynamic environment where a premise for the inference has changed since we last checked.  Below we rejudge `left(a,b)` to have probability `.3` in `s2`.  However, when we query `right(b,a)` in `s2`, we still have the old value we found with the earlier judgement for `left(a,b)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.7)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n",
      "0.7\n",
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.7)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n"
     ]
    }
   ],
   "source": [
    "left_a_b.in_poss(M).judge('s2',.3)\n",
    "print(show(M))\n",
    "print(show(right_b_a.in_poss(M).query('s2')))\n",
    "print(show(M))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is clearly the wrong result for the new circumstances.  A way around this is to `forget()` the probability for `s2` before calling `query()`.  Of course, knowing what needs to be forgotten is a delicate problem related to work on belief revision.  A good strategy is perhaps to always forget before querying when there is any possibility that something relevant might have changed.  `forget()` returns the old probability it found just in case you want to save it and reinstate it later if you are unable to compute a new probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7\n",
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n",
      "0.3\n",
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.3)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(show(right_b_a.in_poss(M).forget('s2')))\n",
    "print(show(M))\n",
    "print(show(right_b_a.in_poss(M).query('s2')))\n",
    "print(show(M))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Non-specific querying"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Non-specific queries are computed on the basis of a sample of objects which have been previously judged or queried in relation to the type (including objects which have been judged to have 0 probability of being a witness for the type).  The size of the sample is controlled by the variable `sample_size` in the file `config.py`.  A list representing a random sample of objects that have been judged for a type, `T`, is returned by `T.sample()`. If the number of objects that have been judged for `T` is less than `sample_size` then a list of all the objects which have been judged is returned.  We can also use `T.sample(n)` to return a sample of as many objects as have been judged up to `n`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[a, b]\n",
      "[b]\n"
     ]
    }
   ],
   "source": [
    "print(show(Ind.sample()))\n",
    "print(show(Ind.sample(1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The query `T.query_nonspec()` asks for the probability that there is something of type `T`, $p(T)$ in the notation of probabilistic TTR, though estimated on the basis of a random sample.  If something in the sample has been judged to be of type `T` with probability 1, then `T.query_nonspec()` returns probability 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(Ind.query_nonspec())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, if there is nothing in the sample but we have made a non-specific judgement using `judge_nonspec`, then the result of that non-specific judgement is returned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'>=0.3&<=0.4'"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(T_new.query_nonspec())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If no non-specific judgement has been made, then what is returned is the disjunctive probability of all the probabilities for objects in the sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.3)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n",
      "0.72\n",
      "0.72\n",
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.3)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(show(M))\n",
    "print(show(right_b_a.in_poss(M).query_nonspec()))\n",
    "print(show(left_a_b.in_poss(M).query_nonspec()))\n",
    "print(show(M))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If a non-specific judgement has been made and there is a non-empty sample, then what is returned is the maximum of the non-specific judgement and the disjunctive probability of the probabilities for the objects in the sample.  Suppose, for example, that I make a judgement that there is .5 probability that there is something of type `Ind`, contrary to what is represented in the sample which shows objects with probability 1.  Then the result from the sample takes precedence.  If on the other hand I have made a judgement that there is certain likelihood of something being of a certain type and this exceeds the evidence represented in the sample, then the non-specific judgement is returned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      ">=0.3&<=0.4\n"
     ]
    }
   ],
   "source": [
    "Ind.judge_nonspec(.5)\n",
    "print(show(Ind.query_nonspec()))\n",
    "T_new.judge('a',.1)\n",
    "print(show(T_new.query_nonspec()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Non-specific queries can also be conditional.  Here we show the probability that there is some situation in the type `right(b,a)` (i.e. that it is true that `b` is to the right of `a`) given that `s3` is a witness of `left(a,b)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.3)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(show(right_b_a.in_poss(M).query_nonspec([('s3',left_a_b.in_poss(M))])))\n",
    "print(show(M))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we show the probability that there is some witness for the type `right(b,a)`, given that there is some witness for the type `left(a,b)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "\n",
      "M:\n",
      "_____________________________________________\n",
      "right(b, a): [(s1, 0.6), (s2, 0.3)]\n",
      "left(a, b): [(s2, 0.3), (s1, 0.6)]\n",
      "_____________________________________________\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(show(right_b_a.in_poss(M).query_nonspec([left_a_b.in_poss(M)])))\n",
    "print(show(M))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the type in the conditions is not a subtype of the type being queried (in the case below because it refers to a different model), then what is returned is the unconditional probability, unless we provide a relevant oracle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.72\n"
     ]
    }
   ],
   "source": [
    "M1 = Possibility('M1')\n",
    "print(show(right_b_a.in_poss(M).query_nonspec([left_a_b.in_poss(M1)])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recall that oracles are Python functions with three arguments: an object/event, `a`, a type `T` and a list of conditions, `c`.  If an oracle is to give us a result for a non-specific query, for example, the probability that there will be a `Rain` event given that there is an event of `DarkClouds`, then the oracle does not need the first argument.  Rather than having oracles with different numbers of arguments and making it difficult to have a single oracle which can provide conditional probabilities for both specific and non-specific queries, we will have the first argument set to `None` in the non-specific case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.85'"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DarkClouds = Type()\n",
    "Rain = Type()\n",
    "def OracleRain(a,T,c):\n",
    "    if a is None and T is Rain and DarkClouds in c:\n",
    "        return PConstraint(.85)\n",
    "show(Rain.query_nonspec([DarkClouds],OracleRain))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Double stroke conditional probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The conditional probabilities represented above represent the probability that there is a situation of the type `right_b_a` given that there is a situation of the type `left_b_a`.  However, given the way we have characterized the relationship between `left` and `right` using the method `learn_witness_fun`, there is a stronger condition we could place on the probability:  we can talk of the probability for any situation, $s$, that $s$ is of the type `right_b_a` given that $s$ is of type `left_b_a`.  In the papers on probabilistic TTR we use $p(T_1||T_2)$ (with the double stroke) to represent the probability for any $s$ that $s:T_1$ given that $s:T_2$ and $p(T_1|T_2)$ (with a single stroke) to represent the probability that there is an $s$, $s:T_1$, given that there is an $s'$, $s':T_2$.  Note that $\\exists s\\ (s:T_1\\wedge s:T_2)$ entails $\\exists s\\ s:T_1 \\wedge \\exists\\ s' s':T_2$ and that for this reason $p(T_1||T_2)\\leq p(T_1|T_2)$ in standard probability models (though, as we will see below, `pyttr` oracles will allow us to use non-standard models in which this does not hold).\n",
    "\n",
    "In `pyttr` the double stroke condition is implemented by the method `query_doublecond` which requires a list of types as its first argument and also allows an optional second argument for an oracle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(right_b_a.in_poss(M).query_doublecond([left_a_b.in_poss(M)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us consider the difference between the two probabilities, $p(T_1|T_2)$ and $p(T_1||T_2)$.  Suppose that we have an experiment where an event of pushing a particular button leads to a reward (say, the appearance of a peanut or a piece of chocolate) 75% of the time.  That is, the probability of a reward event given a button-pushing event is .75: $p(\\text{Reward}|\\text{Button})=.75$.  However, pushing the button is not a reward in itself, that is, the probability of a button-pushing event being a reward event is 0: $p(\\text{Reward}||\\text{Button})=0$.  \n",
    "\n",
    "Let us illustrate this with a small `pyttr` implementation of the example where we interpret $p(\\text{Reward}|\\text{Button})=.75$ as \"the probability of the _next_ event after a button event being a reward event is .75\" and we take a list of events to determine which events have been observed in order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.75\n",
      "0.0\n"
     ]
    }
   ],
   "source": [
    "Button = Type()\n",
    "Reward = Type()\n",
    "events = ['e1','e2','e3','e4','e5','e6','e7']\n",
    "for e in ['e1','e3','e5','e6']:\n",
    "    Button.judge(e)\n",
    "for e in ['e2','e4','e7']:\n",
    "    Reward.judge(e)\n",
    "def OracleButtonReward(eventlist):\n",
    "    def Oracle(a,T,c):\n",
    "        if T == Reward and (a,Button) in c:\n",
    "            return PConstraint(0)\n",
    "        elif T == Button and (a,Reward) in c:\n",
    "            return PConstraint(0)\n",
    "        elif a == None and T == Reward and Button in c:\n",
    "            buttons = [e for e in events if Button.query(e).min==1]\n",
    "            buttonswithrewards = [e for e in buttons \n",
    "                                  if Reward.query(events[events.index(e)+1]).min==1]\n",
    "            return PConstraint(len(buttonswithrewards)/len(buttons))\n",
    "    return Oracle\n",
    "print(show(Reward.query_nonspec([Button],OracleButtonReward(events))))                    \n",
    "print(show(Reward.query_doublecond([Button],OracleButtonReward(events)))) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now consider an example where $p(T_1||T_2)>0$.  Suppose that half of all green apples are tart as opposed to sweet, that is $p(\\textit{TartApple}||\\textit{GreenApple})=.5$.  The probability that there will be a tart apple given that there is a green apple will now be at least $.5$, that is, $p(\\textit{TartApple}|\\textit{GreenApple})\\geq .5$. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<=1.0\n",
      "0.5\n",
      "1.0\n",
      "0.5\n"
     ]
    }
   ],
   "source": [
    "GreenApple = Type()\n",
    "TartApple = Type()\n",
    "apples = ['a1','a2','a3','a4','a5','a6','a7','a8','a9','a10']\n",
    "for a in apples[:-2]:\n",
    "    GreenApple.judge(a)\n",
    "for a in apples[8:]:\n",
    "    GreenApple.judge(a,0)\n",
    "for a in apples[0:4]:\n",
    "    TartApple.judge(a)\n",
    "def OracleApple(applelist):\n",
    "    def Oracle(a,T,c):\n",
    "        if T == TartApple and (a,GreenApple) in c:\n",
    "            greenapples = [a for a in applelist if GreenApple.query(a).min==1]\n",
    "            tartgreenapples = [a for a in greenapples if TartApple.query(a).min==1]\n",
    "            return PConstraint(len(tartgreenapples)/len(greenapples))\n",
    "        elif a == None and T == TartApple and GreenApple in c:\n",
    "            greenapples = DisjProb([(a,GreenApple) for a in applelist])\n",
    "            tartapples = DisjProb([(a,TartApple) for a in applelist])\n",
    "            return PDiv(PTimes(tartapples,greenapples),greenapples)\n",
    "    return Oracle\n",
    "print(show(TartApple.query('a11')))\n",
    "print(show(TartApple.query('a11',[('a11',GreenApple)],OracleApple(apples))))\n",
    "print(show(TartApple.query_nonspec([GreenApple],OracleApple(apples))))\n",
    "print(show(TartApple.query_doublecond([GreenApple],OracleApple(apples))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the example above the first query shows that we don't know whether `a11` is a tart apple. The second query shows that it has 50% probability of being a tart apple if it is a green apple.  The third query shows the probability according to our data that there are tart apples and green apples divided by the probability that there are green apples. (Our data show that there are both green apples and tart apples.) Finally, the fourth query shows the probability that a given apple is tart given that it is green.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the examples above we have shown examples where the method `T1.query_doublecond(T2,oracle)` returns a unique probability.  They are examples where the oracle computes the same conditional probability `T1.query(a,[(a:T2)],oracle)` for each `a` in the data.  Now let us consider an example where this is not the case.  Suppose that we have two types $\\textit{SeriouslyIll}$ and $\\textit{HaveFlu}$.  Now we can ask the question: what is the probability that somebody will be seriously ill given that they have flu?  That is, what is $p(\\textit{SeriouslyIll}||\\textit{HaveFlu})$.  The probability for any particular individual depends on another factor:  the age of the person who has flu.  In probabilistic `pyttr` we will exploit the fact that we are using probability intervals to indicate the range of probabilities in a case like this where there is not unique probability shared by all elements in the data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">=0.05&<=0.6\n",
      ">=0.05\n"
     ]
    }
   ],
   "source": [
    "HaveFlu = Type('HaveFlu')\n",
    "SeriouslyIll = Type('SeriouslyIll')\n",
    "Over80 = Type()\n",
    "Between50and79 = Type()\n",
    "Between2and49 = Type()\n",
    "Under2 = Type()\n",
    "patients = ['a1','a2','a3','a4','a5','a6','a7','a8','a9','a10']\n",
    "for a in patients:\n",
    "    HaveFlu.judge(a)\n",
    "Over80.judge('a10')\n",
    "for a in patients[5:9]:\n",
    "    Between50and79.judge(a)\n",
    "for a in patients[1:5]:\n",
    "    Between2and49.judge(a)\n",
    "Under2.judge('a1')\n",
    "def OracleRiskSeriouslyIll(a,T,c):\n",
    "    if T is SeriouslyIll and (a,HaveFlu) in c:\n",
    "        if Over80.query(a).min == 1:\n",
    "            return PConstraint(.6)\n",
    "        elif Between50and79.query(a).min == 1:\n",
    "            return PConstraint(.3)\n",
    "        elif Between2and49.query(a).min == 1:\n",
    "            return PConstraint(.05)\n",
    "        elif Under2.query(a).min == 1:\n",
    "            return PConstraint(.1)\n",
    "    elif a is None and T is SeriouslyIll and HaveFlu in c:\n",
    "        return PConstraint(T.query_doublecond(c,OracleRiskSeriouslyIll).min,1)\n",
    "print(show(SeriouslyIll.query_doublecond([HaveFlu],OracleRiskSeriouslyIll)))\n",
    "print(show(SeriouslyIll.query_nonspec([HaveFlu],OracleRiskSeriouslyIll)))\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results above show that in our example the probability that somebody will be seriously ill given that they have flu, $p(\\textit{SeriouslyIll}||\\textit{Flu})$, ranges between .05 and .6 whereas the likelihood that somebody will be seriously ill, given that somebody has flu, $p(SeriouslyIll|Flu)$ is at least .05. Note that in the definition of the oracle above, it is the last conditional clause in the definition which requires the relationship between the two probabilities.  If this clause is removed then the final result for the single stroke probability would be [0,1], that is, \"Don't know\", which is not greater than the double stroke probability and thus the oracle would define a non-standard model in the terms we introduced above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Meet types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we judge that the probability of `a` being of type `MeetType(T1,T2)` is `1`, then we also judge the probability of `a` being of `T1` and the probability of `a` being of `T2` to be `1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "Tleft = Type()\n",
    "Tright = Type()\n",
    "Tm = MeetType(Tleft,Tright)\n",
    "Tm.judge('a')\n",
    "print(show(Tleft.query('a')))\n",
    "print(show(Tright.query('a')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Otherwise, we do not currently draw any conclusions about the probabilities for the component types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<=1.0\n",
      "<=1.0\n",
      ">=0.6&<=0.8\n"
     ]
    }
   ],
   "source": [
    "Tleft1 = Type()\n",
    "Tright1 = Type()\n",
    "Tm1 = MeetType(Tleft1,Tright1)\n",
    "Tm1.judge('a',.6,.8)\n",
    "print(show(Tleft1.query('a')))\n",
    "print(show(Tright1.query('a')))\n",
    "print(show(Tm1.query('a')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The user may wish to decide that the `judge` method is not to be used with meet types, only `query`, that is, that the `judge` method is restricted to basic types.  However, making judgements about join types may be useful.  See below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar remarks hold for non-specific judgements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "1.0\n",
      "<=1.0\n",
      "<=1.0\n",
      ">=0.6&<=0.8\n"
     ]
    }
   ],
   "source": [
    "Tleft2 = Type()\n",
    "Tright2 = Type()\n",
    "Tm2 = MeetType(Tleft2,Tright2)\n",
    "Tm2.judge_nonspec()\n",
    "print(show(Tleft2.query_nonspec()))\n",
    "print(show(Tright2.query_nonspec()))\n",
    "Tleft3 = Type()\n",
    "Tright3 = Type()\n",
    "Tm3 = MeetType(Tleft3,Tright3)\n",
    "Tm3.judge_nonspec(.6,.8)\n",
    "print(show(Tleft3.query_nonspec()))\n",
    "print(show(Tright3.query_nonspec()))\n",
    "print(show(Tm3.query_nonspec()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If an object is not in the witness cache of a meet type then the conjunctive probability of the values returned for the two components is returned by the `query()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<=1.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'<=0.6'"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Tleft3.judge('a',.6)\n",
    "print(show(Tright3.query('a')))\n",
    "show(Tm3.query('a'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If an object is in the witness cache then the probability stored there will be returned, even though there may be conflicting evidence in the two components.  In order to get the new value we need to `forget()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<=0.6\n",
      "0.18\n"
     ]
    }
   ],
   "source": [
    "Tright3.judge('a',.3)\n",
    "print(show(Tm3.query('a')))\n",
    "Tm3.forget('a')\n",
    "print(show(Tm3.query('a')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The computation of conjunctive probability uses an adaptation of the Kolmogorov formula for conjunction: $p(a:T_1\\wedge T_2) = p(a:T_1)p(a:T_2\\mid a:T_1)$, as given in Cooper et al. (2015).  This involves a conditional probability and therefore in the implementation and oracle argument creating a dependence between the two types will make a difference to the outcome when querying a meet type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">=0.42&<=0.48\n",
      "0.0\n"
     ]
    }
   ],
   "source": [
    "def Oracle1(a,T,c):\n",
    "    if a == 'a'and T is Tright3 and ('a',Tleft3) in c:\n",
    "        return PConstraint(.7,.8)\n",
    "def Oracle2(a,T,c):\n",
    "    if a == 'a'and T is Tright3 and ('a',Tleft3) in c:\n",
    "        return PConstraint(0)\n",
    "Tm3.forget('a')\n",
    "print(show(Tm3.query('a',oracle=Oracle1)))\n",
    "Tm3.forget('a')\n",
    "print(show(Tm3.query('a',oracle=Oracle2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conditional probabilities can also be queried for meet types.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "Tm3.forget('a')\n",
    "print(show(Tm3.query('a',[('a',Tleft3),('a',Tright3)])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above example shows the need for witness conditions which pass the conditions and oracle arguments to components of the type. The witness condition for meet types is, schematically, \n",
    "`lambda a,c,oracle: ConjProb([(a,<left>),(a,<right>)],c,oracle)`.  (As in non-probabilistic TTR, meet types and other \"logical types\" cannot learn new witness conditions.)  Note that this witness condition has three arguments so that it can pass the conditions, `c`, and the oracle to the function `ConjProb` which computes the conjunctive probability.  Witness conditions in the `probttr` implementation can have one to three arguments and will be applied to the arguments provided to the `query` method from which they are called:  the object, `a`, being queried for one argument, `a` and the conditions, `c`, for two arguments and `a`, `c` and the oracle for three arguments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Join types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Join types work like meet types _mutatis mutandis_.  Schematically, the witness condition is `lambda a,c,oracle: DisjProb([(a,<left>),(a,<right>)],c,oracle)` and no new witness conditions can be learnt.  If we judge something to have 0 probability of being of a join type, then we judge it to have 0 probability of being of the two component types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "0.0\n"
     ]
    }
   ],
   "source": [
    "Tleftd = Type()\n",
    "Trightd = Type()\n",
    "Tmd = JoinType(Tleftd,Trightd)\n",
    "Tmd.judge('a',0)\n",
    "print(show(Tleftd.query('a')))\n",
    "print(show(Trightd.query('a')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly for non-specific judgements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "0.0\n",
      "<=1.0\n",
      "<=1.0\n",
      ">=0.6&<=0.8\n"
     ]
    }
   ],
   "source": [
    "Tleft2d = Type()\n",
    "Tright2d = Type()\n",
    "Tm2d = JoinType(Tleft2d,Tright2d)\n",
    "Tm2d.judge_nonspec(0)\n",
    "print(show(Tleft2d.query_nonspec()))\n",
    "print(show(Tright2d.query_nonspec()))\n",
    "Tleft3d = Type()\n",
    "Tright3d = Type()\n",
    "Tm3d = JoinType(Tleft3,Tright3)\n",
    "Tm3d.judge_nonspec(.6,.8)\n",
    "print(show(Tleft3d.query_nonspec()))\n",
    "print(show(Tright3d.query_nonspec()))\n",
    "print(show(Tm3d.query_nonspec()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Record types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The probability that a record, $r$, is of a record type, $T$, is the conjunctive probability of the probabilities that the objects in the fields of $r$ are of the types in the correspondingly labelled types of $T$.  If there is a label in $T$ which is not in $r$ then the probability that $r$ is of type $T$ is 0 (also if $r$ is not a record at all then the probability is 0)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A simple non-dependent record type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.06\n"
     ]
    }
   ],
   "source": [
    "Tf1 = Type()\n",
    "Tf2 = Type()\n",
    "Tr1 = RecType({'l1':Tf1,'l2':Tf2})\n",
    "Tf1.judge('a',.3)\n",
    "Tf2.judge('b',.2)\n",
    "r1 = Rec({'l1':'a','l2':'b'})\n",
    "print(show(Tr1.query(r1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A simple dependent record type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\left[\\begin{array}{lcl}\n",
       "\\text{x} &:& \\textit{Ind}\\\\\n",
       "\\text{e} &:& \\langle \\lambda v:\\textit{Ind}\\ .\\ \\text{dog}(v), \\langle \\text{x}\\rangle\\rangle\n",
       "\\end{array}\\right]\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dog = Pred('dog',[Ind])\n",
    "a_dog = RecType({'x':Ind,'e':(Fun('v', Ind, PType(dog,['v'])),['x'])})\n",
    "show_latex(a_dog)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.7'"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Ind.judge('d')\n",
    "PType(dog,['d']).judge('s1',.7)\n",
    "r2 = Rec({'x':'d','e':'s1','z':'other_stuff'})\n",
    "show(a_dog.query(r2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A simple record type with a path leading to another record type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\left[\\begin{array}{lcl}\n",
       "\\text{x} &:& \\left[\\begin{array}{lcl}\n",
       "\\text{x} &:& \\textit{Ind}\\\\\n",
       "\\text{e} &:& \\langle \\lambda v:\\textit{Ind}\\ .\\ \\text{dog}(v), \\langle \\text{x}\\rangle\\rangle\n",
       "\\end{array}\\right]\n",
       "\\end{array}\\right]\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Tr2 = RecType({'x':a_dog})\n",
    "show_latex(Tr2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.7'"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r3 = Rec({'x':r2})\n",
    "show(Tr2.query(r3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A slightly more complex record type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\left[\\begin{array}{lcl}\n",
       "\\text{x} &:& \\left[\\begin{array}{lcl}\n",
       "\\text{x} &:& \\textit{Ind}\\\\\n",
       "\\text{e} &:& \\langle \\lambda v:\\textit{Ind}\\ .\\ \\text{dog}(v), \\langle \\text{x}\\rangle\\rangle\n",
       "\\end{array}\\right]\\\\\n",
       "\\text{e} &:& \\langle \\lambda v:\\textit{Ind}\\ .\\ \\text{bark}(v), \\langle \\text{x.x}\\rangle\\rangle\n",
       "\\end{array}\\right]\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bark = Pred('bark',[Ind])\n",
    "a_dog_bark = RecType({'x':a_dog,'e':(Fun('v',Ind,PType(bark,['v'])),['x.x'])})\n",
    "show_latex(a_dog_bark)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\left[\\begin{array}{rcl}\n",
       "\\text{x} &=& \\left[\\begin{array}{rcl}\n",
       "\\text{x} &=& \\text{d}\\\\\n",
       "\\text{e} &=& \\text{s}_{\\text{1}}\\\\\n",
       "\\text{z} &=& \\text{other}_{\\text{stuff}}\n",
       "\\end{array}\\right]\\\\\n",
       "\\text{e} &=& \\text{s}_{\\text{2}}\n",
       "\\end{array}\\right]\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PType(bark,['d']).judge('s2',.3)\n",
    "r4 = Rec({'x':r2,'e':'s2'})\n",
    "show_latex(r4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.21'"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(a_dog_bark.query(r4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conditional probabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.7'"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(a_dog_bark.query(r4,[('s2',PType(bark,['d']))]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0'"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(a_dog_bark.query(r4,[('s1',PType(dog,['d'])),('s2',PType(bark,['d']))]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Non-specific queries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Samples for record types are based not on records that have previously been judge for that type but on judgements that have been made for the types within the record type.  Suppose, for example, that we create a new record type which we have not previously judged but which contains types for which there have been judgements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "\\begin{equation}\\left[\\begin{array}{lcl}\n",
       "\\text{x} &:& \\textit{Ind}\\\\\n",
       "\\text{c} &:& \\langle \\lambda v:\\textit{Ind}\\ .\\ \\text{dog}(v), \\langle \\text{x}\\rangle\\rangle\\\\\n",
       "\\text{e} &:& \\langle \\lambda v:\\textit{Ind}\\ .\\ \\text{bark}(v), \\langle \\text{x}\\rangle\\rangle\n",
       "\\end{array}\\right]\\end{equation}"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adb = RecType({'x':Ind,'c':(Fun('v', Ind, PType(dog,['v'])),['x']),'e':(Fun('v', Ind, PType(bark,['v'])),['x'])})\n",
    "show_latex(adb)               "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We should be able to sample witnesses for this type given that we have a witness for individuals which are dogs and bark, even though we have not judged or queried this particular type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n"
     ]
    }
   ],
   "source": [
    "print(show(adb.sample()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This means that we are able to compute the probability that there is a record of the type `adb` by computing the disjunctive probability of the elements in the sample belonging to the type in the usual way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'<=1.0'"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(adb.query_nonspec())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given that the record types `adb` and `a_dog_bark` contain the same basic information, although structured differently the probability that they have a witness will be the same (or approximately the same given that the random samples chosen for the two types might be different in larger examples) regardless of the history of judgements that have been made for those particular record types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'<=1.0'"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(a_dog_bark.query_nonspec())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Negation of types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The negation of a type is treated as an object of a type class.  `NegType(T)` creates a type which is the negation of `T`.  Probabilities associated with the negation of a type, `T` are computed in terms of `1-p` where `p` is the probability associated with `T`.  For example, `NegType(T).query(a)` has the value computed by: \n",
    "\n",
    ">`PConstraint(1-T.query(a).max,1-T.query(a).min)`\n",
    "\n",
    "Below we show an example using the `query_nonspec` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'<=1.0'"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show(NegType(a_dog_bark).query_nonspec())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}