ML0120EN-Eager_Execution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0120EN-SkillsNetwork/images/IDSN-logo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\">\n",
    "\n",
    "# Eager Execution in TensorFlow 2.X\n",
    "\n",
    "Estimated time needed: **15** minutes\n",
    "\n",
    "## Objectives\n",
    "\n",
    "After completing this lab you will be able to:\n",
    "\n",
    "- Understand the impact of eager execution and the need to enable it\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "<font size = 3><strong>In this notebook we will overview Eager Execution in  TensorFlow 2.x</strong></font>\n",
    "<br>\n",
    "<h2>Table of Contents</h2>\n",
    "<ol>\n",
    "    <li>Instructions</li>\n",
    "    <li>Eager Execution</li>\n",
    "    <li>Tensorflow Operations Without Eager Execution Mode</li>\n",
    "    <li>Tensorflow Operations With Eager Execution Mode</li>\n",
    "    <li>Dynamic Control Flow</li>\n",
    "    <li>Thank You</li>\n",
    "</ol>\n",
    "<p></p>\n",
    "</div>\n",
    "<br>\n",
    "\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Instructions\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing TensorFlow\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We begin by installing TensorFlow version 2.9.0 and its required prerequistes.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install grpcio==1.24.3\n",
    "!pip install tensorflow==2.9.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <b>Notice:</b> This notebook has been created with TensorFlow version 2.9.0, and might not work with other versions. Therefore we check:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "if not tf.__version__ == '2.9.0':\n",
    "    print(tf.__version__)\n",
    "    raise ValueError('please upgrade to TensorFlow 2.9.0, or restart your Kernel (Kernel->Restart & Clear Output)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Eager Execution \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TensorFlow's **eager execution** is an imperative programming environment that evaluates operations immediately, without building graphs, operations return concrete values instead of constructing a computational graph to run later. This makes it easy to get started with TensorFlow and debug models.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With **TensorFlow 2.x**, **Eager Execution is enabled by default**. This allows TensorFlow code to be executed and evaluated line by line. Before version 2.x was released, every graph had to be run wihthin a TensorFlow **session**. This only allowed for the entire graph to be run all at once. This made it hard to debug the computation graph. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Eager execution is a flexible machine learning platform for research and experimentation, providing:\n",
    "\n",
    "- **An intuitive interface**-Structure your code naturally and use Python data structures. Quickly iterate on small models and   small data.\n",
    "\n",
    "\n",
    "- **Easier debugging**- Execute operations directly to inspect code line by line and test changes. Use standard Python debugging tools for immediate error reporting.\n",
    "\n",
    "\n",
    "- **Natural control flow**—Use Python control flow instead of graph control flow, simplifying the specification of dynamic models.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As I mentioned above, in **Tensorflow 2.x**, eager execution is enabled by default. To verify that please run the below code.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.executing_eagerly()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can run TensorFlow operations and the results will return immediately:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But first let me show you how things get done without the eager execution in tensorflow.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensorflow Operations Without Eager Execution Mode\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, there is a **disable_eager_execution()** function in TensorFlow 2.x. You can call the function like this:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.python.framework.ops import disable_eager_execution\n",
    "disable_eager_execution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Note: This function can only be called at the beginning before any Graphs, Ops, or Tensors have been  created.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, verify that the eager execution is disabled or not by running the below code.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.executing_eagerly()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see **False** in the output that means it is disabled now.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just execute the next cell. You will notice that we've created an object **a** of type **tensorflow.python.framework.ops.Tensor**.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "a = tf.constant(np.array([1., 2., 3.]))\n",
    "type(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create another Tensor **b** and apply the dot product between them. This gives us **c**:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b = tf.constant(np.array([4.,5.,6.]))\n",
    "c = tf.tensordot(a, b, 1)\n",
    "type(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that **c** is a **tensorflow.python.framework.ops.Tensor** as well. So any node of the execution graph resembles a Tensor type. **But so far not a single computation happened**. You need to execute the graph. You can pass any graph or subgraph to the TensorFlow runtime for execution. Each TensorFlow graph runs within a TensorFlow Session, therefore we need to create it first:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** Session can be accessed via **tf.compat.v1.Session()** in Tensorflow 2.x.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = tf.compat.v1.Session()\n",
    "output = session.run(c)\n",
    "session.close()\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you see the correct result of 32. But the problem is that debugging is pretty hard if you can only run complete graphs.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So let's actually re-enable the **eager execution**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensorflow Operations With Eager Execution Mode\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IMPORTANT! => Please don't forget restart the kernel by clicking on \"Kernel\" -> \"Restart\" so that the changes take effect.\n",
    "\n",
    "**Enable or Disable Eager execution has to happen on program startup. This is the reason we have to restart the kernel.**\n",
    "\n",
    "**After you have restarted the kernel, you may continue on to the next step.** \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the required libraries again.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the below command to re-enable the eager execution.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.python.framework.ops import enable_eager_execution\n",
    "enable_eager_execution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can run TensorFlow operations and the results will return immediately:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [[4]]\n",
    "m = tf.matmul(x, x)\n",
    "print(\"Result, {}\".format(m))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Enabling eager execution changes how TensorFlow operations behave—now they immediately evaluate and return their values to Python.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since there isn't a computational graph to build and run later in a session, it's easy to inspect results using print() or a debugger.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = tf.constant(np.array([1., 2., 3.]))\n",
    "type(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the very same code created a different type of object. So now **a** is of type **tensorflow.python.framework.ops.EagerTensor**. This is great, because without changing code we obtain a tensor object which allows us to have a look inside, without execting a graph in a session:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(a.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Isn't this amazing? So from now on we can treat Tensors like ordinary python objects, work with them as usual, insert debug statements at any point or even use a debugger. So let's continue this example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b = tf.constant(np.array([4.,5.,6.]))\n",
    "c = tf.tensordot(a, b,1)\n",
    "type(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, **c** is an **tensorflow.python.framework.ops.EagerTensor** object which can be directly read:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(c.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Without creating a session or a graph we obtained the result of the defined computation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dynamic Control Flow\n",
    "\n",
    "A major benefit of eager execution is that all the functionality of the host language is available while your model is executing. So, for example, it is easy to write [fizzbuzz](https://en.wikipedia.org/wiki/Fizz_buzz?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDL0120ENSkillsNetwork954-2023-01-01):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fizzbuzz(max_num):\n",
    "  counter = tf.constant(0)\n",
    "  max_num = tf.convert_to_tensor(max_num)\n",
    "  for num in range(1, max_num.numpy()+1):\n",
    "    num = tf.constant(num)\n",
    "    if int(num % 3) == 0 and int(num % 5) == 0:\n",
    "      print('FizzBuzz')\n",
    "    elif int(num % 3) == 0:\n",
    "      print('Fizz')\n",
    "    elif int(num % 5) == 0:\n",
    "      print('Buzz')\n",
    "    else:\n",
    "      print(num.numpy())\n",
    "    counter += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fizzbuzz(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It prints these values at runtime. It behaves just like any other Python code. It is direct and intuitive. We can use pure Python if, while, and for in the control flow. So that's it for now, stay tuned and have fun.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Thank you for completing this notebook</h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Author\n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/shubham-kumar-yadav-14378768?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDL0120ENSkillsNetwork954-2023-01-01\">Shubham Yadav</a>\n",
    "\n",
    "## Change Log\n",
    "\n",
    "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                 |\n",
    "| ----------------- | ------- | ---------- | ---------------------------------- |\n",
    "| 2020-09-04       | 1.0     | Lavanya    | Added lab to demonstrate Tensorflow eager execution |\n",
    "\n",
    "<hr>\n",
    "\n",
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}