workshops/ww_summer_22/binder/01-data_mocking.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "12baa696",
   "metadata": {},
   "source": [
    "# Hero Data Mocking Workshop\n",
    "## Task Description\n",
    "In this workshop scenario we would like to synthesize two datasets as per the agreed requirements with the team.\n",
    "\n",
    "#### Scenario 1: NYC Crisis Dataset\n",
    "Internally the DS and the DL teams have aligned on a set of different datasets that will be transformed and managed by them. They both agreed that for training the forecaster they will need a dataste of daily occurings/ count of disasters each day in NYC, for the last several days. We will need to synthesize:\n",
    "- Date\n",
    "- Frequency of Events\n",
    "\n",
    "#### Scenario 2: Superhero Timecards\n",
    "Through several rounds of to & fro, the team has finally settled on some data fields that they would like to collect from the prospective user i.e. our superheroes. We've finally settled on collecting:\n",
    "- Full Name\n",
    "- Date\n",
    "- Active-on-duty (a boolean field indicating if the superhero is on duty)\n",
    "\n",
    "## Exercises\n",
    "We will generate all our data samples using pythonic code and a few easy to use packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "21d78fd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import random"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7c02aad0",
   "metadata": {},
   "source": [
    "#### Let's define some arrays with random data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "680a2442",
   "metadata": {},
   "outputs": [],
   "source": [
    "names = [\n",
    "    \"Spiderman\",\n",
    "    \"Daredevil\",\n",
    "    \"Wolverine\",\n",
    "    \"Deadpool\",\n",
    "    \"Luke Cage\",\n",
    "    \"Mr Fantastic\",\n",
    "    \"Dr X\",\n",
    "    \"The Comedian\",\n",
    "    \"Peeping Tom\"\n",
    "] # static list of superhero names\n",
    "\n",
    "availability = [0, 1] # 1's indicate availability\n",
    "\n",
    "events_30_days = pd.date_range(\n",
    "    pd.Timestamp.now().date() - pd.Timedelta(days=30),\n",
    "    pd.Timestamp.now().date()\n",
    ")\n",
    "\n",
    "events_3_days = pd.date_range(\n",
    "    pd.Timestamp.now().date() - pd.Timedelta(days=2),\n",
    "    pd.Timestamp.now().date()\n",
    ")\n",
    "\n",
    "crisis = [0, 1, 3, 5] # distribution of crisis per day"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3f1aed79",
   "metadata": {},
   "source": [
    "#### Creating synthetic data via repeated random sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cadafdc4",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_df = pd.DataFrame({\n",
    "    \"date\": random.choice(events_30_days),\n",
    "    \"crisis\": random.choice(crisis),\n",
    "} for i in range(100))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "747a5657",
   "metadata": {},
   "source": [
    "#### Create more realistic data!\n",
    "Applying multiple levels of transformations and randomization is a good practice in synthesizing realistic samples of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d93795f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>crisis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-05-01</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022-05-02</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022-05-03</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022-05-04</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022-05-05</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2022-05-06</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2022-05-07</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2022-05-08</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2022-05-09</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2022-05-10</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>2022-05-11</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2022-05-12</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2022-05-13</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>2022-05-16</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>2022-05-17</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>2022-05-18</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>2022-05-19</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>2022-05-20</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2022-05-21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>2022-05-22</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>2022-05-23</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>2022-05-24</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>2022-05-25</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>2022-05-26</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>2023-07-11</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>2023-07-12</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>2023-07-13</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>2022-05-27</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>2022-05-28</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>2023-07-18</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date  crisis\n",
       "0  2022-05-01      10\n",
       "1  2022-05-02       6\n",
       "2  2022-05-03       4\n",
       "3  2022-05-04       4\n",
       "4  2022-05-05       6\n",
       "5  2022-05-06       9\n",
       "6  2022-05-07       4\n",
       "7  2022-05-08       6\n",
       "8  2022-05-09      16\n",
       "9  2022-05-10      22\n",
       "10 2022-05-11      18\n",
       "11 2022-05-12       5\n",
       "12 2022-05-13       8\n",
       "13 2022-05-15       6\n",
       "14 2022-05-16       8\n",
       "15 2022-05-17       0\n",
       "16 2022-05-18      14\n",
       "17 2022-05-19       3\n",
       "18 2022-05-20      21\n",
       "19 2022-05-21       0\n",
       "20 2022-05-22      10\n",
       "21 2022-05-23       0\n",
       "22 2022-05-24      13\n",
       "23 2022-05-25       7\n",
       "24 2022-05-26       4\n",
       "25 2022-05-27       3\n",
       "26 2022-05-28       8\n",
       "27 2022-05-29      11\n",
       "28 2022-05-30      10\n",
       "29 2022-05-31       3"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = raw_df.groupby(\"date\").agg(sum).reset_index().sort_values(\"date\")\n",
    "df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "44a89fc9",
   "metadata": {},
   "source": [
    "#### Saving our dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "d5faa24c",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"../data/nyc_crisis_may.csv\", index=False)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1f472a54",
   "metadata": {},
   "source": [
    "### Let's create another dataframe with more complex data!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5a1eb542",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>superhero</th>\n",
       "      <th>date</th>\n",
       "      <th>active_on_duty</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Deadpool</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Dr X</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Wolverine</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Wolverine</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Daredevil</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Deadpool</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Peeping Tom</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Deadpool</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Wolverine</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Spiderman</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Wolverine</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Dr X</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Daredevil</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Peeping Tom</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Dr X</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      superhero       date  active_on_duty\n",
       "0      Deadpool 2022-05-29               0\n",
       "1          Dr X 2022-05-30               0\n",
       "2     Wolverine 2022-05-31               1\n",
       "3     Wolverine 2022-05-29               1\n",
       "4     Daredevil 2022-05-31               1\n",
       "5      Deadpool 2022-05-31               1\n",
       "6   Peeping Tom 2022-05-31               0\n",
       "7     Luke Cage 2022-05-30               0\n",
       "8      Deadpool 2022-05-31               0\n",
       "9     Luke Cage 2022-05-30               0\n",
       "10    Wolverine 2022-05-29               0\n",
       "11    Spiderman 2022-05-30               0\n",
       "12    Wolverine 2022-05-30               1\n",
       "13         Dr X 2022-05-29               0\n",
       "14    Daredevil 2022-05-31               1\n",
       "15  Peeping Tom 2022-05-30               0\n",
       "16    Luke Cage 2022-05-29               0\n",
       "17    Luke Cage 2022-05-29               1\n",
       "18    Luke Cage 2022-05-31               0\n",
       "19         Dr X 2022-05-31               1"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_df = pd.DataFrame({\n",
    "    \"superhero\": random.choice(names),\n",
    "    \"date\": random.choice(events_3_days),\n",
    "    \"active_on_duty\": random.choice(availability),\n",
    "} for i in range(20))\n",
    "\n",
    "raw_df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "34f4a489",
   "metadata": {},
   "source": [
    "#### Once again applying some aggregations to smooth the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c458f55d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>superhero</th>\n",
       "      <th>date</th>\n",
       "      <th>active_on_duty</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Daredevil</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Deadpool</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Deadpool</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Dr X</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Dr X</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Dr X</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2023-07-18</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Luke Cage</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Peeping Tom</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Peeping Tom</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Spiderman</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Wolverine</td>\n",
       "      <td>2022-05-29</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Wolverine</td>\n",
       "      <td>2022-05-30</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Wolverine</td>\n",
       "      <td>2022-05-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>The Comedian</td>\n",
       "      <td>2023-07-18</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      superhero       date  active_on_duty\n",
       "0     Daredevil 2022-05-31               1\n",
       "1      Deadpool 2022-05-29               0\n",
       "2      Deadpool 2022-05-31               1\n",
       "3          Dr X 2022-05-29               0\n",
       "4          Dr X 2022-05-30               0\n",
       "5          Dr X 2022-05-31               1\n",
       "6     Luke Cage 2022-05-29               1\n",
       "7     Luke Cage 2022-05-30               0\n",
       "8     Luke Cage 2022-05-31               0\n",
       "9   Peeping Tom 2022-05-30               0\n",
       "10  Peeping Tom 2022-05-31               0\n",
       "11    Spiderman 2022-05-30               0\n",
       "12    Wolverine 2022-05-29               1\n",
       "13    Wolverine 2022-05-30               1\n",
       "14    Wolverine 2022-05-31               1"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = raw_df.groupby([\"superhero\", \"date\"]).max().reset_index()\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "05428946",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"../data/superhero_timesheets.csv\", index=False)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9b01c12c",
   "metadata": {},
   "source": [
    "## In this section\n",
    "- We fabricated some synthetic datasets to demonstrate how to use programming and randomization libraries to create datasets when none exists.\n",
    "- In the next section, we will attempt to use these datasets towards a simple predictive task.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".env",
   "language": "python",
   "name": ".env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}