travel_package_purchase_prediction.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "private_outputs": true,
      "provenance": [],
      "cell_execution_strategy": "setup",
      "authorship_tag": "ABX9TyOO1r0CuClCJZ2MiPpJ3IIA",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Asrarullah7/Project/blob/main/travel_package_purchase_prediction.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Travel Package Purchase Prediction\n",
        "# Description\n",
        "**Background and Context **\n",
        "You are a Data Scientist for a tourism company named \"Visit with us\". The Policy Maker of the company wants to enable and establish a viable business model to expand the customer base.\n",
        "\n",
        "A viable business model is a central concept that helps you to understand the existing ways of doing the business and how to change the ways for the benefit of the tourism sector.\n",
        "\n",
        "One of the ways to expand the customer base is to introduce a new offering of packages.\n",
        "\n",
        "Currently, there are 5 types of packages the company is offering - Basic, Standard, Deluxe, Super Deluxe, King. Looking at the data of the last year, we observed that 18% of the customers purchased the packages.\n",
        "\n",
        "However, it was difficult to identify the potential customers because customers were contacted at random without looking at the available information.\n",
        "\n",
        "The company is now planning to launch a new product i.e. Wellness Tourism Package. Wellness Tourism is defined as Travel that allows the traveler to maintain, enhance or kick-start a healthy lifestyle, and support or increase one's sense of well-being.\n",
        "\n",
        "This time company wants to harness the available data of existing and potential customers to target the right customers.\n",
        "\n",
        "You as a Data Scientist at \"Visit with us\" travel company has to analyze the customers' data and information to provide recommendations to the Policy Maker and build a model to predict the potential customer who is going to purchase the newly introduced travel package. The model will be built to make predictions before a customer is contacted."
      ],
      "metadata": {
        "id": "9-UX9ervKNmN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Objective\n",
        "To predict which customer is more likely to purchase the newly introduced travel package.\n",
        "\n",
        "Data Dictionary\n",
        "Customer details:\n",
        "* CustomerID: Unique customer ID\n",
        "* ProdTaken: Whether the customer has purchased a package or not (0: No, 1: Yes)\n",
        "* Age: Age of customer\n",
        "* TypeofContact: How customer was contacted (Company Invited or Self Inquiry)\n",
        "* CityTier: City tier depends on the development of a city, population, facilities, and living standards.\n",
        "\n",
        "The categories are ordered i.e. Tier 1 > Tier 2 > Tier 3. It's the city the customer lives in. 6. Occupation: Occupation of customer 7. Gender: Gender of customer 8. NumberOfPersonVisiting: Total number of persons planning to take the trip with the customer 9. PreferredPropertyStar: Preferred hotel property rating by customer 10. MaritalStatus: Marital status of customer 11. NumberOfTrips: Average number of trips in a year by customer 12. Passport: The customer has a passport or not (0: No, 1: Yes) 13. OwnCar: Whether the customers own a car or not (0: No, 1: Yes) 14. NumberOfChildrenVisiting: Total number of children with age less than 5 planning to take the trip with the customer 15. Designation: Designation of the customer in the current organization 16. MonthlyIncome: Gross monthly income of the customer 17. PitchSatisfactionScore: Sales pitch satisfaction score 18. ProductPitched: Product pitched by the salesperson 19. NumberOfFollowups: Total number of follow-ups has been done by the salesperson after the sales pitch 20. DurationOfPitch: Duration of the pitch by a salesperson to the customer"
      ],
      "metadata": {
        "id": "56iqWiyTLJtd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Import Libraries"
      ],
      "metadata": {
        "id": "eArHDsVpLpXu"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YEgzZErzKNBN"
      },
      "outputs": [],
      "source": [
        "# libraries to ignore warnings\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')\n",
        "\n",
        "# libraries to read and manipulate the data\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "# libraries used for data visualization\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "# to enable plotting graphs in Jupyter notebook\n",
        "%matplotlib inline\n",
        "\n",
        "# library to split the data into train and test set\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# library to impute missing values\n",
        "from sklearn.impute import SimpleImputer\n",
        "\n",
        "# Libraries to import decision tree classifier and different ensemble classifiers\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.ensemble import BaggingClassifier\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.ensemble import AdaBoostClassifier, GradientBoostingClassifier\n",
        "from xgboost import XGBClassifier\n",
        "from sklearn.ensemble import StackingClassifier\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.tree import DecisionTreeRegressor\n",
        "from sklearn.ensemble import BaggingRegressor,RandomForestRegressor, GradientBoostingRegressor, AdaBoostRegressor, StackingRegressor\n",
        "from xgboost import XGBRegressor\n",
        "from sklearn import tree\n",
        "import scipy.stats as stats\n",
        "\n",
        "# libraries for model evaluation and metrics\n",
        "from sklearn import metrics\n",
        "from sklearn.metrics import confusion_matrix, classification_report\n",
        "from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score\n",
        "\n",
        "# library used for hyper parameter tuning\n",
        "from sklearn.model_selection import GridSearchCV"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# EDA\n",
        "# Read the data set"
      ],
      "metadata": {
        "id": "JcqIlK6-L5fH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# read the excel file and using Tourism sheet as our data\n",
        "url = 'Tourism.xlsx'\n",
        "data = pd.read_excel(url, sheet_name='Tourism')\n",
        "\n",
        "# copying data to another varaible to avoid any changes to original data\n",
        "travel = data.copy()\n",
        "print(f\"There are {travel.shape[0]} rows and {travel.shape[1]} columns.\")"
      ],
      "metadata": {
        "id": "ti5x2jRaL9WQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# First and last 5 rows of the dataset"
      ],
      "metadata": {
        "id": "rkMfAOstMKvX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel.head()"
      ],
      "metadata": {
        "id": "mDtsg-c4MM7n"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel.tail()"
      ],
      "metadata": {
        "id": "73rlX87dMP_B"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's view a sample of the data\n",
        "travel.sample(n=10, random_state=1)"
      ],
      "metadata": {
        "id": "vg2p6G-uMXrh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Check for duplicates in the data set"
      ],
      "metadata": {
        "id": "5E1msunvMPug"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel[travel.duplicated()].count()"
      ],
      "metadata": {
        "id": "ySpFB8kUMcN4"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "*   There are no duplicate entries.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "wpPDFBJyMkf4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Check the data types of the columns for the dataset"
      ],
      "metadata": {
        "id": "nJEMiFcMMs9L"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()"
      ],
      "metadata": {
        "id": "lmrORWh5MfUI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Dataset has 20 columns, there are 14 numerical columns and 6 object columns.\n",
        "* We can convert object columns into categorical columns, converting \"objects\" to \"category\" reduces the data space required to store the dataframe.\n",
        "* Designation', 'ProdTaken', 'OwnCar', 'Passport', 'CityTier', 'MaritalStatus', * ProductPitched', 'Gender', 'Occupation', 'TypeofContact' columns are categorical columns.\n",
        "* Dataset seems to have missing values in few variables."
      ],
      "metadata": {
        "id": "h7TQq6wpMz_q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Fixing the data types"
      ],
      "metadata": {
        "id": "Wz1PQHAnMfFj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "## Converting the data type of categorical features to 'category'\n",
        "\n",
        "cat_cols = ['Designation','ProdTaken', 'OwnCar', 'Passport', 'CityTier', 'MaritalStatus', 'ProductPitched', 'Gender', 'Occupation']"
      ],
      "metadata": {
        "id": "SpxtnafwNDxC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for i in cat_cols:\n",
        "    travel[i] = travel[i].astype('category')"
      ],
      "metadata": {
        "id": "rb3r4qnmNgPu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()"
      ],
      "metadata": {
        "id": "tmPVO0ylNj4M"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Check the unique values of the columns**"
      ],
      "metadata": {
        "id": "ItifogYMNpIE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# check for number of unique values in each column\n",
        "travel.nunique().sort_values(ascending=False)"
      ],
      "metadata": {
        "id": "XW-eEEadNo4M"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Dropping CustomerID column**"
      ],
      "metadata": {
        "id": "r95uXQHyNx5E"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Dropping columns which are not adding any information.\n",
        "travel.drop('CustomerID',axis=1,inplace=True)\n",
        "travel.shape\n"
      ],
      "metadata": {
        "id": "fbbNVWuLNrG-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's check the value counts of features which has less unique values\n",
        "for col in travel.columns:\n",
        "    if travel[col].nunique() <= 6:\n",
        "        print('***********************', col, '*******************')\n",
        "        print(travel[col].value_counts(normalize=True)*100)\n",
        "        print('-'*50)"
      ],
      "metadata": {
        "id": "qvcIZtzSN1aP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* ProdTaken : This is our Target variable. Only 18% of the customers took the product.\n",
        "* TypeofContact : 70% of the customers contacted by self and only 30% of the customers are contacted by company invite.\n",
        "* CityTier : 65% of the customers are from Tire 1, 30% of the customers are from Tire 3 and only 4% of the customers are from Tire 2.\n",
        "* Occupation : Most of the customer are Salaried(48%) and have small business(42%).\n",
        "* Gender : 60% of the customers are Male. Few Female values are wrongly entered as Fe Male. It should be corrected.\n",
        "NumberOfPersonVisiting : Nearly 80% of the customer are travelling with 3 persons.\n",
        "* NumberOfFollowups : more than 80% of the customers had more than 3 follow ups.\n",
        "* ProductPitched : Basic and Deluxe are the popular products among the 5 products. 70% of the customers uses either Basic or Deluxe package.\n",
        "* PreferredPropertyStar : 61% of the customers prefers 3 star hotel.\n",
        "* MaritalStatus : Most of the customer are Married.\n",
        "* Passport : Most of the customer do not have passport. Only 30% of the customers has passport.\n",
        "* PitchSatisfactionScore : Nearly 70% of the customer rated more than or equal to 3.\n",
        "* OwnCar : 62% of the customers has owncar.\n",
        "* NumberOfChildrenVisiting : 70% of the customers travelled with 1 or 2 children under 5 years. 22% of the customers don't have children under 5 years.\n",
        "* Designation : Most of the customers are Executives or Managers.\n",
        "\n"
      ],
      "metadata": {
        "id": "ltrNSFngN9LU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Gender column correction**\n",
        "* Replacing 'Fe Male' with 'Female'"
      ],
      "metadata": {
        "id": "w9hQ_MdMOcZd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel['Gender'].replace({'Fe Male' : 'Female'}, inplace=True)"
      ],
      "metadata": {
        "id": "AElYs_BKOwCe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel['Gender'].value_counts()"
      ],
      "metadata": {
        "id": "eDHJ1dLfOycm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Check for the missing values:"
      ],
      "metadata": {
        "id": "p-SFJ6e2O11-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel.isnull().sum().sum()"
      ],
      "metadata": {
        "id": "5mKnYKGnO2se"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel.isnull().sum().sort_values(ascending=False)"
      ],
      "metadata": {
        "id": "4J50ye-SPttZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Calculating the percentage of missing values in the dataset\n",
        "travel.isnull().sum().sort_values(ascending=False)/travel.shape[0] * 100"
      ],
      "metadata": {
        "id": "SxAD7Ye7Pxcy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Out of 19 columns, 8 columns has missing values.\n",
        "* DurationOfPitch, MonthlyIncome, Age, NumberOfTrips, NumberOfChildrenVisiting, * NumberOfFollowups, PreferredPropertyStar, TypeofContact columns has missing values.\n",
        "* DurationOfPitch has 5% of missing data, may be some persons duration is not recorded.\n",
        "* MonthlyIncome has 4.7% of missing data.\n",
        "* Age has 4.6% of missing data.\n",
        "* NumberOfTrips, NumberOfChildrenVisiting, NumberOfChildrenVisiting,\n",
        "* NumberOfFollowups, PreferredPropertyStar, TypeofContact has less than 3% of missing data."
      ],
      "metadata": {
        "id": "EgE7KapgP2MJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Missing numeric values can be filled with median (real) values.**"
      ],
      "metadata": {
        "id": "zi9TVnSLQKtK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# we will replace missing values in these columns with its median\n",
        "medianFiller = lambda x: x.fillna(x.median())\n",
        "NaNcolumns = ['DurationOfPitch','MonthlyIncome','Age','NumberOfTrips','NumberOfChildrenVisiting','NumberOfFollowups','PreferredPropertyStar']\n",
        "print (NaNcolumns)"
      ],
      "metadata": {
        "id": "ZP2W6xZRP1v8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# apply the lamda function on NaNcolumns for missing values\n",
        "travel[NaNcolumns] = travel[NaNcolumns].apply(medianFiller,axis=0)"
      ],
      "metadata": {
        "id": "Y6xifwT3QPUi"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# looking at which columns have missing values\n",
        "travel.isnull().sum().sort_values(ascending=False)"
      ],
      "metadata": {
        "id": "erjN8GD0QT0m"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel.TypeofContact.unique()"
      ],
      "metadata": {
        "id": "K2lDNYDvQYBr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Remove missing values for TypeofContact as it's values are of either 'Self Enquiry' or 'Company Invited' and no median possibility.\n",
        "* TypeofContact missing values are only 25, ~ 0.5% of rows so impact is negligible."
      ],
      "metadata": {
        "id": "3SLTb9-UQbIE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Remove the rows of data which have missing value(s)\n",
        "travel.dropna(inplace=True)"
      ],
      "metadata": {
        "id": "KsqnSTUGQha1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# check if there are any missing values\n",
        "travel.isnull().sum().sum()"
      ],
      "metadata": {
        "id": "9IHzOPegQjjU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Missing values are treated.**"
      ],
      "metadata": {
        "id": "Dv7qFsDhQljY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# New dataframe shape\n",
        "print(f\"There are {travel.shape[0]} rows and {travel.shape[1]} columns.\")"
      ],
      "metadata": {
        "id": "gU38VSgSQnL0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "Summary of the dataset"
      ],
      "metadata": {
        "id": "Cn89MZoGQx2D"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()"
      ],
      "metadata": {
        "id": "XrVV_BrNQrY8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# View summary of dataset numerical variables\n",
        "travel.describe().T"
      ],
      "metadata": {
        "id": "6LTh2ihCQ2J3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Observations\n",
        "\n",
        "* Age mean and median is 37.59 and 36 respectively which indicates a roughly normal distribution.\n",
        "* DurationOfPitch mean is 15.38 while median is 13.0 indicating more pitches are under 15.38 with some extreme of 127.0, indicating possible outliers.\n",
        "* NumberOfPersonVisiting mean and median is 2.91 and 3.0 respectively which indicates a roughly normal distribution.\n",
        "* NumberOfFollowups mean and median is 3.71 vs 4.0 indicating a slight left skewed meaning more follow ups are 4.0 and above.\n",
        "* PreferredPropertyStar mean and median is 3.58 vs 3.0 indicating more preferred hotel rating of 3 stars over 4 and 5 stars.\n",
        "* NumberOfTrips mean and median is 3.23 vs 3.0 indicating more number of trips are 3.0 and below with extreme of 22.0 indicating outliers.\n",
        "* PitchSatisfactionScore mean and median is 3.08 vs 3.0 which indicates a roughly normal distribution.\n",
        "* NumberOfChildrenVisiting mean and median is 1.187744 vs 1.0 indicating more number 1.0 or below.\n",
        "* MonthlyIncome mean and median is 23565.41 vs 22347 indicating more with income less than 23565.41."
      ],
      "metadata": {
        "id": "4B1hGldbQ6G0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# View summary of dataset categorical variables\n",
        "travel.describe(include=['category']).T"
      ],
      "metadata": {
        "id": "HZw5cOnARJ3n"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "Observations\n",
        "\n",
        "* 81.1% of the customers in the dataset did not take the package.\n",
        "* 70% of the customers are self enquired.\n",
        "* 65% of the customers are from Tire 1.\n",
        "* 48% of customers are salaried and they form the biggest occupation type of all customers.\n",
        "* 60% of customers are male.\n",
        "* 37.7% of customers were pitched the Basic package which is the most frequent.\n",
        "* 47.8% of customers are married and they form the biggest marital status type of all customers.\n",
        "* 70.8% of customers do not have a passport.\n",
        "* 62% of cusotmers own cars.\n",
        "* 37.7% of customers are executives and they form the biggest designation type of all customers."
      ],
      "metadata": {
        "id": "izOFIkiCRRw9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Univariate Analysis**\n"
      ],
      "metadata": {
        "id": "WHw7zbNeRiCo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# function to plot a boxplot and a histogram along the same scale.\n",
        "\n",
        "\n",
        "def histogram_boxplot(data, feature, figsize=(12, 7), kde=False, bins=None):\n",
        "    \"\"\"\n",
        "    Boxplot and histogram combined\n",
        "\n",
        "    data: dataframe\n",
        "    feature: dataframe column\n",
        "    figsize: size of figure (default (12,7))\n",
        "    kde: whether to show the density curve (default False)\n",
        "    bins: number of bins for histogram (default None)\n",
        "    \"\"\"\n",
        "    f2, (ax_box2, ax_hist2) = plt.subplots(\n",
        "        nrows=2,  # Number of rows of the subplot grid= 2\n",
        "        sharex=True,  # x-axis will be shared among all subplots\n",
        "        gridspec_kw={\"height_ratios\": (0.25, 0.75)},\n",
        "        figsize=figsize,\n",
        "    )  # creating the 2 subplots\n",
        "    sns.boxplot(\n",
        "        data=data, x=feature, ax=ax_box2, showmeans=True, color=\"violet\"\n",
        "    )  # boxplot will be created and a star will indicate the mean value of the column\n",
        "    sns.histplot(\n",
        "        data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins, palette=\"winter\"\n",
        "    ) if bins else sns.histplot(\n",
        "        data=data, x=feature, kde=kde, ax=ax_hist2\n",
        "    )  # For histogram\n",
        "    ax_hist2.axvline(\n",
        "        data[feature].mean(), color=\"green\", linestyle=\"--\"\n",
        "    )  # Add mean to the histogram\n",
        "    ax_hist2.axvline(\n",
        "        data[feature].median(), color=\"black\", linestyle=\"-\"\n",
        "    )  # Add median to the histogram"
      ],
      "metadata": {
        "id": "idQ5M4dtRmig"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()"
      ],
      "metadata": {
        "id": "KXzg9BD5RqBO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Univariate analysis for numerical columns\n",
        "**Observations on Age **"
      ],
      "metadata": {
        "id": "TkwtkjtKRwUU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, 'Age')"
      ],
      "metadata": {
        "id": "wW6zaeymR0NH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* The distribution of age is normal and no outliers.\n",
        "* A significant number of customers are aged around 36."
      ],
      "metadata": {
        "id": "taKUhNDYR8Cw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on DurationOfPitch**"
      ],
      "metadata": {
        "id": "FyUNb2p1SAqY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, \"DurationOfPitch\")\n"
      ],
      "metadata": {
        "id": "dLCbBooZR-sA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "*   A vast majority of customers pitch duration are below 15 with low number of outliers indicating extremely long pitch duration.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "UfrzKH8wSG6Z"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Observations on NumberOfPersonVisiting**"
      ],
      "metadata": {
        "id": "Rvjiqpt9SJ9f"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, \"NumberOfPersonVisiting\")\n"
      ],
      "metadata": {
        "id": "7R7dYV92SJNP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel['NumberOfPersonVisiting'].value_counts()\n"
      ],
      "metadata": {
        "id": "JPYl7KApSSJI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Observations on NumberOfFollowups**"
      ],
      "metadata": {
        "id": "nMaOf2TCSVaq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, \"NumberOfFollowups\")"
      ],
      "metadata": {
        "id": "E0oN7QKYSYpB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel['NumberOfFollowups'].value_counts()\n"
      ],
      "metadata": {
        "id": "LoPsiztVSeqI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* A majority of follow ups are 4 times followed by 3 times with outliers at 1 followup and 6 follow ups."
      ],
      "metadata": {
        "id": "gXzFMG10Sl0K"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on PreferredPropertyStar**"
      ],
      "metadata": {
        "id": "tpipp1GySxny"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, \"PreferredPropertyStar\")\n"
      ],
      "metadata": {
        "id": "YWXp-fr-Sisy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel['PreferredPropertyStar'].value_counts()\n"
      ],
      "metadata": {
        "id": "w3C91P5mS4G0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* A majority of customers preferred a 3 star to 4 star ratings hotel property."
      ],
      "metadata": {
        "id": "-iehubBzS7ba"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on NumberOfTrips**"
      ],
      "metadata": {
        "id": "7bFtnHtLS-S5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, \"NumberOfTrips\")"
      ],
      "metadata": {
        "id": "5sIGt6tHS8ni"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel[\"NumberOfTrips\"].value_counts()"
      ],
      "metadata": {
        "id": "6ixdqrNSS_hp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* A majority of trip numbers are 3 and below with a small number of outliers indicating a significantly higher number of trips."
      ],
      "metadata": {
        "id": "duwEqV7aTH9L"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on PitchSatisfactionScore**"
      ],
      "metadata": {
        "id": "UJdE6zJ8TKny"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, \"PitchSatisfactionScore\")\n"
      ],
      "metadata": {
        "id": "e88mL8yJTJLa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel[\"PitchSatisfactionScore\"].value_counts()\n"
      ],
      "metadata": {
        "id": "zfiaFaUUTQka"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "*   Most pitch satisfaction ratings are 3.0 / 5.0.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "ex4raK-BTbYD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on NumberOfChildrenVisiting**"
      ],
      "metadata": {
        "id": "HqD_FU7wTf08"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "histogram_boxplot(travel, \"NumberOfChildrenVisiting\")"
      ],
      "metadata": {
        "id": "1YbVq5YpTd3k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel[\"NumberOfChildrenVisiting\"].value_counts()"
      ],
      "metadata": {
        "id": "KFfNEh72TnHS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most customers have up to 2 children on the trip with 1 child being the most common."
      ],
      "metadata": {
        "id": "pkpGO9wETqrz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on MonthlyIncome**"
      ],
      "metadata": {
        "id": "uutfBQLoTvMi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "histogram_boxplot(travel, \"MonthlyIncome\")\n"
      ],
      "metadata": {
        "id": "I5FX8EjhTyN1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most customers have monthly income around 24K with a small number of outliers of significantly higher income customers."
      ],
      "metadata": {
        "id": "5xCLwAo-T4PF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Univariate analysis for categorical columns**"
      ],
      "metadata": {
        "id": "Qtd2vDAKTwjE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# function to create labeled barplots\n",
        "\n",
        "\n",
        "def labeled_barplot(data, feature, perc=False, n=None):\n",
        "    \"\"\"\n",
        "    Barplot with percentage at the top\n",
        "\n",
        "    data: dataframe\n",
        "    feature: dataframe column\n",
        "    perc: whether to display percentages instead of count (default is False)\n",
        "    n: displays the top n category levels (default is None, i.e., display all levels)\n",
        "    \"\"\"\n",
        "\n",
        "    total = len(data[feature])  # length of the column\n",
        "    count = data[feature].nunique()\n",
        "    if n is None:\n",
        "        plt.figure(figsize=(count + 1, 5))\n",
        "    else:\n",
        "        plt.figure(figsize=(n + 1, 5))\n",
        "\n",
        "    plt.xticks(rotation=90, fontsize=15)\n",
        "    ax = sns.countplot(\n",
        "        data=data,\n",
        "        x=feature,\n",
        "        palette=\"Paired\",\n",
        "        order=data[feature].value_counts().index[:n].sort_values(),\n",
        "    )\n",
        "\n",
        "    for p in ax.patches:\n",
        "        if perc == True:\n",
        "            label = \"{:.1f}%\".format(\n",
        "                100 * p.get_height() / total\n",
        "            )  # percentage of each class of the category\n",
        "        else:\n",
        "            label = p.get_height()  # count of each level of the category\n",
        "\n",
        "        x = p.get_x() + p.get_width() / 2  # width of the plot\n",
        "        y = p.get_height()  # height of the plot\n",
        "\n",
        "        ax.annotate(\n",
        "            label,\n",
        "            (x, y),\n",
        "            ha=\"center\",\n",
        "            va=\"center\",\n",
        "            size=12,\n",
        "            xytext=(0, 5),\n",
        "            textcoords=\"offset points\",\n",
        "        )  # annotate the percentage\n",
        "\n",
        "    plt.show()  # show the plot"
      ],
      "metadata": {
        "id": "xnqDX92dUA_W"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()"
      ],
      "metadata": {
        "id": "oES4kanCUEwH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken**"
      ],
      "metadata": {
        "id": "Ss5SNuPwUJd0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'ProdTaken', perc=True)"
      ],
      "metadata": {
        "id": "eNLrhOByUG_U"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* 18.9% of customers in dataset took the package while vast majority at 81.1% did not."
      ],
      "metadata": {
        "id": "rytF5rYMUPjQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on TypeofContact**"
      ],
      "metadata": {
        "id": "4VY3rFjdUUYg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'TypeofContact', perc=True)"
      ],
      "metadata": {
        "id": "OII2c1SiURZd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most customers 70.8% are self enquired to contact the company."
      ],
      "metadata": {
        "id": "S7_q_9vfUcxU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on CityTier**"
      ],
      "metadata": {
        "id": "RbGcW9vtUfy1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'CityTier', perc=True)\n"
      ],
      "metadata": {
        "id": "cNALSzcoUeOM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Majority of the customers 65.2% are from Tire 1 city."
      ],
      "metadata": {
        "id": "QcFHA2Z9UlyC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on Occupation**"
      ],
      "metadata": {
        "id": "-ZcQV5BRUpUk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'Occupation', perc=True)"
      ],
      "metadata": {
        "id": "cwIUyKjAUrzs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most customers are salaried employees followed by small business."
      ],
      "metadata": {
        "id": "0q89bwrCUvnA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on Gender**"
      ],
      "metadata": {
        "id": "QsqRgUzLUyp1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'Gender', perc=True)"
      ],
      "metadata": {
        "id": "kKZpeST5UyQ1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* More customers are male."
      ],
      "metadata": {
        "id": "Q7_AsnIsU4pP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched**"
      ],
      "metadata": {
        "id": "Mr-J7O0eU8yu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'ProductPitched', perc=True)"
      ],
      "metadata": {
        "id": "Q5LwiWxbU7OH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most popular packages pitched were Basic and Deluxe packages."
      ],
      "metadata": {
        "id": "Ls7nopMxVC3G"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on MaritalStatus**"
      ],
      "metadata": {
        "id": "Sp3FQuvjVFlH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'MaritalStatus', perc=True)"
      ],
      "metadata": {
        "id": "p9YyFiCJVEI1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Majority of the customers are Married."
      ],
      "metadata": {
        "id": "GCEzdAqTVQK3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on Passport**"
      ],
      "metadata": {
        "id": "N64IZCJhVTFh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'Passport', perc=True)"
      ],
      "metadata": {
        "id": "XWjh2cNyVRjX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most customers 70.8% do not have a passport."
      ],
      "metadata": {
        "id": "q3cS_grUVZL-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on OwnCar**"
      ],
      "metadata": {
        "id": "jo0pYAeuVcJf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'OwnCar', perc=True)"
      ],
      "metadata": {
        "id": "TuEBIQrkVa1W"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most customers(62.1%) own a car."
      ],
      "metadata": {
        "id": "NCRoCPjUVh3q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on Designation**"
      ],
      "metadata": {
        "id": "T5rbvth6Vk7K"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labeled_barplot(travel, 'Designation', perc=True)"
      ],
      "metadata": {
        "id": "RVBvGqVwVjjr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Most customers are executives or managers followed by senior managers."
      ],
      "metadata": {
        "id": "cgR_kGXVVwjN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Bivariate Analysis**"
      ],
      "metadata": {
        "id": "q3-TLOXVVzQN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# function to plot stacked bar chart\n",
        "\n",
        "\n",
        "def stacked_barplot(data, predictor, target):\n",
        "    \"\"\"\n",
        "    Print the category counts and plot a stacked bar chart\n",
        "\n",
        "    data: dataframe\n",
        "    predictor: independent variable\n",
        "    target: target variable\n",
        "    \"\"\"\n",
        "    count = data[predictor].nunique()\n",
        "    sorter = data[target].value_counts().index[-1]\n",
        "    tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(\n",
        "        by=sorter, ascending=False\n",
        "    )\n",
        "    print(tab1)\n",
        "    print(\"-\" * 120)\n",
        "    tab = pd.crosstab(data[predictor], data[target], normalize=\"index\").sort_values(\n",
        "        by=sorter, ascending=False\n",
        "    )\n",
        "    tab.plot(kind=\"bar\", stacked=True, figsize=(count + 1, 5))\n",
        "    plt.legend(\n",
        "        loc=\"lower left\",\n",
        "        frameon=False,\n",
        "    )\n",
        "    plt.legend(loc=\"upper left\", bbox_to_anchor=(1, 1))\n",
        "    plt.show()"
      ],
      "metadata": {
        "id": "_LoWJ93zVwNQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(15, 5))\n",
        "sns.heatmap(travel.corr(), annot=True, vmin=-1, vmax=1, fmt=\".2f\", cmap=\"Spectral\")\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "6uGC87PcV75C"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* NumberOfPersonVisiting and NumberOfChildrenVisiting seemed to be mildly positively correlated as both indicate the number travelling with the customer.\n",
        "* Age and Income shows some correlation.\n",
        "* There does not seem to be any other correlation among numeric variables thus some variables can be converted to categorical: 'NumberOfPersonVisiting', 'NumberOfFollowups', 'PreferredPropertyStar', 'PitchSatisfactionScore', 'NumberOfChildrenVisiting'"
      ],
      "metadata": {
        "id": "_jsqhXJvWDLI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel['NumberOfPersonVisiting'] = travel[\"NumberOfPersonVisiting\"].astype(\"category\")\n",
        "travel[\"NumberOfFollowups\"] = travel[\"NumberOfFollowups\"].astype(\"category\")\n",
        "travel[\"PreferredPropertyStar\"] = travel[\"PreferredPropertyStar\"].astype(\"category\")\n",
        "travel[\"PitchSatisfactionScore\"] = travel[\"PitchSatisfactionScore\"].astype(\"category\")\n",
        "travel[\"NumberOfChildrenVisiting\"] = travel[\"NumberOfChildrenVisiting\"].astype(\"category\")"
      ],
      "metadata": {
        "id": "ZAOs-iO3WMe7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()\n"
      ],
      "metadata": {
        "id": "0EBts1rTWPXr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "sns.pairplot(travel, diag_kind='kde', vars=[\"Age\",\"DurationOfPitch\",\"NumberOfTrips\",\"MonthlyIncome\"], hue=\"ProdTaken\")\n"
      ],
      "metadata": {
        "id": "IKWeMQvPWTJY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* There does not seem to be any pattern in the remaining numeric variables.\n"
      ],
      "metadata": {
        "id": "rhZ2zpQpWbhS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs Numeric Values**"
      ],
      "metadata": {
        "id": "9mflxNeAWfnC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "cols = travel[['Age','DurationOfPitch','NumberOfTrips', 'MonthlyIncome']].columns.tolist()\n",
        "plt.figure(figsize=(12,12))\n",
        "\n",
        "for i, variable in enumerate(cols):\n",
        "                     plt.subplot(3,2,i+1)\n",
        "                     sns.boxplot(data=travel, x = \"ProdTaken\", y = variable, palette=\"PuBu\")\n",
        "                     plt.tight_layout()\n",
        "                     plt.title(variable)\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "cvKpSFtAWdWC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Observations\n",
        "\n",
        "* Age median and distribution from 25th to 75th percentile of package taking customers are lower than non-package taking customers. The age range of customers taking up packages is between ~ 28 to just above 40.\n",
        "\n",
        "* DurationOfPitch median and distribution are rather similar between package and non package takers with non package takers registering a number of outliers with higher pitch durations. Package takers have a marginally higher median and distribution.\n",
        "\n",
        "* NumberOfTrips, for customer number of trips in a year, median and distribution are rather similar between package and non package takers with both registering a number of outliers on the higher end of number of trips.\n",
        "\n",
        "* Monthly Income median and distribution are rather similar between package and non package takers with both registering a number of outliers. Non Package takers have a marginally higher median and distribution as well as lower end and much higher end outliers in monthly income."
      ],
      "metadata": {
        "id": "taXnsxrvWnHh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()"
      ],
      "metadata": {
        "id": "exxNru1HWv0s"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs TypeofContact**"
      ],
      "metadata": {
        "id": "pNfqZ3CfW0n6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'TypeofContact', 'ProdTaken')"
      ],
      "metadata": {
        "id": "tPOjkC2NW1wu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Company invited customers are slightly more likely to take up of a package."
      ],
      "metadata": {
        "id": "gNCMVewMW6aK"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs CityTier**"
      ],
      "metadata": {
        "id": "nL4V9C1GW9mx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'CityTier', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "qFo_sqkWW8DR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Tier 2 and 3 customers are slightly more likely to take up a package than Tier 1 residents."
      ],
      "metadata": {
        "id": "AO6pJnNFXEoD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "**Observations on ProdTaken vs Occupation**"
      ],
      "metadata": {
        "id": "LVGZgmCwXLjM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'Occupation', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "VwLn0JXzXKrq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Free Lancers are very much likely to take up a package followed by Large Business customers. However this might not be statistically significant given that there are only 2 freelancers in the data set."
      ],
      "metadata": {
        "id": "z0MYUmcKxUt2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs Gender**"
      ],
      "metadata": {
        "id": "d5ddmeyUxYf-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'Gender', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "kWIDy2vvxW4h"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* There is not much difference in either gender of customers to determine a higher take up of a package."
      ],
      "metadata": {
        "id": "7btCG4uJzAQz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "# **Observations on ProdTaken vs NumberOfPersonVisiting**"
      ],
      "metadata": {
        "id": "naLZAhoozDgh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'NumberOfPersonVisiting', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "bYyk9cLJzB4T"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* 2 to 4 travel companions with the customer are more likely to take up a package."
      ],
      "metadata": {
        "id": "kWwt1HkKzJ5R"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs NumberOfFollowups**"
      ],
      "metadata": {
        "id": "awJ3uHqHzNhC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'NumberOfFollowups', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "LSG_bEsGzL8k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Among customers, a higher number of follow ups to customers will lead to higher success rate to take up a package."
      ],
      "metadata": {
        "id": "OdPJeDRLzTY7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs ProductPitched**"
      ],
      "metadata": {
        "id": "EbEL5jW2zXKV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'ProductPitched', 'ProdTaken')"
      ],
      "metadata": {
        "id": "8rLaUOrqzVt1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic followed by Standard then Deluxe packages have higher success rate of take up when pitched to customers."
      ],
      "metadata": {
        "id": "YoNAidtSzcxM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs PreferredPropertyStar**"
      ],
      "metadata": {
        "id": "nlfDrUYBzf1r"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'PreferredPropertyStar', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "QIQIOFl2zePc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "*  5 star hotel preferred customers are more likely to take up a package."
      ],
      "metadata": {
        "id": "QgSxt4Y0zo2T"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs MaritalStatus**"
      ],
      "metadata": {
        "id": "m9DyIgT9zr1h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'MaritalStatus', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "XYXUyS50zqSr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Singles are more likely to pick up a package.\n"
      ],
      "metadata": {
        "id": "MNAQ6DvTzzAa"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs Passport**"
      ],
      "metadata": {
        "id": "4zLGXt22z6Cu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'Passport', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "xj87EicPz8En"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Customers with passports are much more likely to pick up a package."
      ],
      "metadata": {
        "id": "Z0borhMLz_z5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs PitchSatisfactionScore**"
      ],
      "metadata": {
        "id": "1OLl5ig_0C5y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'PitchSatisfactionScore', 'ProdTaken')\n"
      ],
      "metadata": {
        "id": "xZnJDxF70BLK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Customers who rated 3.0 or more for pitch satisfaction are more likely to pick up a package."
      ],
      "metadata": {
        "id": "r3JeAcw_0JPB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "**Observations on ProdTaken vs OwnCar**"
      ],
      "metadata": {
        "id": "mt8AzTNh0Mec"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'OwnCar', 'ProdTaken')"
      ],
      "metadata": {
        "id": "JjMmhyLq0Koa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* There is no difference in either owning or not owning a car to determine a higher take up of a package by a customer."
      ],
      "metadata": {
        "id": "6nHiYZp10TSE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs NumberOfChildrenVisiting**"
      ],
      "metadata": {
        "id": "A_b8RpaX0WxB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'NumberOfChildrenVisiting', 'ProdTaken')"
      ],
      "metadata": {
        "id": "9TZen4vT0VH2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* There is no discernible difference in number of children travelling with customer to determine a higher take up of a package by a customer."
      ],
      "metadata": {
        "id": "ktJrsDBJ0dzx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProdTaken vs Designation**"
      ],
      "metadata": {
        "id": "EP5akiN80hKD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_barplot(travel, 'Designation', 'ProdTaken')"
      ],
      "metadata": {
        "id": "xD0y-jkY0fn3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Executives are more likely to take up a package."
      ],
      "metadata": {
        "id": "r4d4NmAm0m-1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Build customer profile**\n",
        "* All who took the different packages are filtered here."
      ],
      "metadata": {
        "id": "2BLWNOw_0qiY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "travelProdTaken = travel.loc[data[\"ProdTaken\"] == 1]"
      ],
      "metadata": {
        "id": "4xz6eOHO0oYR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travelProdTaken.shape"
      ],
      "metadata": {
        "id": "uzqqH1hq0xoy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travelProdTaken.head()"
      ],
      "metadata": {
        "id": "2bRShaYP00lk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's view a sample of the data\n",
        "travelProdTaken.sample(n=10, random_state=1)"
      ],
      "metadata": {
        "id": "3WM_qilv049a"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travelProdTaken.tail()"
      ],
      "metadata": {
        "id": "VrIyPXvf09u5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs Numeric Values**"
      ],
      "metadata": {
        "id": "FgtuKK3i1B8b"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "cols = travelProdTaken[['Age','NumberOfTrips','MonthlyIncome']].columns.tolist()\n",
        "plt.figure(figsize=(12,12))\n",
        "\n",
        "for i, variable in enumerate(cols):\n",
        "                     plt.subplot(3,2,i+1)\n",
        "                     sns.boxplot(data = travelProdTaken, x = \"ProductPitched\", y = variable)\n",
        "                     plt.tight_layout()\n",
        "                     plt.title(variable)\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "YB79FK6g1BDr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* **Age**\n",
        ">* Basic packages are mainly taken by the younger groups from late 20s to late 30s.\n",
        "> * Deluxe package takers mainly age range from early 30s to early 40s.\n",
        "> * Standard package takers mainly age range from mid 30s to late 40s.\n",
        ">* Super Deluxe package takers mainly age range from early 40s to mid 40s.\n",
        ">* King package takers mainly age range from early 40s to mid 50s.\n",
        "\n",
        "\n",
        "* **NumberOfTrips**\n",
        "> * Basic package takers mainly travel between 2 to 3 times a year with several outliers.\n",
        ">* Deluxe package takers mainly travel between 2 to 5 times a year.\n",
        ">* King package takers mainly travel between 2 to 3 times a year with some outliers.\n",
        ">* Standard package takers mainly travel between 2 to 4 times a year with a outlier.\n",
        ">* Super Deluxe package takers travel between 1 to 5 times a year.\n",
        "\n",
        "* **MonthlyIncome**\n",
        "> * Basic package takers income mostly range between 17.5K to ~ 21K.\n",
        ">* Deluxe package takers income mostly range between 21K to just shy of 25K.\n",
        ">* Standard package takers income mostly range between 24K to ~ 29K.\n",
        ">* Super Deluxe takers income mostly range from just below 28K to ~ 32K.\n",
        ">* King package takers income ranges from 35K and above.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "KSMVrMEZ2JTX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travelProdTaken.info()"
      ],
      "metadata": {
        "id": "pt6cekfQ3gvR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs CityTier**"
      ],
      "metadata": {
        "id": "D1JTR6m93mG9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"CityTier\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "gBTbjsan3nU5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic packages are most favored by Tier 1 city residents.\n",
        "* Deluxe packages are most favored by Tier 3 followed by Tier 1 city residents.\n",
        "* Standard to King packages are most favored by Tier 1 and 3 city residents."
      ],
      "metadata": {
        "id": "_wN8yK9i3t3T"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs Occupation**"
      ],
      "metadata": {
        "id": "zzHwXV1B3zxz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"Occupation\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()\n"
      ],
      "metadata": {
        "id": "W_tECqMX3yTz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic packages are most favored by salaried people then small business.\n",
        "* Deluxe packages are most favored by small business then salaried people.\n",
        "* Standard packages are most favored by both salaried people and small business almost equally.\n",
        "* Super Deluxe packages are picked up most by salaried people.\n",
        "* King packages are picked up most by small business."
      ],
      "metadata": {
        "id": "pcI2fOoz36uU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs Gender**"
      ],
      "metadata": {
        "id": "abSHxlDP4EQT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"Gender\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "WZSTDezD36Vr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic packages are more favored by males but remains the most popular among genders.\n",
        "* Deluxe packages are more favored by males but remains 2nd most popular among genders.\n",
        "* Standard packages are more favored by males.\n",
        "* Super Deluxe packages are picked up more by males.\n",
        "* King packages are picked slightly more by females."
      ],
      "metadata": {
        "id": "kKuRj64R359l"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs NumberOfPersonVisiting**"
      ],
      "metadata": {
        "id": "kQL9acXu4TEc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"NumberOfPersonVisiting\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "i1CRJPiw4WPM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic, Deluxe and Standard packages are picked up by customers with 2 to 4 travelling companions with 3 being the most popular.\n",
        "* Super Deluxe and King packages mainly attract 2 or 3 companions groups of customers."
      ],
      "metadata": {
        "id": "_oH-qkH94UpM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs PreferredPropertyStar**"
      ],
      "metadata": {
        "id": "PzCRvZML4fQt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"PreferredPropertyStar\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "zO9LX5804dKv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic, Deluxe and Standard packages are picked up by customers with 2 to 4 travelling companions with 3 being the most popular.\n",
        "* Super Deluxe and King packages mainly attract 2 or 3 companions groups of customers."
      ],
      "metadata": {
        "id": "QEG5yyw_4tON"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs PreferredPropertyStar**"
      ],
      "metadata": {
        "id": "XDpqihNs4068"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"PreferredPropertyStar\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "lMqk8rQI4zXK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Across Basic, Deluxe and Standard packages, most customers who took up packages preferred 3 star hotels followed by 5 stars.\n",
        "* In Super Deluxe and King packages, customers who took up packages tend to prefer 3-4 stars hotels."
      ],
      "metadata": {
        "id": "SCUNYUu84_GY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs MaritalStatus**"
      ],
      "metadata": {
        "id": "Mc9OvhWv5EkS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"MaritalStatus\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "Ay2nn5r15JWW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* In Basic, most customers who took up package are Singles followed by Married customers.\n",
        "* Across Deluxe and Standard, most customers are married and unmarried.\n",
        "* Across Super Deluxe and King, most customers are married and single."
      ],
      "metadata": {
        "id": "mB4Mx1d_5IsT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs Passport**"
      ],
      "metadata": {
        "id": "RM-GeQFO5UJU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"Passport\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "ocm6Ljvm5Tuz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic packages are most likely picked up by a customer with a passport.\n",
        "* Standard package is more picked up by those without passport.\n",
        "* Deluxe, Super Deluxe and King customers have equal numbers having or not having a passport."
      ],
      "metadata": {
        "id": "dEpY37Zy5bIq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs OwnCar**"
      ],
      "metadata": {
        "id": "3OHFCzAB5fv6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"OwnCar\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "dEPCw4rw5hx9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Those who own cars are more likely to pick up a package then those who don't, the disparity is more so for the Basic and King packages."
      ],
      "metadata": {
        "id": "WXXU4vIS5nDL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs NumberOfChildrenVisiting**"
      ],
      "metadata": {
        "id": "0V533_tP5oeN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"NumberOfChildrenVisiting\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "UVAHT9o55tzT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Basic, Deluxe and Super Deluxe packages are most likely picked up by customers who travels with 1-2 kids.\n",
        "* Standard packages are most likely picked up by customers with 0-1 kids.\n",
        "* King packages are most likely picked up by customers with 1 child."
      ],
      "metadata": {
        "id": "85tsDuu25sjE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations on ProductPitched vs Designation**"
      ],
      "metadata": {
        "id": "mYe0HLS151KS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(10,5))\n",
        "sns.countplot(data = travelProdTaken, x = \"ProductPitched\", hue = \"Designation\")\n",
        "plt.legend(bbox_to_anchor=(1, 1))\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "sjR8IPPg53YB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "* Basic Package attracts Executive level customers.\n",
        "* Deluxe package attracts Manager level customers.\n",
        "* Standard package attracts Senior Manager level customers.\n",
        "* Super Deluxe and King packages are picked up by AVPs and VPs respectively.\n",
        "\n"
      ],
      "metadata": {
        "id": "5Jxzno4l58W_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Customer Profiles for package types**\n",
        "* **Basic Package Customer Profile**\n",
        "\n",
        "> * Age ranges from late 20s to 30s.\n",
        "> * Mainly travel between 2 to 3 times a year.\n",
        "> * Monthly Income mostly range from 17.5K to ~ 21K.\n",
        "> * Executive level designation.\n",
        "> * Travels with 1-2 kids.\n",
        "> * Much more likely to own car.\n",
        "> * Most likely owns a passport.\n",
        "> * Most likely Single.\n",
        "> * Preferred 3 star hotels followed by 5 stars hotels.\n",
        "> * With 2 to 4 travelling companions.\n",
        "> * More favored by Males.\n",
        "> * Most favored by salaried people then small business.\n",
        "> * Most favored by Tier 1 city residents.\n",
        "\n",
        "* **Deluxe Package Customer Profile**\n",
        "\n",
        "> * Age range from early 30s to early 40s.\n",
        "> * Mainly travel between 2 to 5 times a year.\n",
        "> * Monthly income mostly range between 21K to just shy of 25K.\n",
        "> * Manager level designation.\n",
        "> * Travels with 1-2 kids.\n",
        "> * Likely to own car.\n",
        "> * Having either a passport or none.\n",
        "> * Most likely Single.\n",
        "> * Preferred 3 star hotels followed by 5 stars hotels.\n",
        "> * With 2 to 4 travelling companions.\n",
        "> * More favored by Males.\n",
        "> * Most favored by small business then salaried people.\n",
        "> * Most favored by Tier 3 followed by Tier 1 city residents.\n",
        "\n",
        "* **Standard Package Customer Profile**\n",
        "\n",
        "> * Age range from mid 30s to late 40s.\n",
        "> * Mainly travel between 2 to 4 times a year.\n",
        "> * Monthly income mostly range between 24K to ~ 29K.\n",
        "> * Senior Manager level designation.\n",
        "> * Travels with 0-1 kids.\n",
        "> * Likely to own car.\n",
        "> * More likely not owning a passport.\n",
        "> * More likely Married.\n",
        "> * Preferred 3 star hotels followed by 5 stars hotels.\n",
        "> * With 2 to 4 travelling companions.\n",
        "> * More favored by Males.\n",
        "> * Favored by both salaried people and small business almost equally.\n",
        "> * Most favored by Tier 1 and 3 city residents.\n",
        "> * Super Deluxe Package Customer Profile\n",
        "\n",
        "* **Age range from early 40s to mid 40s.**\n",
        "> * Travel between 1 to 5 times a year.\n",
        "> * Monthly income mostly range from just below 28K to ~ 32K.\n",
        "> * AVP level designation.\n",
        "> * Travels with 1-2 kids.\n",
        "> * Likely to own car.\n",
        "> * Having either a passport or none.\n",
        "> * Single or Married.\n",
        "> * Prefer 3-4 stars hotels.\n",
        "> * With 2 or 3 travelling companions.\n",
        "> * More favored by Males.\n",
        "> * Favored most by salaried people.\n",
        "> * Most favored by Tier 1 and 3 city residents.\n",
        "> * King Package Customer Profile\n",
        "\n",
        "* **Age range from early 40s to mid 50s.**\n",
        "> * Mainly travel between 2 to 3 times a year.\n",
        "> * Monthly income ranges from 35K and above.\n",
        "> * VP level designation.\n",
        "> * Travels with 1 kid.\n",
        "> * Likely to own car.\n",
        "> * Having either a passport or none.\n",
        "> * Single, Married or Divorced.\n",
        "> * Prefer 3-4 stars hotels.\n",
        "> * With 2 or 3 travelling companions.\n",
        "> * More favored by Females.\n",
        "> * Favored most by small business.\n",
        "> * Most favored by Tier 1 and 3 city residents.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "8c5NZJeF6KYI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**EDA Business Insights**\n",
        "* > It has been shown that the package acceptance is largely among the customers ages 28 to 40. This has made our Basic, Standard and Deluxe packages successful based on marketing the right packages to our customer profiles.\n",
        "\n",
        "* >Tier 2 and Tier 3 city residents are more accepting of packages then Tier 1 city residents. However, Tier 2 residents only constituted of 4.07% of our customer pool. More expansion is needed among Tier 2 residents as well as Tier 3 customer base now at 30.68% only.\n",
        "\n",
        "* >Large Business were also found to be more accepting of packages but only constituted 8.9% of our customer base. More expansion is needed on Large business customers.\n",
        "\n",
        "* >Higher number of followups by our sales teams to the potential customers lead to greater success of the customer accepting a package. This is so for 3 followups and above with each successive followup leading to a greater acceptance level. Sales teams should be briefed to at least have 3 followups with potential customers.\n",
        "\n",
        "* >Basic followed by Standard and Deluxe packages have the most success rates among our customers. The new Wellness package has to have similar features to them and marketed in 3 star or 5 star hotel combinations. This is because although most customers of Basic,Standard or Premium packages preferred 3 star, a number of them also picked 5 star over 4 star within these packages. 5 star preferred customers are also more accepting of a package.\n",
        "\n",
        "* >Singles are much more likely then married or divorcees to take up packages but our current customer base only has 32.8% of singles vs 47.8% married. Perhaps a marketing campaign to attract singles to be our customers can be done.\n",
        "\n",
        "* >Passport holders are much more likely to accept a package than none passport holders but 70.8% of our current customer base do not have passports. Incentives can be launched to attract passport holders to be our customers or encourage our current customers to make a passport.\n",
        "\n",
        "* >Customer rating of Pitch satisfaction score of 3.0 and above are more likely to accept a package. However numbers show that ratings 1.0 is as high as ratings 4.0 as well as 5.0. This has to be further investigated on why the sales team gets such an unusual higher number of rating 1.0 to help improve sales conversion.\n",
        "\n",
        "* >Analysis has shown that least acceptance rate among packages is the Super Deluxe package which ties in with its customer profile of people designated as AVPs. AVPs number the least likely to accept a package and Super Deluxe was marketed at them. The package therefore needs a relook and if needed, it can be discontinued. The new Wellness package should not have features similar to the Super Deluxe package."
      ],
      "metadata": {
        "id": "gjyYulSB8BrR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Data Preprocessing**\n",
        ">* Missing values"
      ],
      "metadata": {
        "id": "k4_RpNJ-8hPe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# checking if there is any missing values in the dataset\n",
        "travel.isnull().sum()"
      ],
      "metadata": {
        "id": "PXOYd8cJ6CE6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        ">* Missing values have been treated earlier.\n",
        "\n",
        "# **Duplicate values**"
      ],
      "metadata": {
        "id": "WCGipaOE8wu4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# checking if there is any duplicate entries in the dataset\n",
        "travel[travel.duplicated()].count()"
      ],
      "metadata": {
        "id": "uu6wrAek816G"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "* > Previously duplicates were not found prior to dropping customer ID.\n",
        "* > We can drop duplicates in the dataset.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "EZuvGHhx87PG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# dropping the duplicate entries in the dataset\n",
        "travel.drop_duplicates(inplace=True)"
      ],
      "metadata": {
        "id": "Fod8mnHe9L-8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# checking duplicates again\n",
        "travel[travel.duplicated()].count()"
      ],
      "metadata": {
        "id": "UgiQQSqt9O3U"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "> Duplicates entries are removed from the dataset.\n",
        "\n"
      ],
      "metadata": {
        "id": "ozunHQCG9VNc"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Treating Outliers**"
      ],
      "metadata": {
        "id": "4C9toNdt9ZHE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Listing all mumerical value boxplots\n",
        "numerical_col = travel.select_dtypes(include=np.number).columns.tolist()\n",
        "plt.figure(figsize=(20,30))\n",
        "\n",
        "for i, variable in enumerate(numerical_col):\n",
        "                     plt.subplot(5,4,i+1)\n",
        "                     plt.boxplot(travel[variable],whis=1.5)\n",
        "                     plt.tight_layout()\n",
        "                     plt.title(variable)\n",
        "\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "QkinZUg99U7F"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "* > Three columns have outliers and they are 'DurationOfPitch', 'NumberOfTrips' and 'MonthlyIncome'.\n",
        "* > Outliers should be treated."
      ],
      "metadata": {
        "id": "jw6yzqMo9gJF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Lets treat outliers by flooring and capping\n",
        "def treat_outliers(dataf,col):\n",
        "    '''\n",
        "    treats outliers in a varaible\n",
        "    col: str, name of the numerical varaible\n",
        "    dataf: data frame\n",
        "    col: name of the column\n",
        "    '''\n",
        "    Q1=dataf[col].quantile(0.25) # 25th quantile\n",
        "    Q3=dataf[col].quantile(0.75)  # 75th quantile\n",
        "    IQR=Q3-Q1\n",
        "    Lower_Whisker = Q1 - 1.5*IQR\n",
        "    Upper_Whisker = Q3 + 1.5*IQR\n",
        "    dataf[col] = np.clip(dataf[col], Lower_Whisker, Upper_Whisker) # all the values smaller than Lower_Whisker will be assigned value of Lower_whisker\n",
        "                                                            # and all the values above upper_whishker will be assigned value of upper_Whisker\n",
        "    return dataf\n",
        "\n",
        "def treat_outliers_all(dataf, col_list):\n",
        "    '''\n",
        "    treat outlier in all numerical varaibles\n",
        "    col_list: list of numerical varaibles\n",
        "    dataf: data frame\n",
        "    '''\n",
        "    for c in col_list:\n",
        "        dataf = treat_outliers(dataf,c)\n",
        "\n",
        "    return dataf"
      ],
      "metadata": {
        "id": "nBtt7NAB9mms"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# treating outliers in travel dataset\n",
        "numerical_col = travel.select_dtypes(include=np.number).columns.tolist()\n",
        "travel = treat_outliers_all(travel, numerical_col)"
      ],
      "metadata": {
        "id": "-CoVIUrq90_G"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Listing all mumerical value boxplots\n",
        "numerical_col = travel.select_dtypes(include=np.number).columns.tolist()\n",
        "plt.figure(figsize=(20,30))\n",
        "\n",
        "for i, variable in enumerate(numerical_col):\n",
        "                     plt.subplot(5,4,i+1)\n",
        "                     plt.boxplot(travel[variable],whis=1.5)\n",
        "                     plt.tight_layout()\n",
        "                     plt.title(variable)\n",
        "\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "jLgtuJAr93D2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "> Outliers are treated.\n",
        "\n"
      ],
      "metadata": {
        "id": "aboFB_uA97fs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()"
      ],
      "metadata": {
        "id": "SMGH7cqL9_oU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's view a sample of the data\n",
        "travel.sample(n=10, random_state=1)"
      ],
      "metadata": {
        "id": "i2dg1oRv-DxD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Feature Engineering**\n",
        "## Age Binning"
      ],
      "metadata": {
        "id": "GGIsUpPJ9_OA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "binned_age = pd.cut(travel['Age'], [15,25,35,45,55,65])\n",
        "binned_age.value_counts(dropna=False)"
      ],
      "metadata": {
        "id": "qeTTaSLN-O2o"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# can add custom labels\n",
        "travel['Age_bin'] = pd.cut(\n",
        "    travel['Age'], [15,25,35,45,55,65],\n",
        "    labels = [\"15-25\" , \"25-35\" , \"35-45\" , \"45-55\" , \"55-65\"]\n",
        ")\n",
        "travel.drop(['Age'], axis=1, inplace=True)\n",
        "travel['Age_bin'].value_counts(dropna=False)"
      ],
      "metadata": {
        "id": "8yIJZhQ--VgU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's view a sample of the data\n",
        "travel.sample(n=5, random_state=1)"
      ],
      "metadata": {
        "id": "b__l-qmL-Xgb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel[\"Age_bin\"] = travel[\"Age_bin\"].astype(\"category\")"
      ],
      "metadata": {
        "id": "JPn510h6-aF_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Age has been put into 5 groups in Age_bin column.\n",
        "* Age column is dropped from the data set.\n",
        "* Age_bin column is converted to categorical variable."
      ],
      "metadata": {
        "id": "JkJxwnPI-fRO"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Income Binning**"
      ],
      "metadata": {
        "id": "gJpFtKvM-lOh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "binned_income = pd.cut(travel['MonthlyIncome'], [10000.0,20000.0,30000.0,40000.0])\n",
        "binned_income.value_counts(dropna=False)"
      ],
      "metadata": {
        "id": "J9StYxeS-kv8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# can add custom labels\n",
        "travel['Income_bin'] = pd.cut(\n",
        "    travel['MonthlyIncome'], [10000.0,20000.0,30000.0,40000.0],\n",
        "    labels = [\"10k-20k\" , \"20k-30k\" , \"30k-40k\"]\n",
        ")\n",
        "travel.drop(['MonthlyIncome'], axis=1, inplace=True)\n",
        "travel['Income_bin'].value_counts(dropna=False)"
      ],
      "metadata": {
        "id": "cqkXQ01L-xIE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's view a sample of the data\n",
        "travel.sample(n=5, random_state=1)"
      ],
      "metadata": {
        "id": "utmpV6ku-1b0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "travel[\"Income_bin\"] = travel[\"Income_bin\"].astype(\"category\")"
      ],
      "metadata": {
        "id": "CHRopByL-8MA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        ">* MonthlyIncome has been put into 3 groups in Income_bin column.\n",
        ">* MonthlyIncome column is dropped from the data set.\n",
        ">* Income_bin column is converted to categorical variable.\n"
      ],
      "metadata": {
        "id": "8E9zXOkB--wy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Check variable datatypes**\n",
        "\n"
      ],
      "metadata": {
        "id": "-4q9h8Ev_SOW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "travel.info()\n"
      ],
      "metadata": {
        "id": "4r2DeaDo_G-b"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Prepare data for modeling**\n",
        "> **Split the data into train and test sets**"
      ],
      "metadata": {
        "id": "sMDHEiLt_Y-f"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "X = travel.drop(\"ProdTaken\" , axis=1)\n",
        "Y = travel[\"ProdTaken\"]\n",
        "\n",
        "# creating dummy variables\n",
        "X = pd.get_dummies(X, drop_first=True)\n",
        "\n",
        "# splitting in training and test set\n",
        "# stratify=Y maintains the ratio of default vs non default in the target variable for both training and testing sets)\n",
        "X_train, X_test, y_train, y_test = train_test_split( X, Y, test_size = 0.3, random_state = 42, stratify=Y)\n",
        "\n",
        "\n",
        "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
      ],
      "metadata": {
        "id": "nqtdJwGh_obb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# check the ratio of 1s to 0s\n",
        "Y.value_counts(1)"
      ],
      "metadata": {
        "id": "2VaMFHL8_rtt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# check the ratio of 1s to 0s\n",
        "y_test.value_counts(1)"
      ],
      "metadata": {
        "id": "0kpGI-FK_vSl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# check the ratio of 1s to 0s\n",
        "y_train.value_counts(1)"
      ],
      "metadata": {
        "id": "n1qNznGI_x5X"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Functions Definitions**"
      ],
      "metadata": {
        "id": "RfrNZTDD_059"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to calculate different metric scores of the model - Accuracy, Recall and Precision\n",
        "def get_metrics_score(model,flag=True):\n",
        "    '''\n",
        "    model : classifier to predict values of X\n",
        "\n",
        "    '''\n",
        "    # defining an empty list to store train and test results\n",
        "    score_list=[]\n",
        "\n",
        "    #Predicting on train and tests\n",
        "    pred_train = model.predict(X_train)\n",
        "    pred_test = model.predict(X_test)\n",
        "\n",
        "    #Accuracy of the model\n",
        "    train_acc = model.score(X_train,y_train)\n",
        "    test_acc = model.score(X_test,y_test)\n",
        "\n",
        "    #Recall of the model\n",
        "    train_recall = metrics.recall_score(y_train,pred_train)\n",
        "    test_recall = metrics.recall_score(y_test,pred_test)\n",
        "\n",
        "    #Precision of the model\n",
        "    train_precision = metrics.precision_score(y_train,pred_train)\n",
        "    test_precision = metrics.precision_score(y_test,pred_test)\n",
        "\n",
        "    #F1 of the model\n",
        "    train_f1 = f1_score(y_train,pred_train)\n",
        "    test_f1 = f1_score(y_test,pred_test)\n",
        "\n",
        "    score_list.extend((train_acc,test_acc,train_recall,test_recall,train_precision,test_precision, train_f1, test_f1))\n",
        "\n",
        "    # If the flag is set to True then only the following print statements will be dispayed. The default value is set to True.\n",
        "    if flag == True:\n",
        "        print(\"Accuracy on training set : \",model.score(X_train,y_train))\n",
        "        print(\"Accuracy on test set : \",model.score(X_test,y_test))\n",
        "        print(\"Recall on training set : \",metrics.recall_score(y_train,pred_train))\n",
        "        print(\"Recall on test set : \",metrics.recall_score(y_test,pred_test))\n",
        "        print(\"Precision on training set : \",metrics.precision_score(y_train,pred_train))\n",
        "        print(\"Precision on test set : \",metrics.precision_score(y_test,pred_test))\n",
        "        print(\"f1 score on training set : \",metrics.f1_score(y_train,pred_train))\n",
        "        print(\"f1 score on test set : \",metrics.f1_score(y_test,pred_test))\n",
        "\n",
        "    return score_list # returning the list with train and test scores"
      ],
      "metadata": {
        "id": "0ZHsbYOi_2sz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        " ## Function to create confusion matrix\n",
        "def make_confusion_matrix(model,y_actual,labels=[1, 0]):\n",
        "    '''\n",
        "    model : classifier to predict values of X\n",
        "    y_actual : ground truth\n",
        "\n",
        "    '''\n",
        "    y_predict = model.predict(X_test)\n",
        "    cm=metrics.confusion_matrix( y_actual, y_predict, labels=[0, 1])\n",
        "    df_cm = pd.DataFrame(cm, index = [i for i in [\"Actual - No\",\"Actual - Yes\"]],\n",
        "                  columns = [i for i in ['Predicted - No','Predicted - Yes']])\n",
        "    group_counts = [\"{0:0.0f}\".format(value) for value in\n",
        "                cm.flatten()]\n",
        "    group_percentages = [\"{0:.2%}\".format(value) for value in\n",
        "                         cm.flatten()/np.sum(cm)]\n",
        "    labels = [f\"{v1}\\n{v2}\" for v1, v2 in\n",
        "              zip(group_counts,group_percentages)]\n",
        "    labels = np.asarray(labels).reshape(2,2)\n",
        "    plt.figure(figsize = (10,7))\n",
        "    sns.heatmap(df_cm, annot=labels,fmt='')\n",
        "    plt.ylabel('True label')\n",
        "    plt.xlabel('Predicted label')"
      ],
      "metadata": {
        "id": "zVD1mcMTADI6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Model Evaluation Criterion**\n",
        "**Model can make wrong predictions as:**\n",
        "1. False Positive: Predicting a customer is a travel package convertable but actually not convertable.\n",
        "2. False Negative: Predicting a customer is a travel package non-convertable but actually convertable.\n",
        "\n",
        "**Which case is more important?**\n",
        "* Both the cases are important as:\n",
        "* If we predict a customer is a travel package convertable but actually not\n",
        "* convertable then a wrong person will be getting the targeted marketing effort wasting resources.\n",
        "* If we predict a customer is a travel package non-convertable but actually convertable, that person will not be able to receive targeted marketing effort and hence may not be aware of the travel package and thus a loss of business.\n",
        "\n",
        "**How to reduce losses?**\n",
        "* We can use accuracy but since the data is imbalanced it would not be the\n",
        "right metric to check the model performance.\n",
        "* Therefore, f1_score should be maximized, the greater the f1_score higher the chances of identifying both the classes correctly."
      ],
      "metadata": {
        "id": "qjxlFo1HAIS6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dtree = DecisionTreeClassifier(class_weight={0:0.19,1:0.81}, random_state = 1)\n",
        "dtree.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "RPvxUFTeA5bg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Model building - Bagging, Random Forest & Decision Tree\n",
        "\n",
        "# Build Decision Tree Model\n",
        "\n",
        "* We will build our model using the DecisionTreeClassifier function. Using default 'gini' criteria to split.\n",
        "* If the frequency of class A is 10% and the frequency of class B is 90%, then class B will become the dominant class and the decision tree will become biased toward the dominant classes.\n",
        "* In this case, we can pass a dictionary {0:0.19,1:0.81} to the model to specify the weight of each class and the decision tree will give more weightage to class 1 based on the dataset distribution.\n",
        "* class_weight is a hyperparameter for the decision tree classifier."
      ],
      "metadata": {
        "id": "aKkPHUZPBC2p"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dtree = DecisionTreeClassifier(class_weight={0:0.19,1:0.81}, random_state = 1)\n",
        "dtree.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "FbM5pUOoBOKC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(dtree, y_test)"
      ],
      "metadata": {
        "id": "Qkmx2sG1BSjV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Confusion Matrix -**\n",
        "\n",
        "* Customer takes up package and the model predicted customer takes up package : True Positive (observed=1,predicted=1)\n",
        "* Customer didn't take up package and the model predicted customer takes up package : False Positive (observed=0,predicted=1)\n",
        "* Customer didn't take up package and the model predicted customer didn't take up package : True Negative (observed=0,predicted=0)\n",
        "* Customer takes up package and the model predicted customer didn't take up package : False Negative (observed=1,predicted=0)"
      ],
      "metadata": {
        "id": "NsdlHl3KBWia"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Using above defined function to get accuracy, recall, precision and f1 score on train and test set\n",
        "dtree_score=get_metrics_score(dtree)"
      ],
      "metadata": {
        "id": "p6MWD8RlBmUc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* Decision tree is working well on the training data but is not able to generalize well on the test data.\n",
        "* This is so as well for the f1 score (train data: 100% ; test data: 70.7%).\n",
        "In fact this is a sign of overfitting.\n",
        "\n",
        "**Build Bagging Classifier Model**"
      ],
      "metadata": {
        "id": "LGR9XqHfBtiR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#base_estimator for bagging classifier is a decision tree by default\n",
        "bagging = BaggingClassifier(base_estimator=DecisionTreeClassifier(class_weight={0:0.19,1:0.81}, random_state=1), random_state=1)\n",
        "bagging.fit(X_train,y_train)"
      ],
      "metadata": {
        "id": "4_7VImqyB9Rp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(bagging,y_test)"
      ],
      "metadata": {
        "id": "pNuEFVx1CA2F"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "bagging_score=get_metrics_score(bagging)"
      ],
      "metadata": {
        "id": "BrX0Fp7nCELZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* Bagging classifier is still overfitting on the training set and is not generalizing well on the test data.\n",
        "* This is so as well for the f1 score (train data: 97.8% ; test data: 66.3%).\n",
        "* When comparing this model with decision tree model: In test data, False positives show good improvement by reducing from 4.73% to 1.62% but False negatives are increased from 5.93% to 8.68%.\n",
        "# **Build Bagging Classifier Model - Logistic Regression as base estimator**"
      ],
      "metadata": {
        "id": "sPsT5dQyCHx_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "bagging_lr=BaggingClassifier(base_estimator=LogisticRegression(class_weight={0:0.19,1:0.81},random_state=1, max_iter=1000),random_state=1)\n",
        "bagging_lr.fit(X_train,y_train)"
      ],
      "metadata": {
        "id": "98g37NLvCTxV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(bagging_lr,y_test)"
      ],
      "metadata": {
        "id": "oJTISn4bCa8X"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "bagging_lr_score=get_metrics_score(bagging_lr)"
      ],
      "metadata": {
        "id": "amlr6H5bCesg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* Bagging classifier with logistic regression as base_estimator is not overfitting the data but the scores are low.\n",
        "* This is so as well for the f1 score (train data: 55.5% ; test data: 51.4%).\n",
        "* Ensemble models are less interpretable than decision tree but bagging classifier is even less interpretable than random forest. It does not even have a feature importance attribute.\n",
        "\n",
        "**Build Random Forest Model**"
      ],
      "metadata": {
        "id": "oBSR3C48CiQ4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "rf = RandomForestClassifier(class_weight={0:0.19,1:0.81},random_state=1)\n",
        "rf.fit(X_train,y_train)"
      ],
      "metadata": {
        "id": "uXy-i9KhCrpa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(rf,y_test)"
      ],
      "metadata": {
        "id": "kRVpBBNlCwe1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "rf_score=get_metrics_score(rf)"
      ],
      "metadata": {
        "id": "OKb6XEVUC0UN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        ">* Random Forest classifier is still overfitting on the training set and is not generalizing well on the test data.\n",
        ">* This is so as well for the f1 score (train data: 100% ; test data: 58.2%).\n",
        "It does not do as well as the lone decision tree or the bagging classifier."
      ],
      "metadata": {
        "id": "TiT-xgSlC35I"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Hyperparameter Tuning**\n",
        "\n",
        "**Decision Tree**\n",
        "\n"
      ],
      "metadata": {
        "id": "uC03aUriDE8q"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Choose the type of classifier.\n",
        "dtree_estimator = DecisionTreeClassifier(class_weight={0:0.19,1:0.81},random_state=1)\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {'max_depth': np.arange(2,30),\n",
        "              'min_samples_leaf': [1, 2, 5, 7, 10],\n",
        "              'max_leaf_nodes' : [2, 3, 5, 10,15],\n",
        "              'min_impurity_decrease': [0.0001,0.001,0.01,0.1]\n",
        "             }\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.f1_score) # using highest f1_score to choose\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(dtree_estimator, parameters, scoring=scorer)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "dtree_estimator = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "dtree_estimator.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "JfdqYO20DEmY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(dtree_estimator,y_test)"
      ],
      "metadata": {
        "id": "DZfeTVSAE7Gu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "dtree_estimator_score=get_metrics_score(dtree_estimator)"
      ],
      "metadata": {
        "id": "DFZKx_JTE_va"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* Overfitting in decision tree tuned has reduced but scores have also reduced.\n",
        "* This is so as well for the f1 score (train data: 53.9% ; test data: 45.6%).\n",
        "\n",
        "**Bagging Classifier**\n",
        "\n",
        "**Some of the important hyperparameters available for bagging classifier are:**\n",
        "\n",
        "* base_estimator: The base estimator to fit on random subsets of the dataset. If None(default), then the base estimator is a decision tree.\n",
        "n_estimators: The number of trees in the forest, default = 100.\n",
        "max_features: The number of features to consider when looking for the best split.\n",
        "* bootstrap: Whether bootstrap samples are used when building trees. If False, the entire dataset is used to build each tree, default=True.\n",
        "bootstrap_features: If it is true, then features are drawn with replacement. Default value is False.\n",
        "* max_samples: If bootstrap is True, then the number of samples to draw from X to train each base estimator. If None (default), then draw N samples, where N is the number of observations in the train data.\n",
        "oob_score: Whether to use out-of-bag samples to estimate the generalization accuracy, default=False."
      ],
      "metadata": {
        "id": "Y4N3rDOsE_M8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Choose the type of classifier.\n",
        "bagging_estimator_tuned = BaggingClassifier(base_estimator=DecisionTreeClassifier(class_weight={0:0.19,1:0.81},random_state=1),random_state=1)\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {'max_samples': [0.7,0.8,0.9,1],\n",
        "              'max_features': [0.7,0.8,0.9,1],\n",
        "              'n_estimators' : [10,20,30,40,50],\n",
        "             }\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "acc_scorer = metrics.make_scorer(metrics.f1_score) # using highest f1_score to choose\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(bagging_estimator_tuned, parameters, scoring=acc_scorer,cv=5)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "bagging_estimator_tuned = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "bagging_estimator_tuned.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "JwJgjOuXFo8f"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(bagging_estimator_tuned,y_test)\n"
      ],
      "metadata": {
        "id": "VVXByhtyFuL8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "bagging_estimator_tuned_score=get_metrics_score(bagging_estimator_tuned)\n"
      ],
      "metadata": {
        "id": "zp0LWxRYFxTZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* Bagging classifier tuned is still overfitting but less so on the training set and is generalizing a little better on the test data for precision.\n",
        "* f1 score's score gap between training and test set remains (train data: 99.3% ; test data: 66.5%).\n",
        "* There is no improvement on other scores but precision score gap between training and test set is smaller by 4% points."
      ],
      "metadata": {
        "id": "xBpgxZE-F9ia"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Bagging Classifier - Logistic Regression as base estimator**"
      ],
      "metadata": {
        "id": "DaJ9EEdFGD5y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Choose the type of classifier.\n",
        "bagging_lr_tuned = BaggingClassifier(base_estimator=LogisticRegression(class_weight={0:0.19,1:0.81},random_state=1, max_iter=1000),random_state=1)\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {'max_samples': [0.7,0.8,0.9,1],\n",
        "              'max_features': [0.7,0.8,0.9,1],\n",
        "              'n_estimators' : [10,20,30,40,50],\n",
        "             }\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "acc_scorer = metrics.make_scorer(metrics.f1_score) # using highest f1_score to choose\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(bagging_lr_tuned, parameters, scoring=acc_scorer,cv=5)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "bagging_lr_tuned = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "bagging_lr_tuned.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "4CG2n0XLF9Nk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(bagging_lr_tuned,y_test)"
      ],
      "metadata": {
        "id": "bORRa0U9GS0b"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "bagging_lr_tuned_score=get_metrics_score(bagging_lr_tuned)\n"
      ],
      "metadata": {
        "id": "ng2_4ppVGVrK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* Bagging classifier tuned with logistic regression as base_estimator is not overfitting the data but the scores performed similarly low to before tuning.\n",
        "* This is so as well for the f1 score (train data: 54.9% ; test data: 51%)."
      ],
      "metadata": {
        "id": "em2-vfF1Ga1t"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Random Forest Classifier**\n",
        "Now, let's see if we can get a better model by tuning the random forest classifier. Some of the important hyperparameters available for random forest classifier are:\n",
        "\n",
        "* n_estimators: The number of trees in the forest, default = 100.\n",
        "max_features: The number of features to consider when looking for the best split.\n",
        "* class_weight: Weights associated with classes in the form {class_label: weight}.If not given, all classes are supposed to have weight one.\n",
        "* For example: If the frequency of class 0 is 80% and the frequency of class 1 is 20% in the data, then class 0 will become the dominant class and the model will become biased toward the dominant classes. In this case, we can pass a dictionary {0:0.2,1:0.8} to the model to specify the weight of each class and the random forest will give more weightage to class 1.\n",
        "* bootstrap: Whether bootstrap samples are used when building trees. If False, the entire dataset is used to build each tree, default=True.\n",
        "* max_samples: If bootstrap is True, then the number of samples to draw from X to train each base estimator. If None (default), then draw N samples, where N is the number of observations in the train data.\n",
        "* oob_score: Whether to use out-of-bag samples to estimate the generalization accuracy, default=False.\n",
        "\n",
        " Note: A lot of hyperparameters of Decision Trees are also available to tune Random Forest like max_depth, min_sample_split etc."
      ],
      "metadata": {
        "id": "VMss_4ZwGkJf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Choose the type of classifier.\n",
        "rf_estimator_tuned = RandomForestClassifier(class_weight={0:0.19,1:0.81},random_state=1)\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {\n",
        "        \"n_estimators\": [50,100,150],\n",
        "        \"min_samples_leaf\": np.arange(5, 10),\n",
        "        \"max_features\": np.arange(0.2, 0.7, 0.1),\n",
        "        \"max_samples\": np.arange(0.3, 0.7, 0.1),\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "acc_scorer = metrics.make_scorer(metrics.f1_score) # using highest f1_score to choose\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(rf_estimator_tuned, parameters, scoring=acc_scorer,cv=5)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "rf_estimator_tuned = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "rf_estimator_tuned.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "xggXSywrHWGD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "make_confusion_matrix(rf_estimator_tuned,y_test)"
      ],
      "metadata": {
        "id": "OXSNM4LTHj2h"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "rf_estimator_tuned_score=get_metrics_score(rf_estimator_tuned)\n"
      ],
      "metadata": {
        "id": "wadYkIrVHl55"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        ">* Random Forest classifier tuned has improved much of the overfitting issue on the training set and is generalizing better on the test data.\n",
        ">* This is so as well for the f1 score (train data: 81.2% ; test data: 60.9%), an improvement of 20% points in the score gap but the scores are low."
      ],
      "metadata": {
        "id": "lDpqrAjDHrhd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Model building - Boosting & Stacking**\n",
        "\n",
        "\n",
        "**Build AdaBoost Classifier Model**"
      ],
      "metadata": {
        "id": "58wGN3erHzNZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Fitting the model\n",
        "ab_classifier = AdaBoostClassifier(random_state=1)\n",
        "ab_classifier.fit(X_train,y_train)\n",
        "\n",
        "#Calculating different metrics\n",
        "get_metrics_score(ab_classifier)\n",
        "\n",
        "#Creating confusion matrix\n",
        "make_confusion_matrix(ab_classifier,y_test)"
      ],
      "metadata": {
        "id": "ZaI90lAEH46o"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "*Adaboost is giving more generalized performance than previous models but the test f1-score is too low.\n",
        "\n",
        "*F1 score is at (train data: 47.6% ; test data: 43.9%).\n",
        "Hyperparameter Tuning"
      ],
      "metadata": {
        "id": "bFAmSqUyIUbG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Hyperparameter Tuning**"
      ],
      "metadata": {
        "id": "br_QOWutQNag"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Choose the type of classifier.\n",
        "abc_tuned = AdaBoostClassifier(random_state=1)\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {\n",
        "    #Let's try different max_depth for base_estimator\n",
        "    \"base_estimator\":[DecisionTreeClassifier(max_depth=1),DecisionTreeClassifier(max_depth=2),\n",
        "                      DecisionTreeClassifier(max_depth=3)],\n",
        "    \"n_estimators\": np.arange(10,110,10),\n",
        "    \"learning_rate\":np.arange(0.1,2,0.1)\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.f1_score)\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(abc_tuned, parameters, scoring=scorer,cv=5)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "abc_tuned = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "abc_tuned.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "Qwj18QXKITXY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#Calculating different metrics\n",
        "get_metrics_score(abc_tuned)\n",
        "\n",
        "#Creating confusion matrix\n",
        "make_confusion_matrix(abc_tuned,y_test)"
      ],
      "metadata": {
        "id": "G9DhX8HuIhI9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* The tuned AdaBoost model performance has increased but the model has started to overfit the training data among several metrics.\n",
        "* F1 score is at (train data: 88.4% ; test data: 56.5%)."
      ],
      "metadata": {
        "id": "vd-VNSjKIkbU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Build Gradient Boosting Classifier Model**"
      ],
      "metadata": {
        "id": "bO_Pfb4YJ8ey"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Fitting the model\n",
        "gb_classifier = GradientBoostingClassifier(random_state=1)\n",
        "gb_classifier.fit(X_train,y_train)\n",
        "\n",
        "#Calculating different metrics\n",
        "get_metrics_score(gb_classifier)\n",
        "\n",
        "#Creating confusion matrix\n",
        "make_confusion_matrix(gb_classifier,y_test)"
      ],
      "metadata": {
        "id": "Z06tWwKrJ-02"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* The Gradient Boosting model does not overfit as much but some of the test data metrics are low.\n",
        "* F1 score is at (train data: 62% ; test data: 50.8%)."
      ],
      "metadata": {
        "id": "7VREb1v2KICx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Hyperparameter Tuning**"
      ],
      "metadata": {
        "id": "LJmKPQbBKObO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Choose the type of classifier.\n",
        "gbc_tuned = GradientBoostingClassifier(init=AdaBoostClassifier(random_state=1), random_state=1)\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {\n",
        "    \"n_estimators\": [100,150,200,250],\n",
        "    \"subsample\":[0.8,0.9,1],\n",
        "    \"max_features\":[0.7,0.8,0.9,1]\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.f1_score)\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(gbc_tuned, parameters, scoring=scorer,cv=5)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "gbc_tuned = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "gbc_tuned.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "9dztoOk5KE8d"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#Calculating different metrics\n",
        "get_metrics_score(gbc_tuned)\n",
        "\n",
        "#Creating confusion matrix\n",
        "make_confusion_matrix(gbc_tuned,y_test)"
      ],
      "metadata": {
        "id": "Vg4E6F9jKj_o"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* The tuned Gradient Boosting model does not overfit yet and all test data metrics improved except for precision but there is more room for improvement.\n",
        "* F1 score is at (train data: 74.9% ; test data: 54.6%)."
      ],
      "metadata": {
        "id": "sskHbwgSKn4z"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Build XGBoost Classifier Model**"
      ],
      "metadata": {
        "id": "dnbi9tm5Ku-D"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Fitting the model\n",
        "xgb_classifier = XGBClassifier(random_state=1, eval_metric='logloss')\n",
        "xgb_classifier.fit(X_train,y_train)\n",
        "\n",
        "#Calculating different metrics\n",
        "get_metrics_score(xgb_classifier)\n",
        "\n",
        "#Creating confusion matrix\n",
        "make_confusion_matrix(xgb_classifier,y_test)"
      ],
      "metadata": {
        "id": "rqc4GH92KtKJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* The XGB model is starting to overfit and all test data metrics have improved significantly.\n",
        "* F1 score is at (train data: 99.7% ; test data: 72%)."
      ],
      "metadata": {
        "id": "9OLxmQ1tK5As"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Hyperparameter Tuning**"
      ],
      "metadata": {
        "id": "8dYLac44LBaQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Choose the type of classifier.\n",
        "xgb_tuned = XGBClassifier(random_state=1, eval_metric='logloss')\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {\n",
        "    \"n_estimators\": [10,30,50],\n",
        "    \"scale_pos_weight\":[1,2,5],\n",
        "    \"subsample\":[0.7,0.9,1],\n",
        "    \"learning_rate\":[0.05, 0.1,0.2],\n",
        "    \"colsample_bytree\":[0.7,0.9,1],\n",
        "    \"colsample_bylevel\":[0.5,0.7,1]\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.f1_score)\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(xgb_tuned, parameters,scoring=scorer,cv=5)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "xgb_tuned = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "xgb_tuned.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "ZIDZWm3NK_vj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#Calculating different metrics\n",
        "get_metrics_score(xgb_tuned)\n",
        "\n",
        "#Creating confusion matrix\n",
        "make_confusion_matrix(xgb_tuned,y_test)"
      ],
      "metadata": {
        "id": "VWilDmIGLMFQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* The tuned XGB model has generalized the test data better but the metrics scores are lower than XGB model.\n",
        "* F1 score is at (train data: 94.4% ; test data: 68.4%)."
      ],
      "metadata": {
        "id": "UQpUe66nLQYl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Build Stacking Classifier Model**"
      ],
      "metadata": {
        "id": "S-iSJlX1LZh-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "estimators = [('Random Forest',rf_estimator_tuned), ('Gradient Boosting',gbc_tuned), ('Decision Tree',dtree_estimator)]\n",
        "\n",
        "final_estimator = xgb_tuned\n",
        "\n",
        "stacking_classifier= StackingClassifier(estimators=estimators,final_estimator=final_estimator)\n",
        "\n",
        "stacking_classifier.fit(X_train,y_train)"
      ],
      "metadata": {
        "id": "Rc5mGCD4LdBq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#Calculating different metrics\n",
        "get_metrics_score(stacking_classifier)\n",
        "\n",
        "#Creating confusion matrix\n",
        "make_confusion_matrix(stacking_classifier,y_test)"
      ],
      "metadata": {
        "id": "takp4eCvLf9O"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* The stacker classifier has generalized the test data similar to XGB tuned model but scored lower on the metrics.\n",
        "* F1 score is at (train data: 84.4% ; test data: 64.7%)."
      ],
      "metadata": {
        "id": "YYlpe7j3LkgA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Comparing all models**"
      ],
      "metadata": {
        "id": "XhColT_lLqw-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# defining list of models\n",
        "models = [dtree, dtree_estimator,rf, rf_estimator_tuned, bagging, bagging_estimator_tuned, bagging_lr, bagging_lr_tuned,\n",
        "          ab_classifier, abc_tuned, gb_classifier, gbc_tuned, xgb_classifier,xgb_tuned, stacking_classifier]\n",
        "\n",
        "# defining empty lists to add train and test results\n",
        "acc_train = []\n",
        "acc_test = []\n",
        "recall_train = []\n",
        "recall_test = []\n",
        "precision_train = []\n",
        "precision_test = []\n",
        "f1_train = []\n",
        "f1_test = []\n",
        "\n",
        "# looping through all the models to get the metrics score - Accuracy, Recall and Precision\n",
        "for model in models:\n",
        "\n",
        "    j = get_metrics_score(model,False)\n",
        "    acc_train.append(j[0])\n",
        "    acc_test.append(j[1])\n",
        "    recall_train.append(j[2])\n",
        "    recall_test.append(j[3])\n",
        "    precision_train.append(j[4])\n",
        "    precision_test.append(j[5])\n",
        "    f1_train.append(j[6])\n",
        "    f1_test.append(j[7])"
      ],
      "metadata": {
        "id": "9hEL4y41Lj-b"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "comparison_frame = pd.DataFrame({'Model':['Decision Tree','Tuned Decision Tree','Random Forest','Tuned Random Forest',\n",
        "                                          'Bagging Classifier','Tuned Bagging Classifier','Bagging Classifier Logistic Regression',\n",
        "                                          'Tuned Bagging Classifier Logistic Regression','AdaBoost Classifier','Tuned AdaBoost Classifier',\n",
        "                                          'Gradient Boosting Classifier', 'Tuned Gradient Boosting Classifier',\n",
        "                                          'XGBoost Classifier',  'Tuned XGBoost Classifier', 'Stacking Classifier'],\n",
        "                                          'Train_Accuracy': acc_train,'Test_Accuracy': acc_test,\n",
        "                                          'Train_Recall':recall_train,'Test_Recall':recall_test,\n",
        "                                          'Train_Precision':precision_train,'Test_Precision':precision_test,\n",
        "                                           'Train_F1-Score':f1_train, 'Test_F1-Score':f1_test})\n",
        "\n",
        "#Sorting models in decreasing order of test recall\n",
        "comparison_frame.sort_values(by='Test_F1-Score',ascending=False)"
      ],
      "metadata": {
        "id": "PTgeq1UpLw_t"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "\n",
        "* The models are either tending towards overfitting or score poorly in terms of f1-score.\n",
        "* XGBoost Classifier should be the model to press ahead as it scored the highest test f1-score.\n",
        "* There may well be other combinations of hyperparameters not attempted yet to improve the metrics of the models. This will however require much more time to find out more comprehensively a better model."
      ],
      "metadata": {
        "id": "oeB1draFL1CJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Feature Importance of XGBoost Classifier Model**"
      ],
      "metadata": {
        "id": "Lmgn3tlKL-03"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "feature_names = X_train.columns\n",
        "importances = xgb_classifier.feature_importances_\n",
        "indices = np.argsort(importances)\n",
        "\n",
        "plt.figure(figsize=(12,12))\n",
        "plt.title('Feature Importances')\n",
        "plt.barh(range(len(indices)), importances[indices], color='violet', align='center')\n",
        "plt.yticks(range(len(indices)), [feature_names[i] for i in indices])\n",
        "plt.xlabel('Relative Importance')\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "4R8bAy_iL8Zb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "*Observations*\n",
        "\n",
        "* In the XGBoost classifier model, 'Designation_Executive' is the most important feature followed by features - 'Passport_1' and MaritalStatus_Single."
      ],
      "metadata": {
        "id": "i_2yjmGGMGec"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Business Insights and Recommendations\n",
        "\n",
        "* Based on the performances of the different models, XGBoost Classifier  \n",
        "* > performed the best using f1_score as the deciding factor due to the\n",
        "* >unbalanced data as well as a lower score gap between training and testing data\n",
        "* >Significant variables include ‘Designation’, ‘Passport’ and ‘MaritalStatus’.\n",
        "Coupled with EDA insights, Executive level, Passport holders and Single customers are more likely to accept a package deal.\n",
        "* >However, the current customer base has 70.8% Non-Passport holders and only 18.4% of customers are single compared to 47.8% are married.\n",
        "* >Therefore more targeted marketing effort at attracting Passport holders and Singles to expand the customer base with these customer segments. Incentives can also be given to encourage existing customer base to obtain a passport or attract new passport holding customers.\n",
        "* >Currently, the Executive level customers dominated the share of customers at 37.9% and more can be done to recruit them and bolster the Executive level numbers.\n",
        "\n",
        "* Comments on additional data sources for model improvement\n",
        "\n",
        "* >Additional data can be obtained from measured feedback of initial targeted marketing efforts to the public in order to strengthen the model.\n",
        "* >Feedback can be gathered from non-package convertible customers for further analysis.\n",
        "* Model implementation in real world and potential business benefits from model\n",
        "\n",
        "* >The model implemented in the real world will help to raise more successful targeted marketing converts of its campaign and reduce the costs of marketing to potential non-converts or miss target marketing to potential converts. This will increase revenue and reduce both variable marketing costs and opportunity costs.\n",
        "* Other Recommendations – From EDA\n",
        "\n",
        "* >More expansion is needed among Tier 2 residents as well as Tier 3 customer base now at 30.77% only.\n",
        "* >More expansion is needed on Large business customers now at 8.9% of our customer base.\n",
        "* >Sales teams should be briefed to at least have 3 follow-ups with potential customers.\n",
        "* >The new Wellness package has to have similar features to Basic, Standard and Deluxe packages and marketed in 3 star or 5 star hotel combinations.\n",
        "Further investigation on unusual high number of rating 1.0 to help improve sales conversion.\n",
        "* >Super Deluxe package and King package performs poorly and these should require a relook at their features and discontinued if needed. The new Wellness package should not have similar features to Super Deluxe package and King package."
      ],
      "metadata": {
        "id": "FSLGY3EsMNM4"
      }
    }
  ]
}