notebooks/basic/Tutorial/2.Building_Your_Predictor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building Your Predictor\n",
    "\n",
    "The next step after preparing and importing your data via `Getting_Data_Ready.ipynb` is to build your first model.\n",
    "\n",
    "The overall process for this is:\n",
    "\n",
    "* Setup\n",
    "* Create a Predictor\n",
    "* Deploy a Predictor\n",
    "* Obtain a Forecast\n",
    "\n",
    "To get started, simply execute the cells below:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the standard Python Libraries that are used in this lesson."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "import time\n",
    "import boto3\n",
    "\n",
    "# importing forecast notebook utility from notebooks/common directory\n",
    "sys.path.insert( 0, os.path.abspath(\"../../common\") )\n",
    "import util"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The line below will retrieve your shared variables from the first notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store -r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last part of the setup process is to validate that your account can communicate with Amazon Forecast, the cell below does just that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = boto3.Session(region_name=region) \n",
    "forecast = session.client(service_name='forecast') \n",
    "forecastquery = session.client(service_name='forecastquery')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Predictor\n",
    "\n",
    "Now in the previous notebook, your data was imported to be used by Forecast, here we will once again define your dataset information and then start building your model or predictor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Forecast horizon is the number of number of time points to predicted in the future. For weekly data, a value of 12 means 12 weeks. Our example is hourly data, we try forecast the next day, so we can set to 24."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictorName= project+'_deeparp_algo'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "forecastHorizon = 24"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "algorithmArn = 'arn:aws:forecast:::algorithm/Deep_AR_Plus'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_predictor_response=forecast.create_predictor(PredictorName=predictorName, \n",
    "                                                  AlgorithmArn=algorithmArn,\n",
    "                                                  ForecastHorizon=forecastHorizon,\n",
    "                                                  PerformAutoML= False,\n",
    "                                                  PerformHPO=False,\n",
    "                                                  EvaluationParameters= {\"NumberOfBacktestWindows\": 1, \n",
    "                                                                         \"BackTestWindowOffset\": 24}, \n",
    "                                                  InputDataConfig= {\"DatasetGroupArn\": datasetGroupArn},\n",
    "                                                  FeaturizationConfig= {\"ForecastFrequency\": \"H\", \n",
    "                                                                        \"Featurizations\": \n",
    "                                                                        [\n",
    "                                                                          {\"AttributeName\": \"target_value\", \n",
    "                                                                           \"FeaturizationPipeline\": \n",
    "                                                                            [\n",
    "                                                                              {\"FeaturizationMethodName\": \"filling\", \n",
    "                                                                               \"FeaturizationMethodParameters\": \n",
    "                                                                                {\"frontfill\": \"none\", \n",
    "                                                                                 \"middlefill\": \"zero\", \n",
    "                                                                                 \"backfill\": \"zero\"}\n",
    "                                                                              }\n",
    "                                                                            ]\n",
    "                                                                          }\n",
    "                                                                        ]\n",
    "                                                                       }\n",
    "                                                 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_arn=create_predictor_response['PredictorArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the status of the predictor. When the status change from **CREATE_IN_PROGRESS** to **ACTIVE**, we can continue to next steps. Depending on data size, model selection and hyper parameters，it can take 10 mins to more than one hour to be **ACTIVE**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "status_indicator = util.StatusIndicator()\n",
    "\n",
    "while True:\n",
    "    status = forecast.describe_predictor(PredictorArn=predictor_arn)['Status']\n",
    "    status_indicator.update(status)\n",
    "    if status in ('ACTIVE', 'CREATE_FAILED'): break\n",
    "    time.sleep(10)\n",
    "\n",
    "status_indicator.end()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get Error Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "forecast.get_accuracy_metrics(PredictorArn=predictor_arn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Forecast\n",
    "\n",
    "Now create a forecast using the model that was trained"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "forecastName= project+'_deeparp_algo_forecast'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_forecast_response=forecast.create_forecast(ForecastName=forecastName,\n",
    "                                                  PredictorArn=predictor_arn)\n",
    "forecast_arn = create_forecast_response['ForecastArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the status of the forecast process, when the status change from **CREATE_IN_PROGRESS** to **ACTIVE**, we can continue to next steps. Depending on data size, model selection and hyper parameters，it can take 10 mins to more than one hour to be **ACTIVE**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "status_indicator = util.StatusIndicator()\n",
    "\n",
    "while True:\n",
    "    status = forecast.describe_forecast(ForecastArn=forecast_arn)['Status']\n",
    "    status_indicator.update(status)\n",
    "    if status in ('ACTIVE', 'CREATE_FAILED'): break\n",
    "    time.sleep(10)\n",
    "\n",
    "status_indicator.end()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get Forecast"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once created, the forecast results are ready and you view them. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(forecast_arn)\n",
    "print()\n",
    "forecastResponse = forecastquery.query_forecast(\n",
    "    ForecastArn=forecast_arn,\n",
    "    Filters={\"item_id\":\"client_12\"}\n",
    ")\n",
    "print(forecastResponse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "Now that your forecast has been created, to evaluate it use `3.Evaluating_Your_Predictor.ipynb` but before opening it, execute the cell below to share your variables again with the next notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store forecast_arn\n",
    "%store predictor_arn"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}