Find_Street_Yellow_Line_Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating an AOI based on road lane feature\n",
    "***\n",
    "\n",
    "In this project, I used Python and OpenCV to find lane lines in the road images then isolate an area of interest for classification.   \n",
    "\n",
    "The following techniques are used:\n",
    "\n",
    "- Color Selection\n",
    "- Canny Edge Detection\n",
    "- Region of Interest Selection\n",
    "- Hough Transform Line Detection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import cv2\n",
    "import os, glob\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading Test Images\n",
    "\n",
    "The following libraries are used.  `cv2` is for OpenCV.\n",
    "The following `show_image` function is used to display images.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def show_images(images, cmap=None):\n",
    "    cols = 2\n",
    "    rows = (len(images)+1)//cols\n",
    "    \n",
    "    plt.figure(figsize=(10, 11))\n",
    "    for i, image in enumerate(images):\n",
    "        plt.subplot(rows, cols, i+1)\n",
    "        # use gray scale color map if there is only one channel\n",
    "        cmap = 'gray' if len(image.shape)==2 else cmap\n",
    "        plt.imshow(image, cmap=cmap)\n",
    "        plt.xticks([])\n",
    "        plt.yticks([])\n",
    "    plt.tight_layout(pad=0, h_pad=0, w_pad=0)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Only know how to handle extensions: [u'png']; with Pillow installed matplotlib can handle more images",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-07a51acf8414>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtest_images\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mpath\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mglob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mglob\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'StreetExamples/*.jpg'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mshow_images\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest_images\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/mmann1123/anaconda/envs/spatial_env/lib/python2.7/site-packages/matplotlib/pyplot.pyc\u001b[0m in \u001b[0;36mimread\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m   2312\u001b[0m \u001b[0;34m@\u001b[0m\u001b[0mdocstring\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopy_dedent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_imread\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2313\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mimread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2314\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0m_imread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2315\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2316\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/mmann1123/anaconda/envs/spatial_env/lib/python2.7/site-packages/matplotlib/image.pyc\u001b[0m in \u001b[0;36mimread\u001b[0;34m(fname, format)\u001b[0m\n\u001b[1;32m   1280\u001b[0m             raise ValueError('Only know how to handle extensions: %s; '\n\u001b[1;32m   1281\u001b[0m                              \u001b[0;34m'with Pillow installed matplotlib can handle '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1282\u001b[0;31m                              'more images' % list(six.iterkeys(handlers)))\n\u001b[0m\u001b[1;32m   1283\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mim\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1284\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: Only know how to handle extensions: [u'png']; with Pillow installed matplotlib can handle more images"
     ]
    }
   ],
   "source": [
    "test_images = [plt.imread(path) for path in glob.glob('StreetExamples/*.jpg')]\n",
    "\n",
    "show_images(test_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Color Selection\n",
    "\n",
    "### HSL and HSV Color Space\n",
    "\n",
    "Using `cv2.cvtColor`, we can convert RGB image into different color space.  For example, [HSL and HSV color space](https://en.wikipedia.org/wiki/HSL_and_HSV).\n",
    "\n",
    "<img src='images/hsl-hsv.png' width='50%'>\n",
    "\n",
    "Image Source: [https://commons.wikimedia.org/wiki/File:Hsl-hsv_models.svg](https://commons.wikimedia.org/wiki/File:Hsl-hsv_models.svg)\n",
    "\n",
    "\n",
    "### HSV Color Space\n",
    "\n",
    "How does it look when RGB images are converted into HSV color space?\n",
    "Is yellow easily detected?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "\n",
    "def convert_hsv(image):\n",
    "    return cv2.cvtColor(image, cv2.COLOR_RGB2HSV)\n",
    "show_images(list(map(convert_hsv, test_images )))\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " \n",
    "Let's build a filter to select those yellow lines. I want to select particular range of each channels (Hue, Saturation and Light). You can use the mouse-over above to see what Hue Saturation and Light to use.\n",
    "  \n",
    "- Use `cv2.inRange` to filter  the yellow color.  \n",
    "  The function returns 255 when the filter conditon is satisfied.  Otherwise, it returns 0.\n",
    "- Use `cv2.bitwise_and` to apply the combined mask onto the original RGB image\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_yellow(image):\n",
    "    converted = convert_hsv(image)\n",
    "    # yellow color mask\n",
    "    #                  Hue  Sat  Light\n",
    "    lower = np.uint8([ 10,  100, 120])  \n",
    "    upper = np.uint8([ 20, 160, 160])     \n",
    " \n",
    "    yellow_mask = cv2.inRange(converted, lower, upper)\n",
    "    # combine the mask\n",
    "    mask = yellow_mask \n",
    "    return cv2.bitwise_and(image, image, mask = mask)\n",
    "\n",
    "yellow_images = list(map(select_yellow, test_images))\n",
    "\n",
    "show_images(yellow_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Now the yellow lines are the dominant feature. Note that these photos were taken in the fall, hence the yellow foliage.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Canny Edge Detection\n",
    "\n",
    "We want to detect edges in order to find straight lines especially lane lines.  For this, \n",
    "\n",
    "- use `cv2.cvtColor` to convert images into gray scale\n",
    "- use `cv2.GaussianBlur` to smooth out rough edges \n",
    "- use `cv2.Canny` to find edges\n",
    "\n",
    "Let's take a look at each step in details.\n",
    "\n",
    "    \n",
    "### Gray Scaling\n",
    "\n",
    "The images should be converted into gray scaled ones in order to detect shapes (edges) in the images.  This is because the Canny edge detection measures the magnitude of pixel intensity changes or gradients (more on this later).\n",
    "\n",
    "Here, I'm converting the white and yellow line images from the above into gray scale for edge detection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_gray_scale(image):\n",
    "    return cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)\n",
    "\n",
    "gray_images = list(map(convert_gray_scale, yellow_images))\n",
    "\n",
    "show_images(gray_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gaussian Smoothing (Gaussian Blur)\n",
    "\n",
    "When there is an edge (i.e. a line), the pixel intensity changes rapidly (i.e. from 0 to 255) which we want to detect.  But before doing so, we should make the edges smoother.  As you can see,  the above images have many rough edges which causes many noisy edges to be detected.\n",
    "\n",
    "I use `cv2.GaussianBlur` to smooth out edges.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def apply_smoothing(image, kernel_size=13):\n",
    "    \"\"\"\n",
    "    kernel_size must be postivie and odd\n",
    "    \"\"\"\n",
    "    return cv2.GaussianBlur(image, (kernel_size, kernel_size), 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The GaussianBlur takes a `kernel_size` parameter which you'll need to play with to find one that works best.  I tried 3, 5, 9, 11, 15, 17 (they must be positive and odd) and check the edge detection (see the next section) result.  The bigger the `kernel_size` value is,  the more blurry the image becomes.  \n",
    "\n",
    "The bigger `kearnel_size` value requires more time to process. It is not noticeable with the test images but we should keep that in mind (later we'll be processing video clips).  So, we should prefer smaller values if the effect is similar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "blurred_images = list(map(lambda image: apply_smoothing(image), gray_images))\n",
    "\n",
    "show_images(blurred_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Edge Detection\n",
    "\n",
    "`cv2.Canny` takes two threshold values which requires some explanation.\n",
    "\n",
    "Wikipedia says:\n",
    "\n",
    "> it is essential to filter out the edge pixel with the weak gradient value and preserve the edge with the high gradient value. Thus two threshold values are set to clarify the different types of edge pixels, one is called high threshold value and the other is called the low threshold value. If the edge pixel’s gradient value is higher than the high threshold value, they are marked as strong edge pixels. If the edge pixel’s gradient value is smaller than the high threshold value and larger than the low threshold value, they are marked as weak edge pixels. If the pixel value is smaller than the low threshold value, they will be suppressed. \n",
    "\n",
    "According to the OpenCV documentation, the double thresholds are used as follows:\n",
    "\n",
    "- If a pixel gradient is higher than the upper threshold, the pixel is accepted as an edge\n",
    "- If a pixel gradient value is below the lower threshold, then it is rejected.\n",
    "- If the pixel gradient is between the two thresholds, then it will be accepted only if it is connected to a pixel that is above the upper threshold.\n",
    "- Canny recommended a upper:lower ratio between 2:1 and 3:1.\n",
    "\n",
    "These two threshold values are empirically determined.  Basically, you will need to define them by trials and errors.\n",
    "\n",
    "I first set the `low_threshold` to zero and then adjust the `high_threshold`.   If `high_threshold` is too high, you find no edges.  If `high_threshold` is too low, you find too many edges.  Once you find a good `high_threshold`, adjust the `low_threshold` to discard the weak edges (noises) connected to the strong edges.\n",
    "\n",
    "- [Canny Edge Detection OpenCV Theory](http://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html)\n",
    "- [cv2.Canny OpenCV API Reference](http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def detect_edges(image, low_threshold=50, high_threshold= 100):\n",
    "    return cv2.Canny(image, low_threshold, high_threshold)\n",
    "\n",
    "edge_images = list(map(lambda image: detect_edges(image), blurred_images))\n",
    "\n",
    "show_images(edge_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Region of Interest Selection\n",
    "\n",
    "When finding lane lines, we don't need to check the sky and the hills. I know from camera placement that it will be in the lower 1/3 of the image  \n",
    "\n",
    "Roughly speaking, we are interested in the aread surrounded by the red lines below:\n",
    "\n",
    "<img src='./images/region-of-interest.png' width='50%'/>\n",
    "\n",
    "So, we exclude outside the region of interest by apply a mask.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def filter_region(image, vertices):\n",
    "    \"\"\"\n",
    "    Create the mask using the vertices and apply it to the input image\n",
    "    \"\"\"\n",
    "    mask = np.zeros_like(image)\n",
    "    if len(mask.shape)==2:\n",
    "        cv2.fillPoly(mask, vertices, 255)\n",
    "    else:\n",
    "        cv2.fillPoly(mask, vertices, (255,)*mask.shape[2]) # in case, the input image has a channel dimension        \n",
    "    return cv2.bitwise_and(image, mask)\n",
    "\n",
    "    \n",
    "def select_region(image):\n",
    "    \"\"\"\n",
    "    It keeps the region surrounded by the `vertices` (i.e. polygon).  Other area is set to 0 (black).\n",
    "    \"\"\"\n",
    "        \n",
    "    # first, define the polygon by vertices\n",
    "    rows, cols = image.shape[:2]\n",
    "    \n",
    "    # first, define the polygon by vertices    \n",
    "    bottom_left  = [cols*0.01, rows*0.99]\n",
    "    top_left     = [cols*0.01, rows*0.7]\n",
    "    bottom_right = [cols*0.9, rows*0.99]\n",
    "    top_right    = [cols*0.9, rows*0.7] \n",
    "    # the vertices are an array of polygons (i.e array of arrays) and the data type must be integer\n",
    "    vertices = np.array([[bottom_left, top_left, top_right, bottom_right]], dtype=np.int32)\n",
    "    return filter_region(image, vertices)\n",
    "\n",
    "\n",
    "# images showing the region of interest only\n",
    "roi_images = list(map(select_region, edge_images))\n",
    "\n",
    "show_images(roi_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hough Transform Line Detection\n",
    "\n",
    "I'm using `cv2.HoughLinesP` to detect lines in the edge images.\n",
    "\n",
    "There are several parameters you'll need to tweak and tune:\n",
    "\n",
    "- rho – Distance resolution of the accumulator in pixels.\n",
    "- theta – Angle resolution of the accumulator in radians.\n",
    "- threshold – Accumulator threshold parameter. Only those lines are returned that get enough votes (> `threshold`).\n",
    "- minLineLength – Minimum line length. Line segments shorter than that are rejected.\n",
    "- maxLineGap – Maximum allowed gap between points on the same line to link them.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def hough_lines(image):\n",
    "    \"\"\"\n",
    "    `image` should be the output of a Canny transform.\n",
    "    \n",
    "    Returns hough lines (not the image with lines)\n",
    "    \"\"\"\n",
    "    return cv2.HoughLinesP(image, rho=1, theta=np.pi/180, threshold=20, minLineLength=20, maxLineGap=300)\n",
    "\n",
    "\n",
    "list_of_lines = list(map(hough_lines, roi_images))\n",
    "print(list_of_lines[0:1] ) # list of lines detected for one image, each line is  x1, y1, x2, y2 coordinates of the line segment\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`list_of_lines` contains a list of lines detected.  With the above parameters, approximately 2-5 lines are detected for each image.\n",
    "\n",
    "Let's draw the lines onto the original images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def draw_lines(image, lines, color=[255, 0, 0], thickness=2, make_copy=True):\n",
    "    # the lines returned by cv2.HoughLinesP has the shape (-1, 1, 4)\n",
    "    if make_copy:\n",
    "        image = np.copy(image) # don't want to modify the original\n",
    "    for line in lines:\n",
    "        for x1,y1,x2,y2 in line:\n",
    "            cv2.line(image, (x1, y1), (x2, y2), color, thickness)\n",
    "    return image\n",
    "\n",
    "\n",
    "line_images = []\n",
    "for image, lines in zip(test_images, list_of_lines):\n",
    "    line_images.append(draw_lines(image, lines))\n",
    "    \n",
    "show_images(line_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculating line slope and intercept\n",
    "\n",
    "There are multiple lines detected for the lane line. We also want to know the intercept and slope of the line so that we can isolate a ROI for our analysis. \n",
    "\n",
    "Also, some lane lines are only partially recognized.  We should extrapolate the line to cover full lane line length.\n",
    " \n",
    "Note: in the image, `y` coordinate is reversed.  The higher `y` value is actually lower in the image. So a possitive slope is actually negative.  \n",
    "\n",
    "\n",
    "<img src='images/coord.png' width='50%'>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_slope_intercept_longest(lines):\n",
    "    ''' Return slope and intercept of the longest line observed'''\n",
    "    intercepts_slopes = [] # (slope, intercept)     \n",
    "    lengths   = [] # (slope, intercept)\n",
    "    #longest_line = []\n",
    "    \n",
    "    for line in lines:\n",
    "        type(line)\n",
    "        for x1, y1, x2, y2 in line:\n",
    "            if x2==x1:\n",
    "                continue # ignore a vertical line\n",
    "            slope = (float(y2)-float(y1))/(float(x2)-float(x1))\n",
    "            intercept = y1 - slope*x1\n",
    "            length = np.sqrt((y2-y1)**2+(x2-x1)**2)\n",
    "             \n",
    "            intercepts_slopes.append((  slope, intercept))\n",
    "            lengths.append(( length))\n",
    "\n",
    "    # return only the longest line \n",
    "    #longest_line.append( intercepts_slopes[lengths.index(max(lengths))]   )\n",
    "            \n",
    "    #return longest_line #(slope, intercept) \n",
    "    return intercepts_slopes[lengths.index(max(lengths))] \n",
    "\n",
    "\n",
    "\n",
    "# capture intercept and slope of longest line for each image\n",
    "long_line_list=[]\n",
    "for line in list_of_lines:\n",
    "    long_line_list.append(get_slope_intercept_longest(line))\n",
    "\n",
    "long_line_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's draw the lanes.  I need to convert the slope and intercept into pixel points.\n",
    "\n",
    "Our `draw_line` except a line as the second parameter.  Each line is a list of 4 values (x1, y1, x2, y2).  The data type needs to be integer for `cv2.line` to work without throwing an error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_line_points(image, line):\n",
    "    \"\"\"\n",
    "    Convert a line represented in slope and intercept into pixel points\n",
    "    \"\"\"\n",
    "    if line is None:\n",
    "        return None\n",
    "    print(line)\n",
    "    slope, intercept = line[0]\n",
    "    \n",
    "    y1 = image.shape[0] # bottom of the image\n",
    "    y2 = y1*0.6         # slightly lower than the middle\n",
    "    \n",
    "    # make sure everything is integer as cv2.line requires it\n",
    "    x1 = int((y1 - intercept)/slope)\n",
    "    x2 = int((y2 - intercept)/slope)\n",
    "    y1 = int(y1)\n",
    "    y2 = int(y2)\n",
    "    \n",
    "    return([x1, y1, x2, y2])\n",
    "\n",
    "\n",
    "def make_line_points(image, line):\n",
    "    \"\"\"\n",
    "    Convert a line represented in slope and intercept into pixel points\n",
    "    \"\"\"\n",
    "    if line is None:\n",
    "        return None\n",
    "    slope, intercept = line  \n",
    "    x1 = int(0) # start far left\n",
    "    x2 = int(image.shape[1]) # end far left\n",
    "    y1 = int(intercept)\n",
    "    y2 = int(intercept + slope*(x2-x1))\n",
    "   \n",
    "    # make sure everything is integer as cv2.line requires it\n",
    "    return([x1, y1, x2, y2])\n",
    "\n",
    "\n",
    "def draw_line(image, line, color=[255, 0, 0], thickness=2, make_copy=True):\n",
    "    if make_copy:\n",
    "        image = np.copy(image) # don't want to modify the original\n",
    "    x1,y1,x2,y2 = line\n",
    "    cv2.line(image, (x1, y1), (x2, y2), color, thickness)\n",
    "    return image\n",
    "\n",
    "\n",
    "line_images = []\n",
    "long_line_points = []\n",
    "for image, lines in zip(test_images, long_line_list):\n",
    "    long_line = make_line_points(image, lines)\n",
    "    long_line_points.append(long_line)\n",
    "    line_images.append(draw_line(image, long_line))\n",
    "    \n",
    "show_images(line_images)\n",
    "print('The x1 y1 x2 y2 points:')\n",
    "long_line_points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this worked under a variety of conditions including very short exposed lane lines, and different orientations of the camera. \n",
    "\n",
    "\n",
    "## Apply adpative AOI based on road \n",
    "\n",
    "In the next step we need to use the road lines created by `get_slope_intercept_longest` to isolate the AOI for our classification algorithm (masking out areas away from the road. Again the mask is applied using the `select_region` function\n",
    "\n",
    "I will do this by adjusting out road line upward by the height of the largest object on the street, a bus, which is a little under 95 pixels tall. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_region(image,line_slope_inter):\n",
    "    \"\"\"  It keeps the region surrounded by the `vertices` (i.e. polygon).  Other area is set to 0 (black).\n",
    "    \"\"\"\n",
    "    # get slope and intercept \n",
    "    slope, intercept = line_slope_inter\n",
    "    \n",
    "    # first, define the polygon by vertices\n",
    "    rows, cols = image.shape[:2]\n",
    "    \n",
    "    bottom_left  = [cols*0, rows]\n",
    "    top_left     = [cols*0, intercept-95]\n",
    "    bottom_right = [cols*1, rows]\n",
    "    top_right    = [cols*1, intercept-95+slope*cols] \n",
    "    #print(bottom_left,top_left,bottom_right,top_right)\n",
    "    # the vertices are an array of polygons (i.e array of arrays) and the data type must be integer\n",
    "    vertices = np.array([[bottom_left, top_left, top_right, bottom_right]], dtype=np.int32)\n",
    "    return filter_region(image, vertices)\n",
    "\n",
    "\n",
    "# images showing the region of interest only\n",
    "roi_images = list(map(select_region, test_images,long_line_list))\n",
    "\n",
    "show_images(roi_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Voilà we have isolated the road from the rest of the image  "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}