08. Data Cleaning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4791161e",
   "metadata": {},
   "source": [
    "## <font color=green> Limpeza e preparação dos dados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d2a58c29",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c397d394",
   "metadata": {},
   "source": [
    "- Durante a análise e a modelagaem dos dados, um período significativo de tempo é gasto em sua preparação: carga, limpeza, transformação e reorganização.\n",
    "- Muitas vezes, o modo como os dados são armazenados em arquivos ou em bancos de dados não constituem o formato correto para uma tarefa em particular.\n",
    "- Pandas, juntos com os recursos embutidos da linguagem Python, oferecem um conjunto de ferramentas de alto nível, rápido e flexível, para permitir que você manipule os dados, deixando-os no formato correto."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5360156",
   "metadata": {},
   "source": [
    "### Tratando dados ausentes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cc8394a",
   "metadata": {},
   "source": [
    "- Dados ausente são comuns em muitas aplicações de análise de dados.\n",
    "- Para dados numéricos, o pandas utiliza o valor de ponto flutuante NaN (Not a Number) para representá-los."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "dbc18f42",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    valor 1\n",
       "1    valor 2\n",
       "2        NaN\n",
       "3    valor 3\n",
       "dtype: object"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data = pd.Series(['valor 1', 'valor 2', np.nan, 'valor 3'])\n",
    "string_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6eef6ced",
   "metadata": {},
   "source": [
    "- isnull é o método que devolve valores booleanos informando quais valores estão ausentes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bfb66457",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2     True\n",
       "3    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.isnull()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ee4cad9",
   "metadata": {},
   "source": [
    "- notnull é a negação de isnull"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7f652611",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2    False\n",
       "3     True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.notnull()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75cc59b9",
   "metadata": {},
   "source": [
    "- No pandas, referenciamos dados ausentes como NA, que quer dizer Not Available (indisponível).\n",
    "- É importante fazer a análise nos próprios dados ausentes a fim de identificar problemas em sua coleta ou possíveis distorções provocadas por dados ausentes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c35e711c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       None\n",
       "1    valor 2\n",
       "2        NaN\n",
       "3    valor 3\n",
       "dtype: object"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data[0] = None\n",
    "string_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "38027e1e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1    False\n",
       "2     True\n",
       "3    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.isnull()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2abce026",
   "metadata": {},
   "source": [
    "- dropna é o método que filtra rótulos de eixos, baseado no fato de os valores para cada rótulo terem dados ausentes, com limites variados para a quantidade de dados ausentes a ser tolerado."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1fa7522e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    valor 2\n",
       "3    valor 3\n",
       "dtype: object"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.dropna()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82e91518",
   "metadata": {},
   "source": [
    "- drona é equivalente a:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "aaa1c57e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    valor 2\n",
       "3    valor 3\n",
       "dtype: object"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data[string_data.notnull()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "376ccff2",
   "metadata": {},
   "source": [
    "- fillna é o método que preenche os dados ausentes com algum valor ou utilizando um método de interpolação como 'ffill' ou 'bfill."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "35598b0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          0\n",
       "1    valor 2\n",
       "2          0\n",
       "3    valor 3\n",
       "dtype: object"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2f04af10",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       None\n",
       "1    valor 2\n",
       "2    valor 2\n",
       "3    valor 3\n",
       "dtype: object"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.fillna(method='ffill')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a7d1372e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    valor 2\n",
       "1    valor 2\n",
       "2    valor 3\n",
       "3    valor 3\n",
       "dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.fillna(method='bfill')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9e7d95e9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1     True\n",
       "2    False\n",
       "3     True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_data.notnull()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68978ab7",
   "metadata": {},
   "source": [
    "- Com objetos DataFrame, a situação é um pouco mais complexa em relação a exclusão de valores NA por linhas, colunas ou pelos dois."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b6196426",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1    2\n",
       "0  1.0  6.5  3.0\n",
       "1  1.0  NaN  NaN\n",
       "2  NaN  NaN  NaN\n",
       "3  NaN  6.5  3.0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.DataFrame([[1, 6.5, 3], [1, np.nan, np.nan], [np.nan, np.nan, np.nan], [np.nan, 6.5, 3]])\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "f2eeff74",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1    2\n",
       "0  1.0  6.5  3.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned = data.dropna()\n",
    "cleaned"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62def9a5",
   "metadata": {},
   "source": [
    "- Passar how='all' descartará apenas as linhas que contenham somente NAs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8f00d5c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1    2\n",
       "0  1.0  6.5  3.0\n",
       "1  1.0  NaN  NaN\n",
       "3  NaN  6.5  3.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.dropna(how='all')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd587c75",
   "metadata": {},
   "source": [
    "- Para descartar colunas do mesmo modo, passe axis='columns' ou axis=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "05f6cc74",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1    2   4\n",
       "0  1.0  6.5  3.0 NaN\n",
       "1  1.0  NaN  NaN NaN\n",
       "2  NaN  NaN  NaN NaN\n",
       "3  NaN  6.5  3.0 NaN"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[4] = np.NaN\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "b924987f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1    2\n",
       "0  1.0  6.5  3.0\n",
       "1  1.0  NaN  NaN\n",
       "2  NaN  NaN  NaN\n",
       "3  NaN  6.5  3.0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.dropna(how='all', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e33fc311",
   "metadata": {},
   "source": [
    "- Suponha que queremos manter somente as linhas contendo determinado número de observações. Podemos representar isso com o argumento thresh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "f85dc7e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1    2   4\n",
       "0  1.0  6.5  3.0 NaN\n",
       "3  NaN  6.5  3.0 NaN"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.dropna(thresh=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c14a755",
   "metadata": {},
   "source": [
    "### Removendo duplicatas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "5ec0d2b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>k1</th>\n",
       "      <th>k2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>one</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>two</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>one</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>two</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>one</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>two</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>two</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    k1  k2\n",
       "0  one   1\n",
       "1  two   1\n",
       "2  one   2\n",
       "3  two   3\n",
       "4  one   3\n",
       "5  two   4\n",
       "6  two   4"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.DataFrame({'k1': ['one', 'two'] * 3 + ['two'], 'k2': [1, 1, 2, 3, 3, 4, 4]})\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "91eb7e1c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "5    False\n",
       "6     True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.duplicated()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "de6aa357",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>k1</th>\n",
       "      <th>k2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>one</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>two</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>one</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>two</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>one</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>two</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    k1  k2\n",
       "0  one   1\n",
       "1  two   1\n",
       "2  one   2\n",
       "3  two   3\n",
       "4  one   3\n",
       "5  two   4"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7eacc44",
   "metadata": {},
   "source": [
    "### Substituindo valores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "37f24b5b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       1\n",
       "1    -999\n",
       "2       2\n",
       "3    -999\n",
       "4       3\n",
       "5   -1000\n",
       "6       4\n",
       "dtype: int64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.Series([1, -999, 2, -999, 3, -1000, 4])\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b13da678",
   "metadata": {},
   "source": [
    "- Supondo que os valores de -999 podem ser valores de sentinela para dados ausentes. Para substituí-los por valores NA, podemos usar replace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "25ee5976",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       1.0\n",
       "1       NaN\n",
       "2       2.0\n",
       "3       NaN\n",
       "4       3.0\n",
       "5   -1000.0\n",
       "6       4.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.replace(-999, np.nan)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d575c931",
   "metadata": {},
   "source": [
    "- Para substituir diversos, basta passar os valores numa lista"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "30b91b9b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1.0\n",
       "1    NaN\n",
       "2    2.0\n",
       "3    NaN\n",
       "4    3.0\n",
       "5    NaN\n",
       "6    4.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.replace([-999, -1000], np.nan)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5a51b63",
   "metadata": {},
   "source": [
    "- Para usar um substituto diferente para cada valor, passe uma lista deles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "954ecea8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1.0\n",
       "1    NaN\n",
       "2    2.0\n",
       "3    NaN\n",
       "4    3.0\n",
       "5    0.0\n",
       "6    4.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.replace([-999, -1000], [np.nan, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96ba10f2",
   "metadata": {},
   "source": [
    "### Detectando e filtrando valores discrepantes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10811734",
   "metadata": {},
   "source": [
    "- Filtrar ou transformar valores discrepantes (outliers) é, em boa medida, em uma questão de aplicar operações de array. Considere um DataFrame com alguns dados normalmente distribuídos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "76173029",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.130507</td>\n",
       "      <td>1.488263</td>\n",
       "      <td>-1.282070</td>\n",
       "      <td>-0.204792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.591077</td>\n",
       "      <td>-1.328322</td>\n",
       "      <td>0.420395</td>\n",
       "      <td>0.588469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.809890</td>\n",
       "      <td>-0.764165</td>\n",
       "      <td>-0.411809</td>\n",
       "      <td>-0.772804</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.670452</td>\n",
       "      <td>-0.587789</td>\n",
       "      <td>1.406915</td>\n",
       "      <td>1.026987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.641791</td>\n",
       "      <td>-0.416728</td>\n",
       "      <td>0.808329</td>\n",
       "      <td>0.908395</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>995</th>\n",
       "      <td>1.856219</td>\n",
       "      <td>-1.191524</td>\n",
       "      <td>-1.498318</td>\n",
       "      <td>-1.017301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>996</th>\n",
       "      <td>1.025834</td>\n",
       "      <td>1.417592</td>\n",
       "      <td>-2.333211</td>\n",
       "      <td>-0.194665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997</th>\n",
       "      <td>-0.559843</td>\n",
       "      <td>1.539932</td>\n",
       "      <td>-0.570945</td>\n",
       "      <td>-1.753647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>0.021811</td>\n",
       "      <td>0.704432</td>\n",
       "      <td>-0.534305</td>\n",
       "      <td>-0.871718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>999</th>\n",
       "      <td>0.061602</td>\n",
       "      <td>0.726847</td>\n",
       "      <td>-0.059413</td>\n",
       "      <td>-1.680357</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            0         1         2         3\n",
       "0    1.130507  1.488263 -1.282070 -0.204792\n",
       "1    0.591077 -1.328322  0.420395  0.588469\n",
       "2   -0.809890 -0.764165 -0.411809 -0.772804\n",
       "3   -0.670452 -0.587789  1.406915  1.026987\n",
       "4    0.641791 -0.416728  0.808329  0.908395\n",
       "..        ...       ...       ...       ...\n",
       "995  1.856219 -1.191524 -1.498318 -1.017301\n",
       "996  1.025834  1.417592 -2.333211 -0.194665\n",
       "997 -0.559843  1.539932 -0.570945 -1.753647\n",
       "998  0.021811  0.704432 -0.534305 -0.871718\n",
       "999  0.061602  0.726847 -0.059413 -1.680357\n",
       "\n",
       "[1000 rows x 4 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.DataFrame(np.random.randn(1000, 4))\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "da659e1d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-0.050407</td>\n",
       "      <td>0.028581</td>\n",
       "      <td>-0.025342</td>\n",
       "      <td>-0.005179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.006232</td>\n",
       "      <td>1.001140</td>\n",
       "      <td>1.050137</td>\n",
       "      <td>1.021996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-3.546045</td>\n",
       "      <td>-2.945722</td>\n",
       "      <td>-2.960806</td>\n",
       "      <td>-3.681865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-0.734890</td>\n",
       "      <td>-0.668024</td>\n",
       "      <td>-0.752436</td>\n",
       "      <td>-0.721785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-0.023592</td>\n",
       "      <td>-0.003778</td>\n",
       "      <td>-0.008145</td>\n",
       "      <td>-0.067217</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.687360</td>\n",
       "      <td>0.726005</td>\n",
       "      <td>0.679199</td>\n",
       "      <td>0.695205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.908475</td>\n",
       "      <td>3.608226</td>\n",
       "      <td>2.788453</td>\n",
       "      <td>3.131224</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 0            1            2            3\n",
       "count  1000.000000  1000.000000  1000.000000  1000.000000\n",
       "mean     -0.050407     0.028581    -0.025342    -0.005179\n",
       "std       1.006232     1.001140     1.050137     1.021996\n",
       "min      -3.546045    -2.945722    -2.960806    -3.681865\n",
       "25%      -0.734890    -0.668024    -0.752436    -0.721785\n",
       "50%      -0.023592    -0.003778    -0.008145    -0.067217\n",
       "75%       0.687360     0.726005     0.679199     0.695205\n",
       "max       2.908475     3.608226     2.788453     3.131224"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d948a00d",
   "metadata": {},
   "source": [
    "- Suponha que quiséssemos encontrar os valores que excedessem 3 em valor absoluto em uma das colunas (no nosso caso, coluna 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "19426317",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Series([], Name: 2, dtype: float64)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col = data[2]\n",
    "col[np.abs(col)>3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fd8ccad",
   "metadata": {},
   "source": [
    "- Para selecionar todas as linhas que tenham um valor que exceda 3 ou -3, podemos utilizat o método any um DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "80bc58e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>300</th>\n",
       "      <td>-3.079510</td>\n",
       "      <td>-0.650906</td>\n",
       "      <td>0.172465</td>\n",
       "      <td>-1.086944</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>363</th>\n",
       "      <td>-0.964514</td>\n",
       "      <td>0.518750</td>\n",
       "      <td>1.822537</td>\n",
       "      <td>-3.681865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>0.689586</td>\n",
       "      <td>-0.014387</td>\n",
       "      <td>0.023096</td>\n",
       "      <td>3.131224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>677</th>\n",
       "      <td>-3.546045</td>\n",
       "      <td>-0.823953</td>\n",
       "      <td>0.011040</td>\n",
       "      <td>-0.638146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>696</th>\n",
       "      <td>-3.221891</td>\n",
       "      <td>-1.080812</td>\n",
       "      <td>-0.259609</td>\n",
       "      <td>0.666280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>797</th>\n",
       "      <td>-3.517268</td>\n",
       "      <td>-0.604442</td>\n",
       "      <td>1.326434</td>\n",
       "      <td>-0.159540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>985</th>\n",
       "      <td>0.750984</td>\n",
       "      <td>3.340691</td>\n",
       "      <td>2.100132</td>\n",
       "      <td>0.500332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>989</th>\n",
       "      <td>0.789814</td>\n",
       "      <td>3.608226</td>\n",
       "      <td>1.204871</td>\n",
       "      <td>0.832472</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            0         1         2         3\n",
       "300 -3.079510 -0.650906  0.172465 -1.086944\n",
       "363 -0.964514  0.518750  1.822537 -3.681865\n",
       "564  0.689586 -0.014387  0.023096  3.131224\n",
       "677 -3.546045 -0.823953  0.011040 -0.638146\n",
       "696 -3.221891 -1.080812 -0.259609  0.666280\n",
       "797 -3.517268 -0.604442  1.326434 -0.159540\n",
       "985  0.750984  3.340691  2.100132  0.500332\n",
       "989  0.789814  3.608226  1.204871  0.832472"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[(np.abs(data) > 3).any(1)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b69d5fb",
   "metadata": {},
   "source": [
    "- Para eliminar os valores que estejam fora do intervalo de -3 a 3 substituido-os por -3 ou 3 respectivamente"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "3ab9458d",
   "metadata": {},
   "outputs": [],
   "source": [
    "data[np.abs(data)>3] = np.sign(data) * 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "3335d939",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-0.049042</td>\n",
       "      <td>0.027632</td>\n",
       "      <td>-0.025342</td>\n",
       "      <td>-0.004628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.001908</td>\n",
       "      <td>0.998073</td>\n",
       "      <td>1.050137</td>\n",
       "      <td>1.019370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-3.000000</td>\n",
       "      <td>-2.945722</td>\n",
       "      <td>-2.960806</td>\n",
       "      <td>-3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-0.734890</td>\n",
       "      <td>-0.668024</td>\n",
       "      <td>-0.752436</td>\n",
       "      <td>-0.721785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-0.023592</td>\n",
       "      <td>-0.003778</td>\n",
       "      <td>-0.008145</td>\n",
       "      <td>-0.067217</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.687360</td>\n",
       "      <td>0.726005</td>\n",
       "      <td>0.679199</td>\n",
       "      <td>0.695205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.908475</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>2.788453</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 0            1            2            3\n",
       "count  1000.000000  1000.000000  1000.000000  1000.000000\n",
       "mean     -0.049042     0.027632    -0.025342    -0.004628\n",
       "std       1.001908     0.998073     1.050137     1.019370\n",
       "min      -3.000000    -2.945722    -2.960806    -3.000000\n",
       "25%      -0.734890    -0.668024    -0.752436    -0.721785\n",
       "50%      -0.023592    -0.003778    -0.008145    -0.067217\n",
       "75%       0.687360     0.726005     0.679199     0.695205\n",
       "max       2.908475     3.000000     2.788453     3.000000"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d1347ac",
   "metadata": {},
   "source": [
    "### Manipulação de Strings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c6af873",
   "metadata": {},
   "source": [
    "- Uma string separada por vírgulas pode ser dividida em partes usando split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "f2522965",
   "metadata": {},
   "outputs": [],
   "source": [
    "val = 'a,b, teste'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "70d2e7e9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', 'b', ' teste']"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val.split(',')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ed59c7a",
   "metadata": {},
   "source": [
    "- Com frequência, split é usado em conjunto com strip para remover espaços em branco (incluindo quebra de linha)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "135471b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', 'b', 'teste']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pieces = [x.strip() for x in val.split(',')]\n",
    "pieces"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "695a5d17",
   "metadata": {},
   "source": [
    "- Replace é usado para substituir as ocorrências de um padrão por outro. É comumente utilizado também para apagar padrões passando uma string vazia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "52e22bea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'a::b:: teste'"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val.replace(',', '::')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "44a943bf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'ab teste'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val.replace(',','')"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}