Skip to content

Annaqin0929/Real-Time-Web-Data-Analysis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 

Repository files navigation

Real Time Web Data Analysis

Project Objective: Automated data scrape and analysis script using Python (Pycharm)

The objective of this project is to develop an automated program using Python that collects the latest rental information on the Trademe platform in Christchurch, New Zealand.

The Python script is designed to significantly save users' time and effort, enabling them to efficiently overview and compare rental options in different areas of Christchurch and help them to understand the market situation and then find the best choice for themselves.

The main functions of this Python program include getting data through web scraping from Trademe, structured data cleaning and storage, and statistical analysis. This Python script is developed in PyCharm (Community Version) and a lot of popular Python packages (requests, random, re, lxml, pandas, etc.) are used. The data visualization contains Box plots (median, quartiles, maximum, minimum), bar charts, pie charts, and the heatmap for rental property.

The integration of Google API and Static Maps allows users to view rental property locations on a static map, enhancing users’ overall rental market exploration experience.

Project Phase

Phase I: Analyze the website URL, the path of key info, and the main python packages that need to be imported;

Phase II: Build the function/code framework and extract info from the website;

Phase III: Check data sets and proceed with functions of data wrangling;

Phase IV: Visualize the data, and output the analysis results;

Phase V: Generate project report

Releases

No releases published

Packages

No packages published

Languages