-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
452fc8c
commit 155475a
Showing
1 changed file
with
28 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
Extracting, Loading, and Transforming Data Really Efficiently with Laravel | ||
--- | ||
By [@thecodeboss](https://thecodeboss.dev) | ||
|
||
|
||
Extract-Transform-Load (ETL) processes have been around for a long time in programming, and they're not too challenging to understand: you take data from one or many sources (legacy DB, SAP, third-party data lake, spreadsheets, etc.), transform it to fit your own application's workflows, and then load it into your own data sources. You can use ETL processes for syncing data across sources, migrating legacy data, aggregating multiple data sources, and more. | ||
|
||
As you might expect, ETL processes involve a lot of data, and it's up to you to ensure that you're building a nice, lean ETL process. No one wants an ETL process that takes hours to run if it's possible to run it in minutes. In this live-demo talk, we'll use Laravel to implement an ETL process where we start with a functioning yet inefficient scenario and step-by-step turn it into a lean, mean ETL process. No prior Laravel knowledge is necessary, just an open mind. We'll review writing efficient database queries, architecting your code to avoid common inefficient pitfalls, and more. | ||
|
||
Aaron Krauss is a senior software engineer at Clevyr where he works with an awesome team building web apps, mobile apps, games, and more. Off work hours, he and his wife just welcomed their first-born son into the world which is taking up all his free time! | ||
|
||
## To Run | ||
``` | ||
cd etl-demo | ||
# Setup the project | ||
cp .env.example .env | ||
composer install | ||
# Run it | ||
docker-compose up | ||
# To run the commands | ||
docker-compose exec app php artisan step1 | ||
docker-compose exec app php artisan step2 | ||
... | ||
docker-compose exec app php artisan step9 | ||
``` |