Skip to content

Commit

Permalink
Merge branch 'louis-e:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
amir16yp authored Aug 16, 2024
2 parents 70f229f + 6cebe5a commit 6bc1fd3
Show file tree
Hide file tree
Showing 10 changed files with 1,363 additions and 946 deletions.
4 changes: 2 additions & 2 deletions .flake8
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[flake8]
ignore = E203, W503
ignore = E203, W503, F405, F403
# line length is intentionally set to 80 here because black uses Bugbear
# See https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length for more details
max-line-length = 95
max-line-length = 100
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is and what you expected to happen.

**Used bbox parameter**
Please provide your input parameters so we can reproduce the issue.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Additional context**
Add any other context about the problem here.
78 changes: 28 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,29 +15,31 @@ This open source project generates any chosen location from the real world in Mi

The raw data obtained from the API *[(see FAQ)](#question-faq)* includes each element (buildings, walls, fountains, farmlands, etc.) with its respective corner coordinates (nodes) and descriptive tags. When you run the script, the following steps are performed automatically to generate a Minecraft world:

#### Processing Pipeline
1. Scraping Data from API: The script fetches geospatial data from the Overpass Turbo API.
2. Determine Coordinate Extremes: Identifies the highest and lowest latitude and longitude values from the dataset.
3. Standardize Coordinate Lengths: Ensures all coordinates are of uniform length and removes the decimal separator.
4. Normalize Data: Adjusts all coordinates to start from zero by subtracting the previously determined lowest values.
5. Parse Data: Transforms the raw data into a standardized structure.
6. Assign IDs to Elements: Iterates through each element and assigns a corresponding ID *[(see FAQ)](#question-faq)* at each coordinate in an array.
6. Sort elements by priority: Enables a layering system with prioritized elements.
7. Optimize Array Size: Focuses on the outermost buildings to reduce array size.
8. Integrate Landuse and Additional Data: Combines the landuse layer with other generated data.
9. Generate Minecraft World: Iterates through the array to create the Minecraft world, including 3D structures like forests, houses, and cemetery graves.
8. Generate Minecraft World: Iterates through the array to create the Minecraft world, including 3D structures like forests, houses, and rivers.

## :keyboard: Usage
```python3 arnis.py --bbox="min_lng,min_lat,max_lng,max_lat" --path="C:/Users/username/AppData/Roaming/.minecraft/saves/worldname"```

Use http://bboxfinder.com/ to draw a rectangle of your wanted area. Then copy the four box coordinates as shown below and use them as the input for the --bbox parameter.
Use http://bboxfinder.com/ to draw a rectangle of your wanted area. Then copy the four box coordinates as shown below and use them as the input for the --bbox parameter.
![How to find area](https://github.com/louis-e/arnis/blob/main/gitassets/bbox-finder.png?raw=true)
The world will always be generated starting from the coordinates 0 0 0.

Manually generate a new Minecraft world (preferably a flat world) before running the script.
The --bbox parameter specifies the bounding box coordinates in the format: min_lng,min_lat,max_lng,max_lat.
Use --path to specify the location of the Minecraft world.
With the --timeout parameter you can set the timeout for the floodfill algorithm in seconds (default: 2).
You can optionally use the parameter --debug to see processed value outputs during runtime.

#### Experimental City/State/Country Input Method
The following method is experimental and may not perform as expected:
The following method is experimental and may not perform as expected. Support is limited.

```python3 arnis.py --city="CityName" --state="StateName" --country="CountryName" --path="C:/Users/username/AppData/Roaming/.minecraft/saves/worldname"```

Expand All @@ -62,55 +64,35 @@ docker cp CONTAINER_ID:/home/region DESTINATION_PATH

## :question: FAQ
- *Why do some cities take so long to generate?*<br>
The script's performance can be significantly affected by large elements, such as extensive farmlands. The current floodfill algorithm can slow down considerably when dealing with such elements, leading to long processing times that can stretch to hours for large cities. Thus there is also a timeout restriction in place. It is recommended to start with smaller towns to get a sense of the script's performance. Future improvements will focus on making the floodfill algorithm multi-threaded to utilize multiple CPU cores and reduce processing times drastically.
The script's performance can be significantly affected by large elements, such as extensive farmlands. The floodfill algorithm can slow down considerably when dealing with such elements, leading to long processing times. Thus there is also a timeout restriction in place, which can be adjusted by the user *[(see Usage)](#keyboard-usage)*. It is recommended to start with smaller areas to get a sense of the script's performance. Continuous improvements on the algorithm especially focus on effiency improvements.
- *Where does the data come from?*<br>
The geographic data is sourced from OpenStreetMap (OSM)[^1], a free, collaborative mapping project that serves as an open-source alternative to commercial mapping services. The data is accessed via the Overpass API, which queries OSM's database.
- *How does the Minecraft world generation work?*<br>
The script uses the [anvil-parser](https://github.com/matcool/anvil-parser) library to interact with Minecraft's world format. This library allows the script to create and manipulate Minecraft region files, enabling the generation of real-world locations within the game.
- *Where does the name come from?*<br>
The project is named after Arnis[^2], the smallest city in Germany. The city's small size made it an ideal test case for developing and debugging the script efficiently.
- *What are the corresponding IDs?*<br>
During the processing stage (*[(see How it works)](#floppy_disk-how-it-works)*), each element is assigned an ID that determines how it will be represented in Minecraft. This system ensures that different layers (e.g., buildings, roads, landuse) are rendered correctly without conflicts.

ID | Name | Note |
--- | --- | --- |
0 | Ground | |
10 | Street | |
11 | Footway | |
12 | Natural path | |
13 | Bridge | |
14 | Railway | |
19 | Street markings | Work in progress *[(see FAQ)](#question-faq)* |
20 | Parking | |
21 | Fountain border | |
22 | Fence | |
30 | Meadow | |
31 | Farmland | |
32 | Forest | |
33 | Cemetery | |
34 | Beach | |
35 | Wetland | |
36 | Pitch | |
37 | Swimming pool | |
38 | Water | |
39 | Raw grass | |
50-59 | House corner | The last digit refers to the building height |
60-69 | House wall | The last digit refers to the building height |
70-79 | House interior | The last digit refers to the building height |

## :memo: ToDo
Feel free to choose an item from the To-Do or Known Bugs list, or bring your own idea to the table. Contributions from everyone are welcome and encouraged to help improve this project.
- [x] Alternative reliable city input options[^3]
- [ ] Split up processData array into several smaller ones for big cities[^4]
- [ ] Floodfill timeout parameters
- [ ] Implement multiprocessing in floodfill algorithm in order to boost CPU bound calculation performance
- [ ] Generate a few big cities using high performance hardware and make them available to download[^3]
- [ ] Add code comments
- [ ] Look into https://github.com/Intergalactyc/anvil-new which seems to have a better support
- [ ] Tool for mapping real coordinates to Minecraft coordinates
- [ ] Fix railway orientation
- [ ] Fix gaps in bridges
- [ ] Full refactoring of variable and function names, establish naming conventions
- [ ] Detection of wrong bbox input
- [ ] Evaluate and implement multiprocessing in the ground layer initialization and floodfill algorithm
- [ ] Implement elevation
- [ ] Find alternative for CV2 package
- [ ] Add interior to buildings
- [ ] Save fountain structure in the code (similar to the tree structure)
- [ ] Add windows to buildings
- [ ] Generate a few big cities using high performance hardware and make them available to download
- [ ] Optimize region file size
- [ ] Street markings
- [ ] Add better code comments
- [x] Alternative reliable city input options
- [x] Split up processData array into several smaller ones for big cities
- [x] Find alternative for CV2 package
- [x] Floodfill timeout parameter
- [x] Automated Tests
- [x] PEP8
- [x] Use f-Strings in print statements
Expand All @@ -119,10 +101,10 @@ Feel free to choose an item from the To-Do or Known Bugs list, or bring your own
- [x] Improve RAM usage

## :bug: Known Bugs
- [ ] 'Noer' bug (occurs when several different digits appear in coordinates before the decimal point)
- [x] 'Nortorf' bug (occurs when there are several elements with a big distance to each other, e.g. the API returns several different cities with the exact same name)
- [ ] Saving step memory overflow
- [ ] Docker image size
- [x] 'Noer' bug (occurs when several different digits appear in coordinates before the decimal point)
- [x] 'Nortorf' bug (occurs when there are several elements with a big distance to each other, e.g. the API returns several different cities with the exact same name)
- [x] Saving step memory overflow
- [x] Non uniform OSM naming standards (dashes) (See name tags at https://overpass-turbo.eu/s/1mMj)

## :trophy: Hall of Fame Contributors
Expand All @@ -139,19 +121,15 @@ This section is dedicated to recognizing and celebrating the outstanding contrib
[![Star History Chart](https://api.star-history.com/svg?repos=louis-e/arnis&type=Date)](https://star-history.com/#louis-e/arnis&Date)

## :copyright: License
MIT License[^5]
MIT License[^3]

Copyright (c) 2022 louis-e

[^1]: https://en.wikipedia.org/wiki/OpenStreetMap

[^2]: https://en.wikipedia.org/wiki/Arnis,_Germany

[^3]: This might require a complete redesign of the algorithm since the current version is based on the city / state / country input. This will be investigated further soon.

[^4]: https://github.com/louis-e/arnis/issues/21#issuecomment-1785801652

[^5]:
[^3]:
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice, the author ("louis-e") and this permission notice shall be included in all copies or substantial portions of the Software.
Expand Down
1 change: 0 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
anvil-parser==0.9.0
matplotlib==3.9.0
numpy==1.26.4
opencv-python==4.9.0.80
pytest==8.2.1
python-polylabel==0.6
requests==2.32.2
Expand Down
95 changes: 95 additions & 0 deletions src/blockDefinitions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import anvil

air = anvil.Block("minecraft", "air")
birch_leaves = anvil.Block("minecraft", "birch_leaves")
birch_log = anvil.Block("minecraft", "birch_log")
black_concrete = anvil.Block("minecraft", "black_concrete")
blue_flower = anvil.Block("minecraft", "blue_orchid")
brick = anvil.Block("minecraft", "bricks")
carrots = anvil.Block("minecraft", "carrots", {"age": 7})
cauldron = anvil.Block.from_numeric_id(118, 0)
cobblestone = anvil.Block("minecraft", "cobblestone")
cobblestone_wall = anvil.Block("minecraft", "cobblestone_wall")
dark_oak_door_lower = anvil.Block("minecraft", "dark_oak_door", {"half": "lower"})
dark_oak_door_upper = anvil.Block("minecraft", "dark_oak_door", {"half": "upper"})
dirt = anvil.Block("minecraft", "dirt")
farmland = anvil.Block("minecraft", "farmland")
glass = anvil.Block("minecraft", "glass_pane")
glowstone = anvil.Block("minecraft", "glowstone")
grass = anvil.Block.from_numeric_id(175, 2)
grass_block = anvil.Block("minecraft", "grass_block")
gravel = anvil.Block("minecraft", "gravel")
gray_concrete = anvil.Block("minecraft", "gray_concrete")
green_stained_hardened_clay = anvil.Block.from_numeric_id(159, 5)
hay_bale = anvil.Block("minecraft", "hay_block")
iron_block = anvil.Block("minecraft", "iron_block")
light_gray_concrete = anvil.Block("minecraft", "light_gray_concrete")
oak_fence = anvil.Block.from_numeric_id(85, 0)
oak_leaves = anvil.Block("minecraft", "oak_leaves")
oak_log = anvil.Block.from_numeric_id(17)
oak_planks = anvil.Block("minecraft", "oak_planks")
podzol = anvil.Block.from_numeric_id(3, 2)
potatoes = anvil.Block("minecraft", "potatoes", {"age": 7})
rail = anvil.Block("minecraft", "rail")
red_flower = anvil.Block.from_numeric_id(38)
sand = anvil.Block("minecraft", "sand")
scaffolding = anvil.Block("minecraft", "scaffolding")
sponge = anvil.Block("minecraft", "sponge")
spruce_log = anvil.Block("minecraft", "spruce_log")
stone = anvil.Block("minecraft", "stone")
stone_block_slab = anvil.Block.from_numeric_id(44, 0)
stone_brick_slab = anvil.Block.from_numeric_id(44, 5)
water = anvil.Block("minecraft", "water")
wheat = anvil.Block("minecraft", "wheat", {"age": 7})
white_concrete = anvil.Block("minecraft", "white_concrete")
white_flower = anvil.Block("minecraft", "azure_bluet")
white_stained_glass = anvil.Block("minecraft", "white_stained_glass")
yellow_flower = anvil.Block("minecraft", "dandelion")

# Variations for building corners
building_corner_variations = [
anvil.Block("minecraft", "stone_bricks"),
anvil.Block("minecraft", "cobblestone"),
anvil.Block("minecraft", "bricks"),
anvil.Block("minecraft", "mossy_cobblestone"),
anvil.Block("minecraft", "sandstone"),
anvil.Block("minecraft", "red_nether_bricks"),
anvil.Block("minecraft", "blackstone"),
anvil.Block("minecraft", "smooth_quartz"),
anvil.Block("minecraft", "chiseled_stone_bricks"),
anvil.Block("minecraft", "polished_basalt"),
anvil.Block("minecraft", "cut_sandstone"),
anvil.Block("minecraft", "polished_blackstone_bricks"),
]

# Variations for building walls
building_wall_variations = [
anvil.Block("minecraft", "white_terracotta"),
anvil.Block("minecraft", "gray_terracotta"),
anvil.Block("minecraft", "bricks"),
anvil.Block("minecraft", "smooth_sandstone"),
anvil.Block("minecraft", "red_terracotta"),
anvil.Block("minecraft", "polished_diorite"),
anvil.Block("minecraft", "smooth_stone"),
anvil.Block("minecraft", "polished_andesite"),
anvil.Block("minecraft", "warped_planks"),
anvil.Block("minecraft", "end_stone_bricks"),
anvil.Block("minecraft", "smooth_red_sandstone"),
anvil.Block("minecraft", "nether_bricks"),
]

# Variations for building floors
building_floor_variations = [
anvil.Block("minecraft", "oak_planks"),
anvil.Block("minecraft", "spruce_planks"),
anvil.Block("minecraft", "dark_oak_planks"),
anvil.Block("minecraft", "stone_bricks"),
anvil.Block("minecraft", "polished_granite"),
anvil.Block("minecraft", "polished_diorite"),
anvil.Block("minecraft", "acacia_planks"),
anvil.Block("minecraft", "jungle_planks"),
anvil.Block("minecraft", "warped_planks"),
anvil.Block("minecraft", "purpur_block"),
anvil.Block("minecraft", "smooth_red_sandstone"),
anvil.Block("minecraft", "polished_blackstone"),
]
108 changes: 0 additions & 108 deletions src/floodFill.py

This file was deleted.

Loading

0 comments on commit 6bc1fd3

Please sign in to comment.