mirror
/
planetiler
kopia lustrzana https://github.com/onthegomap/planetiler
1
0
Forkuj 0
Kod Zgłoszenia Projekty Wydania Wiki Aktywność

Move basemap profile into OpenMapTiles submodule (#258)

pull/310/head^2
Adam Laža 2022-07-29 12:40:15 +02:00 zatwierdzone przez GitHub
rodzic 232834e735
commit 88c7f5650f
Nie znaleziono w bazie danych klucza dla tego podpisu
ID klucza GPG: 4AEE18F83AFDEB23
75 zmienionych plików z 352 dodań i 14329 usunięć
Pokaż statystyki Ściągnij plik aktualizacji Ściągnij plik porównania Expand all files Collapse all files

2
.github/cache-sources-action/action.yml vendored
Wyświetl plik

@ -12,7 +12,7 @@ runs:
- name: Get Date
id: get-data
run: |
echo "::set-output name=hash::${{ hashFiles('**/Planetiler.java', '**/Downloader.java', '**/Geofabrik.java', '**/BasemapMain.java') }}"
echo "::set-output name=hash::${{ hashFiles('**/Planetiler.java', '**/Downloader.java', '**/Geofabrik.java', '**/OpenMapTilesMain.java') }}"
echo "::set-output name=date::$(date -u "+%Y-%m-%d")"
shell: bash
working-directory: ${{ inputs.basedir }}

2
.github/workflows/docs.yml vendored
Wyświetl plik

@ -13,6 +13,8 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v3
with:
submodules: true
- name: Run link check
uses: gaurav-nelson/github-action-markdown-link-check@1.0.13
with:

14
.github/workflows/maven.yml vendored
Wyświetl plik

@ -16,6 +16,8 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
@ -45,6 +47,8 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Set up JDK ${{ matrix.jdk }}
uses: actions/setup-java@v3
with:
@ -66,6 +70,8 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
@ -74,8 +80,8 @@ jobs:
cache: 'maven'
- run: ./scripts/regenerate-openmaptiles.sh
# Skip spotless since that gets checked in a separate task
- run: ./mvnw -Dspotless.check.skip -DskipTests --batch-mode -no-transfer-progress clean install -pl planetiler-basemap -am
- run: ./mvnw -Dspotless.check.skip --batch-mode -no-transfer-progress verify -pl planetiler-basemap
- run: ./mvnw -Dspotless.check.skip -DskipTests --batch-mode -no-transfer-progress clean install -pl planetiler-openmaptiles -am
- run: ./mvnw -Dspotless.check.skip --batch-mode -no-transfer-progress verify -pl planetiler-openmaptiles
examples:
name: Example project
@ -84,6 +90,8 @@ jobs:
continue-on-error: true
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
@ -108,6 +116,8 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Cache data/sources
uses: ./.github/cache-sources-action
- name: Set up JDK

2
.github/workflows/performance.yml vendored
Wyświetl plik

@ -30,11 +30,13 @@ jobs:
uses: actions/checkout@v3
with:
path: branch
submodules: true
- name: 'Checkout base'
uses: actions/checkout@v3
with:
path: base
ref: ${{ github.event.pull_request.base.sha }}
submodules: true
- name: Cache data/sources
uses: ./branch/.github/cache-sources-action
with:

2
.github/workflows/release.yml vendored
Wyświetl plik

@ -26,6 +26,8 @@ jobs:
version = context.payload.inputs.version;
if (/^v/.test(version)) throw new Error("Bad version number: " + version)
- uses: actions/checkout@v3
with:
submodules: true
- name: Cache data/sources
uses: ./.github/cache-sources-action
- uses: actions/setup-java@v3

2
.github/workflows/snapshot.yml vendored
Wyświetl plik

@ -22,6 +22,8 @@ jobs:
packages: write
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Cache data/sources
uses: ./.github/cache-sources-action
- name: Set up JDK

1
.github/workflows/sonar.yml vendored
Wyświetl plik

@ -19,6 +19,7 @@ jobs:
with:
# Disabling shallow clone is recommended for improving relevancy of reporting
fetch-depth: 0
submodules: true
- name: Set up JDK 17
uses: actions/setup-java@v3
with:

2
.github/workflows/update-pr.yml vendored
Wyświetl plik

@ -19,6 +19,8 @@ jobs:
# report status back to pull request
- uses: haya14busa/action-workflow_run-status@v1
- uses: actions/checkout@v3
with:
submodules: true
- name: 'Download branch build info'
uses: dawidd6/action-download-artifact@v2
with:

4
.gitmodules vendored 100644
Wyświetl plik

@ -0,0 +1,4 @@
[submodule "planetiler-openmaptiles"]
path = planetiler-openmaptiles
url = https://github.com/openmaptiles/planetiler-openmaptiles.git
branch = main

2
CONTRIBUTING.md
Wyświetl plik

@ -10,7 +10,7 @@ Pull requests are welcome! Any pull request should:
To set up your local development environment:
- Fork the repo
- Fork the repo [setup submodules](README.md#git-submodules)
- Install Java 16 or later. You can download Java manually from [Adoptium](https://adoptium.net/installation.html) or
use:
- [Windows installer](https://adoptium.net/installation.html#windows-msi)

3
NOTICE.md
Wyświetl plik

@ -39,9 +39,6 @@ The `planetiler-core` module includes the following software:
from [github.com/rawrunprotected/hilbert_curves](https://github.com/rawrunprotected/hilbert_curves) (Public Domain)
- `osmformat.proto` and `fileformat.proto` (generates `Osmformat.java` and `Fileformat.java`)
from [openstreetmap/OSM-binary](https://github.com/openstreetmap/OSM-binary/tree/master/osmpbf) (MIT License)
Additionally, the `planetiler-basemap` module is based on [OpenMapTiles](https://github.com/openmaptiles/openmaptiles):
- Maven Dependencies:
- org.yaml:snakeyaml (Apache license)
- org.commonmark:commonmark (BSD 2-clause license)

5
PLANET.md
Wyświetl plik

@ -1,6 +1,7 @@
# Generating a Map of the World
To generate a map of the world using the built-in [basemap profile](planetiler-basemap), you will need a machine with
To generate a map of the world using the built-in [OpenMapTiles profile](https://github.com/openmaptiles/planetiler-openmaptiles), you will need a
machine with
Java 16 or later installed and at least 10x as much disk space and at least 0.5x as much RAM as the `planet.osm.pbf`
file you start from. All testing has been done using Digital Ocean droplets with dedicated
vCPUs ([referral link](https://m.do.co/c/a947e99aab25)) and OpenJDK 17 installed through `apt`. Planetiler splits work
@ -66,7 +67,7 @@ java -Xmx20g \
Run with `--help` to see all available arguments.
NOTE: The default basemap profile merges nearby buildings at zoom-level 13 (for example,
NOTE: The default OpenMapTiles profile merges nearby buildings at zoom-level 13 (for example,
see [Boston](https://onthegomap.github.io/planetiler-demo/#13.08/42.35474/-71.06597)). This adds about 14 CPU hours (~50
minutes with 16 CPUs) to planet generation time and can be disabled using `--building-merge-z13=false`.

47
README.md
Wyświetl plik

@ -25,12 +25,12 @@ See the [live demo](https://onthegomap.github.io/planetiler-demo/) of vector til
the [OpenStreetMap Americana Project](https://github.com/ZeLonewolf/openstreetmap-americana/).
[![Planetiler Demo Screenshot](./diagrams/demo.png)](https://onthegomap.github.io/planetiler-demo/)
Style [© OpenMapTiles](https://www.openmaptiles.org/)
· Data [© OpenStreetMap contributors](https://www.openstreetmap.org/copyright)
[© OpenMapTiles](https://www.openmaptiles.org/) [© OpenStreetMap contributors](https://www.openstreetmap.org/copyright)
## Usage
To generate a map of an area using the [basemap profile](planetiler-basemap), you will need:
To generate a map of an area using the [OpenMapTiles profile](https://github.com/openmaptiles/planetiler-openmaptiles),
you will need:
- Java 16+ (see [CONTRIBUTING.md](CONTRIBUTING.md)) or [Docker](https://docs.docker.com/get-docker/)
- at least 1GB of free disk space plus 5-10x the size of the `.osm.pbf` file
@ -53,7 +53,8 @@ Or using Docker:
docker run -e JAVA_TOOL_OPTIONS="-Xmx1g" -v "$(pwd)/data":/data ghcr.io/onthegomap/planetiler:latest --download --area=monaco
```
:warning: This starts off by downloading about 1GB of [data sources](NOTICE.md#data) required by the basemap profile
:warning: This starts off by downloading about 1GB of [data sources](NOTICE.md#data) required by the OpenMapTiles
profile
including ~750MB for [ocean polygons](https://osmdata.openstreetmap.de/data/water-polygons.html) and ~240MB
for [Natural Earth Data](https://www.naturalearthdata.com/).
@ -107,6 +108,32 @@ Some common arguments:
- `--force` overwrites the output file
- `--help` shows all of the options and exits
### Git submodules
Planetiler has a submodule dependency
on [planetiler-openmaptiles](https://github.com/openmaptiles/planetiler-openmaptiles). Add `--recurse-submodules`
to `git clone`, `git pull`, or `git checkout` commands to also update submodule dependencies.
To clone the repo with submodules:
```bash
git clone --recurse-submodules https://github.com/onthegomap/planetiler.git
```
If you already pulled the repo, you can initialize submodules with:
```bash
git submodule update --init
```
To force git to always update submodules (recommended), run this command in your local repo:
```bash
git config --local submodule.recurse true
```
Learn more about working with submodules [here](https://git-scm.com/book/en/v2/Git-Tools-Submodules).
## Generating a Map of the World
See [PLANET.md](PLANET.md).
@ -117,7 +144,7 @@ See the [planetiler-examples](planetiler-examples) project.
## Benchmarks
Some example runtimes for the Basemap OpenMapTiles-compatible profile (excluding downloading resources):
Some example runtimes for the OpenMapTiles profile (excluding downloading resources):
| Input | Version | Machine | Time | mbtiles size | Logs |
|-------------------------------------------------------------------------------------------------------------------------------------------|---------|---------------------------------|---------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------|
@ -142,8 +169,9 @@ Merging nearby buildings at z13 is very expensive, when run with `--building-mer
Some other tools that generate vector tiles from OpenStreetMap data:
- [OpenMapTiles](https://github.com/openmaptiles/openmaptiles) is the reference implementation of
the [OpenMapTiles schema](https://openmaptiles.org/schema/) that the [basemap profile](planetiler-basemap) is based
on. It uses an intermediate postgres database and operates in two modes:
the [OpenMapTiles schema](https://openmaptiles.org/schema/) that
the [OpenMapTiles profile](https://github.com/openmaptiles/planetiler-openmaptiles)
is based on. It uses an intermediate postgres database and operates in two modes:
1. Import data into database (~1 day) then serve vector tiles directly from the database. Tile serving is slower and
requires bigger machines, but lets you easily incorporate realtime updates
2. Import data into database (~1 day) then pregenerate every tile for the planet into an mbtiles file which
@ -172,7 +200,7 @@ download regularly-updated tilesets.
using [JTS geometry utilities](https://github.com/locationtech/jts)
- Merge nearby lines or polygons with the same tags before emitting vector tiles
- Automatically fixes self-intersecting polygons
- Built-in basemap profile based on [OpenMapTiles](https://openmaptiles.org/) v3.13.1
- Built-in OpenMapTiles profile based on [OpenMapTiles](https://openmaptiles.org/) v3.13.1
- Optionally download additional name translations for elements from Wikidata
- Export real-time stats to a [prometheus push gateway](https://github.com/prometheus/pushgateway) using
`--pushgateway=http://user:password@ip` argument (and a [grafana dashboard](grafana.json) for viewing)
@ -244,7 +272,8 @@ Planetiler is made possible by these awesome open source projects:
- [OpenMapTiles](https://openmaptiles.org/) for the [schema](https://openmaptiles.org/schema/)
and [reference implementation](https://github.com/openmaptiles/openmaptiles)
that the [basemap profile](planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers)
that
the [openmaptiles profile](https://github.com/openmaptiles/planetiler-openmaptiles/tree/main/src/main/java/com/onthegomap/planetiler/openmaptiles/layers)
is based on
- [Graphhopper](https://www.graphhopper.com/) for basis of utilities to process OpenStreetMap data in Java
- [JTS Topology Suite](https://github.com/locationtech/jts) for working with vector geometries

2
config-example.properties
Wyświetl plik

@ -80,7 +80,7 @@
# mbtiles_version=0.1
# mbtiles_type=baselayer OR overlay
################# Basemap profile config #################
################# OpenMapTiles profile config #################
# Set the input file name and automatically determine the https://download.geofabrik.de/ download URL for a region by name:
# area=monaco

67
planetiler-basemap/README.md
Wyświetl plik

@ -1,67 +0,0 @@
# Planetiler Basemap Profile
This basemap profile is based on [OpenMapTiles](https://github.com/openmaptiles/openmaptiles) v3.13.1.
See [README.md](../README.md) in the parent directory for instructions on how to run.
## Differences from OpenMapTiles
- Road name abbreviations are not implemented yet in the `transportation_name` layer
- `agg_stop` tag not implemented yet in the `poi` layer
- `brunnel` tag is excluded from `transportation_name` layer to avoid breaking apart long `transportation_name`
lines, to revert this behavior set `--transportation-name-brunnel=true`
- `rank` field on `mountain_peak` linestrings only has 3 levels (1: has wikipedia page and name, 2: has name, 3: no name
or wikipedia page or name)
- `rank` field on `mountain_peak` linestrings only has 3 levels (1: has wikipedia page and name, 2: has name, 3: no name
or wikipedia page or name)
- `id` field on `water` polygons is only populated for openstreetmap lakes, not natural earth lakes at lower zoom levels
## Code Layout
[Generate.java](./src/main/java/com/onthegomap/planetiler/basemap/Generate.java) generates code in
the [generated](./src/main/java/com/onthegomap/planetiler/basemap/generated) package from an OpenMapTiles tag in GitHub:
- [OpenMapTilesSchema](./src/main/java/com/onthegomap/planetiler/basemap/generated/OpenMapTilesSchema.java)
contains an interface for each layer with constants for the name, attributes, and allowed values for each tag in that
layer
- [Tables](./src/main/java/com/onthegomap/planetiler/basemap/generated/Tables.java)
contains a record for each table that OpenMapTiles [imposm3](https://github.com/omniscale/imposm3) configuration
generates (along with the tag-filtering expression) so layers can listen on instances of those records instead of
doing the tag filtering and parsing themselves
The [layers](./src/main/java/com/onthegomap/planetiler/basemap/layers) package contains a port of the SQL logic to
generate each layer from OpenMapTiles. Layers define how source features (or parsed imposm3 table rows) map to vector
tile features, and logic for post-processing tile geometries.
[BasemapProfile](./src/main/java/com/onthegomap/planetiler/basemap/BasemapProfile.java) dispatches source features to
layer handlers and merges the results.
[BasemapMain](./src/main/java/com/onthegomap/planetiler/basemap/BasemapMain.java) is the main driver that registers
source data and output location.
## Regenerating Code
To run `Generate.java`, use [scripts/regenerate-openmaptiles.sh](../scripts/regenerate-openmaptiles.sh) script with the
OpenMapTiles release tag:
```bash
./scripts/regenerate-openmaptiles.sh v3.13.1
```
Then follow the instructions it prints for reformatting generated code.
If you want to regenerate from a different repository than the default openmaptiles, you can specify the url like this:
```bash
./scripts/regenerate-openmaptiles.sh v3.13.1 https://raw.githubusercontent.com/openmaptiles/openmaptiles/
```
## License and Attribution
OpenMapTiles code is licensed under the BSD 3-Clause License, which appears at the top of any file ported from
OpenMapTiles.
The OpenMapTiles schema (or "look and feel") is licensed under [CC-BY 4.0](http://creativecommons.org/licenses/by/4.0/),
so any map derived from that schema
must [visibly credit OpenMapTiles](https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md#design-license-cc-by-40)
. It also uses OpenStreetMap data, so you
must [visibly credit OpenStreetMap contributors](https://www.openstreetmap.org/copyright).

56
planetiler-basemap/pom.xml
Wyświetl plik

@ -1,56 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>planetiler-basemap</artifactId>
<parent>
<groupId>com.onthegomap.planetiler</groupId>
<artifactId>planetiler-parent</artifactId>
<version>0.5-SNAPSHOT</version>
</parent>
<dependencies>
<dependency>
<groupId>com.onthegomap.planetiler</groupId>
<artifactId>planetiler-core</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
</dependency>
<dependency>
<groupId>org.commonmark</groupId>
<artifactId>commonmark</artifactId>
<version>0.19.0</version>
</dependency>
<dependency>
<groupId>com.onthegomap.planetiler</groupId>
<artifactId>planetiler-core</artifactId>
<version>${project.parent.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>io.github.zlika</groupId>
<artifactId>reproducible-build-maven-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<configuration>
<!-- we don't want to deploy this module -->
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
</project>

53
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/BasemapMain.java
Wyświetl plik

@ -1,53 +0,0 @@
package com.onthegomap.planetiler.basemap;
import com.onthegomap.planetiler.Planetiler;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.config.Arguments;
import java.nio.file.Path;
/**
* Main entrypoint for generating a map using the basemap schema.
*/
public class BasemapMain {
public static void main(String[] args) throws Exception {
run(Arguments.fromArgsOrConfigFile(args));
}
static void run(Arguments arguments) throws Exception {
Path dataDir = Path.of("data");
Path sourcesDir = dataDir.resolve("sources");
// use --area=... argument, AREA=... env var or area=... in config to set the region of the world to use
// will be ignored if osm_path or osm_url are set
String area = arguments.getString(
"area",
"name of the extract to download if osm_url/osm_path not specified (i.e. 'monaco' 'rhode island' 'australia' or 'planet')",
"monaco"
);
Planetiler.create(arguments)
.setDefaultLanguages(OpenMapTilesSchema.LANGUAGES)
.fetchWikidataNameTranslations(sourcesDir.resolve("wikidata_names.json"))
// defer creation of the profile because it depends on data from the runner
.setProfile(BasemapProfile::new)
// override any of these with arguments: --osm_path=... or --osm_url=...
// or OSM_PATH=... OSM_URL=... environmental argument
// or osm_path=... osm_url=... in a config file
.addShapefileSource("EPSG:3857", BasemapProfile.LAKE_CENTERLINE_SOURCE,
sourcesDir.resolve("lake_centerline.shp.zip"),
// was previously using this old build from 2016: https://github.com/lukasmartinelli/osm-lakelines/releases/download/v0.9/lake_centerline.shp.zip
"https://github.com/acalcutt/osm-lakelines/releases/download/latest/lake_centerline.shp.zip")
.addShapefileSource(BasemapProfile.WATER_POLYGON_SOURCE,
sourcesDir.resolve("water-polygons-split-3857.zip"),
"https://osmdata.openstreetmap.de/download/water-polygons-split-3857.zip")
.addNaturalEarthSource(BasemapProfile.NATURAL_EARTH_SOURCE,
sourcesDir.resolve("natural_earth_vector.sqlite.zip"),
"https://naciscdn.org/naturalearth/packages/natural_earth_vector.sqlite.zip")
.addOsmSource(BasemapProfile.OSM_SOURCE,
sourcesDir.resolve(area.replaceAll("[^a-zA-Z]+", "_") + ".osm.pbf"),
"planet".equalsIgnoreCase(area) ? ("aws:latest") : ("geofabrik:" + area))
// override with --mbtiles=... argument or MBTILES=... env var or mbtiles=... in a config file
.setOutput("mbtiles", dataDir.resolve("output.mbtiles"))
.run();
}
}

267
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/BasemapProfile.java
Wyświetl plik

@ -1,267 +0,0 @@
package com.onthegomap.planetiler.basemap;
import static com.onthegomap.planetiler.geo.GeoUtils.EMPTY_LINE;
import static com.onthegomap.planetiler.geo.GeoUtils.EMPTY_POINT;
import static com.onthegomap.planetiler.geo.GeoUtils.EMPTY_POLYGON;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.ForwardingProfile;
import com.onthegomap.planetiler.Planetiler;
import com.onthegomap.planetiler.Profile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.layers.Transportation;
import com.onthegomap.planetiler.basemap.layers.TransportationName;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.reader.SimpleFeature;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
import java.util.ArrayList;
import java.util.List;
/**
* Delegates the logic for generating a map to individual implementations in the {@code layers} package.
* <p>
* Layer implementations extend these interfaces to subscribe to elements from different sources:
* <ul>
* <li>{@link LakeCenterlineProcessor}</li>
* <li>{@link NaturalEarthProcessor}</li>
* <li>{@link OsmWaterPolygonProcessor}</li>
* <li>{@link OsmAllProcessor} to process every OSM feature</li>
* <li>{@link OsmRelationPreprocessor} to process every OSM relation during first pass through OSM file</li>
* <li>A {@link Tables.RowHandler} implementation in {@code Tables.java} to process input features filtered and parsed
* according to the imposm3 mappings defined in the OpenMapTiles schema. Each element corresponds to a row in the table
* that imposm3 would have generated, with generated methods for accessing the data that would have been in each
* column</li>
* </ul>
* Layers can also subscribe to notifications when we finished processing an input source by implementing
* {@link FinishHandler} or post-process features in that layer before rendering the output tile by implementing
* {@link FeaturePostProcessor}.
*/
public class BasemapProfile extends ForwardingProfile {
// IDs used in stats and logs for each input source, as well as argument/config file overrides to source locations
public static final String LAKE_CENTERLINE_SOURCE = "lake_centerlines";
public static final String WATER_POLYGON_SOURCE = "water_polygons";
public static final String NATURAL_EARTH_SOURCE = "natural_earth";
public static final String OSM_SOURCE = "osm";
/** Index to efficiently find the imposm3 "table row" constructor from an OSM element based on its tags. */
private final MultiExpression.Index<RowDispatch> osmMappings;
/** Index variant that filters out any table only used by layers that implement IgnoreWikidata class. */
private final MultiExpression.Index<Boolean> wikidataMappings;
public BasemapProfile(Planetiler runner) {
this(runner.translations(), runner.config(), runner.stats());
}
public BasemapProfile(Translations translations, PlanetilerConfig config, Stats stats) {
List<String> onlyLayers = config.arguments().getList("only_layers", "Include only certain layers", List.of());
List<String> excludeLayers = config.arguments().getList("exclude_layers", "Exclude certain layers", List.of());
// register release/finish/feature postprocessor/osm relationship handler methods...
List<Handler> layers = new ArrayList<>();
Transportation transportationLayer = null;
TransportationName transportationNameLayer = null;
for (Layer layer : OpenMapTilesSchema.createInstances(translations, config, stats)) {
if ((onlyLayers.isEmpty() || onlyLayers.contains(layer.name())) && !excludeLayers.contains(layer.name())) {
layers.add(layer);
registerHandler(layer);
if (layer instanceof TransportationName transportationName) {
transportationNameLayer = transportationName;
}
}
if (layer instanceof Transportation transportation) {
transportationLayer = transportation;
}
}
// special-case: transportation_name layer depends on transportation layer
if (transportationNameLayer != null) {
transportationNameLayer.needsTransportationLayer(transportationLayer);
if (!layers.contains(transportationLayer)) {
layers.add(transportationLayer);
registerHandler(transportationLayer);
}
}
// register per-source input element handlers
for (Handler handler : layers) {
if (handler instanceof NaturalEarthProcessor processor) {
registerSourceHandler(NATURAL_EARTH_SOURCE,
(source, features) -> processor.processNaturalEarth(source.getSourceLayer(), source, features));
}
if (handler instanceof OsmWaterPolygonProcessor processor) {
registerSourceHandler(WATER_POLYGON_SOURCE, processor::processOsmWater);
}
if (handler instanceof LakeCenterlineProcessor processor) {
registerSourceHandler(LAKE_CENTERLINE_SOURCE, processor::processLakeCenterline);
}
if (handler instanceof OsmAllProcessor processor) {
registerSourceHandler(OSM_SOURCE, processor::processAllOsm);
}
}
// pre-process layers to build efficient indexes for matching OSM elements based on matching expressions
// Map from imposm3 table row class to the layers that implement its handler.
var handlerMap = Tables.generateDispatchMap(layers);
osmMappings = Tables.MAPPINGS
.mapResults(constructor -> {
var handlers = handlerMap.getOrDefault(constructor.rowClass(), List.of()).stream()
.map(r -> {
@SuppressWarnings("unchecked") var handler = (Tables.RowHandler<Tables.Row>) r.handler();
return handler;
})
.toList();
return new RowDispatch(constructor.create(), handlers);
}).simplify().indexAndWarn();
wikidataMappings = Tables.MAPPINGS
.mapResults(constructor -> handlerMap.getOrDefault(constructor.rowClass(), List.of()).stream()
.anyMatch(handler -> !IgnoreWikidata.class.isAssignableFrom(handler.handlerClass()))
).filterResults(b -> b).simplify().index();
// register a handler for all OSM elements that forwards to imposm3 "table row" handler methods
// based on efficient pre-processed index
if (!osmMappings.isEmpty()) {
registerSourceHandler(OSM_SOURCE, (source, features) -> {
for (var match : getTableMatches(source)) {
RowDispatch rowDispatch = match.match();
var row = rowDispatch.constructor.create(source, match.keys().get(0));
for (Tables.RowHandler<Tables.Row> handler : rowDispatch.handlers()) {
handler.process(row, features);
}
}
});
}
}
/** Returns the imposm3 table row constructors that match an input element's tags. */
public List<MultiExpression.Match<RowDispatch>> getTableMatches(SourceFeature input) {
return osmMappings.getMatchesWithTriggers(input);
}
@Override
public boolean caresAboutWikidataTranslation(OsmElement elem) {
var tags = elem.tags();
if (elem instanceof OsmElement.Node) {
return wikidataMappings.getOrElse(SimpleFeature.create(EMPTY_POINT, tags), false);
} else if (elem instanceof OsmElement.Way) {
return wikidataMappings.getOrElse(SimpleFeature.create(EMPTY_POLYGON, tags), false) ||
wikidataMappings.getOrElse(SimpleFeature.create(EMPTY_LINE, tags), false);
} else if (elem instanceof OsmElement.Relation) {
return wikidataMappings.getOrElse(SimpleFeature.create(EMPTY_POLYGON, tags), false);
} else {
return false;
}
}
/*
* Pass-through constants generated from the OpenMapTiles vector schema
*/
@Override
public String name() {
return OpenMapTilesSchema.NAME;
}
@Override
public String description() {
return OpenMapTilesSchema.DESCRIPTION;
}
@Override
public String attribution() {
return OpenMapTilesSchema.ATTRIBUTION;
}
@Override
public String version() {
return OpenMapTilesSchema.VERSION;
}
@Override
public long estimateIntermediateDiskBytes(long osmFileSize) {
// in late 2021, a 60gb OSM file used 200GB for intermediate storage
return osmFileSize * 200 / 60;
}
@Override
public long estimateOutputBytes(long osmFileSize) {
// in late 2021, a 60gb OSM file generated a 100GB output file
return osmFileSize * 100 / 60;
}
@Override
public long estimateRamRequired(long osmFileSize) {
// 20gb for a 67gb OSM file is safe, although less might be OK too
return osmFileSize * 20 / 67;
}
/**
* Layers should implement this interface to subscribe to elements from
* <a href="https://www.naturalearthdata.com/">natural earth</a>.
*/
public interface NaturalEarthProcessor {
/**
* Process an element from {@code table} in the<a href="https://www.naturalearthdata.com/">natural earth source</a>.
*
* @see Profile#processFeature(SourceFeature, FeatureCollector)
*/
void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features);
}
/**
* Layers should implement this interface to subscribe to elements from
* <a href="https://github.com/lukasmartinelli/osm-lakelines">OSM lake centerlines source</a>.
*/
public interface LakeCenterlineProcessor {
/**
* Process an element from the <a href="https://github.com/lukasmartinelli/osm-lakelines">OSM lake centerlines
* source</a>
*
* @see Profile#processFeature(SourceFeature, FeatureCollector)
*/
void processLakeCenterline(SourceFeature feature, FeatureCollector features);
}
/**
* Layers should implement this interface to subscribe to elements from
* <a href="https://osmdata.openstreetmap.de/data/water-polygons.html">OSM water polygons source</a>.
*/
public interface OsmWaterPolygonProcessor {
/**
* Process an element from the <a href="https://osmdata.openstreetmap.de/data/water-polygons.html">OSM water
* polygons source</a>
*
* @see Profile#processFeature(SourceFeature, FeatureCollector)
*/
void processOsmWater(SourceFeature feature, FeatureCollector features);
}
/** Layers should implement this interface to subscribe to every OSM element. */
public interface OsmAllProcessor {
/**
* Process an OSM element during the second pass through the OSM data file.
*
* @see Profile#processFeature(SourceFeature, FeatureCollector)
*/
void processAllOsm(SourceFeature feature, FeatureCollector features);
}
/**
* Layers should implement to indicate they do not need wikidata name translations to avoid downloading more
* translations than are needed.
*/
public interface IgnoreWikidata {}
private record RowDispatch(
Tables.Constructor constructor,
List<Tables.RowHandler<Tables.Row>> handlers
) {}
}

764
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/Generate.java
Wyświetl plik

@ -1,764 +0,0 @@
package com.onthegomap.planetiler.basemap;
import static com.onthegomap.planetiler.expression.Expression.*;
import static java.util.stream.Collectors.joining;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.base.CaseFormat;
import com.onthegomap.planetiler.config.Arguments;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.Expression;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.util.Downloader;
import com.onthegomap.planetiler.util.FileUtils;
import com.onthegomap.planetiler.util.Format;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.TreeMap;
import java.util.stream.Stream;
import org.commonmark.parser.Parser;
import org.commonmark.renderer.html.HtmlRenderer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.yaml.snakeyaml.LoaderOptions;
import org.yaml.snakeyaml.Yaml;
/**
* Generates code in the {@code generated} package from the OpenMapTiles schema crawled from a tag or branch in the
* <a href="https://github.com/openmaptiles/openmaptiles">OpenMapTiles GitHub repo</a>.
* <p>
* {@code OpenMapTilesSchema.java} contains the output layer definitions (i.e. attributes and allowed values) so that
* layer implementations in {@code layers} package can reference them instead of hard-coding.
* <p>
* {@code Tables.java} contains the <a href="https://github.com/omniscale/imposm3">imposm3</a> table definitions from
* mapping.yaml files in the OpenMapTiles repo. Layers in the {@code layer} package can extend the {@code Handler}
* nested class for a table definition to "subscribe" to OSM elements that imposm3 would put in that table.
* <p>
* To run use {@code ./scripts/regenerate-openmaptiles.sh}
*/
public class Generate {
private static final Logger LOGGER = LoggerFactory.getLogger(Generate.class);
private static final ObjectMapper mapper = new ObjectMapper();
private static final Yaml yaml;
private static final String LINE_SEPARATOR = System.lineSeparator();
private static final String GENERATED_FILE_HEADER = """
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
// AUTOGENERATED BY Generate.java -- DO NOT MODIFY
""";
private static final Parser parser = Parser.builder().build();
private static final HtmlRenderer renderer = HtmlRenderer.builder().build();
static {
// bump the default limit of 50
var options = new LoaderOptions();
options.setMaxAliasesForCollections(1_000);
yaml = new Yaml(options);
}
private static <T> T loadAndParseYaml(String url, PlanetilerConfig config, Class<T> clazz) throws IOException {
LOGGER.info("reading {}", url);
try (var stream = Downloader.openStream(url, config)) {
// Jackson yaml parsing does not handle anchors and references, so first parse the input
// using SnakeYAML, then parse SnakeYAML's output using Jackson to get it into our records.
Map<String, Object> parsed = yaml.load(stream);
return mapper.convertValue(parsed, clazz);
}
}
static <T> T parseYaml(String string, Class<T> clazz) {
// Jackson yaml parsing does not handle anchors and references, so first parse the input
// using SnakeYAML, then parse SnakeYAML's output using Jackson to get it into our records.
Map<String, Object> parsed = yaml.load(string);
return mapper.convertValue(parsed, clazz);
}
static JsonNode parseYaml(String string) {
return string == null ? null : parseYaml(string, JsonNode.class);
}
public static void main(String[] args) throws IOException {
Arguments arguments = Arguments.fromArgsOrConfigFile(args);
PlanetilerConfig planetilerConfig = PlanetilerConfig.from(arguments);
String tag = arguments.getString("tag", "openmaptiles tag to use", "v3.13.1");
String baseUrl = arguments.getString("base-url", "the url used to download the openmaptiles.yml",
"https://raw.githubusercontent.com/openmaptiles/openmaptiles/");
String base = baseUrl + tag + "/";
// start crawling from openmaptiles.yaml
// then crawl schema from each layers/<layer>/<layer>.yaml file that it references
// then crawl table definitions from each layers/<layer>/mapping.yaml file that the layer references
String rootUrl = base + "openmaptiles.yaml";
OpenmaptilesConfig config = loadAndParseYaml(rootUrl, planetilerConfig, OpenmaptilesConfig.class);
List<LayerConfig> layers = new ArrayList<>();
Set<String> imposm3MappingFiles = new LinkedHashSet<>();
for (String layerFile : config.tileset.layers) {
String layerURL = base + layerFile;
LayerConfig layer = loadAndParseYaml(layerURL, planetilerConfig, LayerConfig.class);
layers.add(layer);
for (Datasource datasource : layer.datasources) {
if ("imposm3".equals(datasource.type)) {
String mappingPath = Path.of(layerFile).resolveSibling(datasource.mapping_file).normalize().toString();
imposm3MappingFiles.add(base + mappingPath);
} else {
LOGGER.warn("Unknown datasource type: {}", datasource.type);
}
}
}
Map<String, Imposm3Table> tables = new LinkedHashMap<>();
for (String uri : imposm3MappingFiles) {
Imposm3Mapping layer = loadAndParseYaml(uri, planetilerConfig, Imposm3Mapping.class);
tables.putAll(layer.tables);
}
String packageName = "com.onthegomap.planetiler.basemap.generated";
String[] packageParts = packageName.split("\\.");
Path output = Path.of("planetiler-basemap", "src", "main", "java")
.resolve(Path.of(packageParts[0], Arrays.copyOfRange(packageParts, 1, packageParts.length)));
FileUtils.deleteDirectory(output);
Files.createDirectories(output);
emitLayerSchemaDefinitions(config.tileset, layers, packageName, output, tag);
emitTableDefinitions(tables, packageName, output, tag);
LOGGER.info("Done!");
}
/** Generates {@code OpenMapTilesSchema.java} */
private static void emitLayerSchemaDefinitions(OpenmaptilesTileSet info, List<LayerConfig> layers, String packageName,
Path output, String tag)
throws IOException {
StringBuilder schemaClass = new StringBuilder();
schemaClass.append(
"""
%s
package %s;
import static com.onthegomap.planetiler.expression.Expression.*;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.basemap.Layer;
import com.onthegomap.planetiler.util.Translations;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* All vector tile layer definitions, attributes, and allowed values generated from the
* <a href="https://github.com/openmaptiles/openmaptiles/blob/%s/openmaptiles.yaml">OpenMapTiles vector tile schema %s</a>.
*/
@SuppressWarnings("unused")
public class OpenMapTilesSchema {
public static final String NAME = %s;
public static final String DESCRIPTION = %s;
public static final String VERSION = %s;
public static final String ATTRIBUTION = %s;
public static final List<String> LANGUAGES = List.of(%s);
/** Returns a list of expected layer implementation instances from the {@code layers} package. */
public static List<Layer> createInstances(Translations translations, PlanetilerConfig config, Stats stats) {
return List.of(
%s
);
}
"""
.formatted(
GENERATED_FILE_HEADER,
packageName,
escapeJavadoc(tag),
escapeJavadoc(tag),
Format.quote(info.name),
Format.quote(info.description),
Format.quote(info.version),
Format.quote(info.attribution),
info.languages.stream().map(Format::quote).collect(joining(", ")),
layers.stream()
.map(
l -> "new com.onthegomap.planetiler.basemap.layers.%s(translations, config, stats)"
.formatted(lowerUnderscoreToUpperCamel(l.layer.id)))
.collect(joining("," + LINE_SEPARATOR))
.indent(6).trim()
));
for (var layer : layers) {
String layerCode = generateCodeForLayer(tag, layer);
schemaClass.append(layerCode);
}
schemaClass.append("}");
Files.writeString(output.resolve("OpenMapTilesSchema.java"), schemaClass);
}
private static String generateCodeForLayer(String tag, LayerConfig layer) {
String layerName = layer.layer.id;
String className = lowerUnderscoreToUpperCamel(layerName);
StringBuilder fields = new StringBuilder();
StringBuilder fieldValues = new StringBuilder();
StringBuilder fieldMappings = new StringBuilder();
layer.layer.fields.forEach((name, value) -> {
JsonNode valuesNode = value.get("values");
List<String> valuesForComment = valuesNode == null ? List.of() : valuesNode.isArray() ?
iterToList(valuesNode.elements()).stream().map(Objects::toString).toList() :
iterToList(valuesNode.fieldNames());
String javadocDescription = markdownToJavadoc(getFieldDescription(value));
fields.append("""
%s
public static final String %s = %s;
""".formatted(
valuesForComment.isEmpty() ? "/** %s */".formatted(javadocDescription) : """
/**
* %s
* <p>
* allowed values:
* <ul>
* %s
* </ul>
*/
""".stripTrailing().formatted(javadocDescription,
valuesForComment.stream().map(v -> "<li>" + v).collect(joining(LINE_SEPARATOR + " * "))),
name.toUpperCase(Locale.ROOT),
Format.quote(name)
).indent(4));
List<String> values = valuesNode == null ? List.of() : valuesNode.isArray() ?
iterToList(valuesNode.elements()).stream().filter(JsonNode::isTextual).map(JsonNode::textValue)
.map(t -> t.replaceAll(" .*", "")).toList() :
iterToList(valuesNode.fieldNames());
if (values.size() > 0) {
fieldValues.append(values.stream()
.map(v -> "public static final String %s = %s;"
.formatted(name.toUpperCase(Locale.ROOT) + "_" + v.toUpperCase(Locale.ROOT).replace('-', '_'),
Format.quote(v)))
.collect(joining(LINE_SEPARATOR)).indent(2).strip()
.indent(4));
fieldValues.append("public static final Set<String> %s = Set.of(%s);".formatted(
name.toUpperCase(Locale.ROOT) + "_VALUES",
values.stream().map(Format::quote).collect(joining(", "))
).indent(4));
}
if (valuesNode != null && valuesNode.isObject()) {
MultiExpression<String> mapping = generateFieldMapping(valuesNode);
fieldMappings.append(" public static final MultiExpression<String> %s = %s;%n"
.formatted(lowerUnderscoreToUpperCamel(name), generateJavaCode(mapping)));
}
});
return """
/**
* %s
*
* Generated from <a href="https://github.com/openmaptiles/openmaptiles/blob/%s/layers/%s/%s.yaml">%s.yaml</a>
*/
public interface %s extends Layer {
double BUFFER_SIZE = %s;
String LAYER_NAME = %s;
@Override
default String name() {
return LAYER_NAME;
}
/** Attribute names for map elements in the %s layer. */
final class Fields {
%s
}
/** Attribute values for map elements in the %s layer. */
final class FieldValues {
%s
}
/** Complex mappings to generate attribute values from OSM element tags in the %s layer. */
final class FieldMappings {
%s
}
}
""".formatted(
markdownToJavadoc(layer.layer.description),
escapeJavadoc(tag),
escapeJavadoc(layerName),
escapeJavadoc(layerName),
escapeJavadoc(layerName),
className,
layer.layer.buffer_size,
Format.quote(layerName),
escapeJavadoc(layerName),
fields.toString().strip(),
escapeJavadoc(layerName),
fieldValues.toString().strip(),
escapeJavadoc(layerName),
fieldMappings.toString().strip()
).indent(2);
}
/** Generates {@code Tables.java} */
private static void emitTableDefinitions(Map<String, Imposm3Table> tables, String packageName, Path output,
String tag)
throws IOException {
StringBuilder tablesClass = new StringBuilder();
tablesClass.append(
"""
%s
package %s;
import static com.onthegomap.planetiler.expression.Expression.*;
import com.onthegomap.planetiler.expression.Expression;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.reader.SourceFeature;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* OSM element parsers generated from the <a href="https://github.com/omniscale/imposm3">imposm3</a> table definitions
* in the <a href="https://github.com/openmaptiles/openmaptiles/blob/%s/openmaptiles.yaml">OpenMapTiles vector tile schema</a>.
*
* These filter and parse the raw OSM key/value attribute pairs on tags into records with fields that match the
* columns in the tables that imposm3 would generate. Layer implementations can "subscribe" to elements from each
* "table" but implementing the table's {@code Handler} interface and use the element's typed API to access
* attributes.
*/
@SuppressWarnings("unused")
public class Tables {
/** A parsed OSM element that would appear in a "row" of the imposm3 table. */
public interface Row {
/** Returns the original OSM element. */
SourceFeature source();
}
/** A functional interface that the constructor of a new table row can be coerced to. */
@FunctionalInterface
public interface Constructor {
Row create(SourceFeature source, String mappingKey);
}
/** The {@code rowClass} of an imposm3 table row and its constructor coerced to a {@link Constructor}. */
public record RowClassAndConstructor(
Class<? extends Row> rowClass,
Constructor create
) {}
/** A functional interface that the typed handler method that a layer implementation can be coerced to. */
@FunctionalInterface
public interface RowHandler<T extends Row> {
/** Process a typed element according to the profile. */
void process(T element, FeatureCollector features);
}
/** The {@code handlerClass} of a layer handler and it's {@code process} method coerced to a {@link RowHandler}. */
public record RowHandlerAndClass<T extends Row>(
Class<?> handlerClass,
RowHandler<T> handler
) {}
"""
.formatted(GENERATED_FILE_HEADER, packageName, escapeJavadoc(tag)));
List<String> classNames = new ArrayList<>();
Map<String, String> fieldNameToType = new TreeMap<>();
for (var entry : tables.entrySet()) {
String key = entry.getKey();
Imposm3Table table = entry.getValue();
List<OsmTableField> fields = parseTableFields(table);
for (var field : fields) {
String existing = fieldNameToType.get(field.name);
if (existing == null) {
fieldNameToType.put(field.name, field.clazz);
} else if (!existing.equals(field.clazz)) {
throw new IllegalArgumentException(
"Field " + field.name + " has both " + existing + " and " + field.clazz + " types");
}
}
Expression mappingExpression = parseImposm3MappingExpression(table);
String mapping = """
/** Imposm3 "mapping" to filter OSM elements that should appear in this "table". */
public static final Expression MAPPING = %s;
""".formatted(
mappingExpression.generateJavaCode()
);
String tableName = "osm_" + key;
String className = lowerUnderscoreToUpperCamel(tableName);
if (!"relation_member".equals(table.type)) {
classNames.add(className);
tablesClass.append("""
/** An OSM element that would appear in the {@code %s} table generated by imposm3. */
public record %s(%s) implements Row, %s {
public %s(SourceFeature source, String mappingKey) {
this(%s);
}
%s
/**
* Interface for layer implementations to extend to subscribe to OSM elements filtered and parsed as
* {@link %s}.
*/
public interface Handler {
void process(%s element, FeatureCollector features);
}
}
""".formatted(
tableName,
escapeJavadoc(className),
fields.stream().map(c -> "@Override " + c.clazz + " " + lowerUnderscoreToLowerCamel(c.name))
.collect(joining(", ")),
fields.stream().map(c -> lowerUnderscoreToUpperCamel("with_" + c.name))
.collect(joining(", ")),
className,
fields.stream().map(c -> c.extractCode).collect(joining(", ")),
mapping,
escapeJavadoc(className),
className
).indent(2));
}
}
tablesClass.append(fieldNameToType.entrySet().stream().map(e -> {
String attrName = lowerUnderscoreToLowerCamel(e.getKey());
String type = e.getValue();
String interfaceName = lowerUnderscoreToUpperCamel("with_" + e.getKey());
return """
/** Rows with a %s %s attribute. */
public interface %s {
%s %s();
}
""".formatted(
escapeJavadoc(type),
escapeJavadoc(attrName),
interfaceName,
type,
attrName);
}).collect(joining(LINE_SEPARATOR)).indent(2));
tablesClass.append("""
/** Index to efficiently choose which imposm3 "tables" an element should appear in based on its attributes. */
public static final MultiExpression<RowClassAndConstructor> MAPPINGS = MultiExpression.of(List.of(
%s
));
""".formatted(
classNames.stream().map(
className -> "MultiExpression.entry(new RowClassAndConstructor(%s.class, %s::new), %s.MAPPING)".formatted(
className, className, className))
.collect(joining("," + LINE_SEPARATOR)).indent(2).strip()
).indent(2));
String handlerCondition = classNames.stream()
.map(
className -> """
if (handler instanceof %s.Handler typedHandler) {
result.computeIfAbsent(%s.class, cls -> new ArrayList<>()).add(new RowHandlerAndClass<>(typedHandler.getClass(), typedHandler::process));
}"""
.formatted(className, className)
).collect(joining(LINE_SEPARATOR));
tablesClass.append("""
/**
* Returns a map from imposm3 "table row" class to the layers that have a handler for it from a list of layer
* implementations.
*/
public static Map<Class<? extends Row>, List<RowHandlerAndClass<?>>> generateDispatchMap(List<?> handlers) {
Map<Class<? extends Row>, List<RowHandlerAndClass<?>>> result = new HashMap<>();
for (var handler : handlers) {
%s
}
return result;
}
}
""".formatted(handlerCondition.indent(6).trim()));
Files.writeString(output.resolve("Tables.java"), tablesClass);
}
/**
* Returns an {@link Expression} that implements the same logic as the
* <a href="https://imposm.org/docs/imposm3/latest/mapping.html">Imposm3 Data Mapping</a> definition for a table.
*/
static Expression parseImposm3MappingExpression(Imposm3Table table) {
if (table.type_mappings != null) {
return or(
table.type_mappings.entrySet().stream()
.map(entry -> parseImposm3MappingExpression(entry.getKey(), entry.getValue(), table.filters)
).toList()
).simplify();
} else {
return parseImposm3MappingExpression(table.type, table.mapping, table.filters);
}
}
/**
* Returns an {@link Expression} that implements the same logic as the
* <a href="https://imposm.org/docs/imposm3/latest/mapping.html#filters">Imposm3 Data Mapping filters</a> for a table.
*/
static Expression parseImposm3MappingExpression(String type, JsonNode mapping, Imposm3Filters filters) {
return and(
or(parseFieldMappingExpression(mapping).toList()),
and(
filters == null || filters.require == null ? List.of() : parseFieldMappingExpression(filters.require).toList()),
not(or(
filters == null || filters.reject == null ? List.of() : parseFieldMappingExpression(filters.reject).toList())),
matchType(type.replaceAll("s$", ""))
).simplify();
}
private static List<OsmTableField> parseTableFields(Imposm3Table tableDefinition) {
List<OsmTableField> result = new ArrayList<>();
boolean relationMember = "relation_member".equals(tableDefinition.type);
for (Imposm3Column col : tableDefinition.columns) {
if (relationMember && col.from_member) {
// layers process relation info that they need manually
continue;
}
switch (col.type) {
case "id", "validated_geometry", "area", "hstore_tags", "geometry" -> {
// do nothing - already on source feature
}
case "member_id", "member_role", "member_type", "member_index" -> {
// do nothing
}
case "mapping_key" -> result
.add(new OsmTableField("String", col.name, "mappingKey"));
case "mapping_value" -> result
.add(new OsmTableField("String", col.name, "source.getString(mappingKey)"));
case "string" -> result
.add(new OsmTableField("String", col.name,
"source.getString(\"%s\")".formatted(Objects.requireNonNull(col.key, col.toString()))));
case "bool" -> result
.add(new OsmTableField("boolean", col.name,
"source.getBoolean(\"%s\")".formatted(Objects.requireNonNull(col.key, col.toString()))));
case "integer" -> result
.add(new OsmTableField("long", col.name,
"source.getLong(\"%s\")".formatted(Objects.requireNonNull(col.key, col.toString()))));
case "wayzorder" -> result.add(new OsmTableField("int", col.name, "source.getWayZorder()"));
case "direction" -> result.add(new OsmTableField("int", col.name,
"source.getDirection(\"%s\")".formatted(Objects.requireNonNull(col.key, col.toString()))));
default -> throw new IllegalArgumentException("Unhandled column: " + col.type);
}
}
result.add(new OsmTableField("SourceFeature", "source", "source"));
return result;
}
/**
* Returns a {@link MultiExpression} to efficiently determine the value for an output vector tile feature (i.e.
* "class") based on the "field mapping" defined in the layer schema definition.
*/
static MultiExpression<String> generateFieldMapping(JsonNode valuesNode) {
MultiExpression<String> mapping = MultiExpression.of(new ArrayList<>());
valuesNode.fields().forEachRemaining(entry -> {
String field = entry.getKey();
JsonNode node = entry.getValue();
Expression expression = or(parseFieldMappingExpression(node).toList()).simplify();
if (!expression.equals(or()) && !expression.equals(and())) {
mapping.expressions().add(MultiExpression.entry(field, expression));
}
});
return mapping;
}
private static Stream<Expression> parseFieldMappingExpression(JsonNode node) {
if (node.isObject()) {
List<String> keys = iterToList(node.fieldNames());
if (keys.contains("__AND__")) {
if (keys.size() > 1) {
throw new IllegalArgumentException("Cannot combine __AND__ with others");
}
return Stream.of(and(parseFieldMappingExpression(node.get("__AND__")).toList()));
} else if (keys.contains("__OR__")) {
if (keys.size() > 1) {
throw new IllegalArgumentException("Cannot combine __OR__ with others");
}
return Stream.of(or(parseFieldMappingExpression(node.get("__OR__")).toList()));
} else {
return iterToList(node.fields()).stream().map(entry -> {
String field = entry.getKey();
List<String> value = toFlatList(entry.getValue()).map(JsonNode::textValue).filter(Objects::nonNull).toList();
return value.isEmpty() || value.contains("__any__") ? matchField(field) : matchAny(field, value);
});
}
} else if (node.isArray()) {
return iterToList(node.elements()).stream().flatMap(Generate::parseFieldMappingExpression);
} else if (node.isNull()) {
return Stream.empty();
} else {
throw new IllegalArgumentException("parseExpression input not handled: " + node);
}
}
/**
* Returns a flattened list of all the elements in nested arrays from {@code node}.
* <p>
* For example: {@code [[[a, b], c], [d]} becomes {@code [a, b, c, d]}
* <p>
* And {@code a} becomes {@code [a]}
*/
private static Stream<JsonNode> toFlatList(JsonNode node) {
return node.isArray() ? iterToList(node.elements()).stream().flatMap(Generate::toFlatList) : Stream.of(node);
}
/** Returns java code that will recreate an {@link MultiExpression} identical to {@code mapping}. */
private static String generateJavaCode(MultiExpression<String> mapping) {
return "MultiExpression.of(List.of(" + mapping.expressions().stream()
.map(s -> "MultiExpression.entry(%s, %s)".formatted(Format.quote(s.result()), s.expression().generateJavaCode()))
.collect(joining(", ")) + "))";
}
private static String lowerUnderscoreToLowerCamel(String name) {
return CaseFormat.LOWER_UNDERSCORE.to(CaseFormat.LOWER_CAMEL, name);
}
private static String lowerUnderscoreToUpperCamel(String name) {
return CaseFormat.LOWER_UNDERSCORE.to(CaseFormat.UPPER_CAMEL, name);
}
private static <T> List<T> iterToList(Iterator<T> iter) {
List<T> result = new ArrayList<>();
iter.forEachRemaining(result::add);
return result;
}
/** Renders {@code markdown} as HTML and returns comment text safe to insert in generated javadoc. */
private static String markdownToJavadoc(String markdown) {
return Stream.of(markdown.strip().split("[\r\n][\r\n]+"))
.map(p -> parser.parse(p.strip()))
.map(node -> escapeJavadoc(renderer.render(node)))
.map(p -> p.replaceAll("((^<p>)|(</p>$))", "").strip())
.collect(joining(LINE_SEPARATOR + "<p>" + LINE_SEPARATOR));
}
/** Returns {@code comment} text safe to insert in generated javadoc. */
private static String escapeJavadoc(String comment) {
return comment.strip().replaceAll("[\n\r*\\s]+", " ");
}
private static String getFieldDescription(JsonNode value) {
if (value.isTextual()) {
return value.textValue();
} else {
return value.get("description").textValue();
}
}
/*
* Models for deserializing yaml into:
*/
private record OpenmaptilesConfig(
OpenmaptilesTileSet tileset
) {}
@JsonIgnoreProperties(ignoreUnknown = true)
private record OpenmaptilesTileSet(
List<String> layers,
String version,
String attribution,
String name,
String description,
List<String> languages
) {}
@JsonIgnoreProperties(ignoreUnknown = true)
private record LayerDetails(
String id,
String description,
Map<String, JsonNode> fields,
double buffer_size
) {}
private record Datasource(
String type,
String mapping_file
) {}
@JsonIgnoreProperties(ignoreUnknown = true)
private record LayerConfig(
LayerDetails layer,
List<Datasource> datasources
) {}
private record Imposm3Column(
String type,
String name,
String key,
boolean from_member
) {}
record Imposm3Filters(
JsonNode reject,
JsonNode require
) {}
record Imposm3Table(
String type,
@JsonProperty("_resolve_wikidata") boolean resolveWikidata,
List<Imposm3Column> columns,
Imposm3Filters filters,
JsonNode mapping,
Map<String, JsonNode> type_mappings,
@JsonProperty("relation_types") List<String> relationTypes
) {}
@JsonIgnoreProperties(ignoreUnknown = true)
private record Imposm3Mapping(
Map<String, Imposm3Table> tables
) {}
private record OsmTableField(
String clazz,
String name,
String extractCode
) {}
}

8
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/Layer.java
Wyświetl plik

@ -1,8 +0,0 @@
package com.onthegomap.planetiler.basemap;
import com.onthegomap.planetiler.ForwardingProfile;
/** Interface for all vector tile layer implementations that {@link BasemapProfile} delegates to. */
public interface Layer extends
ForwardingProfile.Handler,
ForwardingProfile.HandlerForLayer {}

1906
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/generated/OpenMapTilesSchema.java
Wyświetl plik

Plik diff jest za duży Load Diff

1501
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/generated/Tables.java
Wyświetl plik

Plik diff jest za duży Load Diff

83
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/AerodromeLabel.java
Wyświetl plik

@ -1,83 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import static com.onthegomap.planetiler.basemap.util.Utils.nullOrEmpty;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.basemap.util.Utils;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
/**
* Defines the logic for generating map elements in the {@code aerodrome_label} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/aerodrome_label">OpenMapTiles
* aerodrome_layer sql files</a>.
*/
public class AerodromeLabel implements
OpenMapTilesSchema.AerodromeLabel,
Tables.OsmAerodromeLabelPoint.Handler {
private final MultiExpression.Index<String> classLookup;
private final Translations translations;
public AerodromeLabel(Translations translations, PlanetilerConfig config, Stats stats) {
this.classLookup = FieldMappings.Class.index();
this.translations = translations;
}
@Override
public void process(Tables.OsmAerodromeLabelPoint element, FeatureCollector features) {
String clazz = classLookup.getOrElse(element.source(), FieldValues.CLASS_OTHER);
boolean important = !nullOrEmpty(element.iata()) && FieldValues.CLASS_INTERNATIONAL.equals(clazz);
features.centroid(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setMinZoom(important ? 8 : 10)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.putAttrs(Utils.elevationTags(element.ele()))
.setAttr(Fields.IATA, nullIfEmpty(element.iata()))
.setAttr(Fields.ICAO, nullIfEmpty(element.icao()))
.setAttr(Fields.CLASS, clazz);
}
}

83
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Aeroway.java
Wyświetl plik

@ -1,83 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
/**
* Defines the logic for generating map elements in the {@code aeroway} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/aeroway">OpenMapTiles aeroway sql files</a>.
*/
public class Aeroway implements
OpenMapTilesSchema.Aeroway,
Tables.OsmAerowayLinestring.Handler,
Tables.OsmAerowayPolygon.Handler,
Tables.OsmAerowayPoint.Handler {
public Aeroway(Translations translations, PlanetilerConfig config, Stats stats) {}
@Override
public void process(Tables.OsmAerowayPolygon element, FeatureCollector features) {
features.polygon(LAYER_NAME)
.setMinZoom(10)
.setMinPixelSize(2)
.setAttr(Fields.CLASS, element.aeroway())
.setAttr(Fields.REF, element.ref());
}
@Override
public void process(Tables.OsmAerowayLinestring element, FeatureCollector features) {
features.line(LAYER_NAME)
.setMinZoom(10)
.setAttr(Fields.CLASS, element.aeroway())
.setAttr(Fields.REF, element.ref());
}
@Override
public void process(Tables.OsmAerowayPoint element, FeatureCollector features) {
features.point(LAYER_NAME)
.setMinZoom(14)
.setAttr(Fields.CLASS, element.aeroway())
.setAttr(Fields.REF, element.ref());
}
}

469
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Boundary.java
Wyświetl plik

@ -1,469 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.util.MemoryEstimator.CLASS_HEADER_BYTES;
import static com.onthegomap.planetiler.util.MemoryEstimator.POINTER_BYTES;
import static com.onthegomap.planetiler.util.MemoryEstimator.estimateSize;
import static java.util.stream.Collectors.counting;
import static java.util.stream.Collectors.groupingBy;
import com.carrotsearch.hppc.LongObjectMap;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmRelationInfo;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Format;
import com.onthegomap.planetiler.util.MemoryEstimator;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.stream.Collectors;
import org.locationtech.jts.geom.Coordinate;
import org.locationtech.jts.geom.Geometry;
import org.locationtech.jts.geom.LineString;
import org.locationtech.jts.geom.Point;
import org.locationtech.jts.geom.TopologyException;
import org.locationtech.jts.geom.prep.PreparedGeometry;
import org.locationtech.jts.geom.prep.PreparedGeometryFactory;
import org.locationtech.jts.operation.linemerge.LineMerger;
import org.locationtech.jts.operation.polygonize.Polygonizer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Defines the logic for generating map elements for country, state, and town boundaries in the {@code boundary} layer
* from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/boundary">OpenMapTiles boundary sql
* files</a>.
*/
public class Boundary implements
OpenMapTilesSchema.Boundary,
BasemapProfile.NaturalEarthProcessor,
BasemapProfile.OsmRelationPreprocessor,
BasemapProfile.OsmAllProcessor,
BasemapProfile.FeaturePostProcessor,
BasemapProfile.FinishHandler {
/*
* Uses natural earth at lower zoom levels and OpenStreetMap at higher zoom levels.
*
* For OpenStreetMap data at higher zoom levels:
* 1) Preprocess relations on the first pass to extract info for relations where
* type=boundary and boundary=administrative and store the admin_level for
* later.
* 2) When processing individual ways, take the minimum (most important) admin
* level of every relation they are a part of and use that as the admin level
* for the way.
* 3) If boundary_country_names argument is true and the way is part of a country
* (admin_level=2) boundary, then hold onto it for later
* 4) When we finish processing the OSM source, build country polygons from the
* saved ways and use that to determine which country is on the left and right
* side of each way, then emit the way with ADM0_L and ADM0_R keys set.
* 5) Before emitting boundary lines, merge linestrings with the same tags.
*/
private static final Logger LOGGER = LoggerFactory.getLogger(Boundary.class);
private static final double COUNTRY_TEST_OFFSET = GeoUtils.metersToPixelAtEquator(0, 10) / 256d;
private final Stats stats;
private final boolean addCountryNames;
// may be updated concurrently by multiple threads
private final Map<Long, String> regionNames = new ConcurrentHashMap<>();
// need to synchronize updates to these shared data structures:
private final Map<Long, List<Geometry>> regionGeometries = new HashMap<>();
private final Map<CountryBoundaryComponent, List<Geometry>> boundariesToMerge = new HashMap<>();
private final PlanetilerConfig config;
public Boundary(Translations translations, PlanetilerConfig config, Stats stats) {
this.config = config;
this.addCountryNames = config.arguments().getBoolean(
"boundary_country_names",
"boundary layer: add left/right codes of neighboring countries",
true
);
this.stats = stats;
}
private static boolean isDisputed(Map<String, Object> tags) {
return Parse.bool(tags.get("disputed")) ||
Parse.bool(tags.get("dispute")) ||
"dispute".equals(tags.get("border_status")) ||
tags.containsKey("disputed_by") ||
tags.containsKey("claimed_by");
}
private static String editName(String name) {
return name == null ? null : name.replace(" at ", "")
.replaceAll("\\s+", "")
.replace("Extentof", "");
}
@Override
public void release() {
regionGeometries.clear();
boundariesToMerge.clear();
regionNames.clear();
}
@Override
public void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features) {
boolean disputed = feature.getString("featurecla", "").startsWith("Disputed");
record BoundaryInfo(int adminLevel, int minzoom, int maxzoom) {}
BoundaryInfo info = switch (table) {
case "ne_110m_admin_0_boundary_lines_land" -> new BoundaryInfo(2, 0, 0);
case "ne_50m_admin_0_boundary_lines_land" -> new BoundaryInfo(2, 1, 3);
case "ne_10m_admin_0_boundary_lines_land" -> feature.hasTag("featurecla", "Lease Limit") ? null :
new BoundaryInfo(2, 4, 4);
case "ne_10m_admin_1_states_provinces_lines" -> {
Double minZoom = Parse.parseDoubleOrNull(feature.getTag("min_zoom"));
yield minZoom != null && minZoom <= 7 ? new BoundaryInfo(4, 1, 4) :
minZoom != null && minZoom <= 7.7 ? new BoundaryInfo(4, 4, 4) :
null;
}
default -> null;
};
if (info != null) {
features.line(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setZoomRange(info.minzoom, info.maxzoom)
.setMinPixelSizeAtAllZooms(0)
.setAttr(Fields.ADMIN_LEVEL, info.adminLevel)
.setAttr(Fields.MARITIME, 0)
.setAttr(Fields.DISPUTED, disputed ? 1 : 0);
}
}
@Override
public List<OsmRelationInfo> preprocessOsmRelation(OsmElement.Relation relation) {
if (relation.hasTag("type", "boundary") &&
relation.hasTag("admin_level") &&
relation.hasTag("boundary", "administrative")) {
Integer adminLevelValue = Parse.parseRoundInt(relation.getTag("admin_level"));
String code = relation.getString("ISO3166-1:alpha3");
if (adminLevelValue != null && adminLevelValue >= 2 && adminLevelValue <= 10) {
boolean disputed = isDisputed(relation.tags());
if (code != null) {
regionNames.put(relation.id(), code);
}
return List.of(new BoundaryRelation(
relation.id(),
adminLevelValue,
disputed,
relation.getString("name"),
disputed ? relation.getString("claimed_by") : null,
code
));
}
}
return null;
}
@Override
public void processAllOsm(SourceFeature feature, FeatureCollector features) {
if (!feature.canBeLine()) {
return;
}
var relationInfos = feature.relationInfo(BoundaryRelation.class);
if (!relationInfos.isEmpty()) {
int minAdminLevel = Integer.MAX_VALUE;
String disputedName = null, claimedBy = null;
Set<Long> regionIds = new HashSet<>();
boolean disputed = false;
// aggregate all borders this way is a part of - take the lowest
// admin level, and assume it is disputed if any relation is disputed.
for (var info : relationInfos) {
BoundaryRelation rel = info.relation();
disputed |= rel.disputed;
if (rel.adminLevel < minAdminLevel) {
minAdminLevel = rel.adminLevel;
}
if (rel.disputed) {
disputedName = disputedName == null ? rel.name : disputedName;
claimedBy = claimedBy == null ? rel.claimedBy : claimedBy;
}
if (minAdminLevel == 2 && regionNames.containsKey(info.relation().id)) {
regionIds.add(info.relation().id);
}
}
if (minAdminLevel <= 10) {
boolean wayIsDisputed = isDisputed(feature.tags());
disputed |= wayIsDisputed;
if (wayIsDisputed) {
disputedName = disputedName == null ? feature.getString("name") : disputedName;
claimedBy = claimedBy == null ? feature.getString("claimed_by") : claimedBy;
}
boolean maritime = feature.getBoolean("maritime") ||
feature.hasTag("natural", "coastline") ||
feature.hasTag("boundary_type", "maritime");
int minzoom =
(maritime && minAdminLevel == 2) ? 4 :
minAdminLevel <= 4 ? 5 :
minAdminLevel <= 6 ? 9 :
minAdminLevel <= 8 ? 11 : 12;
if (addCountryNames && !regionIds.isEmpty()) {
// save for later
try {
CountryBoundaryComponent component = new CountryBoundaryComponent(
minAdminLevel,
disputed,
maritime,
minzoom,
feature.line(),
regionIds,
claimedBy,
disputedName
);
// multiple threads may update this concurrently
synchronized (this) {
boundariesToMerge.computeIfAbsent(component.groupingKey(), key -> new ArrayList<>()).add(component.line);
for (var info : relationInfos) {
var rel = info.relation();
if (rel.adminLevel <= 2) {
regionGeometries.computeIfAbsent(rel.id, id -> new ArrayList<>()).add(component.line);
}
}
}
} catch (GeometryException e) {
LOGGER.warn("Cannot extract boundary line from " + feature);
}
} else {
features.line(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.ADMIN_LEVEL, minAdminLevel)
.setAttr(Fields.DISPUTED, disputed ? 1 : 0)
.setAttr(Fields.MARITIME, maritime ? 1 : 0)
.setMinPixelSizeAtAllZooms(0)
.setMinZoom(minzoom)
.setAttr(Fields.CLAIMED_BY, claimedBy)
.setAttr(Fields.DISPUTED_NAME, editName(disputedName));
}
}
}
}
@Override
public void finish(String sourceName, FeatureCollector.Factory featureCollectors,
Consumer<FeatureCollector.Feature> emit) {
if (BasemapProfile.OSM_SOURCE.equals(sourceName)) {
var timer = stats.startStage("boundaries");
LongObjectMap<PreparedGeometry> countryBoundaries = prepareRegionPolygons();
for (var entry : boundariesToMerge.entrySet()) {
CountryBoundaryComponent key = entry.getKey();
LineMerger merger = new LineMerger();
for (Geometry geom : entry.getValue()) {
merger.add(geom);
}
entry.getValue().clear();
for (Object merged : merger.getMergedLineStrings()) {
if (merged instanceof LineString lineString) {
BorderingRegions borderingRegions = getBorderingRegions(countryBoundaries, key.regions, lineString);
var features = featureCollectors.get(SimpleFeature.fromWorldGeometry(lineString));
features.line(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.ADMIN_LEVEL, key.adminLevel)
.setAttr(Fields.DISPUTED, key.disputed ? 1 : 0)
.setAttr(Fields.MARITIME, key.maritime ? 1 : 0)
.setAttr(Fields.CLAIMED_BY, key.claimedBy)
.setAttr(Fields.DISPUTED_NAME, key.disputed ? editName(key.name) : null)
.setAttr(Fields.ADM0_L, borderingRegions.left == null ? null : regionNames.get(borderingRegions.left))
.setAttr(Fields.ADM0_R, borderingRegions.right == null ? null : regionNames.get(borderingRegions.right))
.setMinPixelSizeAtAllZooms(0)
.setMinZoom(key.minzoom);
for (var feature : features) {
emit.accept(feature);
}
}
}
}
timer.stop();
}
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) {
double minLength = config.minFeatureSize(zoom);
double tolerance = config.tolerance(zoom);
return FeatureMerge.mergeLineStrings(items, attrs -> minLength, tolerance, BUFFER_SIZE);
}
/** Returns the left and right country for {@code lineString}. */
private BorderingRegions getBorderingRegions(
LongObjectMap<PreparedGeometry> countryBoundaries,
Set<Long> allRegions,
LineString lineString
) {
Set<Long> validRegions = allRegions.stream()
.filter(countryBoundaries::containsKey)
.collect(Collectors.toSet());
if (validRegions.isEmpty()) {
return BorderingRegions.empty();
}
List<Long> rights = new ArrayList<>();
List<Long> lefts = new ArrayList<>();
int steps = 10;
for (int i = 0; i < steps; i++) {
double ratio = (double) (i + 1) / (steps + 2);
Point right = GeoUtils.pointAlongOffset(lineString, ratio, COUNTRY_TEST_OFFSET);
Point left = GeoUtils.pointAlongOffset(lineString, ratio, -COUNTRY_TEST_OFFSET);
for (Long regionId : validRegions) {
PreparedGeometry geom = countryBoundaries.get(regionId);
if (geom != null) {
if (geom.contains(right)) {
rights.add(regionId);
} else if (geom.contains(left)) {
lefts.add(regionId);
}
}
}
}
var right = mode(rights);
if (right != null) {
lefts.removeAll(List.of(right));
}
var left = mode(lefts);
if (left == null && right == null) {
Coordinate point = GeoUtils.worldToLatLonCoords(GeoUtils.pointAlongOffset(lineString, 0.5, 0)).getCoordinate();
LOGGER.warn("no left or right country for border between OSM country relations: %s around %s"
.formatted(
validRegions,
Format.osmDebugUrl(10, point)
));
}
return new BorderingRegions(left, right);
}
/** Returns a map from region ID to prepared geometry optimized for {@code contains} queries. */
private LongObjectMap<PreparedGeometry> prepareRegionPolygons() {
LOGGER.info("Creating polygons for " + regionGeometries.size() + " boundaries");
LongObjectMap<PreparedGeometry> countryBoundaries = Hppc.newLongObjectHashMap();
for (var entry : regionGeometries.entrySet()) {
Long regionId = entry.getKey();
Polygonizer polygonizer = new Polygonizer();
polygonizer.add(entry.getValue());
try {
Geometry combined = polygonizer.getGeometry().union();
if (combined.isEmpty()) {
LOGGER.warn("Unable to form closed polygon for OSM relation " + regionId + " (likely missing edges)");
} else {
countryBoundaries.put(regionId, PreparedGeometryFactory.prepare(combined));
}
} catch (TopologyException e) {
LOGGER
.warn("Unable to build boundary polygon for OSM relation " + regionId + ": " + e.getMessage());
}
}
LOGGER.info("Finished creating " + countryBoundaries.size() + " country polygons");
return countryBoundaries;
}
/** Returns most frequently-occurring element in {@code list}. */
private static Long mode(List<Long> list) {
return list.stream()
.collect(groupingBy(Function.identity(), counting())).entrySet().stream()
.max(Map.Entry.comparingByValue())
.map(Map.Entry::getKey)
.orElse(null);
}
private record BorderingRegions(Long left, Long right) {
public static BorderingRegions empty() {
return new BorderingRegions(null, null);
}
}
/**
* Minimal set of information extracted from a boundary relation to be used when processing each way in that relation.
*/
private record BoundaryRelation(
long id,
int adminLevel,
boolean disputed,
String name,
String claimedBy,
String iso3166alpha3
) implements OsmRelationInfo {
@Override
public long estimateMemoryUsageBytes() {
return CLASS_HEADER_BYTES + MemoryEstimator.estimateSizeLong(id) + MemoryEstimator.estimateSizeInt(adminLevel) +
estimateSize(disputed) + POINTER_BYTES + estimateSize(name) + POINTER_BYTES + estimateSize(claimedBy) +
POINTER_BYTES + estimateSize(iso3166alpha3);
}
}
/**
* Information to hold onto from processing a way in a boundary relation to determine the left/right region ID later.
*/
private record CountryBoundaryComponent(
int adminLevel,
boolean disputed,
boolean maritime,
int minzoom,
Geometry line,
Set<Long> regions,
String claimedBy,
String name
) {
CountryBoundaryComponent groupingKey() {
return new CountryBoundaryComponent(adminLevel, disputed, maritime, minzoom, null, regions, claimedBy, name);
}
}
}

191
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Building.java
Wyświetl plik

@ -1,191 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.coalesce;
import static com.onthegomap.planetiler.util.MemoryEstimator.CLASS_HEADER_BYTES;
import static com.onthegomap.planetiler.util.Parse.parseDoubleOrNull;
import static java.util.Map.entry;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmRelationInfo;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.MemoryEstimator;
import com.onthegomap.planetiler.util.Translations;
import java.util.List;
import java.util.Locale;
import java.util.Map;
/**
* Defines the logic for generating map elements for buildings in the {@code building} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/building">OpenMapTiles building sql
* files</a>.
*/
public class Building implements
OpenMapTilesSchema.Building,
Tables.OsmBuildingPolygon.Handler,
BasemapProfile.FeaturePostProcessor,
BasemapProfile.OsmRelationPreprocessor {
/*
* Emit all buildings from OSM data at z14.
*
* At z13, emit all buildings at process-time, but then at tile render-time,
* merge buildings that are overlapping or almost touching into combined
* buildings so that entire city blocks show up as a single building polygon.
*
* THIS IS VERY EXPENSIVE! Merging buildings at z13 adds about 50% to the
* total map generation time. To disable it, set building_merge_z13 argument
* to false.
*/
private static final Map<String, String> MATERIAL_COLORS = Map.ofEntries(
entry("cement_block", "#6a7880"),
entry("brick", "#bd8161"),
entry("plaster", "#dadbdb"),
entry("wood", "#d48741"),
entry("concrete", "#d3c2b0"),
entry("metal", "#b7b1a6"),
entry("stone", "#b4a995"),
entry("mud", "#9d8b75"),
entry("steel", "#b7b1a6"), // same as metal
entry("glass", "#5a81a0"),
entry("traditional", "#bd8161"), // same as brick
entry("masonry", "#bd8161"), // same as brick
entry("Brick", "#bd8161"), // same as brick
entry("tin", "#b7b1a6"), // same as metal
entry("timber_framing", "#b3b0a9"),
entry("sandstone", "#b4a995"), // same as stone
entry("clay", "#9d8b75") // same as mud
);
private final boolean mergeZ13Buildings;
public Building(Translations translations, PlanetilerConfig config, Stats stats) {
this.mergeZ13Buildings = config.arguments().getBoolean(
"building_merge_z13",
"building layer: merge nearby buildings at z13",
true
);
}
@Override
public List<OsmRelationInfo> preprocessOsmRelation(OsmElement.Relation relation) {
if (relation.hasTag("type", "building")) {
return List.of(new BuildingRelationInfo(relation.id()));
}
return null;
}
@Override
public void process(Tables.OsmBuildingPolygon element, FeatureCollector features) {
Boolean hide3d = null;
var relations = element.source().relationInfo(BuildingRelationInfo.class);
for (var relation : relations) {
if ("outline".equals(relation.role())) {
hide3d = true;
break;
}
}
String color = element.colour();
if (color == null && element.material() != null) {
color = MATERIAL_COLORS.get(element.material());
}
if (color != null) {
color = color.toLowerCase(Locale.ROOT);
}
Double height = coalesce(
parseDoubleOrNull(element.height()),
parseDoubleOrNull(element.buildingheight())
);
Double minHeight = coalesce(
parseDoubleOrNull(element.minHeight()),
parseDoubleOrNull(element.buildingminHeight())
);
Double levels = coalesce(
parseDoubleOrNull(element.levels()),
parseDoubleOrNull(element.buildinglevels())
);
Double minLevels = coalesce(
parseDoubleOrNull(element.minLevel()),
parseDoubleOrNull(element.buildingminLevel())
);
int renderHeight = (int) Math.ceil(height != null ? height : levels != null ? (levels * 3.66) : 5);
int renderMinHeight = (int) Math.floor(minHeight != null ? minHeight : minLevels != null ? (minLevels * 3.66) : 0);
if (renderHeight < 3660 && renderMinHeight < 3660) {
var feature = features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setMinZoom(13)
.setMinPixelSize(2)
.setAttrWithMinzoom(Fields.RENDER_HEIGHT, renderHeight, 14)
.setAttrWithMinzoom(Fields.RENDER_MIN_HEIGHT, renderMinHeight, 14)
.setAttrWithMinzoom(Fields.COLOUR, color, 14)
.setAttrWithMinzoom(Fields.HIDE_3D, hide3d, 14)
.setSortKey(renderHeight);
if (mergeZ13Buildings) {
feature
.setMinPixelSize(0.1)
.setPixelTolerance(0.25);
}
}
}
@Override
public List<VectorTile.Feature> postProcess(int zoom,
List<VectorTile.Feature> items) throws GeometryException {
return (mergeZ13Buildings && zoom == 13) ? FeatureMerge.mergeNearbyPolygons(items, 4, 4, 0.5, 0.5) : items;
}
private record BuildingRelationInfo(long id) implements OsmRelationInfo {
@Override
public long estimateMemoryUsageBytes() {
return CLASS_HEADER_BYTES + MemoryEstimator.estimateSizeLong(id);
}
}
}

65
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Housenumber.java
Wyświetl plik

@ -1,65 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
/**
* Defines the logic for generating map elements in the {@code housenumber} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/housenumber">OpenMapTiles housenumber sql
* files</a>.
*/
public class Housenumber implements
OpenMapTilesSchema.Housenumber,
Tables.OsmHousenumberPoint.Handler {
public Housenumber(Translations translations, PlanetilerConfig config, Stats stats) {}
@Override
public void process(Tables.OsmHousenumberPoint element, FeatureCollector features) {
features.centroidIfConvex(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.HOUSENUMBER, element.housenumber())
.setMinZoom(14);
}
}

183
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Landcover.java
Wyświetl plik

@ -1,183 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.ZoomFunction;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* Defines the logic for generating map elements for natural land cover polygons like ice, sand, and forest in the
* {@code landcover} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/landcover">OpenMapTiles landcover sql
* files</a>.
*/
public class Landcover implements
OpenMapTilesSchema.Landcover,
BasemapProfile.NaturalEarthProcessor,
Tables.OsmLandcoverPolygon.Handler,
BasemapProfile.FeaturePostProcessor {
/*
* Large ice areas come from natural earth and the rest come from OpenStreetMap at higher zoom
* levels. At render-time, postProcess() merges polygons into larger connected area based
* on the number of points in the original area. Since postProcess() only has visibility into
* features on a single tile, process() needs to pass the number of points the original feature
* had through using a temporary "_numpoints" attribute.
*/
public static final ZoomFunction<Number> MIN_PIXEL_SIZE_THRESHOLDS = ZoomFunction.fromMaxZoomThresholds(Map.of(
13, 8,
10, 4,
9, 2
));
private static final String TEMP_NUM_POINTS_ATTR = "_numpoints";
private static final Set<String> WOOD_OR_FOREST = Set.of(
FieldValues.SUBCLASS_WOOD,
FieldValues.SUBCLASS_FOREST
);
private final MultiExpression.Index<String> classMapping;
public Landcover(Translations translations, PlanetilerConfig config, Stats stats) {
this.classMapping = FieldMappings.Class.index();
}
private String getClassFromSubclass(String subclass) {
return subclass == null ? null : classMapping.getOrElse(Map.of(Fields.SUBCLASS, subclass), null);
}
@Override
public void processNaturalEarth(String table, SourceFeature feature,
FeatureCollector features) {
record LandcoverInfo(String subclass, int minzoom, int maxzoom) {}
LandcoverInfo info = switch (table) {
case "ne_110m_glaciated_areas" -> new LandcoverInfo(FieldValues.SUBCLASS_GLACIER, 0, 1);
case "ne_50m_glaciated_areas" -> new LandcoverInfo(FieldValues.SUBCLASS_GLACIER, 2, 4);
case "ne_10m_glaciated_areas" -> new LandcoverInfo(FieldValues.SUBCLASS_GLACIER, 5, 6);
case "ne_50m_antarctic_ice_shelves_polys" -> new LandcoverInfo("ice_shelf", 2, 4);
case "ne_10m_antarctic_ice_shelves_polys" -> new LandcoverInfo("ice_shelf", 5, 6);
default -> null;
};
if (info != null) {
String clazz = getClassFromSubclass(info.subclass);
if (clazz != null) {
features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, clazz)
.setAttr(Fields.SUBCLASS, info.subclass)
.setZoomRange(info.minzoom, info.maxzoom);
}
}
}
@Override
public void process(Tables.OsmLandcoverPolygon element, FeatureCollector features) {
String subclass = element.subclass();
String clazz = getClassFromSubclass(subclass);
if (clazz != null) {
features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setMinPixelSizeOverrides(MIN_PIXEL_SIZE_THRESHOLDS)
.setAttr(Fields.CLASS, clazz)
.setAttr(Fields.SUBCLASS, subclass)
.setNumPointsAttr(TEMP_NUM_POINTS_ATTR)
.setMinZoom(7);
}
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) throws GeometryException {
if (zoom < 7 || zoom > 13) {
for (var item : items) {
item.attrs().remove(TEMP_NUM_POINTS_ATTR);
}
return items;
} else { // z7-13
// merging only merges polygons with the same attributes, so use this temporary key
// to separate features into layers that will be merged separately
String tempGroupKey = "_group";
List<VectorTile.Feature> result = new ArrayList<>();
List<VectorTile.Feature> toMerge = new ArrayList<>();
for (var item : items) {
Map<String, Object> attrs = item.attrs();
Object numPointsObj = attrs.remove(TEMP_NUM_POINTS_ATTR);
Object subclassObj = attrs.get(Fields.SUBCLASS);
if (numPointsObj instanceof Number num && subclassObj instanceof String subclass) {
long numPoints = num.longValue();
if (zoom >= 10) {
if (WOOD_OR_FOREST.contains(subclass) && numPoints < 300) {
attrs.put(tempGroupKey, "<300");
toMerge.add(item);
} else { // don't merge
result.add(item);
}
} else if (zoom == 9) {
if (WOOD_OR_FOREST.contains(subclass)) {
attrs.put(tempGroupKey, numPoints < 300 ? "<300" : ">300");
toMerge.add(item);
} else { // don't merge
result.add(item);
}
} else { // zoom between 7 and 8
toMerge.add(item);
}
} else {
result.add(item);
}
}
var merged = FeatureMerge.mergeOverlappingPolygons(toMerge, 4);
for (var item : merged) {
item.attrs().remove(tempGroupKey);
}
result.addAll(merged);
return result;
}
}
}

112
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Landuse.java
Wyświetl plik

@ -1,112 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.coalesce;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.ZoomFunction;
import java.util.Map;
import java.util.Set;
/**
* Defines the logic for generating map elements for man-made land use polygons like cemeteries, zoos, and hospitals in
* the {@code landuse} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/landuse">OpenMapTiles landuse sql files</a>.
*/
public class Landuse implements
OpenMapTilesSchema.Landuse,
BasemapProfile.NaturalEarthProcessor,
Tables.OsmLandusePolygon.Handler {
private static final ZoomFunction<Number> MIN_PIXEL_SIZE_THRESHOLDS = ZoomFunction.fromMaxZoomThresholds(Map.of(
13, 4,
7, 2,
6, 1
));
private static final Set<String> Z6_CLASSES = Set.of(
FieldValues.CLASS_RESIDENTIAL,
FieldValues.CLASS_SUBURB,
FieldValues.CLASS_QUARTER,
FieldValues.CLASS_NEIGHBOURHOOD
);
public Landuse(Translations translations, PlanetilerConfig config, Stats stats) {}
@Override
public void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features) {
if ("ne_50m_urban_areas".equals(table)) {
Double scalerank = Parse.parseDoubleOrNull(feature.getTag("scalerank"));
if (scalerank != null && scalerank <= 2) {
features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, FieldValues.CLASS_RESIDENTIAL)
.setZoomRange(4, 5);
}
}
}
@Override
public void process(Tables.OsmLandusePolygon element, FeatureCollector features) {
String clazz = coalesce(
nullIfEmpty(element.landuse()),
nullIfEmpty(element.amenity()),
nullIfEmpty(element.leisure()),
nullIfEmpty(element.tourism()),
nullIfEmpty(element.place()),
nullIfEmpty(element.waterway())
);
if (clazz != null) {
if ("grave_yard".equals(clazz)) {
clazz = FieldValues.CLASS_CEMETERY;
}
features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, clazz)
.setMinPixelSizeOverrides(MIN_PIXEL_SIZE_THRESHOLDS)
.setMinZoom(Z6_CLASSES.contains(clazz) ? 6 : 9);
}
}
}

203
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/MountainPeak.java
Wyświetl plik

@ -1,203 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.elevationTags;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import com.carrotsearch.hppc.LongIntMap;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import java.util.List;
import java.util.concurrent.atomic.AtomicBoolean;
import org.locationtech.jts.geom.Geometry;
import org.locationtech.jts.geom.Point;
import org.locationtech.jts.geom.prep.PreparedGeometry;
import org.locationtech.jts.geom.prep.PreparedGeometryFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Defines the logic for generating map elements for mountain peak label points in the {@code mountain_peak} layer from
* source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/mountain_peak">OpenMapTiles mountain_peak
* sql files</a>.
*/
public class MountainPeak implements
BasemapProfile.NaturalEarthProcessor,
OpenMapTilesSchema.MountainPeak,
Tables.OsmPeakPoint.Handler,
Tables.OsmMountainLinestring.Handler,
BasemapProfile.FeaturePostProcessor {
/*
* Mountain peaks come from OpenStreetMap data and are ranked by importance (based on if they
* have a name or wikipedia page) then by elevation. Uses the "label grid" feature to limit
* label density by only taking the top 5 most important mountain peaks within each 100x100px
* square.
*/
private static final Logger LOGGER = LoggerFactory.getLogger(MountainPeak.class);
private final Translations translations;
private final Stats stats;
// keep track of areas that prefer feet to meters to set customary_ft=1 (just U.S.)
private PreparedGeometry unitedStates = null;
private final AtomicBoolean loggedNoUS = new AtomicBoolean(false);
public MountainPeak(Translations translations, PlanetilerConfig config, Stats stats) {
this.translations = translations;
this.stats = stats;
}
@Override
public void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features) {
if ("ne_10m_admin_0_countries".equals(table) && feature.hasTag("iso_a2", "US")) {
// multiple threads call this method concurrently, US polygon *should* only be found
// once, but just to be safe synchronize updates to that field
synchronized (this) {
try {
Geometry boundary = feature.polygon();
unitedStates = PreparedGeometryFactory.prepare(boundary);
} catch (GeometryException e) {
LOGGER.error("Failed to get United States Polygon for mountain_peak layer: " + e);
}
}
}
}
@Override
public void process(Tables.OsmPeakPoint element, FeatureCollector features) {
Double meters = Parse.meters(element.ele());
if (meters != null && Math.abs(meters) < 10_000) {
var feature = features.point(LAYER_NAME)
.setAttr(Fields.CLASS, element.source().getTag("natural"))
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.putAttrs(elevationTags(meters))
.setSortKeyDescending(
meters.intValue() +
(nullIfEmpty(element.wikipedia()) != null ? 10_000 : 0) +
(nullIfEmpty(element.name()) != null ? 10_000 : 0)
)
.setMinZoom(7)
// need to use a larger buffer size to allow enough points through to not cut off
// any label grid squares which could lead to inconsistent label ranks for a feature
// in adjacent tiles. postProcess() will remove anything outside the desired buffer.
.setBufferPixels(100)
.setPointLabelGridSizeAndLimit(13, 100, 5);
if (peakInAreaUsingFeet(element)) {
feature.setAttr(Fields.CUSTOMARY_FT, 1);
}
}
}
@Override
public void process(Tables.OsmMountainLinestring element, FeatureCollector features) {
// TODO rank is approximate to sort important/named ridges before others, should switch to labelgrid for linestrings later
int rank = 3 -
(nullIfEmpty(element.wikipedia()) != null ? 1 : 0) -
(nullIfEmpty(element.name()) != null ? 1 : 0);
features.line(LAYER_NAME)
.setAttr(Fields.CLASS, element.source().getTag("natural"))
.setAttr(Fields.RANK, rank)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setSortKey(rank)
.setMinZoom(13)
.setBufferPixels(100);
}
/** Returns true if {@code element} is a point in an area where feet are used insead of meters (the US). */
private boolean peakInAreaUsingFeet(Tables.OsmPeakPoint element) {
if (unitedStates == null) {
if (!loggedNoUS.get() && loggedNoUS.compareAndSet(false, true)) {
LOGGER.warn("No US polygon for inferring mountain_peak customary_ft tag");
}
} else {
try {
Geometry wayGeometry = element.source().worldGeometry();
return unitedStates.intersects(wayGeometry);
} catch (GeometryException e) {
e.log(stats, "omt_mountain_peak_us_test",
"Unable to test mountain_peak against US polygon: " + element.source().id());
}
}
return false;
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) {
LongIntMap groupCounts = Hppc.newLongIntHashMap();
for (int i = 0; i < items.size(); i++) {
VectorTile.Feature feature = items.get(i);
int gridrank = groupCounts.getOrDefault(feature.group(), 1);
groupCounts.put(feature.group(), gridrank + 1);
// now that we have accurate ranks, remove anything outside the desired buffer
if (!insideTileBuffer(feature)) {
items.set(i, null);
} else if (!feature.attrs().containsKey(Fields.RANK)) {
feature.attrs().put(Fields.RANK, gridrank);
}
}
return items;
}
private static boolean insideTileBuffer(double xOrY) {
return xOrY >= -BUFFER_SIZE && xOrY <= 256 + BUFFER_SIZE;
}
private boolean insideTileBuffer(VectorTile.Feature feature) {
try {
Geometry geom = feature.geometry().decode();
return !(geom instanceof Point point) || (insideTileBuffer(point.getX()) && insideTileBuffer(point.getY()));
} catch (GeometryException e) {
e.log(stats, "mountain_peak_decode_point", "Error decoding mountain peak point: " + feature.attrs());
return false;
}
}
}

161
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Park.java
Wyświetl plik

@ -1,161 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.coalesce;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import static com.onthegomap.planetiler.collection.FeatureGroup.SORT_KEY_BITS;
import com.carrotsearch.hppc.LongIntMap;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.geo.GeometryType;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.SortKey;
import com.onthegomap.planetiler.util.Translations;
import java.util.List;
import java.util.Locale;
/**
* Defines the logic for generating map elements for designated parks polygons and their label points in the {@code
* park} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/park">OpenMapTiles park sql files</a>.
*/
public class Park implements
OpenMapTilesSchema.Park,
Tables.OsmParkPolygon.Handler,
BasemapProfile.FeaturePostProcessor {
// constants for packing the minimum zoom ordering of park labels into the sort-key field
private static final int PARK_NATIONAL_PARK_BOOST = 1 << (SORT_KEY_BITS - 1);
private static final int PARK_WIKIPEDIA_BOOST = 1 << (SORT_KEY_BITS - 2);
// constants for determining the minimum zoom level for a park label based on its area
private static final double WORLD_AREA_FOR_70K_SQUARE_METERS =
Math.pow(GeoUtils.metersToPixelAtEquator(0, Math.sqrt(70_000)) / 256d, 2);
private static final double LOG2 = Math.log(2);
private static final int PARK_AREA_RANGE = 1 << (SORT_KEY_BITS - 3);
private static final double SMALLEST_PARK_WORLD_AREA = Math.pow(4, -26); // 2^14 tiles, 2^12 pixels per tile
private final Translations translations;
private final Stats stats;
public Park(Translations translations, PlanetilerConfig config, Stats stats) {
this.stats = stats;
this.translations = translations;
}
@Override
public void process(Tables.OsmParkPolygon element, FeatureCollector features) {
String protectionTitle = element.protectionTitle();
if (protectionTitle != null) {
protectionTitle = protectionTitle.replace(' ', '_').toLowerCase(Locale.ROOT);
}
String clazz = coalesce(
nullIfEmpty(protectionTitle),
nullIfEmpty(element.boundary()),
nullIfEmpty(element.leisure())
);
// park shape
var outline = features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttrWithMinzoom(Fields.CLASS, clazz, 5)
.setMinPixelSize(2)
.setMinZoom(4);
// park name label point (if it has one)
if (element.name() != null) {
try {
double area = element.source().area();
int minzoom = getMinZoomForArea(area);
var names = LanguageUtils.getNamesWithoutTranslations(element.source().tags());
outline.putAttrsWithMinzoom(names, 5);
features.pointOnSurface(LAYER_NAME).setBufferPixels(256)
.setAttr(Fields.CLASS, clazz)
.putAttrs(names)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setPointLabelGridPixelSize(14, 100)
.setSortKey(SortKey
.orderByTruesFirst("national_park".equals(clazz))
.thenByTruesFirst(element.source().hasTag("wikipedia") || element.source().hasTag("wikidata"))
.thenByLog(area, 1d, SMALLEST_PARK_WORLD_AREA, 1 << (SORT_KEY_BITS - 2) - 1)
.get()
).setMinZoom(minzoom);
} catch (GeometryException e) {
e.log(stats, "omt_park_area", "Unable to get park area for " + element.source().id());
}
}
}
private int getMinZoomForArea(double area) {
// sql filter: area > 70000*2^(20-zoom_level)
// simplifies to: zoom_level > 20 - log(area / 70000) / log(2)
int minzoom = (int) Math.floor(20 - Math.log(area / WORLD_AREA_FOR_70K_SQUARE_METERS) / LOG2);
minzoom = Math.min(14, Math.max(5, minzoom));
return minzoom;
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) throws GeometryException {
// infer the "rank" attribute from point ordering within each label grid square
LongIntMap counts = Hppc.newLongIntHashMap();
for (VectorTile.Feature feature : items) {
if (feature.geometry().geomType() == GeometryType.POINT && feature.hasGroup()) {
int count = counts.getOrDefault(feature.group(), 0) + 1;
feature.attrs().put("rank", count);
counts.put(feature.group(), count);
}
}
if (zoom <= 4) {
items = FeatureMerge.mergeOverlappingPolygons(items, 0);
}
return items;
}
}

426
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Place.java
Wyświetl plik

@ -1,426 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.coalesce;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import static com.onthegomap.planetiler.basemap.util.Utils.nullOrEmpty;
import static com.onthegomap.planetiler.collection.FeatureGroup.SORT_KEY_BITS;
import com.carrotsearch.hppc.LongIntMap;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.geo.PointIndex;
import com.onthegomap.planetiler.geo.PolygonIndex;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.SortKey;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.ZoomFunction;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.TreeMap;
import java.util.stream.Collectors;
import java.util.stream.DoubleStream;
import java.util.stream.Stream;
import org.apache.commons.lang3.StringUtils;
import org.locationtech.jts.geom.Point;
/**
* Defines the logic for generating label points for populated places like continents, countries, cities, and towns in
* the {@code place} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/place">OpenMapTiles place sql files</a>.
*/
public class Place implements
OpenMapTilesSchema.Place,
BasemapProfile.NaturalEarthProcessor,
Tables.OsmContinentPoint.Handler,
Tables.OsmCountryPoint.Handler,
Tables.OsmStatePoint.Handler,
Tables.OsmIslandPoint.Handler,
Tables.OsmIslandPolygon.Handler,
Tables.OsmCityPoint.Handler,
BasemapProfile.FeaturePostProcessor {
/*
* Place labels locations and names come from OpenStreetMap, but we also join with natural
* earth state/country geographic areas and city point labels to give a hint for what rank
* and minimum zoom level to use for those points.
*/
private static final TreeMap<Double, Integer> ISLAND_AREA_RANKS = new TreeMap<>(Map.of(
Double.MAX_VALUE, 3,
squareMetersToWorldArea(40_000_000), 4,
squareMetersToWorldArea(15_000_000), 5,
squareMetersToWorldArea(1_000_000), 6
));
private static final double MIN_ISLAND_WORLD_AREA = Math.pow(4, -26); // 2^14 tiles, 2^12 pixels per tile
private static final double CITY_JOIN_DISTANCE = GeoUtils.metersToPixelAtEquator(0, 50_000) / 256d;
// constants for packing place label precedence into the sort-key field
private static final double MAX_CITY_POPULATION = 100_000_000d;
private static final Set<String> MAJOR_CITY_PLACES = Set.of("city", "town", "village");
private static final ZoomFunction<Number> LABEL_GRID_LIMITS = ZoomFunction.fromMaxZoomThresholds(Map.of(
8, 4,
9, 8,
10, 12,
12, 14
), 0);
private final Translations translations;
private final Stats stats;
// spatial indexes for joining natural earth place labels with their corresponding points
// from openstreetmap
private PolygonIndex<NaturalEarthRegion> countries = PolygonIndex.create();
private PolygonIndex<NaturalEarthRegion> states = PolygonIndex.create();
private PointIndex<NaturalEarthPoint> cities = PointIndex.create();
public Place(Translations translations, PlanetilerConfig config, Stats stats) {
this.translations = translations;
this.stats = stats;
}
/** Returns the portion of the world that {@code squareMeters} covers where 1 is the entire planet. */
private static double squareMetersToWorldArea(double squareMeters) {
double oneSideMeters = Math.sqrt(squareMeters);
double oneSideWorld = GeoUtils.metersToPixelAtEquator(0, oneSideMeters) / 256d;
return Math.pow(oneSideWorld, 2);
}
/**
* Packs place precedence ordering ({@code rank asc, place asc, population desc, name.length asc}) into an integer for
* the sort-key field.
*/
static int getSortKey(Integer rank, PlaceType place, long population, String name) {
return SortKey
// ORDER BY "rank" ASC NULLS LAST,
.orderByInt(rank == null ? 15 : rank, 0, 15) // 4 bits
// place ASC NULLS LAST,
.thenByInt(place == null ? 15 : place.ordinal(), 0, 15) // 4 bits
// population DESC NULLS LAST,
.thenByLog(population, MAX_CITY_POPULATION, 1, 1 << (SORT_KEY_BITS - 13) - 1)
// length(name) ASC
.thenByInt(name == null ? 0 : name.length(), 0, 31) // 5 bits
.get();
}
@Override
public void release() {
countries = null;
states = null;
cities = null;
}
@Override
public void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features) {
// store data from natural earth to help with ranks and min zoom levels when actually
// emitting features from openstreetmap data.
try {
switch (table) {
case "ne_10m_admin_0_countries" -> countries.put(feature.worldGeometry(), new NaturalEarthRegion(
feature.getString("name"), 6,
feature.getLong("scalerank"),
feature.getLong("labelrank")
));
case "ne_10m_admin_1_states_provinces" -> {
Double scalerank = Parse.parseDoubleOrNull(feature.getTag("scalerank"));
Double labelrank = Parse.parseDoubleOrNull(feature.getTag("labelrank"));
if (scalerank != null && scalerank <= 6 && labelrank != null && labelrank <= 7) {
states.put(feature.worldGeometry(), new NaturalEarthRegion(
feature.getString("name"), 6,
scalerank,
labelrank,
feature.getLong("datarank")
));
}
}
case "ne_10m_populated_places" -> cities.put(feature.worldGeometry(), new NaturalEarthPoint(
feature.getString("name"),
feature.getString("wikidataid"),
(int) feature.getLong("scalerank"),
Stream.of("name", "namealt", "meganame", "gn_ascii", "nameascii").map(feature::getString)
.filter(Objects::nonNull)
.map(s -> s.toLowerCase(Locale.ROOT))
.collect(Collectors.toSet())
));
}
} catch (GeometryException e) {
e.log(stats, "omt_place_ne",
"Error getting geometry for natural earth feature " + table + " " + feature.getTag("ogc_fid"));
}
}
@Override
public void process(Tables.OsmContinentPoint element, FeatureCollector features) {
if (!nullOrEmpty(element.name())) {
features.point(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setAttr(Fields.CLASS, FieldValues.CLASS_CONTINENT)
.setAttr(Fields.RANK, 1)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setZoomRange(0, 3);
}
}
@Override
public void process(Tables.OsmCountryPoint element, FeatureCollector features) {
if (nullOrEmpty(element.name())) {
return;
}
String isoA2 = coalesce(
nullIfEmpty(element.countryCodeIso31661Alpha2()),
nullIfEmpty(element.iso31661Alpha2()),
nullIfEmpty(element.iso31661())
);
if (isoA2 == null) {
return;
}
try {
// set country rank to 6, unless there is a match in natural earth that indicates it
// should be lower
int rank = 7;
NaturalEarthRegion country = countries.get(element.source().worldGeometry().getCentroid());
var names = LanguageUtils.getNames(element.source().tags(), translations);
if (country != null) {
if (nullOrEmpty(names.get(Fields.NAME_EN))) {
names.put(Fields.NAME_EN, country.name);
}
rank = country.rank;
}
rank = Math.max(1, Math.min(6, rank));
features.point(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(names)
.setAttr(Fields.ISO_A2, isoA2)
.setAttr(Fields.CLASS, FieldValues.CLASS_COUNTRY)
.setAttr(Fields.RANK, rank)
.setMinZoom(rank - 1)
.setSortKey(rank);
} catch (GeometryException e) {
e.log(stats, "omt_place_country",
"Unable to get point for OSM country " + element.source().id());
}
}
@Override
public void process(Tables.OsmStatePoint element, FeatureCollector features) {
try {
// want the containing (not nearest) state polygon since we pre-filter the states in the polygon index
// use natural earth to filter out any spurious states, and to set the rank field
NaturalEarthRegion state = states.getOnlyContaining(element.source().worldGeometry().getCentroid());
if (state != null) {
var names = LanguageUtils.getNames(element.source().tags(), translations);
if (nullOrEmpty(names.get(Fields.NAME_EN))) {
names.put(Fields.NAME_EN, state.name);
}
int rank = Math.min(6, Math.max(1, state.rank));
features.point(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(names)
.setAttr(Fields.CLASS, element.place())
.setAttr(Fields.RANK, rank)
.setMinZoom(2)
.setSortKey(rank);
}
} catch (GeometryException e) {
e.log(stats, "omt_place_state",
"Unable to get point for OSM state " + element.source().id());
}
}
@Override
public void process(Tables.OsmIslandPolygon element, FeatureCollector features) {
try {
double area = element.source().area();
int rank = ISLAND_AREA_RANKS.ceilingEntry(area).getValue();
int minzoom = rank <= 3 ? 8 : rank <= 4 ? 9 : 10;
features.pointOnSurface(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setAttr(Fields.CLASS, "island")
.setAttr(Fields.RANK, rank)
.setMinZoom(minzoom)
.setSortKey(SortKey.orderByLog(area, 1d, MIN_ISLAND_WORLD_AREA).get());
} catch (GeometryException e) {
e.log(stats, "omt_place_island_poly",
"Unable to get point for OSM island polygon " + element.source().id());
}
}
@Override
public void process(Tables.OsmIslandPoint element, FeatureCollector features) {
features.point(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setAttr(Fields.CLASS, "island")
.setAttr(Fields.RANK, 7)
.setMinZoom(12);
}
@Override
public void process(Tables.OsmCityPoint element, FeatureCollector features) {
Integer rank = null;
if (MAJOR_CITY_PLACES.contains(element.place())) {
// only for major cities, attempt to find a nearby natural earth label with a similar
// name and use that to set a rank from OSM that causes the label to be shown at lower
// zoom levels
try {
Point point = element.source().worldGeometry().getCentroid();
List<NaturalEarthPoint> neCities = cities.getWithin(point, CITY_JOIN_DISTANCE);
String rawName = coalesce(element.name(), "");
String name = coalesce(rawName, "").toLowerCase(Locale.ROOT);
String nameEn = coalesce(element.nameEn(), "").toLowerCase(Locale.ROOT);
String normalizedName = StringUtils.stripAccents(rawName);
String wikidata = element.source().getString("wikidata", "");
for (var neCity : neCities) {
if (wikidata.equals(neCity.wikidata) ||
neCity.names.contains(name) ||
neCity.names.contains(nameEn) ||
normalizedName.equals(neCity.name)) {
rank = neCity.scaleRank <= 5 ? neCity.scaleRank + 1 : neCity.scaleRank;
break;
}
}
} catch (GeometryException e) {
e.log(stats, "omt_place_city",
"Unable to get point for OSM city " + element.source().id());
}
}
String capital = element.capital();
PlaceType placeType = PlaceType.forName(element.place());
int minzoom = rank != null && rank == 1 ? 2 :
rank != null && rank <= 8 ? Math.max(3, rank - 1) :
placeType.ordinal() <= PlaceType.TOWN.ordinal() ? 7 :
placeType.ordinal() <= PlaceType.VILLAGE.ordinal() ? 8 :
placeType.ordinal() <= PlaceType.SUBURB.ordinal() ? 11 : 14;
var feature = features.point(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setAttr(Fields.CLASS, element.place())
.setAttr(Fields.RANK, rank)
.setMinZoom(minzoom)
.setSortKey(getSortKey(rank, placeType, element.population(), element.name()))
.setPointLabelGridPixelSize(12, 128);
if (rank == null) {
feature.setPointLabelGridLimit(LABEL_GRID_LIMITS);
}
if ("2".equals(capital) || "yes".equals(capital)) {
feature.setAttr(Fields.CAPITAL, 2);
} else if ("4".equals(capital)) {
feature.setAttr(Fields.CAPITAL, 4);
}
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) {
// infer the rank field from ordering of the place labels with each label grid square
LongIntMap groupCounts = Hppc.newLongIntHashMap();
for (VectorTile.Feature feature : items) {
int gridrank = groupCounts.getOrDefault(feature.group(), 1);
groupCounts.put(feature.group(), gridrank + 1);
if (!feature.attrs().containsKey(Fields.RANK)) {
feature.attrs().put(Fields.RANK, 10 + gridrank);
}
}
return items;
}
/** Ordering defines the precedence of place classes. */
enum PlaceType {
CITY("city"),
TOWN("town"),
VILLAGE("village"),
HAMLET("hamlet"),
SUBURB("suburb"),
QUARTER("quarter"),
NEIGHBORHOOD("neighbourhood"),
ISOLATED_DWELLING("isolated_dwelling"),
UNKNOWN("unknown");
private static final Map<String, PlaceType> byName = new HashMap<>();
static {
for (PlaceType place : values()) {
byName.put(place.name, place);
}
}
private final String name;
PlaceType(String name) {
this.name = name;
}
public static PlaceType forName(String name) {
return byName.getOrDefault(name, UNKNOWN);
}
}
/**
* Information extracted from a natural earth geographic region that will be inspected when joining with OpenStreetMap
* data.
*/
private record NaturalEarthRegion(String name, int rank) {
NaturalEarthRegion(String name, int maxRank, double... ranks) {
this(name, (int) Math.ceil(DoubleStream.of(ranks).average().orElse(maxRank)));
}
}
/**
* Information extracted from a natural earth place label that will be inspected when joining with OpenStreetMap data.
*/
private record NaturalEarthPoint(String name, String wikidata, int scaleRank, Set<String> names) {}
}

194
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Poi.java
Wyświetl plik

@ -1,194 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.coalesce;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfLong;
import static com.onthegomap.planetiler.basemap.util.Utils.nullOrEmpty;
import static java.util.Map.entry;
import com.carrotsearch.hppc.LongIntMap;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.ForwardingProfile;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import java.util.List;
import java.util.Map;
/**
* Defines the logic for generating map elements for things like shops, parks, and schools in the {@code poi} layer from
* source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/poi">OpenMapTiles poi sql files</a>.
*/
public class Poi implements
OpenMapTilesSchema.Poi,
Tables.OsmPoiPoint.Handler,
Tables.OsmPoiPolygon.Handler,
ForwardingProfile.FeaturePostProcessor {
/*
* process() creates the raw POI feature from OSM elements and postProcess()
* assigns the feature rank from order in the tile at render-time.
*/
private static final Map<String, Integer> CLASS_RANKS = Map.ofEntries(
entry(FieldValues.CLASS_HOSPITAL, 20),
entry(FieldValues.CLASS_RAILWAY, 40),
entry(FieldValues.CLASS_BUS, 50),
entry(FieldValues.CLASS_ATTRACTION, 70),
entry(FieldValues.CLASS_HARBOR, 75),
entry(FieldValues.CLASS_COLLEGE, 80),
entry(FieldValues.CLASS_SCHOOL, 85),
entry(FieldValues.CLASS_STADIUM, 90),
entry("zoo", 95),
entry(FieldValues.CLASS_TOWN_HALL, 100),
entry(FieldValues.CLASS_CAMPSITE, 110),
entry(FieldValues.CLASS_CEMETERY, 115),
entry(FieldValues.CLASS_PARK, 120),
entry(FieldValues.CLASS_LIBRARY, 130),
entry("police", 135),
entry(FieldValues.CLASS_POST, 140),
entry(FieldValues.CLASS_GOLF, 150),
entry(FieldValues.CLASS_SHOP, 400),
entry(FieldValues.CLASS_GROCERY, 500),
entry(FieldValues.CLASS_FAST_FOOD, 600),
entry(FieldValues.CLASS_CLOTHING_STORE, 700),
entry(FieldValues.CLASS_BAR, 800)
);
private final MultiExpression.Index<String> classMapping;
private final Translations translations;
public Poi(Translations translations, PlanetilerConfig config, Stats stats) {
this.classMapping = FieldMappings.Class.index();
this.translations = translations;
}
static int poiClassRank(String clazz) {
return CLASS_RANKS.getOrDefault(clazz, 1_000);
}
private String poiClass(String subclass, String mappingKey) {
subclass = coalesce(subclass, "");
return classMapping.getOrElse(Map.of(
"subclass", subclass,
"mapping_key", coalesce(mappingKey, "")
), subclass);
}
private int minzoom(String subclass, String mappingKey) {
boolean lowZoom = ("station".equals(subclass) && "railway".equals(mappingKey)) ||
"halt".equals(subclass) || "ferry_terminal".equals(subclass);
return lowZoom ? 12 : 14;
}
@Override
public void process(Tables.OsmPoiPoint element, FeatureCollector features) {
// TODO handle uic_ref => agg_stop
setupPoiFeature(element, features.point(LAYER_NAME));
}
@Override
public void process(Tables.OsmPoiPolygon element, FeatureCollector features) {
setupPoiFeature(element, features.centroidIfConvex(LAYER_NAME));
}
private <T extends Tables.WithSubclass & Tables.WithStation & Tables.WithFunicular & Tables.WithSport & Tables.WithInformation & Tables.WithReligion & Tables.WithMappingKey & Tables.WithName & Tables.WithIndoor & Tables.WithLayer & Tables.WithSource & Tables.WithOperator & Tables.WithNetwork> void setupPoiFeature(
T element, FeatureCollector.Feature output) {
String rawSubclass = element.subclass();
if ("station".equals(rawSubclass) && "subway".equals(element.station())) {
rawSubclass = "subway";
}
if ("station".equals(rawSubclass) && "yes".equals(element.funicular())) {
rawSubclass = "halt";
}
// ATM names fall back to operator, or else network
String name = element.name();
var tags = element.source().tags();
if ("atm".equals(rawSubclass) && nullOrEmpty(name)) {
name = coalesce(nullIfEmpty(element.operator()), nullIfEmpty(element.network()));
if (name != null) {
tags.put("name", name);
}
}
String subclass = switch (rawSubclass) {
case "information" -> nullIfEmpty(element.information());
case "place_of_worship" -> nullIfEmpty(element.religion());
case "pitch" -> nullIfEmpty(element.sport());
default -> rawSubclass;
};
String poiClass = poiClass(rawSubclass, element.mappingKey());
int poiClassRank = poiClassRank(poiClass);
int rankOrder = poiClassRank + ((nullOrEmpty(name)) ? 2000 : 0);
output.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, poiClass)
.setAttr(Fields.SUBCLASS, subclass)
.setAttr(Fields.LAYER, nullIfLong(element.layer(), 0))
.setAttr(Fields.LEVEL, Parse.parseLongOrNull(element.source().getTag("level")))
.setAttr(Fields.INDOOR, element.indoor() ? 1 : null)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setPointLabelGridPixelSize(14, 64)
.setSortKey(rankOrder)
.setMinZoom(minzoom(element.subclass(), element.mappingKey()));
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) {
// infer the "rank" field from the order of features within each label grid square
LongIntMap groupCounts = Hppc.newLongIntHashMap();
for (VectorTile.Feature feature : items) {
int gridrank = groupCounts.getOrDefault(feature.group(), 1);
groupCounts.put(feature.group(), gridrank + 1);
if (!feature.attrs().containsKey(Fields.RANK)) {
feature.attrs().put(Fields.RANK, gridrank);
}
}
return items;
}
}

569
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Transportation.java
Wyświetl plik

@ -1,569 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.*;
import static com.onthegomap.planetiler.util.MemoryEstimator.CLASS_HEADER_BYTES;
import static com.onthegomap.planetiler.util.MemoryEstimator.POINTER_BYTES;
import static com.onthegomap.planetiler.util.MemoryEstimator.estimateSize;
import static java.util.Map.entry;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmReader;
import com.onthegomap.planetiler.reader.osm.OsmRelationInfo;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.MemoryEstimator;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.ZoomFunction;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.locationtech.jts.geom.Geometry;
import org.locationtech.jts.geom.LineString;
import org.locationtech.jts.geom.prep.PreparedGeometry;
import org.locationtech.jts.geom.prep.PreparedGeometryFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Defines the logic for generating map elements for roads, shipways, railroads, and paths in the {@code transportation}
* layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/transportation">OpenMapTiles transportation
* sql files</a>.
*/
public class Transportation implements
OpenMapTilesSchema.Transportation,
Tables.OsmAerialwayLinestring.Handler,
Tables.OsmHighwayLinestring.Handler,
Tables.OsmRailwayLinestring.Handler,
Tables.OsmShipwayLinestring.Handler,
Tables.OsmHighwayPolygon.Handler,
BasemapProfile.NaturalEarthProcessor,
BasemapProfile.FeaturePostProcessor,
BasemapProfile.OsmRelationPreprocessor,
BasemapProfile.IgnoreWikidata {
/*
* Generates the shape for roads, trails, ferries, railways with detailed
* attributes for rendering, but not any names. The transportation_name
* layer includes names, but less detailed attributes.
*/
private static final Logger LOGGER = LoggerFactory.getLogger(Transportation.class);
private static final Pattern GREAT_BRITAIN_REF_NETWORK_PATTERN = Pattern.compile("^[AM][0-9AM()]+");
private static final MultiExpression.Index<String> classMapping = FieldMappings.Class.index();
private static final Set<String> RAILWAY_RAIL_VALUES = Set.of(
FieldValues.SUBCLASS_RAIL,
FieldValues.SUBCLASS_NARROW_GAUGE,
FieldValues.SUBCLASS_PRESERVED,
FieldValues.SUBCLASS_FUNICULAR
);
private static final Set<String> RAILWAY_TRANSIT_VALUES = Set.of(
FieldValues.SUBCLASS_SUBWAY,
FieldValues.SUBCLASS_LIGHT_RAIL,
FieldValues.SUBCLASS_MONORAIL,
FieldValues.SUBCLASS_TRAM
);
private static final Set<String> SERVICE_VALUES = Set.of(
FieldValues.SERVICE_SPUR,
FieldValues.SERVICE_YARD,
FieldValues.SERVICE_SIDING,
FieldValues.SERVICE_CROSSOVER,
FieldValues.SERVICE_DRIVEWAY,
FieldValues.SERVICE_ALLEY,
FieldValues.SERVICE_PARKING_AISLE
);
private static final Set<String> SURFACE_UNPAVED_VALUES = Set.of(
"unpaved", "compacted", "dirt", "earth", "fine_gravel", "grass", "grass_paver", "gravel", "gravel_turf", "ground",
"ice", "mud", "pebblestone", "salt", "sand", "snow", "woodchips"
);
private static final Set<String> SURFACE_PAVED_VALUES = Set.of(
"paved", "asphalt", "cobblestone", "concrete", "concrete:lanes", "concrete:plates", "metal",
"paving_stones", "sett", "unhewn_cobblestone", "wood"
);
private static final Set<String> ACCESS_NO_VALUES = Set.of(
"private", "no"
);
private static final ZoomFunction.MeterToPixelThresholds MIN_LENGTH = ZoomFunction.meterThresholds()
.put(7, 50)
.put(6, 100)
.put(5, 500)
.put(4, 1_000);
// ORDER BY network_type, network, LENGTH(ref), ref)
private static final Comparator<RouteRelation> RELATION_ORDERING = Comparator
.<RouteRelation>comparingInt(
r -> r.networkType() != null ? r.networkType.ordinal() : Integer.MAX_VALUE)
.thenComparing(routeRelation -> coalesce(routeRelation.network(), ""))
.thenComparingInt(r -> r.ref().length())
.thenComparing(RouteRelation::ref);
private final AtomicBoolean loggedNoGb = new AtomicBoolean(false);
private final boolean z13Paths;
private PreparedGeometry greatBritain = null;
private final Map<String, Integer> MINZOOMS;
private final Stats stats;
private final PlanetilerConfig config;
public Transportation(Translations translations, PlanetilerConfig config, Stats stats) {
this.config = config;
this.stats = stats;
z13Paths = config.arguments().getBoolean(
"transportation_z13_paths",
"transportation(_name) layer: show all paths on z13",
false
);
MINZOOMS = Map.ofEntries(
entry(FieldValues.CLASS_PATH, z13Paths ? 13 : 14),
entry(FieldValues.CLASS_TRACK, 14),
entry(FieldValues.CLASS_SERVICE, 13),
entry(FieldValues.CLASS_MINOR, 13),
entry(FieldValues.CLASS_RACEWAY, 12),
entry(FieldValues.CLASS_TERTIARY, 11),
entry(FieldValues.CLASS_BUSWAY, 11),
entry(FieldValues.CLASS_SECONDARY, 9),
entry(FieldValues.CLASS_PRIMARY, 7),
entry(FieldValues.CLASS_TRUNK, 5),
entry(FieldValues.CLASS_MOTORWAY, 4)
);
}
@Override
public void processNaturalEarth(String table, SourceFeature feature,
FeatureCollector features) {
if ("ne_10m_admin_0_countries".equals(table) && feature.hasTag("iso_a2", "GB")) {
// multiple threads call this method concurrently, GB polygon *should* only be found
// once, but just to be safe synchronize updates to that field
synchronized (this) {
try {
Geometry boundary = feature.polygon().buffer(GeoUtils.metersToPixelAtEquator(0, 10_000) / 256d);
greatBritain = PreparedGeometryFactory.prepare(boundary);
} catch (GeometryException e) {
LOGGER.error("Failed to get Great Britain Polygon: " + e);
}
}
}
}
/** Returns a value for {@code surface} tag constrained to a small set of known values from raw OSM data. */
private static String surface(String value) {
return value == null ? null : SURFACE_PAVED_VALUES.contains(value) ? FieldValues.SURFACE_PAVED :
SURFACE_UNPAVED_VALUES.contains(value) ? FieldValues.SURFACE_UNPAVED : null;
}
/** Returns a value for {@code access} tag constrained to a small set of known values from raw OSM data. */
private static String access(String value) {
return value == null ? null : ACCESS_NO_VALUES.contains(value) ? "no" : null;
}
/** Returns a value for {@code service} tag constrained to a small set of known values from raw OSM data. */
private static String service(String value) {
return (value == null || !SERVICE_VALUES.contains(value)) ? null : value;
}
private static String railwayClass(String value) {
return value == null ? null :
RAILWAY_RAIL_VALUES.contains(value) ? "rail" :
RAILWAY_TRANSIT_VALUES.contains(value) ? "transit" : null;
}
static String highwayClass(String highway, String publicTransport, String construction, String manMade) {
return (!nullOrEmpty(highway) || !nullOrEmpty(publicTransport)) ? classMapping.getOrElse(Map.of(
"highway", coalesce(highway, ""),
"public_transport", coalesce(publicTransport, ""),
"construction", coalesce(construction, "")
), null) : isBridgeOrPier(manMade) ? manMade : null;
}
static String highwaySubclass(String highwayClass, String publicTransport, String highway) {
return FieldValues.CLASS_PATH.equals(highwayClass) ? coalesce(nullIfEmpty(publicTransport), highway) : null;
}
static boolean isFootwayOrSteps(String highway) {
return "footway".equals(highway) || "steps".equals(highway);
}
static boolean isLink(String highway) {
return highway != null && highway.endsWith("_link");
}
private static boolean isResidentialOrUnclassified(String highway) {
return "residential".equals(highway) || "unclassified".equals(highway);
}
private static boolean isDrivewayOrParkingAisle(String service) {
return FieldValues.SERVICE_PARKING_AISLE.equals(service) || FieldValues.SERVICE_DRIVEWAY.equals(service);
}
private static boolean isBridgeOrPier(String manMade) {
return "bridge".equals(manMade) || "pier".equals(manMade);
}
enum RouteNetwork {
US_INTERSTATE("us-interstate"),
US_HIGHWAY("us-highway"),
US_STATE("us-state"),
CA_TRANSCANADA("ca-transcanada"),
GB_MOTORWAY("gb-motorway"),
GB_TRUNK("gb-trunk");
final String name;
RouteNetwork(String name) {
this.name = name;
}
}
@Override
public List<OsmRelationInfo> preprocessOsmRelation(OsmElement.Relation relation) {
if (relation.hasTag("route", "road", "hiking")) {
RouteNetwork networkType = null;
String network = relation.getString("network");
String ref = relation.getString("ref");
if ("US:I".equals(network)) {
networkType = RouteNetwork.US_INTERSTATE;
} else if ("US:US".equals(network)) {
networkType = RouteNetwork.US_HIGHWAY;
} else if (network != null && network.length() == 5 && network.startsWith("US:")) {
networkType = RouteNetwork.US_STATE;
} else if (network != null && network.startsWith("CA:transcanada")) {
networkType = RouteNetwork.CA_TRANSCANADA;
}
int rank = switch (coalesce(network, "")) {
case "iwn", "nwn", "rwn" -> 1;
case "lwn" -> 2;
default -> (relation.hasTag("osmc:symbol") || relation.hasTag("colour")) ? 2 : 3;
};
if (network != null || rank < 3) {
return List.of(new RouteRelation(coalesce(ref, ""), network, networkType, (byte) rank, relation.id()));
}
}
return null;
}
List<RouteRelation> getRouteRelations(Tables.OsmHighwayLinestring element) {
String ref = element.ref();
List<OsmReader.RelationMember<RouteRelation>> relations = element.source().relationInfo(RouteRelation.class);
List<RouteRelation> result = new ArrayList<>(relations.size() + 1);
for (var relationMember : relations) {
var relation = relationMember.relation();
// avoid duplicates - list should be very small and usually only one
if (!result.contains(relation)) {
result.add(relation);
}
}
if (ref != null) {
// GB doesn't use regular relations like everywhere else, so if we are
// in GB then use a naming convention instead.
Matcher refMatcher = GREAT_BRITAIN_REF_NETWORK_PATTERN.matcher(ref);
if (refMatcher.find()) {
if (greatBritain == null) {
if (!loggedNoGb.get() && loggedNoGb.compareAndSet(false, true)) {
LOGGER.warn("No GB polygon for inferring route network types");
}
} else {
try {
Geometry wayGeometry = element.source().worldGeometry();
if (greatBritain.intersects(wayGeometry)) {
Transportation.RouteNetwork networkType =
"motorway".equals(element.highway()) ? Transportation.RouteNetwork.GB_MOTORWAY :
Transportation.RouteNetwork.GB_TRUNK;
String network = "motorway".equals(element.highway()) ? "omt-gb-motorway" : "omt-gb-trunk";
result.add(new RouteRelation(refMatcher.group(), network, networkType, (byte) -1,
0));
}
} catch (GeometryException e) {
e.log(stats, "omt_transportation_name_gb_test",
"Unable to test highway against GB route network: " + element.source().id());
}
}
}
}
Collections.sort(result);
return result;
}
RouteRelation getRouteRelation(Tables.OsmHighwayLinestring element) {
List<RouteRelation> all = getRouteRelations(element);
return all.isEmpty() ? null : all.get(0);
}
@Override
public void process(Tables.OsmHighwayLinestring element, FeatureCollector features) {
if (element.isArea()) {
return;
}
RouteRelation routeRelation = getRouteRelation(element);
RouteNetwork networkType = routeRelation != null ? routeRelation.networkType : null;
String highway = element.highway();
String highwayClass = highwayClass(element.highway(), element.publicTransport(), element.construction(),
element.manMade());
String service = service(element.service());
if (highwayClass != null) {
if (isPierPolygon(element)) {
return;
}
int minzoom = getMinzoom(element, highwayClass);
boolean highwayRamp = isLink(highway);
Integer rampAboveZ12 = (highwayRamp || element.isRamp()) ? 1 : null;
Integer rampBelowZ12 = highwayRamp ? 1 : null;
FeatureCollector.Feature feature = features.line(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
// main attributes at all zoom levels (used for grouping <= z8)
.setAttr(Fields.CLASS, highwayClass)
.setAttr(Fields.SUBCLASS, highwaySubclass(highwayClass, element.publicTransport(), highway))
.setAttr(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()))
.setAttr(Fields.NETWORK, networkType != null ? networkType.name : null)
// z8+
.setAttrWithMinzoom(Fields.EXPRESSWAY, element.expressway() && !"motorway".equals(highway) ? 1 : null, 8)
// z9+
.setAttrWithMinzoom(Fields.LAYER, nullIfLong(element.layer(), 0), 9)
.setAttrWithMinzoom(Fields.BICYCLE, nullIfEmpty(element.bicycle()), 9)
.setAttrWithMinzoom(Fields.FOOT, nullIfEmpty(element.foot()), 9)
.setAttrWithMinzoom(Fields.HORSE, nullIfEmpty(element.horse()), 9)
.setAttrWithMinzoom(Fields.MTB_SCALE, nullIfEmpty(element.mtbScale()), 9)
.setAttrWithMinzoom(Fields.ACCESS, access(element.access()), 9)
.setAttrWithMinzoom(Fields.TOLL, element.toll() ? 1 : null, 9)
// sometimes z9+, sometimes z12+
.setAttr(Fields.RAMP, minzoom >= 12 ? rampAboveZ12 :
((ZoomFunction<Integer>) z -> z < 9 ? null : z >= 12 ? rampAboveZ12 : rampBelowZ12))
// z12+
.setAttrWithMinzoom(Fields.SERVICE, service, 12)
.setAttrWithMinzoom(Fields.ONEWAY, nullIfInt(element.isOneway(), 0), 12)
.setAttrWithMinzoom(Fields.SURFACE, surface(element.surface()), 12)
.setMinPixelSize(0) // merge during post-processing, then limit by size
.setSortKey(element.zOrder())
.setMinZoom(minzoom);
if (isFootwayOrSteps(highway)) {
feature
.setAttr(Fields.LEVEL, Parse.parseLongOrNull(element.source().getTag("level")))
.setAttr(Fields.INDOOR, element.indoor() ? 1 : null);
}
}
}
int getMinzoom(Tables.OsmHighwayLinestring element, String highwayClass) {
List<RouteRelation> routeRelations = getRouteRelations(element);
int routeRank = 3;
for (var rel : routeRelations) {
if (rel.intRank() < routeRank) {
routeRank = rel.intRank();
}
}
String highway = element.highway();
int minzoom;
if ("pier".equals(element.manMade())) {
minzoom = 13;
} else if (isResidentialOrUnclassified(highway)) {
minzoom = 12;
} else {
String baseClass = highwayClass.replace("_construction", "");
minzoom = switch (baseClass) {
case FieldValues.CLASS_SERVICE -> isDrivewayOrParkingAisle(service(element.service())) ? 14 : 13;
case FieldValues.CLASS_TRACK, FieldValues.CLASS_PATH -> routeRank == 1 ? 12 :
(z13Paths || !nullOrEmpty(element.name()) || routeRank <= 2 || !nullOrEmpty(element.sacScale())) ? 13 : 14;
default -> MINZOOMS.get(baseClass);
};
}
if (isLink(highway)) {
minzoom = Math.max(minzoom, 9);
}
return minzoom;
}
private boolean isPierPolygon(Tables.OsmHighwayLinestring element) {
if ("pier".equals(element.manMade())) {
try {
if (element.source().worldGeometry()instanceof LineString lineString && lineString.isClosed()) {
// ignore this because it's a polygon
return true;
}
} catch (GeometryException e) {
e.log(stats, "omt_transportation_pier",
"Unable to decode pier geometry for " + element.source().id());
return true;
}
}
return false;
}
@Override
public void process(Tables.OsmRailwayLinestring element, FeatureCollector features) {
String railway = element.railway();
String clazz = railwayClass(railway);
if (clazz != null) {
String service = nullIfEmpty(element.service());
int minzoom;
if (service != null) {
minzoom = 14;
} else if (FieldValues.SUBCLASS_RAIL.equals(railway)) {
minzoom = "main".equals(element.usage()) ? 8 : 10;
} else if (FieldValues.SUBCLASS_NARROW_GAUGE.equals(railway)) {
minzoom = 10;
} else if (FieldValues.SUBCLASS_LIGHT_RAIL.equals(railway)) {
minzoom = 11;
} else {
minzoom = 14;
}
features.line(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, clazz)
.setAttr(Fields.SUBCLASS, railway)
.setAttr(Fields.SERVICE, service(service))
.setAttr(Fields.ONEWAY, nullIfInt(element.isOneway(), 0))
.setAttr(Fields.RAMP, element.isRamp() ? 1L : null)
.setAttrWithMinzoom(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()), 10)
.setAttrWithMinzoom(Fields.LAYER, nullIfLong(element.layer(), 0), 9)
.setSortKey(element.zOrder())
.setMinPixelSize(0) // merge during post-processing, then limit by size
.setMinZoom(minzoom);
}
}
@Override
public void process(Tables.OsmAerialwayLinestring element, FeatureCollector features) {
features.line(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, "aerialway")
.setAttr(Fields.SUBCLASS, element.aerialway())
.setAttr(Fields.SERVICE, service(element.service()))
.setAttr(Fields.ONEWAY, nullIfInt(element.isOneway(), 0))
.setAttr(Fields.RAMP, element.isRamp() ? 1L : null)
.setAttr(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()))
.setAttr(Fields.LAYER, nullIfLong(element.layer(), 0))
.setSortKey(element.zOrder())
.setMinPixelSize(0) // merge during post-processing, then limit by size
.setMinZoom(12);
}
@Override
public void process(Tables.OsmShipwayLinestring element, FeatureCollector features) {
features.line(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, element.shipway()) // "ferry"
// no subclass
.setAttr(Fields.SERVICE, service(element.service()))
.setAttr(Fields.ONEWAY, nullIfInt(element.isOneway(), 0))
.setAttr(Fields.RAMP, element.isRamp() ? 1L : null)
.setAttr(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()))
.setAttr(Fields.LAYER, nullIfLong(element.layer(), 0))
.setSortKey(element.zOrder())
.setMinPixelSize(0) // merge during post-processing, then limit by size
.setMinZoom(11);
}
@Override
public void process(Tables.OsmHighwayPolygon element, FeatureCollector features) {
String manMade = element.manMade();
if (isBridgeOrPier(manMade) ||
// only allow closed ways where area=yes, and multipolygons
// and ignore underground pedestrian areas
(!element.source().canBeLine() && element.layer() >= 0)) {
String highwayClass = highwayClass(element.highway(), element.publicTransport(), null, element.manMade());
if (highwayClass != null) {
features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, highwayClass)
.setAttr(Fields.SUBCLASS, highwaySubclass(highwayClass, element.publicTransport(), element.highway()))
.setAttr(Fields.BRUNNEL, brunnel("bridge".equals(manMade), false, false))
.setAttr(Fields.LAYER, nullIfLong(element.layer(), 0))
.setSortKey(element.zOrder())
.setMinZoom(13);
}
}
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) {
double tolerance = config.tolerance(zoom);
double minLength = coalesce(MIN_LENGTH.apply(zoom), 0).doubleValue();
// TODO preserve direction for one-way?
return FeatureMerge.mergeLineStrings(items, minLength, tolerance, BUFFER_SIZE);
}
/** Information extracted from route relations to use when processing ways in that relation. */
record RouteRelation(
String ref,
String network,
RouteNetwork networkType,
byte rank,
@Override long id
) implements OsmRelationInfo, Comparable<RouteRelation> {
@Override
public long estimateMemoryUsageBytes() {
return CLASS_HEADER_BYTES +
MemoryEstimator.estimateSize(rank) +
POINTER_BYTES + estimateSize(ref) +
POINTER_BYTES + estimateSize(network) +
POINTER_BYTES + // networkType
MemoryEstimator.estimateSizeLong(id);
}
public int intRank() {
return rank;
}
@Override
public int compareTo(RouteRelation o) {
return RELATION_ORDERING.compare(this, o);
}
}
}

398
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/TransportationName.java
Wyświetl plik

@ -1,398 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.layers.Transportation.highwayClass;
import static com.onthegomap.planetiler.basemap.layers.Transportation.highwaySubclass;
import static com.onthegomap.planetiler.basemap.layers.Transportation.isFootwayOrSteps;
import static com.onthegomap.planetiler.basemap.util.Utils.*;
import com.carrotsearch.hppc.LongArrayList;
import com.carrotsearch.hppc.LongByteMap;
import com.carrotsearch.hppc.LongHashSet;
import com.carrotsearch.hppc.LongSet;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.ForwardingProfile;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.ZoomFunction;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
/**
* Defines the logic for generating map elements for road, shipway, rail, and path names in the {@code
* transportation_name} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/transportation_name">OpenMapTiles
* transportation_name sql files</a>.
*/
public class TransportationName implements
OpenMapTilesSchema.TransportationName,
Tables.OsmHighwayPoint.Handler,
Tables.OsmHighwayLinestring.Handler,
Tables.OsmAerialwayLinestring.Handler,
Tables.OsmShipwayLinestring.Handler,
BasemapProfile.FeaturePostProcessor,
BasemapProfile.IgnoreWikidata,
ForwardingProfile.OsmNodePreprocessor,
ForwardingProfile.OsmWayPreprocessor {
/*
* Generate road names from OSM data. Route networkType and ref are copied
* from relations that roads are a part of - except in Great Britain which
* uses a naming convention instead of relations.
*
* The goal is to make name linestrings as long as possible to give clients
* the best chance of showing road names at different zoom levels, so do not
* limit linestrings by length at process time and merge them at tile
* render-time.
*
* Any 3-way nodes and intersections break line merging so set the
* transportation_name_limit_merge argument to true to add temporary
* "is link" and "relation" keys to prevent opposite directions of a
* divided highway or on/off ramps from getting merged for main highways.
*/
// extra temp key used to group on/off-ramps separately from main highways
private static final String LINK_TEMP_KEY = "__islink";
private static final String RELATION_ID_TEMP_KEY = "__relid";
private static final ZoomFunction.MeterToPixelThresholds MIN_LENGTH = ZoomFunction.meterThresholds()
.put(6, 20_000)
.put(7, 20_000)
.put(8, 14_000)
.put(9, 8_000)
.put(10, 8_000)
.put(11, 8_000);
private static final List<String> CONCURRENT_ROUTE_KEYS = List.of(
Fields.ROUTE_1,
Fields.ROUTE_2,
Fields.ROUTE_3,
Fields.ROUTE_4,
Fields.ROUTE_5,
Fields.ROUTE_6
);
private final boolean brunnel;
private final boolean sizeForShield;
private final boolean limitMerge;
private final PlanetilerConfig config;
private final boolean minorRefs;
private Transportation transportation;
private final LongByteMap motorwayJunctionHighwayClasses = Hppc.newLongByteHashMap();
private final LongSet motorwayJunctionNodes = new LongHashSet();
public TransportationName(Translations translations, PlanetilerConfig config, Stats stats) {
this.config = config;
this.brunnel = config.arguments().getBoolean(
"transportation_name_brunnel",
"transportation_name layer: set to false to omit brunnel and help merge long highways",
false
);
this.sizeForShield = config.arguments().getBoolean(
"transportation_name_size_for_shield",
"transportation_name layer: allow road names on shorter segments (ie. they will have a shield)",
false
);
this.limitMerge = config.arguments().getBoolean(
"transportation_name_limit_merge",
"transportation_name layer: limit merge so we don't combine different relations to help merge long highways",
false
);
this.minorRefs = config.arguments().getBoolean(
"transportation_name_minor_refs",
"transportation_name layer: include name and refs from minor road networks if not present on a way",
false
);
}
public void needsTransportationLayer(Transportation transportation) {
this.transportation = transportation;
}
@Override
public void preprocessOsmNode(OsmElement.Node node) {
if (node.hasTag("highway", "motorway_junction")) {
synchronized (motorwayJunctionNodes) {
motorwayJunctionNodes.add(node.id());
}
}
}
@Override
public void preprocessOsmWay(OsmElement.Way way) {
String highway = way.getString("highway");
if (highway != null) {
HighwayClass cls = HighwayClass.from(highway);
if (cls != HighwayClass.UNKNOWN) {
LongArrayList nodes = way.nodes();
for (int i = 0; i < nodes.size(); i++) {
long node = nodes.get(i);
if (motorwayJunctionNodes.contains(node)) {
synchronized (motorwayJunctionHighwayClasses) {
byte oldValue = motorwayJunctionHighwayClasses.getOrDefault(node, HighwayClass.UNKNOWN.value);
byte newValue = cls.value;
if (newValue > oldValue) {
motorwayJunctionHighwayClasses.put(node, newValue);
}
}
}
}
}
}
}
@Override
public void process(Tables.OsmHighwayPoint element, FeatureCollector features) {
long id = element.source().id();
byte value = motorwayJunctionHighwayClasses.getOrDefault(id, (byte) -1);
if (value > 0) {
HighwayClass cls = HighwayClass.from(value);
if (cls != HighwayClass.UNKNOWN) {
String subclass = FieldValues.SUBCLASS_JUNCTION;
String ref = element.ref();
features.point(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.putAttrs(LanguageUtils.getNamesWithoutTranslations(element.source().tags()))
.setAttr(Fields.REF, ref)
.setAttr(Fields.REF_LENGTH, ref != null ? ref.length() : null)
.setAttr(Fields.CLASS, highwayClass(cls.highwayValue, null, null, null))
.setAttr(Fields.SUBCLASS, subclass)
.setAttr(Fields.LAYER, nullIfLong(element.layer(), 0))
.setSortKeyDescending(element.zOrder())
.setMinZoom(10);
}
}
}
@Override
public void process(Tables.OsmHighwayLinestring element, FeatureCollector features) {
String ref = element.ref();
List<Transportation.RouteRelation> relations = transportation.getRouteRelations(element);
Transportation.RouteRelation firstRelationWithNetwork = relations.stream()
.filter(rel -> rel.networkType() != null)
.findFirst()
.orElse(null);
if (firstRelationWithNetwork != null && !nullOrEmpty(firstRelationWithNetwork.ref())) {
ref = firstRelationWithNetwork.ref();
}
// if transportation_name_minor_refs flag set and we don't have a ref yet, then pull it from any network
if (nullOrEmpty(ref) && minorRefs && !relations.isEmpty()) {
ref = relations.stream()
.map(r -> r.ref())
.filter(r -> !nullOrEmpty(r))
.findFirst()
.orElse(null);
}
String name = nullIfEmpty(element.name());
ref = nullIfEmpty(ref);
String highway = nullIfEmpty(element.highway());
String highwayClass = highwayClass(element.highway(), null, element.construction(), element.manMade());
if (element.isArea() || highway == null || highwayClass == null || (name == null && ref == null)) {
return;
}
boolean isLink = Transportation.isLink(highway);
String baseClass = highwayClass.replace("_construction", "");
int minzoom = FieldValues.CLASS_TRUNK.equals(baseClass) ? 8 :
FieldValues.CLASS_MOTORWAY.equals(baseClass) ? 6 :
isLink ? 13 : 12; // fallback - get from line minzoom, but floor at 12
// inherit min zoom threshold from visible road, and ensure we never show a label on a road that's not visible yet.
minzoom = Math.max(minzoom, transportation.getMinzoom(element, highwayClass));
FeatureCollector.Feature feature = features.line(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setBufferPixelOverrides(MIN_LENGTH)
// TODO abbreviate road names - can't port osml10n because it is AGPL
.putAttrs(LanguageUtils.getNamesWithoutTranslations(element.source().tags()))
.setAttr(Fields.REF, ref)
.setAttr(Fields.REF_LENGTH, ref != null ? ref.length() : null)
.setAttr(Fields.NETWORK,
firstRelationWithNetwork != null ? firstRelationWithNetwork.networkType().name : !nullOrEmpty(ref) ? "road" :
null)
.setAttr(Fields.CLASS, highwayClass)
.setAttr(Fields.SUBCLASS, highwaySubclass(highwayClass, null, highway))
.setMinPixelSize(0)
.setSortKey(element.zOrder())
.setMinZoom(minzoom);
// populate route_1, route_2, ... tags
for (int i = 0; i < Math.min(CONCURRENT_ROUTE_KEYS.size(), relations.size()); i++) {
Transportation.RouteRelation routeRelation = relations.get(i);
feature.setAttr(CONCURRENT_ROUTE_KEYS.get(i), routeRelation.network() == null ? null :
routeRelation.network() + "=" + coalesce(routeRelation.ref(), ""));
}
if (brunnel) {
feature.setAttr(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()));
}
/*
* to help group roads into longer segments, add temporary tags to limit which segments get grouped together. Since
* a divided highway typically has a separate relation for each direction, this ends up keeping segments going
* opposite directions group getting grouped together and confusing the line merging process
*/
if (limitMerge) {
feature
.setAttr(LINK_TEMP_KEY, isLink ? 1 : 0)
.setAttr(RELATION_ID_TEMP_KEY, firstRelationWithNetwork == null ? null : firstRelationWithNetwork.id());
}
if (isFootwayOrSteps(highway)) {
feature
.setAttrWithMinzoom(Fields.LAYER, nullIfLong(element.layer(), 0), 12)
.setAttrWithMinzoom(Fields.LEVEL, Parse.parseLongOrNull(element.source().getTag("level")), 12)
.setAttrWithMinzoom(Fields.INDOOR, element.indoor() ? 1 : null, 12);
}
}
@Override
public void process(Tables.OsmAerialwayLinestring element, FeatureCollector features) {
if (!nullOrEmpty(element.name())) {
features.line(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setBufferPixelOverrides(MIN_LENGTH)
.putAttrs(LanguageUtils.getNamesWithoutTranslations(element.source().tags()))
.setAttr(Fields.CLASS, "aerialway")
.setAttr(Fields.SUBCLASS, element.aerialway())
.setMinPixelSize(0)
.setSortKey(element.zOrder())
.setMinZoom(12);
}
}
@Override
public void process(Tables.OsmShipwayLinestring element, FeatureCollector features) {
if (!nullOrEmpty(element.name())) {
features.line(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setBufferPixelOverrides(MIN_LENGTH)
.putAttrs(LanguageUtils.getNamesWithoutTranslations(element.source().tags()))
.setAttr(Fields.CLASS, element.shipway())
.setMinPixelSize(0)
.setSortKey(element.zOrder())
.setMinZoom(12);
}
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) {
double tolerance = config.tolerance(zoom);
double minLength = coalesce(MIN_LENGTH.apply(zoom), 0).doubleValue();
// TODO tolerances:
// z6: (tolerance: 500)
// z7: (tolerance: 200)
// z8: (tolerance: 120)
// z9-11: (tolerance: 50)
Function<Map<String, Object>, Double> lengthLimitCalculator =
zoom >= 14 ? (p -> 0d) :
minLength > 0 ? (p -> minLength) :
this::getMinLengthForName;
var result = FeatureMerge.mergeLineStrings(items, lengthLimitCalculator, tolerance, BUFFER_SIZE);
if (limitMerge) {
// remove temp keys that were just used to improve line merging
for (var feature : result) {
feature.attrs().remove(LINK_TEMP_KEY);
feature.attrs().remove(RELATION_ID_TEMP_KEY);
}
}
return result;
}
/** Returns the minimum pixel length that a name will fit into. */
private double getMinLengthForName(Map<String, Object> attrs) {
Object ref = attrs.get(Fields.REF);
Object name = coalesce(attrs.get(Fields.NAME), ref);
return (sizeForShield && ref instanceof String) ? 6 :
name instanceof String str ? str.length() * 6 : Double.MAX_VALUE;
}
private enum HighwayClass {
MOTORWAY("motorway", 6),
TRUNK("trunk", 5),
PRIMARY("primary", 4),
SECONDARY("secondary", 3),
TERTIARY("tertiary", 2),
UNCLASSIFIED("unclassified", 1),
UNKNOWN("", 0);
private static final Map<String, HighwayClass> indexByString = new HashMap<>();
private static final Map<Byte, HighwayClass> indexByByte = new HashMap<>();
final byte value;
final String highwayValue;
HighwayClass(String highwayValue, int id) {
this.highwayValue = highwayValue;
this.value = (byte) id;
}
static {
Arrays.stream(values()).forEach(cls -> {
indexByString.put(cls.highwayValue, cls);
indexByByte.put(cls.value, cls);
});
}
static HighwayClass from(String highway) {
return indexByString.getOrDefault(highway, UNKNOWN);
}
static HighwayClass from(byte value) {
return indexByByte.getOrDefault(value, UNKNOWN);
}
}
}

132
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Water.java
Wyświetl plik

@ -1,132 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.ForwardingProfile;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.Utils;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
import java.util.List;
/**
* Defines the logic for generating map elements for oceans and lakes in the {@code water} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/water">OpenMapTiles water sql files</a>.
*/
public class Water implements
OpenMapTilesSchema.Water,
Tables.OsmWaterPolygon.Handler,
BasemapProfile.NaturalEarthProcessor,
BasemapProfile.OsmWaterPolygonProcessor,
ForwardingProfile.FeaturePostProcessor {
/*
* At low zoom levels, use natural earth for oceans and major lakes, and at high zoom levels
* use OpenStreetMap data. OpenStreetMap data contains smaller bodies of water, but not
* large ocean polygons. For oceans, use https://osmdata.openstreetmap.de/data/water-polygons.html
* which infers ocean polygons by preprocessing all coastline elements.
*/
private final MultiExpression.Index<String> classMapping;
private final PlanetilerConfig config;
public Water(Translations translations, PlanetilerConfig config, Stats stats) {
this.classMapping = FieldMappings.Class.index();
this.config = config;
}
@Override
public void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features) {
record WaterInfo(int minZoom, int maxZoom, String clazz) {}
WaterInfo info = switch (table) {
case "ne_110m_ocean" -> new WaterInfo(0, 1, FieldValues.CLASS_OCEAN);
case "ne_50m_ocean" -> new WaterInfo(2, 4, FieldValues.CLASS_OCEAN);
case "ne_10m_ocean" -> new WaterInfo(5, 5, FieldValues.CLASS_OCEAN);
// TODO: get OSM ID from low-zoom natural earth lakes
case "ne_110m_lakes" -> new WaterInfo(0, 1, FieldValues.CLASS_LAKE);
case "ne_50m_lakes" -> new WaterInfo(2, 3, FieldValues.CLASS_LAKE);
case "ne_10m_lakes" -> new WaterInfo(4, 5, FieldValues.CLASS_LAKE);
default -> null;
};
if (info != null) {
features.polygon(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setZoomRange(info.minZoom, info.maxZoom)
.setAttr(Fields.CLASS, info.clazz);
}
}
@Override
public void processOsmWater(SourceFeature feature, FeatureCollector features) {
features.polygon(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, FieldValues.CLASS_OCEAN)
.setMinZoom(6);
}
@Override
public void process(Tables.OsmWaterPolygon element, FeatureCollector features) {
if (!"bay".equals(element.natural())) {
String clazz = "riverbank".equals(element.waterway()) ? FieldValues.CLASS_RIVER :
classMapping.getOrElse(element.source(), FieldValues.CLASS_LAKE);
features.polygon(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setMinPixelSizeBelowZoom(11, 2)
.setMinZoom(6)
.setAttr(Fields.ID, element.source().id())
.setAttr(Fields.INTERMITTENT, element.isIntermittent() ? 1 : 0)
.setAttrWithMinzoom(Fields.BRUNNEL, Utils.brunnel(element.isBridge(), element.isTunnel()), 12)
.setAttr(Fields.CLASS, clazz);
}
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) throws GeometryException {
return items.size() > 1 ? FeatureMerge.mergeOverlappingPolygons(items, config.minFeatureSize(zoom)) : items;
}
}

202
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/WaterName.java
Wyświetl plik

@ -1,202 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import com.carrotsearch.hppc.LongObjectMap;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import java.util.Map;
import java.util.concurrent.ConcurrentSkipListMap;
import org.locationtech.jts.geom.Geometry;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Defines the logic for generating map elements for ocean and lake names in the {@code water_name} layer from source
* features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/water_name">OpenMapTiles water_name sql
* files</a>.
*/
public class WaterName implements
OpenMapTilesSchema.WaterName,
Tables.OsmMarinePoint.Handler,
Tables.OsmWaterPolygon.Handler,
BasemapProfile.NaturalEarthProcessor,
BasemapProfile.LakeCenterlineProcessor {
/*
* Labels for lakes and oceans come primarily from OpenStreetMap data, but we also join
* with the lake centerlines source to get linestring geometries for prominent lakes.
* We also join with natural earth to make certain important lake/ocean labels visible
* at lower zoom levels.
*/
private static final Logger LOGGER = LoggerFactory.getLogger(WaterName.class);
private static final double WORLD_AREA_FOR_70K_SQUARE_METERS =
Math.pow(GeoUtils.metersToPixelAtEquator(0, Math.sqrt(70_000)) / 256d, 2);
private static final double LOG2 = Math.log(2);
private final Translations translations;
// need to synchronize updates from multiple threads
private final LongObjectMap<Geometry> lakeCenterlines = Hppc.newLongObjectHashMap();
// may be updated concurrently by multiple threads
private final ConcurrentSkipListMap<String, Integer> importantMarinePoints = new ConcurrentSkipListMap<>();
private final Stats stats;
public WaterName(Translations translations, PlanetilerConfig config, Stats stats) {
this.translations = translations;
this.stats = stats;
}
@Override
public void release() {
lakeCenterlines.release();
importantMarinePoints.clear();
}
@Override
public void processLakeCenterline(SourceFeature feature, FeatureCollector features) {
// TODO pull lake centerline computation into planetiler?
long osmId = Math.abs(feature.getLong("OSM_ID"));
if (osmId == 0L) {
LOGGER.warn("Bad lake centerline. Tags: {}", feature.tags());
} else {
try {
// multiple threads call this concurrently
synchronized (this) {
// if we already have a centerline for this OSM_ID, then merge the existing one with this one
var newGeometry = feature.worldGeometry();
var oldGeometry = lakeCenterlines.get(osmId);
if (oldGeometry != null) {
newGeometry = GeoUtils.combine(oldGeometry, newGeometry);
}
lakeCenterlines.put(osmId, newGeometry);
}
} catch (GeometryException e) {
e.log(stats, "omt_water_name_lakeline", "Bad lake centerline: " + feature);
}
}
}
@Override
public void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features) {
// use natural earth named polygons just as a source of name to zoom-level mappings for later
if ("ne_10m_geography_marine_polys".equals(table)) {
String name = feature.getString("name");
Integer scalerank = Parse.parseIntOrNull(feature.getTag("scalerank"));
if (name != null && scalerank != null) {
name = name.replaceAll("\\s+", " ").trim().toLowerCase();
importantMarinePoints.put(name, scalerank);
}
}
}
@Override
public void process(Tables.OsmMarinePoint element, FeatureCollector features) {
if (!element.name().isBlank()) {
String place = element.place();
var source = element.source();
// use name from OSM, but get min zoom from natural earth based on fuzzy name match...
Integer rank = Parse.parseIntOrNull(source.getTag("rank"));
String name = element.name().toLowerCase();
Integer nerank;
if ((nerank = importantMarinePoints.get(name)) != null) {
rank = nerank;
} else if ((nerank = importantMarinePoints.get(source.getString("name:en", "").toLowerCase())) != null) {
rank = nerank;
} else if ((nerank = importantMarinePoints.get(source.getString("name:es", "").toLowerCase())) != null) {
rank = nerank;
} else {
Map.Entry<String, Integer> next = importantMarinePoints.ceilingEntry(name);
if (next != null && next.getKey().startsWith(name)) {
rank = next.getValue();
}
}
int minZoom = "ocean".equals(place) ? 0 : rank != null ? rank : 8;
features.point(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.putAttrs(LanguageUtils.getNames(source.tags(), translations))
.setAttr(Fields.CLASS, place)
.setAttr(Fields.INTERMITTENT, element.isIntermittent() ? 1 : 0)
.setMinZoom(minZoom);
}
}
@Override
public void process(Tables.OsmWaterPolygon element, FeatureCollector features) {
if (nullIfEmpty(element.name()) != null) {
try {
Geometry centerlineGeometry = lakeCenterlines.get(element.source().id());
FeatureCollector.Feature feature;
int minzoom = 9;
if (centerlineGeometry != null) {
// prefer lake centerline if it exists
feature = features.geometry(LAYER_NAME, centerlineGeometry)
.setMinPixelSizeBelowZoom(13, 6d * element.name().length());
} else {
// otherwise just use a label point inside the lake
feature = features.pointOnSurface(LAYER_NAME);
Geometry geometry = element.source().worldGeometry();
double area = geometry.getArea();
minzoom = (int) Math.floor(20 - Math.log(area / WORLD_AREA_FOR_70K_SQUARE_METERS) / LOG2);
minzoom = Math.min(14, Math.max(9, minzoom));
}
feature
.setAttr(Fields.CLASS, FieldValues.CLASS_LAKE)
.setBufferPixels(BUFFER_SIZE)
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setAttr(Fields.INTERMITTENT, element.isIntermittent() ? 1 : 0)
.setMinZoom(minzoom);
} catch (GeometryException e) {
e.log(stats, "omt_water_polygon", "Unable to get geometry for water polygon " + element.source().id());
}
}
}
}

231
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/layers/Waterway.java
Wyświetl plik

@ -1,231 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import com.carrotsearch.hppc.LongObjectHashMap;
import com.google.common.util.concurrent.AtomicDouble;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.FeatureMerge;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.basemap.generated.OpenMapTilesSchema;
import com.onthegomap.planetiler.basemap.generated.Tables;
import com.onthegomap.planetiler.basemap.util.LanguageUtils;
import com.onthegomap.planetiler.basemap.util.Utils;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmReader;
import com.onthegomap.planetiler.reader.osm.OsmRelationInfo;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.ZoomFunction;
import java.util.List;
import java.util.Map;
/**
* Defines the logic for generating river map elements in the {@code waterway} layer from source features.
* <p>
* This class is ported to Java from
* <a href="https://github.com/openmaptiles/openmaptiles/tree/master/layers/waterway">OpenMapTiles waterway sql
* files</a>.
*/
public class Waterway implements
OpenMapTilesSchema.Waterway,
Tables.OsmWaterwayLinestring.Handler,
BasemapProfile.FeaturePostProcessor,
BasemapProfile.NaturalEarthProcessor,
BasemapProfile.OsmRelationPreprocessor,
BasemapProfile.OsmAllProcessor {
/*
* Uses Natural Earth at lower zoom-levels and OpenStreetMap at higher zoom levels.
*
* For OpenStreetMap, attempts to merge disconnected linestrings with the same name
* at lower zoom levels so that clients can more easily render the name. We also
* limit their length at merge-time which only has visibilty into that feature in a
* single tile, so at render-time we need to allow through features far enough outside
* the tile boundary enough to not accidentally filter out a long river only because a
* short segment of it goes through this tile.
*/
private static final Map<String, Integer> CLASS_MINZOOM = Map.of(
"river", 12,
"canal", 12,
"stream", 13,
"drain", 13,
"ditch", 13
);
private static final String TEMP_REL_ID_ADDR = "_relid";
private final Translations translations;
private final PlanetilerConfig config;
private final Stats stats;
private final LongObjectHashMap<AtomicDouble> riverRelationLengths = Hppc.newLongObjectHashMap();
public Waterway(Translations translations, PlanetilerConfig config, Stats stats) {
this.config = config;
this.translations = translations;
this.stats = stats;
}
private static final ZoomFunction.MeterToPixelThresholds MIN_PIXEL_LENGTHS = ZoomFunction.meterThresholds()
.put(6, 500_000)
.put(7, 400_000)
.put(8, 300_000)
.put(9, 8_000)
.put(10, 4_000)
.put(11, 1_000);
// zoom-level 3-5 come from natural earth
@Override
public void processNaturalEarth(String table, SourceFeature feature, FeatureCollector features) {
if (feature.hasTag("featurecla", "River")) {
record ZoomRange(int min, int max) {}
ZoomRange zoom = switch (table) {
case "ne_110m_rivers_lake_centerlines" -> new ZoomRange(3, 3);
case "ne_50m_rivers_lake_centerlines" -> new ZoomRange(4, 5);
default -> null;
};
if (zoom != null) {
features.line(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, FieldValues.CLASS_RIVER)
.setZoomRange(zoom.min, zoom.max);
}
}
}
// zoom-level 6-8 come from OSM river relations
private record WaterwayRelation(
long id,
Map<String, Object> names
) implements OsmRelationInfo {}
@Override
public List<OsmRelationInfo> preprocessOsmRelation(OsmElement.Relation relation) {
if (relation.hasTag("waterway", "river") && !Utils.nullOrEmpty(relation.getString("name"))) {
synchronized (riverRelationLengths) {
riverRelationLengths.put(relation.id(), new AtomicDouble());
}
return List.of(new WaterwayRelation(relation.id(), LanguageUtils.getNames(relation.tags(), translations)));
}
return null;
}
@Override
public void processAllOsm(SourceFeature feature, FeatureCollector features) {
List<OsmReader.RelationMember<WaterwayRelation>> waterways = feature.relationInfo(WaterwayRelation.class);
if (waterways != null && !waterways.isEmpty() && feature.canBeLine()) {
for (var waterway : waterways) {
String role = waterway.role();
if (Utils.nullOrEmpty(role) || "main_stream".equals(role)) {
long relId = waterway.relation().id();
try {
AtomicDouble counter = riverRelationLengths.get(relId);
if (counter != null) {
counter.addAndGet(feature.length());
}
} catch (GeometryException e) {
e.log(stats, "waterway_decode", "Unable to get waterway length for " + feature.id());
}
features.line(LAYER_NAME)
.setAttr(TEMP_REL_ID_ADDR, relId)
.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, FieldValues.CLASS_RIVER)
.putAttrs(waterway.relation().names())
.setZoomRange(6, 8)
.setMinPixelSize(0);
}
}
}
}
// zoom-level 9+ come from OSM river ways
@Override
public void process(Tables.OsmWaterwayLinestring element, FeatureCollector features) {
String waterway = element.waterway();
String name = nullIfEmpty(element.name());
boolean important = "river".equals(waterway) && name != null;
int minzoom = important ? 9 : CLASS_MINZOOM.getOrDefault(element.waterway(), 14);
features.line(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, element.waterway())
.putAttrs(LanguageUtils.getNames(element.source().tags(), translations))
.setMinZoom(minzoom)
// details only at higher zoom levels so that named rivers can be merged more aggressively
.setAttrWithMinzoom(Fields.BRUNNEL, Utils.brunnel(element.isBridge(), element.isTunnel()), 12)
.setAttrWithMinzoom(Fields.INTERMITTENT, element.isIntermittent() ? 1 : 0, 12)
// at lower zoom levels, we'll merge linestrings and limit length/clip afterwards
.setBufferPixelOverrides(MIN_PIXEL_LENGTHS).setMinPixelSizeBelowZoom(11, 0);
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) {
if (zoom >= 6 && zoom <= 8) {
// remove ways for river relations if relation is not long enough
double minSizeAtZoom = MIN_PIXEL_LENGTHS.apply(zoom).doubleValue() / Math.pow(2, zoom) / 256d;
for (int i = 0; i < items.size(); i++) {
Object relIdObj = items.get(i).attrs().remove(TEMP_REL_ID_ADDR);
if (relIdObj instanceof Long relId && riverRelationLengths.get(relId).get() < minSizeAtZoom) {
items.set(i, null);
}
}
return FeatureMerge.mergeLineStrings(
items,
config.minFeatureSize(zoom),
config.tolerance(zoom),
BUFFER_SIZE
);
} else if (zoom >= 9 && zoom <= 11) {
return FeatureMerge.mergeLineStrings(
items,
MIN_PIXEL_LENGTHS.apply(zoom).doubleValue(),
config.tolerance(zoom),
BUFFER_SIZE
);
}
return items;
}
}

180
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/util/LanguageUtils.java
Wyświetl plik

@ -1,180 +0,0 @@
/*
Copyright (c) 2021, MapTiler.com & OpenMapTiles contributors.
All rights reserved.
Code license: BSD 3-Clause License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Design license: CC-BY 4.0
See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for details on usage
*/
package com.onthegomap.planetiler.basemap.util;
import static com.onthegomap.planetiler.basemap.util.Utils.coalesce;
import static com.onthegomap.planetiler.basemap.util.Utils.nullIfEmpty;
import com.onthegomap.planetiler.util.Translations;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.function.Predicate;
import java.util.regex.Pattern;
import java.util.stream.Stream;
/**
* Utilities to extract common name fields (name, name_en, name_de, name:latin, name:nonlatin, name_int) that the
* OpenMapTiles schema uses across any map element with a name.
* <p>
* Ported from
* <a href="https://github.com/openmaptiles/openmaptiles-tools/blob/master/sql/zzz_language.sql">openmaptiles-tools</a>.
*/
public class LanguageUtils {
// Name tags that should be eligible for finding a latin name.
// See https://wiki.openstreetmap.org/wiki/Multilingual_names
private static final Predicate<String> VALID_NAME_TAGS =
Pattern
.compile("^name:[a-z]{2,3}(-[a-z]{4})?([-_](x-)?[a-z]{2,})?(-([a-z]{2}|[0-9]{3}))?$", Pattern.CASE_INSENSITIVE)
.asMatchPredicate();
// See https://github.com/onthegomap/planetiler/issues/86
// Match strings that only contain latin characters.
private static final Predicate<String> ONLY_LATIN = Pattern
.compile("^[\\P{IsLetter}[\\p{IsLetter}&&\\p{IsLatin}]]+$")
.asMatchPredicate();
// Match only latin letters
private static final Pattern LATIN_LETTER = Pattern.compile("[\\p{IsLetter}&&\\p{IsLatin}]+");
private static final Pattern EMPTY_PARENS = Pattern.compile("(\\([ -.]*\\)|\\[[ -.]*])");
private static final Pattern LEADING_TRAILING_JUNK = Pattern.compile("((^[\\s./-]*)|([\\s./-]*$))");
private static final Pattern WHITESPACE = Pattern.compile("\\s+");
private static final Set<String> EN_DE_NAME_KEYS = Set.of("name:en", "name:de");
private LanguageUtils() {}
private static void putIfNotEmpty(Map<String, Object> dest, String key, Object value) {
if (value != null && !value.equals("")) {
dest.put(key, value);
}
}
private static String string(Object obj) {
return nullIfEmpty(obj == null ? null : obj.toString());
}
static boolean containsOnlyLatinCharacters(String string) {
return string != null && ONLY_LATIN.test(string);
}
private static String transliteratedName(Map<String, Object> tags) {
return Translations.transliterate(string(tags.get("name")));
}
static String removeLatinCharacters(String name) {
if (name == null) {
return null;
}
var matcher = LATIN_LETTER.matcher(name);
if (matcher.find()) {
String result = matcher.replaceAll("");
// if the name was "<nonlatin text> (<latin description)"
// or "<nonlatin text> - <latin description>"
// then remove any of those extra characters now
result = EMPTY_PARENS.matcher(result).replaceAll("");
result = LEADING_TRAILING_JUNK.matcher(result).replaceAll("");
return WHITESPACE.matcher(result).replaceAll(" ");
}
return name.trim();
}
/**
* Returns a map with default name attributes (name, name_en, name_de, name:latin, name:nonlatin, name_int) that every
* element should have, derived from name, int_name, name:en, and name:de tags on the input element.
*
* <ul>
* <li>name is the original name value from the element</li>
* <li>name_en is the original name:en value from the element, or name if missing</li>
* <li>name_de is the original name:de value from the element, or name/ name_en if missing</li>
* <li>name:latin is the first of name, int_name, or any name: attribute that contains only latin characters</li>
* <li>name:nonlatin is any nonlatin part of name if present</li>
* <li>name_int is the first of int_name name:en name:latin name</li>
* </ul>
*/
public static Map<String, Object> getNamesWithoutTranslations(Map<String, Object> tags) {
return getNames(tags, null);
}
/**
* Returns a map with default name attributes that {@link #getNamesWithoutTranslations(Map)} adds, but also
* translations for every language that {@code translations} is configured to handle.
*/
public static Map<String, Object> getNames(Map<String, Object> tags, Translations translations) {
Map<String, Object> result = new HashMap<>();
String name = string(tags.get("name"));
String intName = string(tags.get("int_name"));
String nameEn = string(tags.get("name:en"));
String nameDe = string(tags.get("name:de"));
boolean isLatin = containsOnlyLatinCharacters(name);
String latin = isLatin ? name :
Stream
.concat(Stream.of(nameEn, intName, nameDe), getAllNameTranslationsBesidesEnglishAndGerman(tags))
.filter(LanguageUtils::containsOnlyLatinCharacters)
.findFirst().orElse(null);
if (latin == null && translations != null && translations.getShouldTransliterate()) {
latin = transliteratedName(tags);
}
String nonLatin = isLatin ? null : removeLatinCharacters(name);
if (coalesce(nonLatin, "").equals(latin)) {
nonLatin = null;
}
putIfNotEmpty(result, "name", name);
putIfNotEmpty(result, "name_en", coalesce(nameEn, name));
putIfNotEmpty(result, "name_de", coalesce(nameDe, name, nameEn));
putIfNotEmpty(result, "name:latin", latin);
putIfNotEmpty(result, "name:nonlatin", nonLatin);
putIfNotEmpty(result, "name_int", coalesce(
intName,
nameEn,
latin,
name
));
if (translations != null) {
translations.addTranslations(result, tags);
}
return result;
}
private static Stream<String> getAllNameTranslationsBesidesEnglishAndGerman(Map<String, Object> tags) {
return tags.entrySet().stream()
.filter(e -> !EN_DE_NAME_KEYS.contains(e.getKey()) && VALID_NAME_TAGS.test(e.getKey()))
.map(Map.Entry::getValue)
.map(LanguageUtils::string);
}
}

78
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/util/Utils.java
Wyświetl plik

@ -1,78 +0,0 @@
package com.onthegomap.planetiler.basemap.util;
import com.onthegomap.planetiler.util.Parse;
import java.util.Map;
/**
* Common utilities for working with data and the OpenMapTiles schema in {@code layers} implementations.
*/
public class Utils {
public static <T> T coalesce(T a, T b) {
return a != null ? a : b;
}
public static <T> T coalesce(T a, T b, T c) {
return a != null ? a : b != null ? b : c;
}
public static <T> T coalesce(T a, T b, T c, T d) {
return a != null ? a : b != null ? b : c != null ? c : d;
}
public static <T> T coalesce(T a, T b, T c, T d, T e) {
return a != null ? a : b != null ? b : c != null ? c : d != null ? d : e;
}
public static <T> T coalesce(T a, T b, T c, T d, T e, T f) {
return a != null ? a : b != null ? b : c != null ? c : d != null ? d : e != null ? e : f;
}
/** Boxes {@code a} into an {@link Integer}, or {@code null} if {@code a} is {@code nullValue}. */
public static Long nullIfLong(long a, long nullValue) {
return a == nullValue ? null : a;
}
/** Boxes {@code a} into a {@link Long}, or {@code null} if {@code a} is {@code nullValue}. */
public static Integer nullIfInt(int a, int nullValue) {
return a == nullValue ? null : a;
}
/** Returns {@code a}, or null if {@code a} is "". */
public static String nullIfEmpty(String a) {
return (a == null || a.isEmpty()) ? null : a;
}
/** Returns true if {@code a} is null, or its {@link Object#toString()} value is "". */
public static boolean nullOrEmpty(Object a) {
return a == null || a.toString().isEmpty();
}
/** Returns a map with {@code ele} (meters) and {ele_ft} attributes from an elevation in meters. */
public static Map<String, Object> elevationTags(double meters) {
return Map.of(
"ele", (int) Math.round(meters),
"ele_ft", (int) Math.round(meters * 3.2808399)
);
}
/**
* Returns a map with {@code ele} (meters) and {ele_ft} attributes from an elevation string in meters, if {@code
* meters} can be parsed as a valid number.
*/
public static Map<String, Object> elevationTags(String meters) {
Double ele = Parse.meters(meters);
return ele == null ? Map.of() : elevationTags(ele);
}
/** Returns "bridge" or "tunnel" string used for "brunnel" attribute by OpenMapTiles schema. */
public static String brunnel(boolean isBridge, boolean isTunnel) {
return brunnel(isBridge, isTunnel, false);
}
/** Returns "bridge" or "tunnel" or "ford" string used for "brunnel" attribute by OpenMapTiles schema. */
public static String brunnel(boolean isBridge, boolean isTunnel, boolean isFord) {
return isBridge ? "bridge" : isTunnel ? "tunnel" : isFord ? "ford" : null;
}
}

44
planetiler-basemap/src/main/java/com/onthegomap/planetiler/basemap/util/VerifyMonaco.java
Wyświetl plik

@ -1,44 +0,0 @@
package com.onthegomap.planetiler.basemap.util;
import com.onthegomap.planetiler.mbtiles.Mbtiles;
import com.onthegomap.planetiler.mbtiles.Verify;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Map;
import org.locationtech.jts.geom.Envelope;
import org.locationtech.jts.geom.LineString;
import org.locationtech.jts.geom.Point;
import org.locationtech.jts.geom.Polygon;
/**
* A utility to check the contents of an mbtiles file generated for Monaco.
*/
public class VerifyMonaco {
public static final Envelope MONACO_BOUNDS = new Envelope(7.40921, 7.44864, 43.72335, 43.75169);
/**
* Returns a verification result with a basic set of checks against an openmaptiles map built from an extract for
* Monaco.
*/
public static Verify verify(Mbtiles mbtiles) {
Verify verify = Verify.verify(mbtiles);
verify.checkMinFeatureCount(MONACO_BOUNDS, "building", Map.of(), 13, 14, 100, Polygon.class);
verify.checkMinFeatureCount(MONACO_BOUNDS, "transportation", Map.of(), 10, 14, 5, LineString.class);
verify.checkMinFeatureCount(MONACO_BOUNDS, "landcover", Map.of(
"class", "grass",
"subclass", "park"
), 14, 10, Polygon.class);
verify.checkMinFeatureCount(MONACO_BOUNDS, "water", Map.of("class", "ocean"), 0, 14, 1, Polygon.class);
verify.checkMinFeatureCount(MONACO_BOUNDS, "place", Map.of("class", "country"), 2, 14, 1, Point.class);
return verify;
}
public static void main(String[] args) throws IOException {
try (var mbtiles = Mbtiles.newReadOnlyDatabase(Path.of(args[0]))) {
var result = verify(mbtiles);
result.print();
result.failIfErrors();
}
}
}

38
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/BasemapProfileTest.java
Wyświetl plik

@ -1,38 +0,0 @@
package com.onthegomap.planetiler.basemap;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.Wikidata;
import java.util.List;
import org.junit.jupiter.api.Test;
class BasemapProfileTest {
private final Wikidata.WikidataTranslations wikidataTranslations = new Wikidata.WikidataTranslations();
private final Translations translations = Translations.defaultProvider(List.of("en", "es", "de"))
.addTranslationProvider(wikidataTranslations);
private final BasemapProfile profile = new BasemapProfile(translations, PlanetilerConfig.defaults(),
Stats.inMemory());
@Test
void testCaresAboutWikidata() {
var node = new OsmElement.Node(1, 1, 1);
node.setTag("aeroway", "gate");
assertTrue(profile.caresAboutWikidataTranslation(node));
node.setTag("aeroway", "other");
assertFalse(profile.caresAboutWikidataTranslation(node));
}
@Test
void testDoesntCareAboutWikidataForRoads() {
var way = new OsmElement.Way(1);
way.setTag("highway", "footway");
assertFalse(profile.caresAboutWikidataTranslation(way));
}
}

225
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/BasemapTest.java
Wyświetl plik

@ -1,225 +0,0 @@
package com.onthegomap.planetiler.basemap;
import static com.onthegomap.planetiler.TestUtils.assertContains;
import static com.onthegomap.planetiler.TestUtils.assertFeatureNear;
import static com.onthegomap.planetiler.basemap.util.VerifyMonaco.MONACO_BOUNDS;
import static com.onthegomap.planetiler.util.Gzip.gunzip;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import com.onthegomap.planetiler.TestUtils;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.util.VerifyMonaco;
import com.onthegomap.planetiler.config.Arguments;
import com.onthegomap.planetiler.mbtiles.Mbtiles;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Map;
import java.util.Set;
import java.util.stream.Stream;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestFactory;
import org.junit.jupiter.api.io.TempDir;
import org.locationtech.jts.geom.Geometry;
import org.locationtech.jts.geom.LineString;
import org.locationtech.jts.geom.Point;
import org.locationtech.jts.geom.Polygon;
/**
* End-to-end tests for basemap generation.
* <p>
* Generates an entire map for the smallest openstreetmap extract available (Monaco) and asserts that expected output
* features exist
*/
class BasemapTest {
@TempDir
static Path tmpDir;
private static Mbtiles mbtiles;
@BeforeAll
public static void runPlanetiler() throws Exception {
Path dbPath = tmpDir.resolve("output.mbtiles");
BasemapMain.run(Arguments.of(
// Override input source locations
"osm_path", TestUtils.pathToResource("monaco-latest.osm.pbf"),
"natural_earth_path", TestUtils.pathToResource("natural_earth_vector.sqlite.zip"),
"water_polygons_path", TestUtils.pathToResource("water-polygons-split-3857.zip"),
// no centerlines in monaco - so fake it out with an empty source
"lake_centerlines_path", TestUtils.pathToResource("water-polygons-split-3857.zip"),
// Override temp dir location
"tmp", tmpDir.toString(),
// Override output location
"mbtiles", dbPath.toString()
));
mbtiles = Mbtiles.newReadOnlyDatabase(dbPath);
}
@AfterAll
public static void close() throws IOException {
mbtiles.close();
}
@Test
void testMetadata() {
Map<String, String> metadata = mbtiles.metadata().getAll();
assertEquals("OpenMapTiles", metadata.get("name"));
assertEquals("0", metadata.get("minzoom"));
assertEquals("14", metadata.get("maxzoom"));
assertEquals("baselayer", metadata.get("type"));
assertEquals("pbf", metadata.get("format"));
assertEquals("7.40921,43.72335,7.44864,43.75169", metadata.get("bounds"));
assertEquals("7.42892,43.73752,14", metadata.get("center"));
assertContains("openmaptiles.org", metadata.get("description"));
assertContains("openmaptiles.org", metadata.get("attribution"));
assertContains("www.openstreetmap.org/copyright", metadata.get("attribution"));
}
@Test
void ensureValidGeometries() throws Exception {
Set<Mbtiles.TileEntry> parsedTiles = TestUtils.getAllTiles(mbtiles);
for (var tileEntry : parsedTiles) {
var decoded = VectorTile.decode(gunzip(tileEntry.bytes()));
for (VectorTile.Feature feature : decoded) {
TestUtils.validateGeometry(feature.geometry().decode());
}
}
}
@Test
void testContainsOceanPolyons() {
assertFeatureNear(mbtiles, "water", Map.of(
"class", "ocean"
), 7.4484, 43.70783, 0, 14);
}
@Test
void testContainsCountryName() {
assertFeatureNear(mbtiles, "place", Map.of(
"class", "country",
"iso_a2", "MC",
"name", "Monaco"
), 7.42769, 43.73235, 2, 14);
}
@Test
void testContainsSuburb() {
assertFeatureNear(mbtiles, "place", Map.of(
"name", "Les Moneghetti",
"class", "suburb"
), 7.41746, 43.73638, 11, 14);
}
@Test
void testContainsBuildings() {
assertFeatureNear(mbtiles, "building", Map.of(), 7.41919, 43.73401, 13, 14);
assertNumFeatures("building", Map.of(), 14, 1316, Polygon.class);
assertNumFeatures("building", Map.of(), 13, 196, Polygon.class);
}
@Test
void testContainsHousenumber() {
assertFeatureNear(mbtiles, "housenumber", Map.of(
"housenumber", "27"
), 7.42117, 43.73652, 14, 14);
assertNumFeatures("housenumber", Map.of(), 14, 274, Point.class);
}
@Test
void testBoundary() {
assertFeatureNear(mbtiles, "boundary", Map.of(
"admin_level", 2L,
"maritime", 1L,
"disputed", 0L
), 7.41884, 43.72396, 4, 14);
}
@Test
void testAeroway() {
assertNumFeatures("aeroway", Map.of(
"class", "heliport"
), 14, 1, Polygon.class);
assertNumFeatures("aeroway", Map.of(
"class", "helipad"
), 14, 11, Polygon.class);
}
@Test
void testLandcover() {
assertNumFeatures("landcover", Map.of(
"class", "grass",
"subclass", "park"
), 14, 20, Polygon.class);
assertNumFeatures("landcover", Map.of(
"class", "grass",
"subclass", "garden"
), 14, 33, Polygon.class);
}
@Test
void testPoi() {
assertNumFeatures("poi", Map.of(
"class", "restaurant",
"subclass", "restaurant"
), 14, 217, Point.class);
assertNumFeatures("poi", Map.of(
"class", "art_gallery",
"subclass", "artwork"
), 14, 132, Point.class);
}
@Test
void testLanduse() {
assertNumFeatures("landuse", Map.of(
"class", "residential"
), 14, 8, Polygon.class);
assertNumFeatures("landuse", Map.of(
"class", "hospital"
), 14, 4, Polygon.class);
}
@Test
void testTransportation() {
assertNumFeatures("transportation", Map.of(
"class", "path",
"subclass", "footway"
), 14, 909, LineString.class);
assertNumFeatures("transportation", Map.of(
"class", "primary"
), 14, 170, LineString.class);
}
@Test
void testTransportationName() {
assertNumFeatures("transportation_name", Map.of(
"name", "Boulevard du Larvotto",
"class", "primary"
), 14, 12, LineString.class);
}
@Test
void testWaterway() {
assertNumFeatures("waterway", Map.of(
"class", "stream"
), 14, 6, LineString.class);
}
@TestFactory
Stream<DynamicTest> testVerifyChecks() {
return VerifyMonaco.verify(mbtiles).results().stream()
.map(check -> dynamicTest(check.name(), () -> {
check.error().ifPresent(Assertions::fail);
}));
}
private static void assertNumFeatures(String layer, Map<String, Object> attrs, int zoom,
int expected, Class<? extends Geometry> clazz) {
TestUtils.assertNumFeatures(mbtiles, layer, zoom, attrs, MONACO_BOUNDS, expected, clazz);
}
}

228
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/GenerateTest.java
Wyświetl plik

@ -1,228 +0,0 @@
package com.onthegomap.planetiler.basemap;
import static com.onthegomap.planetiler.basemap.Generate.parseYaml;
import static com.onthegomap.planetiler.expression.Expression.*;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import com.fasterxml.jackson.databind.JsonNode;
import com.onthegomap.planetiler.expression.Expression;
import com.onthegomap.planetiler.expression.MultiExpression;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestFactory;
class GenerateTest {
@Test
void testParseSimple() {
MultiExpression<String> parsed = Generate.generateFieldMapping(parseYaml("""
output:
key: value
key2:
- value2
- '%value3%'
"""));
assertEquals(MultiExpression.of(List.of(
MultiExpression.entry("output", or(
matchAny("key", "value"),
matchAny("key2", "value2", "%value3%")
))
)), parsed);
}
@Test
void testParseAnd() {
MultiExpression<String> parsed = Generate.generateFieldMapping(parseYaml("""
output:
__AND__:
key1: val1
key2: val2
"""));
assertEquals(MultiExpression.of(List.of(
MultiExpression.entry("output", and(
matchAny("key1", "val1"),
matchAny("key2", "val2")
))
)), parsed);
}
@Test
void testParseAndWithOthers() {
MultiExpression<String> parsed = Generate.generateFieldMapping(parseYaml("""
output:
- key0: val0
- __AND__:
key1: val1
key2: val2
"""));
assertEquals(MultiExpression.of(List.of(
MultiExpression.entry("output", or(
matchAny("key0", "val0"),
and(
matchAny("key1", "val1"),
matchAny("key2", "val2")
)
))
)), parsed);
}
@Test
void testParseAndContainingOthers() {
MultiExpression<String> parsed = Generate.generateFieldMapping(parseYaml("""
output:
__AND__:
- key1: val1
- __OR__:
key2: val2
key3: val3
"""));
assertEquals(MultiExpression.of(List.of(
MultiExpression.entry("output", and(
matchAny("key1", "val1"),
or(
matchAny("key2", "val2"),
matchAny("key3", "val3")
)
))
)), parsed);
}
@Test
void testParseContainsKey() {
MultiExpression<String> parsed = Generate.generateFieldMapping(parseYaml("""
output:
key1: val1
key2:
"""));
assertEquals(MultiExpression.of(List.of(
MultiExpression.entry("output", or(
matchAny("key1", "val1"),
matchField("key2")
))
)), parsed);
}
@TestFactory
Stream<DynamicTest> testParseImposm3Mapping() {
record TestCase(String name, String mapping, String require, String reject, Expression expected) {
TestCase(String mapping, Expression expected) {
this(mapping, mapping, null, null, expected);
}
}
return Stream.of(
new TestCase(
"key: val", matchAny("key", "val")
),
new TestCase(
"key: [val1, val2]", matchAny("key", "val1", "val2")
),
new TestCase(
"key: [\"__any__\"]", matchField("key")
),
new TestCase("reject",
"key: val",
"mustkey: mustval",
null,
and(
matchAny("key", "val"),
matchAny("mustkey", "mustval")
)
),
new TestCase("require",
"key: val",
null,
"badkey: badval",
and(
matchAny("key", "val"),
not(matchAny("badkey", "badval"))
)
),
new TestCase("require and reject complex",
"""
key: val
key2:
- val1
- val2
""",
"""
mustkey: mustval
mustkey2:
- mustval1
- mustval2
""",
"""
notkey: notval
notkey2:
- notval1
- notval2
""",
and(
or(
matchAny("key", "val"),
matchAny("key2", "val1", "val2")
),
matchAny("mustkey", "mustval"),
matchAny("mustkey2", "mustval1", "mustval2"),
not(matchAny("notkey", "notval")),
not(matchAny("notkey2", "notval1", "notval2"))
)
)
).map(test -> dynamicTest(test.name, () -> {
Expression parsed = Generate
.parseImposm3MappingExpression("point", parseYaml(test.mapping), new Generate.Imposm3Filters(
parseYaml(test.reject),
parseYaml(test.require)
));
assertEquals(test.expected, parsed.replace(matchType("point"), TRUE).simplify());
}));
}
@Test
void testTypeMappingTopLevelType() {
Expression parsed = Generate
.parseImposm3MappingExpression("point", parseYaml("""
key: val
"""), new Generate.Imposm3Filters(null, null));
assertEquals(and(
matchAny("key", "val"),
matchType("point")
), parsed);
}
@Test
void testTypeMappings() {
Map<String, JsonNode> props = new LinkedHashMap<>();
props.put("points", parseYaml("""
key: val
"""));
props.put("polygons", parseYaml("""
key2: val2
"""));
Expression parsed = Generate
.parseImposm3MappingExpression(new Generate.Imposm3Table(
"geometry",
false,
List.of(),
null,
null,
props,
List.of()
));
assertEquals(or(
and(
matchAny("key", "val"),
matchType("point")
),
and(
matchAny("key2", "val2"),
matchType("polygon")
)
), parsed);
}
}

243
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/AbstractLayerTest.java
Wyświetl plik

@ -1,243 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.assertSubmap;
import static com.onthegomap.planetiler.TestUtils.newLineString;
import static com.onthegomap.planetiler.TestUtils.newPoint;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import static com.onthegomap.planetiler.basemap.util.Utils.coalesce;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.TestUtils;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.reader.osm.OsmReader;
import com.onthegomap.planetiler.reader.osm.OsmRelationInfo;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.Wikidata;
import java.util.Arrays;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.StreamSupport;
public abstract class AbstractLayerTest {
final Wikidata.WikidataTranslations wikidataTranslations = new Wikidata.WikidataTranslations();
final Translations translations = Translations.defaultProvider(List.of("en", "es", "de"))
.addTranslationProvider(wikidataTranslations);
final PlanetilerConfig params = PlanetilerConfig.defaults();
final BasemapProfile profile = new BasemapProfile(translations, PlanetilerConfig.defaults(),
Stats.inMemory());
final Stats stats = Stats.inMemory();
final FeatureCollector.Factory featureCollectorFactory = new FeatureCollector.Factory(params, stats);
static void assertFeatures(int zoom, List<Map<String, Object>> expected, Iterable<FeatureCollector.Feature> actual) {
// ensure both are sorted by layer
var expectedList =
expected.stream().sorted(Comparator.comparing(d -> coalesce(d.get("_layer"), "").toString())).toList();
var actualList = StreamSupport.stream(actual.spliterator(), false)
.sorted(Comparator.comparing(FeatureCollector.Feature::getLayer))
.toList();
assertEquals(expectedList.size(), actualList.size(), () -> "size: " + actualList);
for (int i = 0; i < expectedList.size(); i++) {
assertSubmap(expectedList.get(i), TestUtils.toMap(actualList.get(i), zoom));
}
}
static void assertDescending(int... vals) {
for (int i = 1; i < vals.length; i++) {
if (vals[i - 1] < vals[i]) {
fail("element at " + (i - 1) + " is less than element at " + i);
}
}
}
static void assertAscending(int... vals) {
for (int i = 1; i < vals.length; i++) {
if (vals[i - 1] > vals[i]) {
fail(
Arrays.toString(vals) +
System.lineSeparator() + "element at " + (i - 1) + " (" + vals[i - 1] + ") is greater than element at " +
i + " (" + vals[i] + ")");
}
}
}
VectorTile.Feature pointFeature(String layer, Map<String, Object> map, int group) {
return new VectorTile.Feature(
layer,
1,
VectorTile.encodeGeometry(newPoint(0, 0)),
new HashMap<>(map),
group
);
}
FeatureCollector process(SourceFeature feature) {
var collector = featureCollectorFactory.get(feature);
profile.processFeature(feature, collector);
return collector;
}
void assertCoversZoomRange(int minzoom, int maxzoom, String layer, FeatureCollector... featureCollectors) {
Map<?, ?>[] zooms = new Map[Math.max(15, maxzoom + 1)];
for (var features : featureCollectors) {
for (var feature : features) {
if (feature.getLayer().equals(layer)) {
for (int zoom = feature.getMinZoom(); zoom <= feature.getMaxZoom(); zoom++) {
Map<String, Object> map = TestUtils.toMap(feature, zoom);
if (zooms[zoom] != null) {
fail("Multiple features at z" + zoom + ":" + System.lineSeparator() + zooms[zoom] + "\n" + map);
}
zooms[zoom] = map;
}
}
}
}
for (int zoom = 0; zoom <= 14; zoom++) {
if (zoom < minzoom || zoom > maxzoom) {
if (zooms[zoom] != null) {
fail("Expected nothing at z" + zoom + " but found: " + zooms[zoom]);
}
} else {
if (zooms[zoom] == null) {
fail("No feature at z" + zoom);
}
}
}
}
SourceFeature pointFeature(Map<String, Object> props) {
return SimpleFeature.create(
newPoint(0, 0),
new HashMap<>(props),
OSM_SOURCE,
null,
0
);
}
SourceFeature lineFeature(Map<String, Object> props) {
return SimpleFeature.create(
newLineString(0, 0, 1, 1),
new HashMap<>(props),
OSM_SOURCE,
null,
0
);
}
SourceFeature closedWayFeature(Map<String, Object> props) {
return SimpleFeature.createFakeOsmFeature(
newLineString(0, 0, 1, 0, 1, 1, 0, 1, 0, 0),
new HashMap<>(props),
OSM_SOURCE,
null,
0,
null
);
}
SourceFeature polygonFeatureWithArea(double area, Map<String, Object> props) {
return SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(area))),
new HashMap<>(props),
OSM_SOURCE,
null,
0
);
}
SourceFeature polygonFeature(Map<String, Object> props) {
return polygonFeatureWithArea(1, props);
}
protected SimpleFeature lineFeatureWithRelation(List<OsmRelationInfo> relationInfos,
Map<String, Object> map) {
return SimpleFeature.createFakeOsmFeature(
newLineString(0, 0, 1, 1),
map,
OSM_SOURCE,
null,
0,
(relationInfos == null ? List.<OsmRelationInfo>of() : relationInfos).stream()
.map(r -> new OsmReader.RelationMember<>("", r)).toList()
);
}
protected void testMergesLinestrings(Map<String, Object> attrs, String layer,
double length, int zoom) throws GeometryException {
var line1 = new VectorTile.Feature(
layer,
1,
VectorTile.encodeGeometry(newLineString(0, 0, length / 2, 0)),
attrs,
0
);
var line2 = new VectorTile.Feature(
layer,
1,
VectorTile.encodeGeometry(newLineString(length / 2, 0, length, 0)),
attrs,
0
);
var connected = new VectorTile.Feature(
layer,
1,
VectorTile.encodeGeometry(newLineString(0, 0, length, 0)),
attrs,
0
);
assertEquals(
List.of(connected),
profile.postProcessLayerFeatures(layer, zoom, List.of(line1, line2))
);
}
protected void testDoesNotMergeLinestrings(Map<String, Object> attrs, String layer,
double length, int zoom) throws GeometryException {
var line1 = new VectorTile.Feature(
layer,
1,
VectorTile.encodeGeometry(newLineString(0, 0, length / 2, 0)),
attrs,
0
);
var line2 = new VectorTile.Feature(
layer,
1,
VectorTile.encodeGeometry(newLineString(length / 2, 0, length, 0)),
attrs,
0
);
assertEquals(
List.of(line1, line2),
profile.postProcessLayerFeatures(layer, zoom, List.of(line1, line2))
);
}
public static Map<String, Object> mapOf(Object... args) {
assert args.length % 2 == 0;
Map<String, Object> result = new HashMap<>();
for (int i = 0; i < args.length; i += 2) {
String key = args[i].toString();
Object value = args[i + 1];
result.put(key, value == null ? "<null>" : value);
}
return result;
}
}

156
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/AerodromeLabelTest.java
Wyświetl plik

@ -1,156 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
class AerodromeLabelTest extends AbstractLayerTest {
@BeforeEach
public void setupWikidataTranslation() {
wikidataTranslations.put(123, "es", "es wd name");
}
@Test
void testIntlWithIata() {
assertFeatures(14, List.of(Map.of(
"class", "international",
"ele", 100,
"ele_ft", 328,
"name", "osm name",
"name:es", "es wd name",
"_layer", "aerodrome_label",
"_type", "point",
"_minzoom", 8,
"_maxzoom", 14,
"_buffer", 64d
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"name", "osm name",
"wikidata", "Q123",
"ele", "100",
"aerodrome", "international",
"iata", "123",
"icao", "1234"
))));
}
@Test
void testElevationFeet() {
assertFeatures(14, List.of(Map.of(
"ele", 100,
"ele_ft", 328
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"name", "osm name",
"ele", "328'",
"aerodrome", "international",
"iata", "123",
"icao", "1234"
))));
}
@Test
void testElevationFeetInches() {
assertFeatures(14, List.of(Map.of(
"ele", 100,
"ele_ft", 328
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"name", "osm name",
"ele", "328' 1\"",
"aerodrome", "international",
"iata", "123",
"icao", "1234"
))));
}
@Test
void testInternational() {
assertFeatures(14, List.of(Map.of(
"class", "international",
"_layer", "aerodrome_label",
"_minzoom", 10 // no IATA
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"aerodrome_type", "international"
))));
}
@Test
void testPublic() {
assertFeatures(14, List.of(Map.of(
"class", "public",
"_layer", "aerodrome_label",
"_minzoom", 10
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"aerodrome_type", "public airport"
))));
assertFeatures(14, List.of(Map.of(
"class", "public",
"_layer", "aerodrome_label"
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"aerodrome_type", "civil"
))));
}
@Test
void testMilitary() {
assertFeatures(14, List.of(Map.of(
"class", "military",
"_layer", "aerodrome_label",
"_minzoom", 10
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"aerodrome_type", "military airport"
))));
assertFeatures(14, List.of(Map.of(
"class", "military",
"_layer", "aerodrome_label"
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"military", "airfield"
))));
}
@Test
void testPrivate() {
assertFeatures(14, List.of(Map.of(
"class", "private",
"_layer", "aerodrome_label",
"_minzoom", 10
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"aerodrome_type", "private"
))));
assertFeatures(14, List.of(Map.of(
"class", "private",
"_layer", "aerodrome_label"
)), process(pointFeature(Map.of(
"aeroway", "aerodrome",
"aerodrome", "private"
))));
}
@Test
void testOther() {
assertFeatures(14, List.of(Map.of(
"class", "other",
"_layer", "aerodrome_label",
"_minzoom", 10
)), process(pointFeature(Map.of(
"aeroway", "aerodrome"
))));
}
@Test
void testIgnoreNonPoints() {
assertFeatures(14, List.of(), process(lineFeature(Map.of(
"aeroway", "aerodrome"
))));
}
}

92
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/AerowayTest.java
Wyświetl plik

@ -1,92 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class AerowayTest extends AbstractLayerTest {
@Test
void aerowayGate() {
assertFeatures(14, List.of(Map.of(
"class", "gate",
"ref", "123",
"_layer", "aeroway",
"_type", "point",
"_minzoom", 14,
"_maxzoom", 14,
"_buffer", 4d
)), process(pointFeature(Map.of(
"aeroway", "gate",
"ref", "123"
))));
assertFeatures(14, List.of(), process(lineFeature(Map.of(
"aeroway", "gate"
))));
assertFeatures(14, List.of(), process(polygonFeature(Map.of(
"aeroway", "gate"
))));
}
@Test
void aerowayLine() {
assertFeatures(14, List.of(Map.of(
"class", "runway",
"ref", "123",
"_layer", "aeroway",
"_type", "line",
"_minzoom", 10,
"_maxzoom", 14,
"_buffer", 4d
)), process(lineFeature(Map.of(
"aeroway", "runway",
"ref", "123"
))));
assertFeatures(14, List.of(), process(pointFeature(Map.of(
"aeroway", "runway"
))));
}
@Test
void aerowayPolygon() {
assertFeatures(14, List.of(Map.of(
"class", "runway",
"ref", "123",
"_layer", "aeroway",
"_type", "polygon",
"_minzoom", 10,
"_maxzoom", 14,
"_buffer", 4d
)), process(polygonFeature(Map.of(
"aeroway", "runway",
"ref", "123"
))));
assertFeatures(14, List.of(Map.of(
"class", "runway",
"ref", "123",
"_layer", "aeroway",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"area:aeroway", "runway",
"ref", "123"
))));
assertFeatures(14, List.of(Map.of(
"class", "heliport",
"ref", "123",
"_layer", "aeroway",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"aeroway", "heliport",
"ref", "123"
))));
assertFeatures(14, List.of(), process(lineFeature(Map.of(
"aeroway", "heliport"
))));
assertFeatures(14, List.of(), process(pointFeature(Map.of(
"aeroway", "heliport"
))));
}
}

641
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/BoundaryTest.java
Wyświetl plik

@ -1,641 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.newLineString;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmReader;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
import org.junit.jupiter.api.Test;
class BoundaryTest extends AbstractLayerTest {
@Test
void testNaturalEarthCountryBoundaries() {
assertCoversZoomRange(
0, 4, "boundary",
process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_110m_admin_0_boundary_lines_land",
0
)),
process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_50m_admin_0_boundary_lines_land",
1
)),
process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_0_boundary_lines_land",
2
))
);
assertFeatures(0, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 0,
"maritime", 0,
"admin_level", 2,
"_minzoom", 0,
"_buffer", 4d
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"featurecla", "International boundary (verify)"
),
NATURAL_EARTH_SOURCE,
"ne_110m_admin_0_boundary_lines_land",
0
)));
assertFeatures(0, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 1,
"maritime", 0,
"admin_level", 2,
"_buffer", 4d
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"featurecla", "Disputed (please verify)"
),
NATURAL_EARTH_SOURCE,
"ne_110m_admin_0_boundary_lines_land",
0
)));
assertFeatures(0, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"admin_level", 2
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"featurecla", "International boundary (verify)"
),
NATURAL_EARTH_SOURCE,
"ne_50m_admin_0_boundary_lines_land",
0
)));
assertFeatures(0, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"admin_level", 2
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"featurecla", "International boundary (verify)"
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_0_boundary_lines_land",
0
)));
assertFeatures(0, List.of(), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"featurecla", "Lease Limit"
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_0_boundary_lines_land",
0
)));
}
@Test
void testNaturalEarthStateBoundaries() {
assertFeatures(0, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 0,
"maritime", 0,
"admin_level", 4,
"_minzoom", 1,
"_maxzoom", 4,
"_buffer", 4d
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"min_zoom", 7d
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_1_states_provinces_lines",
0
)));
assertFeatures(0, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 0,
"maritime", 0,
"admin_level", 4,
"_minzoom", 4,
"_maxzoom", 4,
"_buffer", 4d
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"min_zoom", 7.6d
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_1_states_provinces_lines",
0
)));
assertFeatures(0, List.of(), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"min_zoom", 7.9d
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_1_states_provinces_lines",
0
)));
assertFeatures(0, List.of(), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_1_states_provinces_lines",
0
)));
}
@Test
void testMergesDisconnectedLineFeatures() throws GeometryException {
testMergesLinestrings(Map.of("admin_level", 2), Boundary.LAYER_NAME, 10, 13);
testMergesLinestrings(Map.of("admin_level", 2), Boundary.LAYER_NAME, 10, 14);
}
@Test
void testOsmTownBoundary() {
var relation = new OsmElement.Relation(1);
relation.setTag("type", "boundary");
relation.setTag("admin_level", "10");
relation.setTag("boundary", "administrative");
assertFeatures(14, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 0,
"maritime", 0,
"admin_level", 10,
"_minzoom", 12,
"_maxzoom", 14,
"_buffer", 4d,
"_minpixelsize", 0d
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation),
Map.of())));
}
@Test
void testOsmBoundaryLevelTwoAndAHalf() {
var relation = new OsmElement.Relation(1);
relation.setTag("type", "boundary");
relation.setTag("admin_level", "2.5");
relation.setTag("boundary", "administrative");
assertFeatures(14, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 0,
"maritime", 0,
"admin_level", 3,
"_minzoom", 5,
"_maxzoom", 14,
"_buffer", 4d,
"_minpixelsize", 0d
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation),
Map.of())));
}
@Test
void testOsmBoundaryTakesMinAdminLevel() {
var relation1 = new OsmElement.Relation(1);
relation1.setTag("type", "boundary");
relation1.setTag("admin_level", "10");
relation1.setTag("name", "Town");
relation1.setTag("boundary", "administrative");
var relation2 = new OsmElement.Relation(2);
relation2.setTag("type", "boundary");
relation2.setTag("admin_level", "4");
relation2.setTag("name", "State");
relation2.setTag("boundary", "administrative");
assertFeatures(14, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 0,
"maritime", 0,
"admin_level", 4
)), process(lineFeatureWithRelation(
Stream.concat(
profile.preprocessOsmRelation(relation2).stream(),
profile.preprocessOsmRelation(relation1).stream()
).toList(),
Map.of())));
}
@Test
void testOsmBoundarySetsMaritimeFromWay() {
var relation1 = new OsmElement.Relation(1);
relation1.setTag("type", "boundary");
relation1.setTag("admin_level", "10");
relation1.setTag("boundary", "administrative");
assertFeatures(14, List.of(Map.of(
"maritime", 1
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation1),
Map.of(
"maritime", "yes"
))
));
assertFeatures(14, List.of(Map.of(
"maritime", 1
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation1),
Map.of(
"natural", "coastline"
))
));
assertFeatures(14, List.of(Map.of(
"maritime", 1
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation1),
Map.of(
"boundary_type", "maritime"
))
));
}
@Test
void testIgnoresProtectedAreas() {
var relation1 = new OsmElement.Relation(1);
relation1.setTag("type", "boundary");
relation1.setTag("admin_level", "10");
relation1.setTag("boundary", "protected_area");
assertNull(profile.preprocessOsmRelation(relation1));
}
@Test
void testIgnoresProtectedAdminLevelOver10() {
var relation1 = new OsmElement.Relation(1);
relation1.setTag("type", "boundary");
relation1.setTag("admin_level", "11");
relation1.setTag("boundary", "administrative");
assertNull(profile.preprocessOsmRelation(relation1));
}
@Test
void testOsmBoundaryDisputed() {
var relation = new OsmElement.Relation(1);
relation.setTag("type", "boundary");
relation.setTag("admin_level", "5");
relation.setTag("boundary", "administrative");
relation.setTag("disputed", "yes");
relation.setTag("name", "Border A - B");
relation.setTag("claimed_by", "A");
assertFeatures(14, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed_name", "BorderA-B",
"claimed_by", "A",
"disputed", 1,
"maritime", 0,
"admin_level", 5
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation),
Map.of())
));
}
@Test
void testOsmBoundaryDisputedFromWay() {
var relation = new OsmElement.Relation(1);
relation.setTag("type", "boundary");
relation.setTag("admin_level", "5");
relation.setTag("boundary", "administrative");
assertFeatures(14, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 1,
"maritime", 0,
"admin_level", 5
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation),
Map.of(
"disputed", "yes"
))
));
assertFeatures(14, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 1,
"maritime", 0,
"admin_level", 5,
"claimed_by", "A",
"disputed_name", "AB"
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation),
Map.of(
"disputed", "yes",
"claimed_by", "A",
"name", "AB"
))
));
}
@Test
void testCountryBoundaryEmittedIfNoName() {
var relation = new OsmElement.Relation(1);
relation.setTag("type", "boundary");
relation.setTag("admin_level", "2");
relation.setTag("boundary", "administrative");
assertFeatures(14, List.of(Map.of(
"_layer", "boundary",
"_type", "line",
"disputed", 0,
"maritime", 0,
"admin_level", 2
)), process(lineFeatureWithRelation(
profile.preprocessOsmRelation(relation),
Map.of())
));
}
@Test
void testCountryLeftRightName() {
var country1 = new OsmElement.Relation(1);
country1.setTag("type", "boundary");
country1.setTag("admin_level", "2");
country1.setTag("boundary", "administrative");
country1.setTag("ISO3166-1:alpha3", "C1");
var country2 = new OsmElement.Relation(2);
country2.setTag("type", "boundary");
country2.setTag("admin_level", "2");
country2.setTag("boundary", "administrative");
country2.setTag("ISO3166-1:alpha3", "C2");
// shared edge
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
newLineString(0, 0, 0, 10),
Map.of(),
OSM_SOURCE,
null,
3,
Stream.concat(
profile.preprocessOsmRelation(country1).stream(),
profile.preprocessOsmRelation(country2).stream()
).map(r -> new OsmReader.RelationMember<>("", r)).toList()
)
));
// other 2 edges of country 1
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
newLineString(0, 0, 5, 10),
Map.of(),
OSM_SOURCE,
null,
4,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)
));
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
newLineString(0, 10, 5, 10),
Map.of(),
OSM_SOURCE,
null,
4,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)
));
// other 2 edges of country 2
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
newLineString(0, 0, -5, 10),
Map.of(),
OSM_SOURCE,
null,
4,
profile.preprocessOsmRelation(country2).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)
));
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
newLineString(0, 10, -5, 10),
Map.of(),
OSM_SOURCE,
null,
4,
profile.preprocessOsmRelation(country2).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)
));
List<FeatureCollector.Feature> features = new ArrayList<>();
profile.finish(OSM_SOURCE, new FeatureCollector.Factory(params, stats), features::add);
assertEquals(3, features.size());
// ensure shared edge has country labels on right sides
var sharedEdge = features.stream()
.filter(c -> c.getAttrsAtZoom(0).containsKey("adm0_l") && c.getAttrsAtZoom(0).containsKey("adm0_r")).findFirst()
.get();
if (sharedEdge.getGeometry().getCoordinate().y == 0.5) { // going up
assertEquals("C1", sharedEdge.getAttrsAtZoom(0).get("adm0_r"));
assertEquals("C2", sharedEdge.getAttrsAtZoom(0).get("adm0_l"));
} else { // going down
assertEquals("C2", sharedEdge.getAttrsAtZoom(0).get("adm0_r"));
assertEquals("C1", sharedEdge.getAttrsAtZoom(0).get("adm0_l"));
}
var c1 = features.stream()
.filter(c -> c.getGeometry().getEnvelopeInternal().getMaxX() > 0.5).findFirst()
.get();
if (c1.getGeometry().getCoordinate().y == 0.5) { // going up
assertEquals("C1", c1.getAttrsAtZoom(0).get("adm0_l"));
} else { // going down
assertEquals("C1", c1.getAttrsAtZoom(0).get("adm0_r"));
}
var c2 = features.stream()
.filter(c -> c.getGeometry().getEnvelopeInternal().getMinX() < 0.5).findFirst()
.get();
if (c2.getGeometry().getCoordinate().y == 0.5) { // going up
assertEquals("C2", c2.getAttrsAtZoom(0).get("adm0_r"));
} else { // going down
assertEquals("C2", c2.getAttrsAtZoom(0).get("adm0_l"));
}
}
@Test
void testCountryBoundaryNotClosed() {
var country1 = new OsmElement.Relation(1);
country1.setTag("type", "boundary");
country1.setTag("admin_level", "2");
country1.setTag("boundary", "administrative");
country1.setTag("ISO3166-1:alpha3", "C1");
// shared edge
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
newLineString(0, 0, 0, 10, 5, 5),
Map.of(),
OSM_SOURCE,
null,
3,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)));
List<FeatureCollector.Feature> features = new ArrayList<>();
profile.finish(OSM_SOURCE, new FeatureCollector.Factory(params, stats), features::add);
assertFeatures(0, List.of(Map.of(
"adm0_r", "<null>",
"adm0_l", "<null>",
"maritime", 0,
"disputed", 0,
"admin_level", 2,
"_layer", "boundary"
)), features);
}
@Test
void testNestedCountry() throws GeometryException {
var country1 = new OsmElement.Relation(1);
country1.setTag("type", "boundary");
country1.setTag("admin_level", "2");
country1.setTag("boundary", "administrative");
country1.setTag("ISO3166-1:alpha3", "C1");
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
GeoUtils.polygonToLineString(rectangle(0, 10)),
Map.of(),
OSM_SOURCE,
null,
3,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)));
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
GeoUtils.polygonToLineString(rectangle(1, 9)),
Map.of(),
OSM_SOURCE,
null,
3,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)));
List<FeatureCollector.Feature> features = new ArrayList<>();
profile.finish(OSM_SOURCE, new FeatureCollector.Factory(params, stats), features::add);
assertFeatures(0, List.of(Map.of(
"adm0_l", "C1",
"adm0_r", "<null>"
), Map.of(
"adm0_r", "C1",
"adm0_l", "<null>"
)), features);
}
@Test
void testDontLabelBadPolygon() {
var country1 = new OsmElement.Relation(1);
country1.setTag("type", "boundary");
country1.setTag("admin_level", "2");
country1.setTag("boundary", "administrative");
country1.setTag("ISO3166-1:alpha3", "C1");
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
GeoUtils.worldToLatLonCoords(newLineString(0, 0, 0.1, 0, 0.1, 0.1, 0.02, 0.1, 0.02, -0.02)),
Map.of(),
OSM_SOURCE,
null,
3,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)));
List<FeatureCollector.Feature> features = new ArrayList<>();
profile.finish(OSM_SOURCE, new FeatureCollector.Factory(params, stats), features::add);
assertFeatures(0, List.of(Map.of(
"adm0_l", "<null>",
"adm0_r", "<null>"
)), features);
}
@Test
void testIgnoreBadPolygonAndLabelGoodPart() throws GeometryException {
var country1 = new OsmElement.Relation(1);
country1.setTag("type", "boundary");
country1.setTag("admin_level", "2");
country1.setTag("boundary", "administrative");
country1.setTag("ISO3166-1:alpha3", "C1");
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
GeoUtils.worldToLatLonCoords(newLineString(0, 0, 0.1, 0, 0.1, 0.1, 0.2, 0.1, 0.2, -0.2)),
Map.of(),
OSM_SOURCE,
null,
3,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)));
assertFeatures(14, List.of(), process(SimpleFeature.createFakeOsmFeature(
GeoUtils.worldToLatLonCoords(GeoUtils.polygonToLineString(rectangle(0.2, 0.3))),
Map.of(),
OSM_SOURCE,
null,
3,
profile.preprocessOsmRelation(country1).stream().map(r -> new OsmReader.RelationMember<>("", r))
.toList()
)));
List<FeatureCollector.Feature> features = new ArrayList<>();
profile.finish(OSM_SOURCE, new FeatureCollector.Factory(params, stats), features::add);
assertFeatures(0, List.of(Map.of(
"adm0_l", "<null>",
"adm0_r", "<null>"
), Map.of(
"adm0_l", "<null>",
"adm0_r", "C1"
)), features);
}
}

179
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/BuildingTest.java
Wyświetl plik

@ -1,179 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import static org.junit.jupiter.api.Assertions.assertEquals;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmReader;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class BuildingTest extends AbstractLayerTest {
@Test
void testBuilding() {
assertFeatures(13, List.of(Map.of(
"colour", "<null>",
"hide_3d", "<null>",
"_layer", "building",
"_type", "polygon",
"_minzoom", 13,
"_maxzoom", 14,
"_buffer", 4d,
"_minpixelsize", 0.1d
)), process(polygonFeature(Map.of(
"building", "yes"
))));
assertFeatures(13, List.of(Map.of(
"_layer", "building",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"building:part", "yes"
))));
assertFeatures(13, List.of(), process(polygonFeature(Map.of(
"building", "no"
))));
}
@Test
void testIgnoreUndergroundBuilding() {
assertFeatures(14, List.of(), process(polygonFeature(Map.of(
"building", "yes",
"location", "underground"
))));
}
@Test
void testAirportBuildings() {
assertFeatures(13, List.of(Map.of(
"_layer", "building",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"aeroway", "terminal"
))));
assertFeatures(13, List.of(Map.of(
"_layer", "building",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"aeroway", "hangar"
))));
}
@Test
void testRenderHeights() {
assertFeatures(13, List.of(Map.of(
"render_height", "<null>",
"render_min_height", "<null>"
)), process(polygonFeature(Map.of(
"building", "yes"
))));
assertFeatures(14, List.of(Map.of(
"render_height", 5,
"render_min_height", 0
)), process(polygonFeature(Map.of(
"building", "yes"
))));
assertFeatures(14, List.of(Map.of(
"render_height", 12,
"render_min_height", 3
)), process(polygonFeature(Map.of(
"building", "yes",
"building:min_height", "3",
"building:height", "12"
))));
assertFeatures(14, List.of(Map.of(
"render_height", 44,
"render_min_height", 10
)), process(polygonFeature(Map.of(
"building", "yes",
"building:min_level", "3",
"building:levels", "12"
))));
assertFeatures(14, List.of(), process(polygonFeature(Map.of(
"building", "yes",
"building:min_level", "1500",
"building:levels", "1500"
))));
}
@Test
void testOutlineHides3d() {
var relation = new OsmElement.Relation(1);
relation.setTag("type", "building");
var relationInfos = profile.preprocessOsmRelation(relation).stream()
.map(i -> new OsmReader.RelationMember<>("outline", i)).toList();
assertFeatures(14, List.of(Map.of(
"_layer", "building",
"hide_3d", true
)), process(SimpleFeature.createFakeOsmFeature(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of(
"building", "yes"
),
OSM_SOURCE,
null,
0,
relationInfos
)));
}
@Test
void testMergePolygonsZ13() throws GeometryException {
var poly1 = new VectorTile.Feature(
Building.LAYER_NAME,
1,
VectorTile.encodeGeometry(rectangle(10, 20)),
Map.of(),
0
);
var poly2 = new VectorTile.Feature(
Building.LAYER_NAME,
1,
VectorTile.encodeGeometry(rectangle(20, 10, 22, 20)),
Map.of(),
0
);
assertEquals(
2,
profile.postProcessLayerFeatures(Building.LAYER_NAME, 14, List.of(poly1, poly2)).size()
);
assertEquals(
1,
profile.postProcessLayerFeatures(Building.LAYER_NAME, 13, List.of(poly1, poly2)).size()
);
}
@Test
void testColor() {
assertFeatures(14, List.of(Map.of(
"colour", "#ff0000"
)), process(polygonFeature(Map.of(
"building", "yes",
"building:colour", "#ff0000",
"building:material", "brick"
))));
assertFeatures(14, List.of(Map.of(
"colour", "#bd8161"
)), process(polygonFeature(Map.of(
"building", "yes",
"building:building", "yes",
"building:material", "brick"
))));
assertFeatures(13, List.of(Map.of(
"colour", "<null>"
)), process(polygonFeature(Map.of(
"building", "yes",
"building:building", "yes",
"building:colour", "#ff0000"
))));
}
}

30
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/HousenumberTest.java
Wyświetl plik

@ -1,30 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class HousenumberTest extends AbstractLayerTest {
@Test
void testHousenumber() {
assertFeatures(14, List.of(Map.of(
"_layer", "housenumber",
"_type", "point",
"_minzoom", 14,
"_maxzoom", 14,
"_buffer", 8d
)), process(pointFeature(Map.of(
"addr:housenumber", "10"
))));
assertFeatures(14, List.of(Map.of(
"_layer", "housenumber",
"_type", "point",
"_minzoom", 14,
"_maxzoom", 14,
"_buffer", 8d
)), process(polygonFeature(Map.of(
"addr:housenumber", "10"
))));
}
}

209
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/LandcoverTest.java
Wyświetl plik

@ -1,209 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import static org.junit.jupiter.api.Assertions.assertEquals;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class LandcoverTest extends AbstractLayerTest {
@Test
void testNaturalEarthGlaciers() {
var glacier1 = process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_110m_glaciated_areas",
0
));
var glacier2 = process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_50m_glaciated_areas",
0
));
var glacier3 = process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_glaciated_areas",
0
));
assertFeatures(0, List.of(Map.of(
"_layer", "landcover",
"subclass", "glacier",
"class", "ice",
"_buffer", 4d
)), glacier1);
assertFeatures(0, List.of(Map.of(
"_layer", "landcover",
"subclass", "glacier",
"class", "ice",
"_buffer", 4d
)), glacier2);
assertFeatures(0, List.of(Map.of(
"_layer", "landcover",
"subclass", "glacier",
"class", "ice",
"_buffer", 4d
)), glacier3);
assertCoversZoomRange(0, 6, "landcover",
glacier1,
glacier2,
glacier3
);
}
@Test
void testNaturalEarthAntarcticIceShelves() {
var ice1 = process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_50m_antarctic_ice_shelves_polys",
0
));
var ice2 = process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_antarctic_ice_shelves_polys",
0
));
assertFeatures(0, List.of(Map.of(
"_layer", "landcover",
"subclass", "ice_shelf",
"class", "ice",
"_buffer", 4d
)), ice1);
assertFeatures(0, List.of(Map.of(
"_layer", "landcover",
"subclass", "ice_shelf",
"class", "ice",
"_buffer", 4d
)), ice2);
assertCoversZoomRange(2, 6, "landcover",
ice1,
ice2
);
}
@Test
void testOsmLandcover() {
assertFeatures(13, List.of(Map.of(
"_layer", "landcover",
"subclass", "wood",
"class", "wood",
"_minpixelsize", 8d,
"_numpointsattr", "_numpoints",
"_minzoom", 7,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"natural", "wood"
))));
assertFeatures(12, List.of(Map.of(
"_layer", "landcover",
"subclass", "forest",
"class", "wood",
"_minpixelsize", 8d,
"_minzoom", 7,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"landuse", "forest"
))));
assertFeatures(10, List.of(Map.of(
"_layer", "landcover",
"subclass", "dune",
"class", "sand",
"_minpixelsize", 4d,
"_minzoom", 7,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"natural", "dune"
))));
assertFeatures(10, List.of(), process(polygonFeature(Map.of(
"landuse", "park"
))));
}
@Test
void testMergeForestsBuNumPointsZ9to13() throws GeometryException {
Map<String, Object> map = Map.of("subclass", "wood");
assertMerges(List.of(map, map, map, map, map, map), List.of(
// don't merge any
feature(rectangle(10, 20), Map.of("_numpoints", 48, "subclass", "wood")),
feature(rectangle(10, 20), Map.of("_numpoints", 49, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 50, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 299, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 300, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 301, "subclass", "wood"))
), 14);
assertMerges(List.of(map, map, map), List.of(
// < 300 - merge
feature(rectangle(10, 20), Map.of("_numpoints", 48, "subclass", "wood")),
feature(rectangle(10, 20), Map.of("_numpoints", 49, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 50, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 299, "subclass", "wood")),
// >= 300 - don't merge
feature(rectangle(12, 18), Map.of("_numpoints", 300, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 301, "subclass", "wood"))
), 13);
assertMerges(List.of(map, map), List.of(
// < 300 - merge
feature(rectangle(10, 20), Map.of("_numpoints", 48, "subclass", "wood")),
feature(rectangle(10, 20), Map.of("_numpoints", 49, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 50, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 299, "subclass", "wood")),
// >= 300 - merge
feature(rectangle(12, 18), Map.of("_numpoints", 300, "subclass", "wood")),
feature(rectangle(12, 18), Map.of("_numpoints", 301, "subclass", "wood"))
), 9);
}
@Test
void testMergeNonForestsBelowZ9() throws GeometryException {
Map<String, Object> map = Map.of("subclass", "dune");
assertMerges(List.of(map, map), List.of(
feature(rectangle(10, 20), Map.of("_numpoints", 48, "subclass", "dune")),
feature(rectangle(12, 18), Map.of("_numpoints", 301, "subclass", "dune"))
), 9);
assertMerges(List.of(map), List.of(
feature(rectangle(10, 20), Map.of("_numpoints", 48, "subclass", "dune")),
feature(rectangle(12, 18), Map.of("_numpoints", 301, "subclass", "dune"))
), 8);
assertMerges(List.of(map, map), List.of(
feature(rectangle(10, 20), Map.of("_numpoints", 48, "subclass", "dune")),
feature(rectangle(12, 18), Map.of("_numpoints", 301, "subclass", "dune"))
), 6);
}
private VectorTile.Feature feature(org.locationtech.jts.geom.Polygon geom, Map<String, Object> m) {
return new VectorTile.Feature(
"landcover",
1,
VectorTile.encodeGeometry(geom),
new HashMap<>(m),
0
);
}
private void assertMerges(List<Map<String, Object>> expected, List<VectorTile.Feature> in, int zoom)
throws GeometryException {
assertEquals(expected,
profile.postProcessLayerFeatures("landcover", zoom, in).stream().map(
VectorTile.Feature::attrs)
.toList());
}
}

94
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/LanduseTest.java
Wyświetl plik

@ -1,94 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.reader.SimpleFeature;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class LanduseTest extends AbstractLayerTest {
@Test
void testNaturalEarthUrbanAreas() {
assertFeatures(0, List.of(Map.of(
"_layer", "landuse",
"class", "residential",
"_buffer", 4d
)), process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of("scalerank", 1.9),
NATURAL_EARTH_SOURCE,
"ne_50m_urban_areas",
0
)));
assertFeatures(0, List.of(), process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
Map.of("scalerank", 2.1),
NATURAL_EARTH_SOURCE,
"ne_50m_urban_areas",
0
)));
}
@Test
void testOsmLanduse() {
assertFeatures(13, List.of(
Map.of("_layer", "poi"),
Map.of(
"_layer", "landuse",
"class", "railway",
"_minpixelsize", 4d,
"_minzoom", 9,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"landuse", "railway",
"amenity", "school"
))));
assertFeatures(13, List.of(Map.of("_layer", "poi"), Map.of(
"_layer", "landuse",
"class", "school",
"_minpixelsize", 4d,
"_minzoom", 9,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"amenity", "school"
))));
}
@Test
void testGraveYardBecomesCemetery() {
assertFeatures(14, List.of(
Map.of("_layer", "poi"),
Map.of(
"_layer", "landuse",
"class", "cemetery"
)), process(polygonFeature(Map.of(
"amenity", "grave_yard"
))));
}
@Test
void testOsmLanduseLowerZoom() {
assertFeatures(6, List.of(Map.of(
"_layer", "landuse",
"class", "suburb",
"_minzoom", 6,
"_maxzoom", 14,
"_minpixelsize", 1d
)), process(polygonFeature(Map.of(
"place", "suburb"
))));
assertFeatures(7, List.of(Map.of(
"_layer", "landuse",
"class", "residential",
"_minzoom", 6,
"_maxzoom", 14,
"_minpixelsize", 2d
)), process(polygonFeature(Map.of(
"landuse", "residential"
))));
}
}

324
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/MountainPeakTest.java
Wyświetl plik

@ -1,324 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.newPoint;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import static org.junit.jupiter.api.Assertions.assertEquals;
import com.google.common.collect.Lists;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
class MountainPeakTest extends AbstractLayerTest {
@BeforeEach
public void setupWikidataTranslation() {
wikidataTranslations.put(123, "es", "es wd name");
}
@Test
void testHappyPath() {
var peak = process(pointFeature(Map.of(
"natural", "peak",
"name", "test",
"ele", "100",
"wikidata", "Q123"
)));
assertFeatures(14, List.of(Map.of(
"class", "peak",
"ele", 100,
"ele_ft", 328,
"customary_ft", "<null>",
"_layer", "mountain_peak",
"_type", "point",
"_minzoom", 7,
"_maxzoom", 14,
"_buffer", 100d
)), peak);
assertFeatures(14, List.of(Map.of(
"name:latin", "test",
"name", "test",
"name:es", "es wd name"
)), peak);
}
@Test
void testLabelGrid() {
var peak = process(pointFeature(Map.of(
"natural", "peak",
"ele", "100"
)));
assertFeatures(14, List.of(Map.of(
"_labelgrid_limit", 0
)), peak);
assertFeatures(13, List.of(Map.of(
"_labelgrid_limit", 5,
"_labelgrid_size", 100d
)), peak);
}
@Test
void testVolcano() {
assertFeatures(14, List.of(Map.of(
"class", "volcano"
)), process(pointFeature(Map.of(
"natural", "volcano",
"ele", "100"
))));
}
@Test
void testElevationFeet() {
assertFeatures(14, List.of(Map.of(
"class", "volcano",
"ele", 30,
"ele_ft", 100
)), process(pointFeature(Map.of(
"natural", "volcano",
"ele", "100'"
))));
}
@Test
void testElevationFeetInches() {
assertFeatures(14, List.of(Map.of(
"class", "volcano",
"ele", 31,
"ele_ft", 101
)), process(pointFeature(Map.of(
"natural", "volcano",
"ele", "100' 11\""
))));
}
@Test
void testSaddle() {
assertFeatures(14, List.of(Map.of(
"class", "saddle"
)), process(pointFeature(Map.of(
"natural", "saddle",
"ele", "100"
))));
}
@Test
void testNoElevation() {
assertFeatures(14, List.of(), process(pointFeature(Map.of(
"natural", "volcano"
))));
}
@Test
void testBogusElevation() {
assertFeatures(14, List.of(), process(pointFeature(Map.of(
"natural", "volcano",
"ele", "11000"
))));
}
@Test
void testIgnorePeakLines() {
assertFeatures(14, List.of(), process(lineFeature(Map.of(
"natural", "peak",
"name", "name",
"ele", "100"
))));
}
@Test
void testMountainLinestring() {
assertFeatures(14, List.of(Map.of(
"class", "ridge",
"name", "Ridge",
"_layer", "mountain_peak",
"_type", "line",
"_minzoom", 13,
"_maxzoom", 14,
"_buffer", 100d
)), process(lineFeature(Map.of(
"natural", "ridge",
"name", "Ridge"
))));
}
@Test
void testCustomaryFt() {
process(SimpleFeature.create(
rectangle(0, 0.1),
Map.of("iso_a2", "US"),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_0_countries",
0
));
// inside US - customary_ft=1
assertFeatures(14, List.of(Map.of(
"class", "volcano",
"customary_ft", 1,
"ele", 100,
"ele_ft", 328
)), process(SimpleFeature.create(
newPoint(0, 0),
new HashMap<>(Map.<String, Object>of(
"natural", "volcano",
"ele", "100"
)),
OSM_SOURCE,
null,
0
)));
// outside US - customary_ft omitted
assertFeatures(14, List.of(Map.of(
"class", "volcano",
"customary_ft", "<null>",
"ele", 100,
"ele_ft", 328
)), process(SimpleFeature.create(
newPoint(1, 1),
new HashMap<>(Map.<String, Object>of(
"natural", "volcano",
"ele", "100"
)),
OSM_SOURCE,
null,
0
)));
}
private int getSortKey(Map<String, Object> tags) {
return process(pointFeature(Map.of(
"natural", "peak",
"ele", "100"
))).iterator().next().getSortKey();
}
@Test
void testSortKey() {
assertAscending(
getSortKey(Map.of(
"natural", "peak",
"name", "name",
"wikipedia", "wikilink",
"ele", "100"
)),
getSortKey(Map.of(
"natural", "peak",
"name", "name",
"ele", "100"
)),
getSortKey(Map.of(
"natural", "peak",
"ele", "100"
))
);
}
@Test
void testMountainPeakPostProcessing() throws GeometryException {
assertEquals(List.of(), profile.postProcessLayerFeatures(MountainPeak.LAYER_NAME, 13, List.of()));
assertEquals(List.of(pointFeature(
MountainPeak.LAYER_NAME,
Map.of("rank", 1),
1
)), profile.postProcessLayerFeatures(MountainPeak.LAYER_NAME, 13, List.of(pointFeature(
MountainPeak.LAYER_NAME,
Map.of(),
1
))));
assertEquals(List.of(
pointFeature(
MountainPeak.LAYER_NAME,
Map.of("rank", 1, "name", "a"),
1
), pointFeature(
MountainPeak.LAYER_NAME,
Map.of("rank", 2, "name", "b"),
1
), pointFeature(
MountainPeak.LAYER_NAME,
Map.of("rank", 1, "name", "c"),
2
)
), profile.postProcessLayerFeatures(MountainPeak.LAYER_NAME, 13, List.of(
pointFeature(
MountainPeak.LAYER_NAME,
Map.of("name", "a"),
1
),
pointFeature(
MountainPeak.LAYER_NAME,
Map.of("name", "b"),
1
),
pointFeature(
MountainPeak.LAYER_NAME,
Map.of("name", "c"),
2
)
)));
}
@Test
void testMountainPeakPostProcessingLimitsFeaturesOutsideZoom() throws GeometryException {
assertEquals(Lists.newArrayList(
new VectorTile.Feature(
MountainPeak.LAYER_NAME,
1,
VectorTile.encodeGeometry(newPoint(-64, -64)),
Map.of("rank", 1),
1
),
null,
new VectorTile.Feature(
MountainPeak.LAYER_NAME,
3,
VectorTile.encodeGeometry(newPoint(256 + 64, 256 + 64)),
Map.of("rank", 1),
2
),
null
), profile.postProcessLayerFeatures(MountainPeak.LAYER_NAME, 13, Lists.newArrayList(
new VectorTile.Feature(
MountainPeak.LAYER_NAME,
1,
VectorTile.encodeGeometry(newPoint(-64, -64)),
new HashMap<>(),
1
),
new VectorTile.Feature(
MountainPeak.LAYER_NAME,
2,
VectorTile.encodeGeometry(newPoint(-65, -65)),
new HashMap<>(),
1
),
new VectorTile.Feature(
MountainPeak.LAYER_NAME,
3,
VectorTile.encodeGeometry(newPoint(256 + 64, 256 + 64)),
new HashMap<>(),
2
),
new VectorTile.Feature(
MountainPeak.LAYER_NAME,
4,
VectorTile.encodeGeometry(newPoint(256 + 65, 256 + 65)),
new HashMap<>(),
2
)
)));
}
}

135
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/ParkTest.java
Wyświetl plik

@ -1,135 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import com.onthegomap.planetiler.geo.GeoUtils;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class ParkTest extends AbstractLayerTest {
@Test
void testNationalPark() {
assertFeatures(13, List.of(Map.of(
"_layer", "park",
"_type", "polygon",
"class", "national_park",
"name", "Grand Canyon National Park",
"_minpixelsize", 2d,
"_minzoom", 4,
"_maxzoom", 14
), Map.of(
"_layer", "park",
"_type", "point",
"class", "national_park",
"name", "Grand Canyon National Park",
"name_int", "Grand Canyon National Park",
"name:latin", "Grand Canyon National Park",
// "name:es", "es name", // don't include all translations
"_minzoom", 5,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"boundary", "national_park",
"name", "Grand Canyon National Park",
"name:es", "es name",
"protection_title", "National Park",
"wikipedia", "en:Grand Canyon National Park"
))));
// needs a name
assertFeatures(13, List.of(Map.of(
"_layer", "park",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"boundary", "national_park",
"protection_title", "National Park"
))));
}
@Test
void testSmallerPark() {
double z11area = Math.pow((GeoUtils.metersToPixelAtEquator(0, Math.sqrt(70_000)) / 256d), 2) * Math.pow(2, 20 - 11);
assertFeatures(13, List.of(Map.of(
"_layer", "park",
"_type", "polygon",
"class", "protected_area",
"name", "Small park",
"_minpixelsize", 2d,
"_minzoom", 4,
"_maxzoom", 14
), Map.of(
"_layer", "park",
"_type", "point",
"class", "protected_area",
"name", "Small park",
"name_int", "Small park",
"_minzoom", 11,
"_maxzoom", 14
)), process(polygonFeatureWithArea(z11area, Map.of(
"boundary", "protected_area",
"name", "Small park",
"wikipedia", "en:Small park"
))));
assertFeatures(13, List.of(Map.of(
"_layer", "park",
"_type", "polygon"
), Map.of(
"_layer", "park",
"_type", "point",
"_minzoom", 5,
"_maxzoom", 14
)), process(polygonFeatureWithArea(1, Map.of(
"boundary", "protected_area",
"name", "Small park",
"wikidata", "Q123"
))));
}
@Test
void testSortKeys() {
assertAscending(
getLabelSortKey(1, Map.of(
"boundary", "national_park",
"name", "a",
"wikipedia", "en:park"
)),
getLabelSortKey(1e-10, Map.of(
"boundary", "national_park",
"name", "a",
"wikipedia", "en:Park"
)),
getLabelSortKey(1, Map.of(
"boundary", "national_park",
"name", "a"
)),
getLabelSortKey(1e-10, Map.of(
"boundary", "national_park",
"name", "a"
)),
getLabelSortKey(1, Map.of(
"boundary", "protected_area",
"name", "a",
"wikipedia", "en:park"
)),
getLabelSortKey(1e-10, Map.of(
"boundary", "protected_area",
"name", "a",
"wikipedia", "en:Park"
)),
getLabelSortKey(1, Map.of(
"boundary", "protected_area",
"name", "a"
)),
getLabelSortKey(1e-10, Map.of(
"boundary", "protected_area",
"name", "a"
))
);
}
private int getLabelSortKey(double area, Map<String, Object> tags) {
var iter = process(polygonFeatureWithArea(area, tags)).iterator();
iter.next();
return iter.next().getSortKey();
}
}

523
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/PlaceTest.java
Wyświetl plik

@ -1,523 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.newPoint;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import static com.onthegomap.planetiler.basemap.layers.Place.getSortKey;
import static com.onthegomap.planetiler.collection.FeatureGroup.SORT_KEY_MAX;
import static com.onthegomap.planetiler.collection.FeatureGroup.SORT_KEY_MIN;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class PlaceTest extends AbstractLayerTest {
@Test
void testContinent() {
wikidataTranslations.put(49, "es", "América del Norte y América Central");
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "continent",
"name", "North America",
"name:en", "North America",
"name:es", "América del Norte y América Central",
"name:latin", "North America",
"rank", 1,
"_type", "point",
"_minzoom", 0,
"_maxzoom", 3
)), process(pointFeature(Map.of(
"place", "continent",
"wikidata", "Q49",
"name:es", "América del Norte",
"name", "North America",
"name:en", "North America"
))));
}
@Test
void testCountry() {
wikidataTranslations.put(30, "es", "Estados Unidos");
process(SimpleFeature.create(
rectangle(0, 0.25),
Map.of(
"name", "United States",
"scalerank", 0,
"labelrank", 2
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_0_countries",
0
));
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "country",
"name", "United States of America",
"name_en", "United States of America",
"name:es", "Estados Unidos",
"name:latin", "United States of America",
"iso_a2", "US",
"rank", 6,
"_type", "point",
"_minzoom", 5
)), process(SimpleFeature.create(
newPoint(0.5, 0.5),
Map.of(
"place", "country",
"wikidata", "Q30",
"name:es", "Estados Unidos de América",
"name", "United States of America",
"name:en", "United States of America",
"country_code_iso3166_1_alpha_2", "US"
),
OSM_SOURCE,
null,
0
)));
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "country",
"name", "United States of America",
"name_en", "United States of America",
"name:es", "Estados Unidos",
"name:latin", "United States of America",
"iso_a2", "US",
"rank", 1,
"_type", "point",
"_minzoom", 0
)), process(SimpleFeature.create(
newPoint(0.1, 0.1),
Map.of(
"place", "country",
"wikidata", "Q30",
"name:es", "Estados Unidos de América",
"name", "United States of America",
"name:en", "United States of America",
"country_code_iso3166_1_alpha_2", "US"
),
OSM_SOURCE,
null,
0
)));
}
@Test
void testState() {
wikidataTranslations.put(771, "es", "Massachusetts es");
process(SimpleFeature.create(
rectangle(0, 0.25),
Map.of(
"name", "Massachusetts",
"scalerank", 0,
"labelrank", 2,
"datarank", 1
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_1_states_provinces",
0
));
process(SimpleFeature.create(
rectangle(0.4, 0.6),
Map.of(
"name", "Massachusetts - not important",
"scalerank", 8,
"labelrank", 8,
"datarank", 1
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_1_states_provinces",
0
));
// no match
assertFeatures(0, List.of(), process(SimpleFeature.create(
newPoint(1, 1),
Map.of(
"place", "state",
"wikidata", "Q771",
"name", "Massachusetts",
"name:en", "Massachusetts"
),
OSM_SOURCE,
null,
0
)));
// unimportant match
assertFeatures(0, List.of(), process(SimpleFeature.create(
newPoint(0.5, 0.5),
Map.of(
"place", "state",
"wikidata", "Q771",
"name", "Massachusetts",
"name:en", "Massachusetts"
),
OSM_SOURCE,
null,
0
)));
// important match
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "state",
"name", "Massachusetts",
"name_en", "Massachusetts",
"name:es", "Massachusetts es",
"name:latin", "Massachusetts",
"rank", 1,
"_type", "point",
"_minzoom", 2
)), process(SimpleFeature.create(
newPoint(0.1, 0.1),
Map.of(
"place", "state",
"wikidata", "Q771",
"name", "Massachusetts",
"name:en", "Massachusetts"
),
OSM_SOURCE,
null,
0
)));
}
@Test
void testProvince() {
wikidataTranslations.put(95027, "es", "provincia de Lugo");
process(SimpleFeature.create(
rectangle(0, 0.25),
Map.of(
"name", "Nova Scotia",
"scalerank", 3,
"labelrank", 3,
"datarank", 3
),
NATURAL_EARTH_SOURCE,
"ne_10m_admin_1_states_provinces",
0
));
assertFeatures(4, List.of(Map.of(
"_layer", "place",
"class", "province",
"name", "Lugo",
"name:es", "provincia de Lugo",
"rank", 3,
"_type", "point",
"_minzoom", 2
)), process(SimpleFeature.create(
newPoint(0.1, 0.1),
Map.of(
"place", "province",
"wikidata", "Q95027",
"name", "Lugo"
),
OSM_SOURCE,
null,
0
)));
}
@Test
void testIslandPoint() {
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "island",
"name", "Nantucket",
"name_en", "Nantucket",
"name:latin", "Nantucket",
"rank", 7,
"_type", "point",
"_minzoom", 12
)), process(pointFeature(
Map.of(
"place", "island",
"name", "Nantucket",
"name:en", "Nantucket"
))));
}
@Test
void testIslandPolygon() {
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "island",
"name", "Nantucket",
"name_en", "Nantucket",
"name:latin", "Nantucket",
"rank", 3,
"_type", "point",
"_minzoom", 8
)), process(polygonFeatureWithArea(1,
Map.of(
"place", "island",
"name", "Nantucket",
"name:en", "Nantucket"
))));
double rank4area = Math.pow(GeoUtils.metersToPixelAtEquator(0, Math.sqrt(40_000_000 - 1)) / 256d, 2);
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "island",
"name", "Nantucket",
"rank", 4,
"_type", "point",
"_minzoom", 9
)), process(polygonFeatureWithArea(rank4area,
Map.of(
"place", "island",
"name", "Nantucket",
"name:en", "Nantucket"
))));
}
@Test
void testPlaceSortKeyRanking() {
int[] sortKeys = new int[]{
// max
getSortKey(0, Place.PlaceType.CITY, 1_000_000_000, "name"),
getSortKey(0, Place.PlaceType.CITY, 1_000_000_000, "name longer"),
getSortKey(0, Place.PlaceType.CITY, 1_000_000_000, "x".repeat(32)),
getSortKey(0, Place.PlaceType.CITY, 10_000_000, "name"),
getSortKey(0, Place.PlaceType.CITY, 0, "name"),
getSortKey(0, Place.PlaceType.TOWN, 1_000_000_000, "name"),
getSortKey(0, Place.PlaceType.ISOLATED_DWELLING, 1_000_000_000, "name"),
getSortKey(0, null, 1_000_000_000, "name"),
getSortKey(1, Place.PlaceType.CITY, 1_000_000_000, "name"),
getSortKey(10, Place.PlaceType.CITY, 1_000_000_000, "name"),
getSortKey(null, Place.PlaceType.CITY, 1_000_000_000, "name"),
// min
getSortKey(null, null, 0, null),
};
for (int i = 0; i < sortKeys.length; i++) {
if (sortKeys[i] < SORT_KEY_MIN) {
fail("Item at index " + i + " is < " + SORT_KEY_MIN + ": " + sortKeys[i]);
}
if (sortKeys[i] > SORT_KEY_MAX) {
fail("Item at index " + i + " is > " + SORT_KEY_MAX + ": " + sortKeys[i]);
}
}
assertAscending(sortKeys);
}
@Test
void testCountryCapital() {
process(SimpleFeature.create(
newPoint(0, 0),
Map.of(
"name", "Washington, D.C.",
"scalerank", 0,
"wikidataid", "Q61"
),
NATURAL_EARTH_SOURCE,
"ne_10m_populated_places",
0
));
assertFeatures(7, List.of(Map.of(
"_layer", "place",
"class", "city",
"name", "Washington, D.C.",
"rank", 1,
"capital", 2,
"_labelgrid_limit", 0,
"_labelgrid_size", 128d,
"_type", "point",
"_minzoom", 2
)), process(pointFeature(
Map.of(
"place", "city",
"name", "Washington, D.C.",
"population", "672228",
"wikidata", "Q61",
"capital", "yes"
))));
}
@Test
void testStateCapital() {
process(SimpleFeature.create(
newPoint(0, 0),
Map.of(
"name", "Boston",
"scalerank", 2,
"wikidataid", "Q100"
),
NATURAL_EARTH_SOURCE,
"ne_10m_populated_places",
0
));
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "city",
"name", "Boston",
"rank", 3,
"capital", 4,
"_type", "point",
"_minzoom", 3
)), process(pointFeature(
Map.of(
"place", "city",
"name", "Boston",
"population", "667137",
"capital", "4"
))));
// no match when far away
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "city",
"name", "Boston",
"rank", "<null>"
)), process(SimpleFeature.create(
newPoint(1, 1),
Map.of(
"place", "city",
"name", "Boston",
"wikidata", "Q100",
"population", "667137",
"capital", "4"
),
OSM_SOURCE,
null,
0
)));
// unaccented name match
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "city",
"rank", 3
)), process(pointFeature(
Map.of(
"place", "city",
"name", "Böston",
"population", "667137",
"capital", "4"
))));
// wikidata only match
assertFeatures(0, List.of(Map.of(
"_layer", "place",
"class", "city",
"rank", 3
)), process(pointFeature(
Map.of(
"place", "city",
"name", "Other name",
"population", "667137",
"wikidata", "Q100",
"capital", "4"
))));
}
@Test
void testCityWithoutNaturalEarthMatch() {
assertFeatures(7, List.of(Map.of(
"_layer", "place",
"class", "city",
"rank", "<null>",
"_minzoom", 7,
"_labelgrid_limit", 4,
"_labelgrid_size", 128d
)), process(pointFeature(
Map.of(
"place", "city",
"name", "City name"
))));
assertFeatures(13, List.of(Map.of(
"_layer", "place",
"class", "isolated_dwelling",
"rank", "<null>",
"_labelgrid_limit", 0,
"_labelgrid_size", 0d,
"_minzoom", 14
)), process(pointFeature(
Map.of(
"place", "isolated_dwelling",
"name", "City name"
))));
assertFeatures(12, List.of(Map.of(
"_layer", "place",
"class", "isolated_dwelling",
"rank", "<null>",
"_labelgrid_limit", 14,
"_labelgrid_size", 128d,
"_minzoom", 14
)), process(pointFeature(
Map.of(
"place", "isolated_dwelling",
"name", "City name"
))));
}
@Test
void testCitySetRankFromGridrank() throws GeometryException {
var layerName = Place.LAYER_NAME;
assertEquals(List.of(), profile.postProcessLayerFeatures(layerName, 13, List.of()));
assertEquals(List.of(pointFeature(
layerName,
Map.of("rank", 11),
1
)), profile.postProcessLayerFeatures(layerName, 13, List.of(pointFeature(
layerName,
Map.of(),
1
))));
assertEquals(List.of(
pointFeature(
layerName,
Map.of("rank", 11, "name", "a"),
1
), pointFeature(
layerName,
Map.of("rank", 12, "name", "b"),
1
), pointFeature(
layerName,
Map.of("rank", 11, "name", "c"),
2
)
), profile.postProcessLayerFeatures(layerName, 13, List.of(
pointFeature(
layerName,
Map.of("name", "a"),
1
),
pointFeature(
layerName,
Map.of("name", "b"),
1
),
pointFeature(
layerName,
Map.of("name", "c"),
2
)
)));
}
}

240
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/PoiTest.java
Wyświetl plik

@ -1,240 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static org.junit.jupiter.api.Assertions.assertEquals;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SourceFeature;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
class PoiTest extends AbstractLayerTest {
private SourceFeature feature(boolean area, Map<String, Object> tags) {
return area ? polygonFeature(tags) : pointFeature(tags);
}
@Test
void testFenwayPark() {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "stadium",
"subclass", "stadium",
"name", "Fenway Park",
"rank", "<null>",
"_minzoom", 14,
"_labelgrid_size", 64d
)), process(pointFeature(Map.of(
"leisure", "stadium",
"name", "Fenway Park"
))));
}
@ParameterizedTest
@ValueSource(booleans = {false, true})
void testFunicularHalt(boolean area) {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "railway",
"subclass", "halt",
"rank", "<null>",
"_minzoom", 12
)), process(feature(area, Map.of(
"railway", "station",
"funicular", "yes",
"name", "station"
))));
}
@ParameterizedTest
@ValueSource(booleans = {false, true})
void testSubway(boolean area) {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "railway",
"subclass", "subway",
"rank", "<null>",
"_minzoom", 12
)), process(feature(area, Map.of(
"railway", "station",
"station", "subway",
"name", "station"
))));
}
@ParameterizedTest
@ValueSource(booleans = {false, true})
void testPlaceOfWorshipFromReligionTag(boolean area) {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "place_of_worship",
"subclass", "religion value",
"rank", "<null>",
"_minzoom", 14
)), process(feature(area, Map.of(
"amenity", "place_of_worship",
"religion", "religion value",
"name", "station"
))));
}
@Test
void testPitchFromSportTag() {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "pitch",
"subclass", "soccer",
"rank", "<null>"
)), process(pointFeature(Map.of(
"leisure", "pitch",
"sport", "soccer",
"name", "station"
))));
}
@ParameterizedTest
@ValueSource(booleans = {false, true})
void testInformation(boolean area) {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "information",
"subclass", "infotype",
"layer", 3L,
"level", 2L,
"indoor", 1,
"rank", "<null>"
)), process(feature(area, Map.of(
"tourism", "information",
"information", "infotype",
"name", "station",
"layer", "3",
"level", "2",
"indoor", "yes"
))));
}
@ParameterizedTest
@ValueSource(booleans = {false, true})
void testFerryTerminal(boolean area) {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "ferry_terminal",
"subclass", "ferry_terminal",
"name", "Water Taxi",
"_minzoom", 12
)), process(feature(area, Map.of(
"amenity", "ferry_terminal",
"information", "infotype",
"name", "Water Taxi",
"layer", "3",
"level", "2",
"indoor", "yes"
))));
}
@Test
void testGridRank() throws GeometryException {
var layerName = Poi.LAYER_NAME;
assertEquals(List.of(), profile.postProcessLayerFeatures(layerName, 13, List.of()));
assertEquals(List.of(pointFeature(
layerName,
Map.of("rank", 1),
1
)), profile.postProcessLayerFeatures(layerName, 14, List.of(pointFeature(
layerName,
Map.of(),
1
))));
assertEquals(List.of(
pointFeature(
layerName,
Map.of("rank", 1, "name", "a"),
1
), pointFeature(
layerName,
Map.of("rank", 2, "name", "b"),
1
), pointFeature(
layerName,
Map.of("rank", 1, "name", "c"),
2
)
), profile.postProcessLayerFeatures(layerName, 14, List.of(
pointFeature(
layerName,
Map.of("name", "a"),
1
),
pointFeature(
layerName,
Map.of("name", "b"),
1
),
pointFeature(
layerName,
Map.of("name", "c"),
2
)
)));
}
@Test
void testEmbassy() {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "diplomatic",
"subclass", "diplomatic",
"name", "The Embassy"
)), process(pointFeature(Map.of(
"office", "diplomatic",
"name", "The Embassy"
))));
}
@Test
void testLocksmith() {
assertFeatures(7, List.of(Map.of(
"_layer", "poi",
"class", "shop",
"subclass", "locksmith",
"name", "The Locksmith"
)), process(pointFeature(Map.of(
"shop", "locksmith",
"name", "The Locksmith"
))));
}
@Test
void testAtm() {
List<Map<String, Object>> expected = List.of(Map.of(
"_layer", "poi",
"class", "atm",
"subclass", "atm",
"name", "ATM name"
));
// prefer name, otherwise fall back to operator, or else network
assertFeatures(14, expected, process(pointFeature(Map.of(
"amenity", "atm",
"name", "ATM name"
))));
assertFeatures(14, expected, process(pointFeature(Map.of(
"amenity", "atm",
"name", "ATM name",
"operator", "ATM operator",
"network", "ATM network"
))));
assertFeatures(14, expected, process(pointFeature(Map.of(
"amenity", "atm",
"operator", "ATM name",
"network", "ATM network"
))));
assertFeatures(14, expected, process(pointFeature(Map.of(
"amenity", "atm",
"network", "ATM name"
))));
}
}

1249
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/TransportationTest.java
Wyświetl plik

Plik diff jest za duży Load Diff

206
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/WaterNameTest.java
Wyświetl plik

@ -1,206 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.newLineString;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.LAKE_CENTERLINE_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import com.onthegomap.planetiler.TestUtils;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.reader.SimpleFeature;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import org.locationtech.jts.geom.Geometry;
class WaterNameTest extends AbstractLayerTest {
@Test
void testWaterNamePoint() {
assertFeatures(11, List.of(Map.of(
"_layer", "water"
), Map.of(
"class", "lake",
"name", "waterway",
"name:es", "waterway es",
"intermittent", 1,
"_layer", "water_name",
"_type", "point",
"_minzoom", 9,
"_maxzoom", 14
)), process(polygonFeatureWithArea(1, Map.of(
"name", "waterway",
"name:es", "waterway es",
"natural", "water",
"water", "pond",
"intermittent", "1"
))));
double z11area = Math.pow((GeoUtils.metersToPixelAtEquator(0, Math.sqrt(70_000)) / 256d), 2) * Math.pow(2, 20 - 11);
assertFeatures(10, List.of(Map.of(
"_layer", "water"
), Map.of(
"_layer", "water_name",
"_type", "point",
"_minzoom", 11,
"_maxzoom", 14
)), process(polygonFeatureWithArea(z11area, Map.of(
"name", "waterway",
"natural", "water",
"water", "pond"
))));
}
@Test
void testWaterNameLakeline() {
assertFeatures(11, List.of(), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
new HashMap<>(Map.<String, Object>of(
"OSM_ID", -10
)),
LAKE_CENTERLINE_SOURCE,
null,
0
)));
assertFeatures(10, List.of(Map.of(
"_layer", "water"
), Map.of(
"name", "waterway",
"name:es", "waterway es",
"_layer", "water_name",
"_type", "line",
"_geom", new TestUtils.NormGeometry(GeoUtils.latLonToWorldCoords(newLineString(0, 0, 1, 1))),
"_minzoom", 9,
"_maxzoom", 14,
"_minpixelsize", "waterway".length() * 6d
)), process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
new HashMap<>(Map.<String, Object>of(
"name", "waterway",
"name:es", "waterway es",
"natural", "water",
"water", "pond"
)),
OSM_SOURCE,
null,
10
)));
}
@Test
void testWaterNameMultipleLakelines() {
assertFeatures(11, List.of(), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
new HashMap<>(Map.<String, Object>of(
"OSM_ID", -10
)),
LAKE_CENTERLINE_SOURCE,
null,
0
)));
assertFeatures(11, List.of(), process(SimpleFeature.create(
newLineString(2, 2, 3, 3),
new HashMap<>(Map.<String, Object>of(
"OSM_ID", -10
)),
LAKE_CENTERLINE_SOURCE,
null,
0
)));
assertFeatures(10, List.of(Map.of(
"_layer", "water"
), Map.of(
"name", "waterway",
"name:es", "waterway es",
"_layer", "water_name",
"_geom",
new TestUtils.NormGeometry(
GeoUtils.latLonToWorldCoords(GeoUtils.JTS_FACTORY.createGeometryCollection(new Geometry[]{
newLineString(0, 0, 1, 1),
newLineString(2, 2, 3, 3)
}))),
"_minzoom", 9,
"_maxzoom", 14,
"_minpixelsize", "waterway".length() * 6d
)), process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
new HashMap<>(Map.<String, Object>of(
"name", "waterway",
"name:es", "waterway es",
"natural", "water",
"water", "pond"
)),
OSM_SOURCE,
null,
10
)));
}
@Test
void testMarinePoint() {
assertFeatures(11, List.of(), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
new HashMap<>(Map.<String, Object>of(
"scalerank", 1,
"name", "Black sea"
)),
NATURAL_EARTH_SOURCE,
"ne_10m_geography_marine_polys",
0
)));
// name match - use scale rank from NE
assertFeatures(10, List.of(Map.of(
"name", "Black Sea",
"name:es", "Mar Negro",
"_layer", "water_name",
"_type", "point",
"_minzoom", 1,
"_maxzoom", 14
)), process(pointFeature(Map.of(
"rank", 9,
"name", "Black Sea",
"name:es", "Mar Negro",
"place", "sea"
))));
// name match but ocean - use min zoom=0
assertFeatures(10, List.of(Map.of(
"_layer", "water_name",
"_type", "point",
"_minzoom", 0,
"_maxzoom", 14
)), process(pointFeature(Map.of(
"rank", 9,
"name", "Black Sea",
"place", "ocean"
))));
// no name match - use OSM rank
assertFeatures(10, List.of(Map.of(
"_layer", "water_name",
"_type", "point",
"_minzoom", 9,
"_maxzoom", 14
)), process(pointFeature(Map.of(
"rank", 9,
"name", "Atlantic",
"place", "sea"
))));
// no rank at all, default to 8
assertFeatures(10, List.of(Map.of(
"_layer", "water_name",
"_type", "point",
"_minzoom", 8,
"_maxzoom", 14
)), process(pointFeature(Map.of(
"name", "Atlantic",
"place", "sea"
))));
}
}

282
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/WaterTest.java
Wyświetl plik

@ -1,282 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.rectangle;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.WATER_POLYGON_SOURCE;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.reader.SimpleFeature;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class WaterTest extends AbstractLayerTest {
@Test
void testWaterNaturalEarth() {
assertFeatures(0, List.of(Map.of(
"class", "lake",
"intermittent", "<null>",
"_layer", "water",
"_type", "polygon",
"_minzoom", 0
)), process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_110m_lakes",
0
)));
assertFeatures(0, List.of(Map.of(
"class", "ocean",
"intermittent", "<null>",
"_layer", "water",
"_type", "polygon",
"_minzoom", 0
)), process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_110m_ocean",
0
)));
assertFeatures(6, List.of(Map.of(
"class", "lake",
"_layer", "water",
"_type", "polygon",
"_maxzoom", 5
)), process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_lakes",
0
)));
assertFeatures(6, List.of(Map.of(
"class", "ocean",
"_layer", "water",
"_type", "polygon",
"_maxzoom", 5
)), process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_ocean",
0
)));
}
@Test
void testWaterOsmWaterPolygon() {
assertFeatures(0, List.of(Map.of(
"class", "ocean",
"intermittent", "<null>",
"_layer", "water",
"_type", "polygon",
"_minzoom", 6,
"_maxzoom", 14
)), process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
WATER_POLYGON_SOURCE,
null,
0
)));
}
@Test
void testWaterOsmId() {
long id = 123;
assertFeatures(14, List.of(Map.of(
"class", "lake",
"id", id,
"_layer", "water",
"_type", "polygon",
"_minzoom", 6,
"_maxzoom", 14
)), process(SimpleFeature.create(
GeoUtils.worldToLatLonCoords(rectangle(0, Math.sqrt(1))),
new HashMap<>(Map.<String, Object>of(
"natural", "water",
"water", "reservoir"
)),
OSM_SOURCE,
null,
id
)));
}
@Test
void testWater() {
assertFeatures(14, List.of(Map.of(
"class", "lake",
"_layer", "water",
"_type", "polygon",
"_minzoom", 6,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"natural", "water",
"water", "reservoir"
))));
assertFeatures(14, List.of(
Map.of("_layer", "poi"),
Map.of(
"class", "swimming_pool",
"_layer", "water",
"_type", "polygon",
"_minzoom", 6,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"leisure", "swimming_pool"
))));
assertFeatures(14, List.of(), process(polygonFeature(Map.of(
"natural", "bay"
))));
assertFeatures(14, List.of(Map.of()), process(polygonFeature(Map.of(
"natural", "water"
))));
assertFeatures(14, List.of(), process(polygonFeature(Map.of(
"natural", "water",
"covered", "yes"
))));
assertFeatures(14, List.of(Map.of(
"class", "river",
"brunnel", "bridge",
"intermittent", 1,
"_layer", "water",
"_type", "polygon",
"_minzoom", 6,
"_maxzoom", 14
)), process(polygonFeature(Map.of(
"waterway", "riverbank",
"bridge", "1",
"intermittent", "1"
))));
assertFeatures(11, List.of(Map.of(
"class", "lake",
"brunnel", "<null>",
"intermittent", 0,
"_layer", "water",
"_type", "polygon",
"_minzoom", 6,
"_maxzoom", 14,
"_minpixelsize", 2d
)), process(polygonFeature(Map.of(
"landuse", "salt_pond",
"bridge", "1"
))));
}
@Test
void testRiverbank() {
assertFeatures(11, List.of(Map.of(
"class", "river",
"_layer", "water",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"waterway", "riverbank"
))));
}
@Test
void testRiver() {
assertFeatures(11, List.of(Map.of(
"class", "river",
"_layer", "water",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"water", "river"
))));
}
@Test
void testSpring() {
assertFeatures(11, List.of(Map.of(
"class", "lake",
"_layer", "water",
"_type", "polygon"
)), process(polygonFeature(Map.of(
"natural", "spring"
))));
}
@Test
void testOceanZoomLevels() {
assertCoversZoomRange(0, 14, "water",
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_110m_ocean",
0
)),
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_50m_ocean",
0
)),
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_ocean",
0
)),
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
WATER_POLYGON_SOURCE,
null,
0
))
);
}
@Test
void testLakeZoomLevels() {
assertCoversZoomRange(0, 14, "water",
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_110m_lakes",
0
)),
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_50m_lakes",
0
)),
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(),
NATURAL_EARTH_SOURCE,
"ne_10m_lakes",
0
)),
process(SimpleFeature.create(
rectangle(0, 10),
Map.of(
"natural", "water",
"water", "reservoir"
),
OSM_SOURCE,
null,
0
))
);
}
}

236
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/layers/WaterwayTest.java
Wyświetl plik

@ -1,236 +0,0 @@
package com.onthegomap.planetiler.basemap.layers;
import static com.onthegomap.planetiler.TestUtils.newLineString;
import static com.onthegomap.planetiler.basemap.BasemapProfile.NATURAL_EARTH_SOURCE;
import static com.onthegomap.planetiler.basemap.BasemapProfile.OSM_SOURCE;
import static org.junit.jupiter.api.Assertions.assertEquals;
import com.onthegomap.planetiler.FeatureCollector;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmReader;
import com.onthegomap.planetiler.reader.osm.OsmRelationInfo;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
class WaterwayTest extends AbstractLayerTest {
@ParameterizedTest
@ValueSource(booleans = {false, true})
void testOsmWaterwayRelation(boolean isLongEnough) throws GeometryException {
var rel = new OsmElement.Relation(1);
rel.setTag("name", "River Relation");
rel.setTag("name:es", "ES name");
rel.setTag("waterway", "river");
List<OsmRelationInfo> relationInfos = profile.preprocessOsmRelation(rel);
FeatureCollector features = process(SimpleFeature.createFakeOsmFeature(
newLineString(0, 0, 0, isLongEnough ? 3 : 1),
Map.of(),
OSM_SOURCE,
null,
0,
(relationInfos == null ? List.<OsmRelationInfo>of() : relationInfos).stream()
.map(r -> new OsmReader.RelationMember<>("", r)).toList()
));
assertFeatures(14, List.of(Map.of(
"class", "river",
"name", "River Relation",
"name:es", "ES name",
"_relid", 1L,
"_layer", "waterway",
"_type", "line",
"_minzoom", 6,
"_maxzoom", 8,
"_buffer", 4d
)), features);
// ensure that post-processing combines waterways, and filters out ones that
// belong to rivers that are not long enough to be shown
var line1 = new VectorTile.Feature(
Waterway.LAYER_NAME,
1,
VectorTile.encodeGeometry(newLineString(0, 0, 10, 0)),
mapOf("name", "river", "_relid", 1L),
0
);
var line2 = new VectorTile.Feature(
Waterway.LAYER_NAME,
1,
VectorTile.encodeGeometry(newLineString(10, 0, 20, 0)),
mapOf("name", "river", "_relid", 1L),
0
);
var connected = new VectorTile.Feature(
Waterway.LAYER_NAME,
1,
VectorTile.encodeGeometry(newLineString(0, 0, 20, 0)),
mapOf("name", "river"),
0
);
assertEquals(
isLongEnough ? List.of(connected) : List.of(),
profile.postProcessLayerFeatures(Waterway.LAYER_NAME, 8, new ArrayList<>(List.of(line1, line2)))
);
}
@Test
void testWaterwayImportantRiverProcess() {
var charlesRiver = process(lineFeature(Map.of(
"waterway", "river",
"name", "charles river",
"name:es", "es name"
)));
assertFeatures(14, List.of(Map.of(
"class", "river",
"name", "charles river",
"name:es", "es name",
"intermittent", 0,
"_layer", "waterway",
"_type", "line",
"_minzoom", 9,
"_maxzoom", 14,
"_buffer", 4d
)), charlesRiver);
assertFeatures(11, List.of(Map.of(
"class", "river",
"name", "charles river",
"name:es", "es name",
"intermittent", "<null>",
"_buffer", 13.082664546679323
)), charlesRiver);
assertFeatures(10, List.of(Map.of(
"class", "river",
"_buffer", 26.165329093358647
)), charlesRiver);
assertFeatures(9, List.of(Map.of(
"class", "river",
"_buffer", 26.165329093358647
)), charlesRiver);
}
@Test
void testWaterwayImportantRiverPostProcess() throws GeometryException {
var line1 = new VectorTile.Feature(
Waterway.LAYER_NAME,
1,
VectorTile.encodeGeometry(newLineString(0, 0, 10, 0)),
Map.of("name", "river"),
0
);
var line2 = new VectorTile.Feature(
Waterway.LAYER_NAME,
1,
VectorTile.encodeGeometry(newLineString(10, 0, 20, 0)),
Map.of("name", "river"),
0
);
var connected = new VectorTile.Feature(
Waterway.LAYER_NAME,
1,
VectorTile.encodeGeometry(newLineString(0, 0, 20, 0)),
Map.of("name", "river"),
0
);
assertEquals(
List.of(),
profile.postProcessLayerFeatures(Waterway.LAYER_NAME, 11, List.of())
);
assertEquals(
List.of(line1, line2),
profile.postProcessLayerFeatures(Waterway.LAYER_NAME, 12, List.of(line1, line2))
);
assertEquals(
List.of(connected),
profile.postProcessLayerFeatures(Waterway.LAYER_NAME, 11, List.of(line1, line2))
);
}
@Test
void testWaterwaySmaller() {
// river with no name is not important
assertFeatures(14, List.of(Map.of(
"class", "river",
"brunnel", "bridge",
"_layer", "waterway",
"_type", "line",
"_minzoom", 12
)), process(lineFeature(Map.of(
"waterway", "river",
"bridge", "1"
))));
assertFeatures(14, List.of(Map.of(
"class", "canal",
"_layer", "waterway",
"_type", "line",
"_minzoom", 12
)), process(lineFeature(Map.of(
"waterway", "canal",
"name", "name"
))));
assertFeatures(14, List.of(Map.of(
"class", "stream",
"_layer", "waterway",
"_type", "line",
"_minzoom", 13
)), process(lineFeature(Map.of(
"waterway", "stream",
"name", "name"
))));
}
@Test
void testWaterwayNaturalEarth() {
assertFeatures(3, List.of(Map.of(
"class", "river",
"name", "<null>",
"intermittent", "<null>",
"_layer", "waterway",
"_type", "line",
"_minzoom", 3,
"_maxzoom", 3
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"featurecla", "River",
"name", "name"
),
NATURAL_EARTH_SOURCE,
"ne_110m_rivers_lake_centerlines",
0
)));
assertFeatures(6, List.of(Map.of(
"class", "river",
"intermittent", "<null>",
"_layer", "waterway",
"_type", "line",
"_minzoom", 4,
"_maxzoom", 5
)), process(SimpleFeature.create(
newLineString(0, 0, 1, 1),
Map.of(
"featurecla", "River",
"name", "name"
),
NATURAL_EARTH_SOURCE,
"ne_50m_rivers_lake_centerlines",
0
)));
}
}

265
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/util/LanguageUtilsTest.java
Wyświetl plik

@ -1,265 +0,0 @@
package com.onthegomap.planetiler.basemap.util;
import static com.onthegomap.planetiler.TestUtils.assertSubmap;
import static com.onthegomap.planetiler.basemap.util.LanguageUtils.containsOnlyLatinCharacters;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import com.onthegomap.planetiler.util.Translations;
import com.onthegomap.planetiler.util.Wikidata;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.junit.jupiter.params.provider.ValueSource;
class LanguageUtilsTest {
private final Wikidata.WikidataTranslations wikidataTranslations = new Wikidata.WikidataTranslations();
private final Translations translations = Translations.defaultProvider(List.of("en", "es", "de"))
.addTranslationProvider(wikidataTranslations);
@Test
void testSimpleExample() {
assertSubmap(Map.of(
"name", "name",
"name_en", "english name",
"name_de", "german name"
), LanguageUtils.getNames(Map.of(
"name", "name",
"name:en", "english name",
"name:de", "german name"
), translations));
assertSubmap(Map.of(
"name", "name",
"name_en", "name",
"name_de", "german name"
), LanguageUtils.getNames(Map.of(
"name", "name",
"name:de", "german name"
), translations));
assertSubmap(Map.of(
"name", "name",
"name_en", "english name",
"name_de", "name"
), LanguageUtils.getNames(Map.of(
"name", "name",
"name:en", "english name"
), translations));
}
@ParameterizedTest
@CsvSource({
"abc, true",
"5!, true",
"5~, true",
"é, true",
"éś, true",
"ɏə, true",
"ɐ, true",
"ᵿἀ, false",
"Ḁỿ, true",
"\u02ff\u0370, false",
"\u0030\u036f, true",
"日本, false",
"abc本123, false",
})
void testIsLatin(String in, boolean isLatin) {
if (!isLatin) {
assertFalse(containsOnlyLatinCharacters(in));
} else {
assertEquals(in, LanguageUtils.getNames(Map.of(
"name", in
), translations).get("name:latin"));
}
}
@ParameterizedTest
@CsvSource(value = {
"abcaāíìś+, null",
"abca日āíìś+, 日+",
"(abc), null",
"日本 (Japan), 日本",
"日本 [Japan - Nippon], 日本",
" Japan - Nippon (Japan) - Japan - 日本 - Japan - Nippon (Japan), 日本",
"Japan - 日本~+ , 日本~+",
"Japan / 日本 / Japan , 日本",
}, nullValues = "null")
void testRemoveNonLatin(String in, String out) {
assertEquals(out, LanguageUtils.getNames(Map.of(
"name", in
), translations).get("name:nonlatin"));
}
@ParameterizedTest
@ValueSource(strings = {
// OSM tags that SHOULD be eligible for name:latin feature in the output
"name:en",
"name:en-US",
"name:en-010",
"int_name",
"name:fr",
"name:es",
"name:pt",
"name:de",
"name:ar",
"name:it",
"name:ko-Latn",
"name:be-tarask",
// https://wiki.openstreetmap.org/wiki/Multilingual_names#Japan
"name:ja",
"name:ja-Latn",
"name:ja_rm",
"name:ja_kana",
// https://wiki.openstreetmap.org/wiki/Multilingual_names#China
"name:zh-CN",
"name:zh-hant-CN",
"name:zh_pinyin",
"name:zh_zhuyin",
"name:zh-Latn-tongyong",
"name:zh-Latn-pinyin",
"name:zh-Latn-wadegiles",
"name:yue-Latn-jyutping",
// https://wiki.openstreetmap.org/wiki/Multilingual_names#France
"name:fr",
"name:fr-x-gallo",
"name:br",
"name:oc",
"name:vls",
"name:frp",
"name:gcf",
"name:gsw",
})
void testLatinFallbacks(String key) {
assertEquals("a", LanguageUtils.getNames(Map.of(
key, "a"
), translations).get("name:latin"));
assertNull(LanguageUtils.getNames(Map.of(
key, "ア"
), translations).get("name:latin"));
assertNull(LanguageUtils.getNames(Map.of(
key, "غ"
), translations).get("name:latin"));
}
@ParameterizedTest
@ValueSource(strings = {
// OSM tags that should NOT be eligible for name:latin feature in the output
"name:signed",
"name:prefix",
"name:abbreviation",
"name:source",
"name:full",
"name:adjective",
"name:proposed",
"name:pronunciation",
"name:etymology",
"name:etymology:wikidata",
"name:etymology:wikipedia",
"name:etymology:right",
"name:etymology:left",
"name:genitive",
})
void testNoLatinFallback(String key) {
assertSubmap(Map.of(
"name", "Branch Hill–Loveland Road",
"name_en", "Branch Hill–Loveland Road",
"name_de", "Branch Hill–Loveland Road",
"name:latin", "Branch Hill–Loveland Road",
"name_int", "Branch Hill–Loveland Road"
), LanguageUtils.getNames(Map.of(
"name", "Branch Hill–Loveland Road",
key, "Q22133584;Q843993"
), translations));
assertSubmap(Map.of(
"name", "日",
"name_en", "日",
"name_de", "日",
"name:latin", "rì",
"name_int", "rì"
), LanguageUtils.getNames(Map.of(
"name", "日",
key, "other" // don't use this latin string with invalid name keys
), translations));
}
@ParameterizedTest
@CsvSource({
"キャンパス, kyanpasu",
"Αλφαβητικός Κατάλογος, Alphabētikós Katálogos",
"биологическом, biologičeskom",
})
void testTransliterate(String in, String out) {
assertEquals(out, LanguageUtils.getNames(Map.of(
"name", in
), translations).get("name:latin"));
translations.setShouldTransliterate(false);
assertNull(LanguageUtils.getNames(Map.of(
"name", in
), translations).get("name:latin"));
}
@Test
void testUseWikidata() {
wikidataTranslations.put(123, "es", "es name");
assertSubmap(Map.of(
"name:es", "es name"
), LanguageUtils.getNames(Map.of(
"name", "name",
"wikidata", "Q123"
), translations));
}
@Test
void testUseOsm() {
assertSubmap(Map.of(
"name:es", "es name osm"
), LanguageUtils.getNames(Map.of(
"name", "name",
"wikidata", "Q123",
"name:es", "es name osm"
), translations));
}
@Test
void testPreferWikidata() {
wikidataTranslations.put(123, "es", "wd es name");
assertSubmap(Map.of(
"name:es", "wd es name",
"name:de", "de name osm"
), LanguageUtils.getNames(Map.of(
"name", "name",
"wikidata", "Q123",
"name:es", "es name osm",
"name:de", "de name osm"
), translations));
}
@Test
void testDontUseTranslationsWhenNotSpecified() {
var result = LanguageUtils.getNamesWithoutTranslations(Map.of(
"name", "name",
"wikidata", "Q123",
"name:es", "es name osm",
"name:de", "de name osm"
));
assertNull(result.get("name:es"));
assertNull(result.get("name:de"));
assertEquals("name", result.get("name"));
}
@Test
void testIgnoreLanguages() {
wikidataTranslations.put(123, "ja", "ja name wd");
var result = LanguageUtils.getNamesWithoutTranslations(Map.of(
"name", "name",
"wikidata", "Q123",
"name:ja", "ja name osm"
));
assertNull(result.get("name:ja"));
}
}

64
planetiler-basemap/src/test/java/com/onthegomap/planetiler/basemap/util/VerifyMonacoTest.java
Wyświetl plik

@ -1,64 +0,0 @@
package com.onthegomap.planetiler.basemap.util;
import static com.onthegomap.planetiler.geo.GeoUtils.point;
import static com.onthegomap.planetiler.util.Gzip.gzip;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.geo.TileCoord;
import com.onthegomap.planetiler.mbtiles.Mbtiles;
import com.onthegomap.planetiler.mbtiles.TileEncodingResult;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.OptionalLong;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
class VerifyMonacoTest {
private Mbtiles mbtiles;
@BeforeEach
public void setup() {
mbtiles = Mbtiles.newInMemoryDatabase();
}
@AfterEach
public void teardown() throws IOException {
mbtiles.close();
}
@Test
void testEmptyFileInvalid() {
assertInvalid(mbtiles);
}
@Test
void testEmptyTablesInvalid() {
mbtiles.createTablesWithIndexes();
assertInvalid(mbtiles);
}
@Test
void testStilInvalidWithOneTile() throws IOException {
mbtiles.createTablesWithIndexes();
mbtiles.metadata().setName("name");
try (var writer = mbtiles.newBatchedTileWriter()) {
VectorTile tile = new VectorTile();
tile.addLayerFeatures("layer", List.of(new VectorTile.Feature(
"layer",
1,
VectorTile.encodeGeometry(point(0, 0)),
Map.of()
)));
writer.write(new TileEncodingResult(TileCoord.ofXYZ(0, 0, 0), gzip(tile.encode()), OptionalLong.empty()));
}
assertInvalid(mbtiles);
}
private void assertInvalid(Mbtiles mbtiles) {
assertTrue(VerifyMonaco.verify(mbtiles).numErrors() > 0);
}
}

2
planetiler-benchmarks/pom.xml
Wyświetl plik

@ -20,7 +20,7 @@
</dependency>
<dependency>
<groupId>com.onthegomap.planetiler</groupId>
<artifactId>planetiler-basemap</artifactId>
<artifactId>planetiler-openmaptiles</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>

6
planetiler-benchmarks/src/main/java/com/onthegomap/planetiler/benchmarks/BasemapMapping.java → planetiler-benchmarks/src/main/java/com/onthegomap/planetiler/benchmarks/OpenMapTilesMapping.java
Wyświetl plik

@ -1,8 +1,8 @@
package com.onthegomap.planetiler.benchmarks;
import com.onthegomap.planetiler.basemap.BasemapProfile;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.openmaptiles.OpenMapTilesProfile;
import com.onthegomap.planetiler.reader.SourceFeature;
import com.onthegomap.planetiler.reader.osm.OsmElement;
import com.onthegomap.planetiler.reader.osm.OsmInputFile;
@ -20,10 +20,10 @@ import org.locationtech.jts.geom.Geometry;
* Performance tests for {@link MultiExpression}. Times how long a sample of elements from an OSM input file take to
* match.
*/
public class BasemapMapping {
public class OpenMapTilesMapping {
public static void main(String[] args) {
var profile = new BasemapProfile(Translations.nullProvider(List.of()), PlanetilerConfig.defaults(),
var profile = new OpenMapTilesProfile(Translations.nullProvider(List.of()), PlanetilerConfig.defaults(),
Stats.inMemory());
var random = new Random(0);
List<SourceFeature> inputs = new ArrayList<>();

2
planetiler-core/src/main/java/com/onthegomap/planetiler/ForwardingProfile.java
Wyświetl plik

@ -22,7 +22,7 @@ import java.util.function.Consumer;
* <li>{@link FinishHandler} to be notified whenever we finish processing each source</li>
* <li>{@link FeaturePostProcessor} to post-process features in a layer before rendering the output tile</li>
* </ul>
* See {@code BasemapProfile} for a full implementation using this framework.
* See {@code OpenMapTilesProfile} for a full implementation using this framework.
*/
public abstract class ForwardingProfile implements Profile {

98
planetiler-core/src/main/java/com/onthegomap/planetiler/util/LanguageUtils.java 100644
Wyświetl plik

@ -0,0 +1,98 @@
package com.onthegomap.planetiler.util;
import java.util.Map;
import java.util.Set;
import java.util.function.Predicate;
import java.util.regex.Pattern;
import java.util.stream.Stream;
public class LanguageUtils {
// Name tags that should be eligible for finding a latin name.
// See https://wiki.openstreetmap.org/wiki/Multilingual_names
public static final Predicate<String> VALID_NAME_TAGS =
Pattern
.compile("^name:[a-z]{2,3}(-[a-z]{4})?([-_](x-)?[a-z]{2,})?(-([a-z]{2}|[0-9]{3}))?$", Pattern.CASE_INSENSITIVE)
.asMatchPredicate();
// See https://github.com/onthegomap/planetiler/issues/86
// Match strings that only contain latin characters.
private static final Predicate<String> ONLY_LATIN = Pattern
.compile("^[\\P{IsLetter}[\\p{IsLetter}&&\\p{IsLatin}]]+$")
.asMatchPredicate();
// Match only latin letters
private static final Pattern LATIN_LETTER = Pattern.compile("[\\p{IsLetter}&&\\p{IsLatin}]+");
private static final Pattern EMPTY_PARENS = Pattern.compile("(\\([ -.]*\\)|\\[[ -.]*])");
private static final Pattern LEADING_TRAILING_JUNK = Pattern.compile("((^[\\s./-]*)|([\\s./-]*$))");
private static final Pattern WHITESPACE = Pattern.compile("\\s+");
public static final Set<String> EN_DE_NAME_KEYS = Set.of("name:en", "name:de");
private LanguageUtils() {
throw new IllegalStateException("Utility class");
}
public static void putIfNotEmpty(Map<String, Object> dest, String key, Object value) {
if (value != null && !value.equals("")) {
dest.put(key, value);
}
}
public static String nullIfEmpty(String a) {
return (a == null || a.isEmpty()) ? null : a;
}
public static String string(Object obj) {
return nullIfEmpty(obj == null ? null : obj.toString());
}
public static boolean containsOnlyLatinCharacters(String string) {
return string != null && ONLY_LATIN.test(string);
}
public static String transliteratedName(Map<String, Object> tags) {
return Translations.transliterate(string(tags.get("name")));
}
public static String removeLatinCharacters(String name) {
if (name == null) {
return null;
}
var matcher = LATIN_LETTER.matcher(name);
if (matcher.find()) {
String result = matcher.replaceAll("");
// if the name was "<nonlatin text> (<latin description)"
// or "<nonlatin text> - <latin description>"
// then remove any of those extra characters now
result = EMPTY_PARENS.matcher(result).replaceAll("");
result = LEADING_TRAILING_JUNK.matcher(result).replaceAll("");
result = WHITESPACE.matcher(result).replaceAll(" ").trim();
return result.isBlank() ? null : result;
}
return name.trim();
}
public static boolean isValidOsmNameTag(String tag) {
return VALID_NAME_TAGS.test(tag);
}
public static String getLatinName(Map<String, Object> tags, boolean transliterate) {
String name = string(tags.get("name"));
if (containsOnlyLatinCharacters(name)) {
return name;
} else {
return getNameTranslations(tags)
.filter(LanguageUtils::containsOnlyLatinCharacters)
.findFirst()
.orElse(transliterate ? Translations.transliterate(name) : null);
}
}
private static Stream<String> getNameTranslations(Map<String, Object> tags) {
return Stream.concat(
Stream.of("name:en", "int_name", "name:de").map(tag -> string(tags.get(tag))),
tags.entrySet().stream()
.filter(e -> !EN_DE_NAME_KEYS.contains(e.getKey()) && VALID_NAME_TAGS.test(e.getKey()))
.map(Map.Entry::getValue)
.map(LanguageUtils::string)
);
}
}

159
planetiler-core/src/test/java/com/onthegomap/planetiler/util/LanguageUtilsTest.java 100644
Wyświetl plik

@ -0,0 +1,159 @@
package com.onthegomap.planetiler.util;
import static com.onthegomap.planetiler.util.LanguageUtils.containsOnlyLatinCharacters;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.Map;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.junit.jupiter.params.provider.ValueSource;
class LanguageUtilsTest {
@ParameterizedTest
@CsvSource({
"abc, true",
"5!, true",
"5~, true",
"é, true",
"éś, true",
"ɏə, true",
"ɐ, true",
"ᵿἀ, false",
"Ḁỿ, true",
"\u02ff\u0370, false",
"\u0030\u036f, true",
"日本, false",
"abc本123, false",
})
void testIsLatin(String in, boolean isLatin) {
if (!isLatin) {
assertFalse(containsOnlyLatinCharacters(in));
} else {
assertEquals(in, LanguageUtils.getLatinName(Map.of(
"name", in
), true));
}
}
@ParameterizedTest
@CsvSource(value = {
"null,null",
"abcaāíìś+, +",
"abcaāíìś, null",
"日本, 日本",
"abca日āíìś+, 日+",
"(abc), null",
"日本 (Japan), 日本",
"日本 [Japan - Nippon], 日本",
" Japan - Nippon (Japan) - Japan - 日本 - Japan - Nippon (Japan), 日本",
"Japan - 日本~+ , 日本~+",
"Japan / 日本 / Japan , 日本",
}, nullValues = "null")
void testRemoveNonLatin(String in, String out) {
assertEquals(out, LanguageUtils.removeLatinCharacters(in));
}
@ParameterizedTest
@ValueSource(strings = {
// OSM tags that SHOULD be eligible for name:latin feature in the output
"name:en",
"name:en-US",
"name:en-010",
"int_name",
"name:fr",
"name:es",
"name:pt",
"name:de",
"name:ar",
"name:it",
"name:ko-Latn",
"name:be-tarask",
// https://wiki.openstreetmap.org/wiki/Multilingual_names#Japan
"name:ja",
"name:ja-Latn",
"name:ja_rm",
"name:ja_kana",
// https://wiki.openstreetmap.org/wiki/Multilingual_names#China
"name:zh-CN",
"name:zh-hant-CN",
"name:zh_pinyin",
"name:zh_zhuyin",
"name:zh-Latn-tongyong",
"name:zh-Latn-pinyin",
"name:zh-Latn-wadegiles",
"name:yue-Latn-jyutping",
// https://wiki.openstreetmap.org/wiki/Multilingual_names#France
"name:fr",
"name:fr-x-gallo",
"name:br",
"name:oc",
"name:vls",
"name:frp",
"name:gcf",
"name:gsw",
})
void testLatinFallbacks(String key) {
if (key.startsWith("name:")) {
assertTrue(LanguageUtils.isValidOsmNameTag(key));
}
assertEquals("a", LanguageUtils.getLatinName(Map.of(
key, "a"
), true));
assertNull(LanguageUtils.getLatinName(Map.of(
key, "ア"
), true));
assertNull(LanguageUtils.getLatinName(Map.of(
key, "غ"
), true));
}
@ParameterizedTest
@ValueSource(strings = {
// OSM tags that should NOT be eligible for name:latin feature in the output
"name:signed",
"name:prefix",
"name:abbreviation",
"name:source",
"name:full",
"name:adjective",
"name:proposed",
"name:pronunciation",
"name:etymology",
"name:etymology:wikidata",
"name:etymology:wikipedia",
"name:etymology:right",
"name:etymology:left",
"name:genitive",
})
void testNoLatinFallback(String key) {
assertFalse(LanguageUtils.isValidOsmNameTag(key));
assertEquals("Branch Hill–Loveland Road", LanguageUtils.getLatinName(Map.of(
"name", "Branch Hill–Loveland Road",
key, "Q22133584;Q843993"
), true));
assertEquals("rì", LanguageUtils.getLatinName(Map.of(
"name", "日",
key, "other"
), true));
}
@ParameterizedTest
@CsvSource({
"キャンパス, kyanpasu",
"Αλφαβητικός Κατάλογος, Alphabētikós Katálogos",
"биологическом, biologičeskom",
})
void testTransliterate(String in, String out) {
assertEquals(out, LanguageUtils.getLatinName(Map.of(
"name", in
), true));
assertNull(LanguageUtils.getLatinName(Map.of(
"name", in
), false));
}
}

2
planetiler-dist/pom.xml
Wyświetl plik

@ -29,7 +29,7 @@
</dependency>
<dependency>
<groupId>com.onthegomap.planetiler</groupId>
<artifactId>planetiler-basemap</artifactId>
<artifactId>planetiler-openmaptiles</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>

14
planetiler-dist/src/main/java/com/onthegomap/planetiler/Main.java
Wyświetl plik

@ -2,16 +2,16 @@ package com.onthegomap.planetiler;
import static java.util.Map.entry;
import com.onthegomap.planetiler.basemap.BasemapMain;
import com.onthegomap.planetiler.basemap.util.VerifyMonaco;
import com.onthegomap.planetiler.benchmarks.BasemapMapping;
import com.onthegomap.planetiler.benchmarks.LongLongMapBench;
import com.onthegomap.planetiler.benchmarks.OpenMapTilesMapping;
import com.onthegomap.planetiler.custommap.ConfiguredMapMain;
import com.onthegomap.planetiler.examples.BikeRouteOverlay;
import com.onthegomap.planetiler.examples.OsmQaTiles;
import com.onthegomap.planetiler.examples.ToiletsOverlay;
import com.onthegomap.planetiler.examples.ToiletsOverlayLowLevelApi;
import com.onthegomap.planetiler.mbtiles.Verify;
import com.onthegomap.planetiler.openmaptiles.OpenMapTilesMain;
import com.onthegomap.planetiler.openmaptiles.util.VerifyMonaco;
import java.util.Arrays;
import java.util.Locale;
import java.util.Map;
@ -22,17 +22,17 @@ import java.util.Map;
*/
public class Main {
private static final EntryPoint DEFAULT_TASK = BasemapMain::main;
private static final EntryPoint DEFAULT_TASK = OpenMapTilesMain::main;
private static final Map<String, EntryPoint> ENTRY_POINTS = Map.ofEntries(
entry("generate-basemap", BasemapMain::main),
entry("generate-openmaptiles", OpenMapTilesMain::main),
entry("generate-custom", ConfiguredMapMain::main),
entry("basemap", BasemapMain::main),
entry("openmaptiles", OpenMapTilesMain::main),
entry("example-bikeroutes", BikeRouteOverlay::main),
entry("example-toilets", ToiletsOverlay::main),
entry("example-toilets-lowlevel", ToiletsOverlayLowLevelApi::main),
entry("example-qa", OsmQaTiles::main),
entry("osm-qa", OsmQaTiles::main),
entry("benchmark-mapping", BasemapMapping::main),
entry("benchmark-mapping", OpenMapTilesMapping::main),
entry("benchmark-longlongmap", LongLongMapBench::main),
entry("verify-mbtiles", Verify::main),
entry("verify-monaco", VerifyMonaco::main)

4
planetiler-examples/README.md
Wyświetl plik

@ -163,8 +163,8 @@ for a complete unit and integration test.
Check out:
- The other [minimal examples](./src/main/java/com/onthegomap/planetiler/examples)
- The [basemap profile](../planetiler-basemap) for a full-featured example of a complex profile with processing broken
out into a handler per layer
- The [OpenMapTiles profile](https://github.com/openmaptiles/planetiler-openmaptiles) for a full-featured example of a
complex profile with processing broken out into a handler per layer
- [Planetiler](../planetiler-core/src/main/java/com/onthegomap/planetiler/Planetiler.java) for more options when
invoking the program
- [FeatureCollector](../planetiler-core/src/main/java/com/onthegomap/planetiler/FeatureCollector.java)

1
planetiler-openmaptiles 160000

@ -0,0 +1 @@
Subproject commit 62d420811bb11a1c0cbc0f2a32af41015f27dea8

6
pom.xml
Wyświetl plik

@ -26,8 +26,10 @@
<sonar.organization>onthegomap</sonar.organization>
<sonar.projectKey>onthegomap_planetiler</sonar.projectKey>
<sonar.moduleKey>${project.artifactId}</sonar.moduleKey>
<sonar.exclusions>planetiler-benchmarks/**/*, **/VectorTileProto.java, **/crosby/binary/*.java,
<sonar.exclusions>
planetiler-benchmarks/**/*, **/VectorTileProto.java, **/crosby/binary/*.java,
**/generated/*.java
, planetiler-openmaptiles/**/*
</sonar.exclusions>
</properties>
@ -85,7 +87,7 @@
<modules>
<module>planetiler-core</module>
<module>planetiler-basemap</module>
<module>planetiler-openmaptiles</module>
<module>planetiler-custommap</module>
<module>planetiler-benchmarks</module>
<module>planetiler-examples</module>

2
scripts/regenerate-openmaptiles.sh
Wyświetl plik

@ -14,7 +14,7 @@ echo "Building..."
./mvnw -DskipTests=true --projects planetiler-dist -am package
echo "Running..."
java -cp planetiler-dist/target/*-with-deps.jar com.onthegomap.planetiler.basemap.Generate -tag="${TAG}" -base-url="${BASE_URL}"
java -cp planetiler-dist/target/*-with-deps.jar com.onthegomap.planetiler.openmaptiles.Generate -tag="${TAG}" -base-url="${BASE_URL}"
echo "Formatting..."
./scripts/format.sh

4
scripts/test-release.sh
Wyświetl plik

@ -13,7 +13,7 @@ else
fi
echo "Test java build"
echo "::group::Basemap monaco (java)"
echo "::group::OpenMapTiles monaco (java)"
rm -f data/out.mbtiles
java -jar planetiler-dist/target/*with-deps.jar --download --area=monaco --mbtiles=data/out.mbtiles
./scripts/check-monaco.sh data/out.mbtiles
@ -25,7 +25,7 @@ java -jar planetiler-dist/target/*with-deps.jar example-toilets --download --are
echo "::endgroup::"
echo "::endgroup::"
echo "::group::Basemap monaco (docker)"
echo "::group::OpenMapTiles monaco (docker)"
rm -f data/out.mbtiles
docker run -v "$(pwd)/data":/data ghcr.io/onthegomap/planetiler:"${version}" --area=monaco --mbtiles=data/out.mbtiles
./scripts/check-monaco.sh data/out.mbtiles

6
sonar-project.properties
Wyświetl plik

@ -1,4 +1,4 @@
sonar.issue.ignore.multicriteria=js1659,js3358,js1172,js106,js125,js2699,js3776,js1121,js107,js1192
sonar.issue.ignore.multicriteria=js1659,js3358,js106,js125,js2699,js3776,js1121,js107,js1192
# subjective
sonar.issue.ignore.multicriteria.js1659.ruleKey=java:S1659
sonar.issue.ignore.multicriteria.js1659.resourceKey=**/*.java
@ -17,10 +17,6 @@ sonar.issue.ignore.multicriteria.js107.resourceKey=**/*.java
sonar.issue.ignore.multicriteria.js1192.ruleKey=java:S1192
sonar.issue.ignore.multicriteria.js1192.resourceKey=**/*.java
# layer constructors need same signatures
sonar.issue.ignore.multicriteria.js1172.ruleKey=java:S1172
sonar.issue.ignore.multicriteria.js1172.resourceKey=**/basemap/layers/*.java
# tests use assertion helpers
sonar.issue.ignore.multicriteria.js2699.ruleKey=java:S2699
sonar.issue.ignore.multicriteria.js2699.resourceKey=**/*.java