mirror
/
planetiler
kopia lustrzana https://github.com/onthegomap/planetiler
1
0
Forkuj 0
Kod Zgłoszenia Projekty Wydania Wiki Aktywność
873 Commity
87 Gałęzie
10 Tagi
57 MiB
Wykres commitów

69 Commity (main)

Autor SHA1 Wiadomość Data
Michael Barry f69882addb
Fix tile size stats memory leak (#861) 2024-04-02 20:34:42 -04:00
Peter Hanecak d703b626ad
Attempt to fix IOException/GOAWAY happening within parseResults() (#818) 2024-02-16 07:46:17 -05:00
Brandon Liu 0cb2645639
Fix comparison of multipolygons bounds check [#780] (#793) 2024-01-16 08:11:03 -05:00
Michael Barry bf24ef3961
Check for deterministic output in CI (#792) 2024-01-15 14:14:17 -05:00
Michael Barry 5a7757435d
Deterministic IDs on SimpleFeatures (#791) 2024-01-15 06:22:30 -05:00
Michael Barry 062528b1ee
Make OSM polygon construction deterministic (#788) 2024-01-12 09:01:09 -05:00
Michael Barry 0dc2ee82e1
Deterministic merging (#785) 2024-01-11 08:42:16 -05:00
Michael Barry 96eae6110b
Archive compare (#780) 2024-01-10 05:21:03 -05:00
Björn Bilger df804879ef
remove log4j-CloseShieldOutputStream usage (#779) 2024-01-09 18:14:28 -05:00
Björn Bilger c480b35f1c
Add support for "files"-archive (#761) 
* Add support for "files"-archive

i.e. write individual pbf-files to disk in the format <base>/z/x/y.pbf

in order to use that format it must be passed as "--ouput=/path/to/tiles?format=files"

Fixes #536

* default to files format

...if no explict format query param given, path ends with a slash, or no extension given

* output metadata.json in files-archive

and refactor TileArchiveMetadata
1. put zoom into center (CoordinateXY->Coordinate) - in sync with mbtiles-format
2. add (De-)Serializer for Coordinate+Envelop => avoid duplication and cleaner
3. change the json and proto output for TileArchiveMetadata to be (more) in sync with mbtiles-format

* add support for custom tile scheme in files-archive

{z}/{x}/{y}.pbf is the default and can be configured as needed - e.g.:
- different order: {x}/{y}/{z}.pbf
- with intermediate dirs: {x}/a/{y}/b/{z}.pbf
- with different extension: {z}/{y}/{y}.pbf.gz

instead of {x} and {y}, {xs} and {xy} can be used which breaks up
x and y into 2 directories each and ensures that each directory has <1000 children

* fix issues with multiple writers

1. call finish archive only once after all writers are finished
   ...and not every time a writer finishes
2. log "zoom-progress" for the first tile write only
   (Finished z11 ... now starting z12)
3. remove file/dir-size progress logger bottleneck for files archive
   => each archive now reports the bytes written, which also fixes
   the issues of stream-archives reporting the size incorrectly
4. introduce printStats-hook on archive-level

* add async file write support to files archive

...allow to use virtual threads ExecturService (bound only!) for tile writing

also add some benchmark for writing tiles to disk: fixed, bound virtual, async, unbound virtual

* Revert "add async file write support to files archive"

This reverts commit b8cfa56977d98520aa8b62252c3a2726d440afe0.

* few improvements

- extract TileSchemeEncoding
- use Counter.MultithreadCounter rather than LongAdder to count bytes written
- add some JavaDoc

* simplify files archive usage

1. allow to pass tile scheme directly via output: --output=tiles/{x}/{y}/{z}.pbf
2. auto-encode { (%7B) and } (%7D) => no need to encode it the URI on CLI

* few more adjustments according to PR feeback

1. use WriteableTileArchive#bytesWritten in summmary as well
2. call WriteableTileArchive#init in a safer manner

..and a few more adjustments

* more PR feedback
 2024-01-02 20:37:49 -05:00
Michael Barry cbb092a1f7
Compute layer attr stats from actual vector tile features (#752) 2023-12-14 19:26:27 -05:00
Michael Barry 44f22b27c5
Rewrite downloader using virtual threads (#702) 2023-11-01 19:30:28 -04:00
Michael Barry 9f960022b8
Logging fix (#699) 2023-10-27 20:40:45 -04:00
Michael Barry 1be2fca45f
Use some new Java 21 features (#695) 2023-10-27 20:29:26 -04:00
Michael Barry 2bb05a0797
fix log4j log format (#697) 2023-10-26 07:30:56 -04:00
Michael Barry e7e18d5f17
Sort merged vector tile features by hilbert order (#673) 2023-09-26 20:56:04 -04:00
Michael Barry 0a241e17e1
Fix downloading tile weights in CI (#668) 2023-09-22 06:30:17 -04:00
Michael Barry 1f23b55274
Tile stats (#656) 2023-09-21 21:44:09 -04:00
Michael Barry a48a9c8391
Page through S3 index (#608) 2023-06-21 20:43:37 -04:00
Erik Price cced1920d4
Add support for Overture maps downloads. (#599) 2023-06-08 09:01:01 -07:00
Michael Barry 72ea82cd50
Fall back to slower transliterator (#543) 2023-03-31 06:33:12 -04:00
Michael Barry 509795401e
Fix geometry errors (#526) 2023-03-20 16:41:18 -04:00
Michael Barry 97231d18ef
Add --keep-unzipped option to avoid unzipping sources each time (#522) 2023-03-19 14:01:17 -04:00
Michael Barry 74db638dbc
Expose pmtiles writer (#520) 2023-03-18 14:38:04 -04:00
Brandon Liu d5b404d8e2
Finish PMTiles write implementation and basic reader (#502) 2023-03-13 13:56:11 -04:00
Michael Barry 1375ee7abb
Support disabling colors in terminal output (#501) 2023-02-25 07:45:45 -05:00
Michael Barry 9a704e773e
Convert get all tile coords to iterator (#463) 2023-02-05 14:16:05 -05:00
Michael Barry 78129905e5
Add `--download-max-bandwidth` option (#467) 2023-01-30 13:38:09 -05:00
Brandon Liu 88daeb4d0b
Speed up VarInt encoding, remove emitTilesInOrder option [#98] (#460) 2023-01-27 08:12:54 -05:00
Brandon Liu f772180fb7
refactor VectorLayers from mbtiles into util.LayerStats [#98] (#456) 2023-01-25 21:15:43 -05:00
Erik Price a0f8c67c78
Support unzipping GeoPackage sources at runtime (#430) 2023-01-25 20:56:30 -05:00
Brandon Liu b97d9d0e79
fix CloseableConsumer class name (#454) 2023-01-23 06:06:57 -05:00
Brandon Liu 1afb5de820
add VarInt encoder for unsigned longs [#98] (#449) 2023-01-23 05:58:23 -05:00
Michael Barry dad2073f61
Add planetiler-specific metadata (#428) 2023-01-02 11:26:00 -05:00
Erik Price 184f950051
Extract ZIP file walker out of ShapefileReader. (#423) 2023-01-01 17:29:00 -05:00
Michael Barry 926bc9210b
Switch ordering of translation providers (#403) 2022-12-01 17:26:04 -05:00
Erik Price ccf098115e
Move NumberFormat to thread-local variable (#387) 
* Move NumberFormat to thread-local variable

The NumberFormat class isn't thread safe, and sharing one instance
across threads allowed for races which resulted in otherwise valid OSM
tags not being parsed correctly.

* Add supression for ThreadLocal use

* Use ThreadLocal for NumberFormat objects in `Format`

* Wrap `NumberFormat` instances rather than hashmap
 2022-11-21 20:34:28 -05:00
Michael Barry 0eb148ee3c
Expose arguments via config (#363) 2022-10-04 19:57:59 -04:00
Michael Barry 5296d1772e
Shortbread schema (#323) 2022-09-23 06:49:09 -04:00
Michael Barry 4cc139a603
Improve geofabrik area search (#317) 2022-08-09 07:13:45 -04:00
Michael Barry 93fe75782e
Improve shapefile zip reads (#314) 2022-07-31 07:17:42 -04:00
Adam Laža 88c7f5650f
Move basemap profile into OpenMapTiles submodule (#258) 2022-07-29 06:40:15 -04:00
Michael Barry 621a8ed759
Support zoom levels up to 15 (#303) 2022-07-26 07:51:31 -04:00
farfromrefuge 76c7880d6b
feat: added `maxzoomForRendering` and `minzoomForRendering` (#272) 2022-06-20 06:31:50 -04:00
Brian Sperlongano 3fd094ddae
Allow more than 256 attribute keys (#275) 2022-06-20 05:37:40 -04:00
Michael Barry b0f634bcaf
Use fnv1a64 for tile hash (#253) 2022-06-03 20:44:49 -04:00
Michael Barry 2ccef9f7f6
Unsynchronize ICU4j transliterate (#246) 2022-06-01 21:31:49 -04:00
Björn Bilger f5206b3a73
add compact DB mode (--compact-db) to de-duplicate mbtiles output (#219) 2022-05-24 17:46:56 -04:00
Michael Barry dab37f572e
Handle elevations in units besides meters (#226) 2022-05-17 20:23:11 -04:00
Michael Barry 726e6d0107
Parallel temp feature writes using `--write-threads` argument (#213) 2022-05-05 22:02:18 -04:00