Imagico.de

blog

September 13, 2021
by chris
0 comments

Satellite image news

The title of this section should be interpreted loosely this time – i have not discussed new developments in the domain in general for some time now so some of the news are more than a year old already.

There were a number of of recent and are a number of upcoming satellite launches both in the Open Data domain (in particular Landsat 9 of course) and in commercial Earth observation. In the latter we have had in particular the first two of four Pleiades Neo satellites launched (May 20 and August 16) which are going to offer increased resolution and higher recording capacity compared to the existing two Pleiades satellites. Maxar has been showing their plans for a new constellation of more lightweight high resolution satellites making use of economics of scale in building them and offering more flexibility in recording than single and more heavy system like they are predominantly using right now. But there seems no definitive launch date yet. Still the Pleiades Neo launches evidently will put pressure on them. Planet Labs has also started launching an updated version of their micro-satellites offering additional spectral bands. As usual they have been very tight-lipped in public about this though.

In terms of newly available open data satellite imagery i want to mention in particular the Brazilian CBERS-4A (launched December 20, 2019) and Amazonia-1 (launched February 28, 2021) satellites. It seems the data released by the National Institute for Space Research (INPE) is all available under CC-BY-SA which is more restrictive than the licenses of other open data image providers but still the most notable entry in the open data imagery domain from outside Europe, North America and Japan so far. What is not clear is if the data published by the INPE is all the data recorded by the satellites or if they release only data selected to be published as open data and there is also unreleased data that is not available freely. Considering the data published is mostly limited to South America that is not an unlikely scenario.

Amazonia-1 image example over southern Peru/Bolivia - whole scene

Amazonia-1 image example over southern Peru/Bolivia – whole scene

The Amazonia-1 satellite with its 850km swath and 60m resolution fills a noticeable gap in the European, North American and Japanese systems between the high resolution systems (resolution better than about 50m but swath width less than about 300km) and the low/moderate resolution systems with swath width of more than 1000km but resolutions of 250m or worse.

Amazonia-1 image sample - magnified crop

Amazonia-1 image sample – magnified crop

The actual Amazonia-1 image sensor is not particularly advanced by today’s standards – dynamic range seems to be comparable to Landsat 7 – with the same tendency to saturate the sensor on bright features. It seems to be using 10-bit AD conversion.

Amazonia-1 image sample - full resolution crop

Amazonia-1 image sample – full resolution crop

There also seems to no radiometric calibration data available so far – just the raw sensor values. But still this is a fairly impressive source of imagery available there with the combination of a fairly wide field of view with a moderately high resolution and it would be great to have this available also for areas outside the region in South America covered so far.

Targeting the same gap in available imaging system is another experimental system – SeaHawk – that was launched in 2018 already and that became more prominently publicized recently. SeaHawk is a less than 5kg micro-satellite that records a 216km swath at 120m resolution. That is less ambitious than Amazonia-1 and provides less nominal coverage than a single Sentinel-2 satellite but at that form factor this is pretty impressive, especially also in terms of image quality.

SeaHawk image sample from Eastern Greenland

SeaHawk image sample from Eastern Greenland

SeaHawk image sample from New Caledonia

SeaHawk image sample from New Caledonia

If you look at the images available more closely you can see various issues and flaws, in particular:

  • a significant level of banding and striping due to non-uniformity in the sensor. This could probably be partly compensated by better per-pixel calibration but still this is on a level significantly worse than larger current satellite sensors.
  • a significant decline in resolution towards the swath edges which is clearly the result of limitation of the optics and not of the view geometry (larger viewing distance at the edges – well known for wide angle systems like MODIS and VIIRS) alone.
  • a significant mismatch between the spectral bands leading to color rims at sharp contrast edges.
SeaHawk image sample magnified crop

SeaHawk image sample magnified crop

The most significant operational issue however seems to be the poor out-of-the-box geolocation accuracy of the images. The raw Level-1 data you can download comes without any geolocation data and so far they seem to be manually geocoding the data on the coastline. Micro-satellites like this have only very limited ability to determine their own orientation in space and therefore limited ability to know where exactly they are looking. That lack of accurate information severely limits the ability to properly and accurately geocode the image data. A more detailed discussion of the matter is provided by the operators.

In a quite similar direction of miniaturization goes the PROBA-V plus one project – meant to be a follow-up on the ended PROBA-V – which unfortunately despite being paid with tax money has never been fully open data. It will be a good test case to see if the organizational culture at ESA is changing in light of the Copernicus program (where the open data policy has been imposed from the top by the EU Commission) or if the influential people in and around ESA still prefer to keep the bulk of the data exclusive to themselves and those deemed worthy.

Following up on my posts on Sentinel-3 image use and the severe issues with geolocation data in the SLSTR files – ESA has in late 2019 apparently finally noticed and fixed the problem. Newer files produced (produced after early 2020, indicated by a ‘004’ version string in the file name) do not contain this error any more. Older Sentinel-3B data has also been reprocessed but Sentinel-3A data not yet.

I will follow up on this in a bit more detail in a separate post probably.

Of all the optical earth observations systems in the EU Copernicus program i had not yet looked at the Sentinel-5P satellite. It is of relatively little value for the work i am doing but for completeness reasons i had a closer look. Sentinel-5P features a low spatial resolution hyperspectral sensor which records data in particular about the atmosphere and focuses on the UV and blue domain. It would be great for considering and demonstrating the effect of different spectral band designs of multispectral sensors on visualization of the Earth but unfortunately its spectral coverage has a huge gap right in the middle of the visual range where you would typically have the green band.

Sentinel-5P L1B data false color visualization showing data from Sept 10, 2021 at around 305nm, 390nm and 475nm

Sentinel-5P L1B data false color visualization showing data from Sept 10, 2021 at around 305nm, 390nm and 475nm

The illustration above shows the visualization of the data of a single day in the UV-Range and in the blue spectrum in a false color display. The strips along the flight direction of the satellite are caused by the different scattering characteristics depending on illumination and view direction. A large part of the signal in the ultraviolet is caused by scattering in the atmosphere. Different gases and aerosols have different scattering and absorption characteristics depending on the wavelength – visible on that day in particular in the mid west of the USA where the smoke from fires shows up in strong red-brown tones. The Sentinel-5P sensor offers very detailed spectra of every pixel recorded. Overall more than 150GB of L1B data were processed for the production of this illustration (and that is just three of eight spectral bands).

Sentinel-5P by the way is the only satellite from the Copernicus program that covers the whole planet surface during a single day.

Finally a bit of seriously bad news – NASA has last year announced that the Terra satellite is nearing the end of its life and will – due to the lack of fuel (i discussed this in more detail for EO-1 in the past) – start to drift in its orbit. It has meanwhile exceeded nominal specifications for the recordings and will probably reach a 15 minute difference in timing/position of its images by September 2022 and will likely cease producing usable images by 2025 latest.

I will probably write a bit more about the Terra satellite and its significance for the historic development of Earth observation from space as well as its present day significance. In particular i would also like to discuss the MISR instrument. Most people know Terra in particular for the MODIS and ASTER sensors and MISR – being somewhat in the shadow of the more popular MODIS is one of the most underrated and underused Earth observation systems currently available.

I have also updated my diagram of Multispectral Earth Observation satellites with many of the systems mentioned above and a few others as well as some updates and fixes for existing data.

Multispectral Earth observation satellite sensors - September 2021

Multispectral Earth observation satellite sensors – September 2021

September 3, 2021
by chris
1 Comment

Green Marble version 2.1

I am pleased to announce an update to my global satellite image product Green Marble.

The Green Marble is a medium resolution whole earth image offering an unsurpassed uniformity in quality across the whole planet surface on both water and land coloring, based on recent data and a hundred percent cloud free.

The Caspian Sea depicted by the Green Marble 2.1

The Caspian Sea depicted by the Green Marble 2.1

The version 2.1 i introduce here is an update of the data basis used for water rendering, which is generated from Sentinel-3 OLCI data. The data basis for that has been more than doubled compared to version 2, meaning that more than 100TB of Sentinel-3 data were downloaded and processed for the production of this image.

This broader data basis resulted in a significant reduction in artefacts and noise levels in water surface rendering, which demonstrates the excellent convergence characteristics of the processing methods used.

Contrast enhanced visualization of the noise level reduction in depicting open ocean areas

Contrast enhanced visualization of the noise level reduction in depicting open ocean areas

To give some idea of the volume of data processed – here a visualization of the number of ultimately useful data samples (likely water, likely free of cloud, sea ice and sun glint) that went into processing.

Number of usable samples of water color data processed in production of the Green Marble 2.1

Number of usable samples of water color data processed in production of the Green Marble 2.1

Here a few examples of comparison between version 2 and version 2.1 illustrating the improvement in quality.


North Sea example (large: GM 2/GM 2.1)


Indonesia example (large: GM 2/GM 2.1)


Korea example (large: GM 2/GM 2.1)


South Sandwich Islands example (large: GM 2/GM 2.1)

The reduced sea ice coverage in the new image is the result of some adjustments made to processing and the additional water data used and does not reflect a reduction in actual sea ice coverage during the past years.

I updated the preview map on maps.imagico.de where you can look at the image in more detail. The product page on services.imagico.de is updated to the new version as well. If you are interested in using this image in your own project you can contact me through the form on the product page or via email. Here some additional samples featuring the improved water rendering.

Torres Strait

Torres Strait

Great Barrier Reef

Great Barrier Reef

Svalbard

Svalbard

Caribbean Sea

Caribbean Sea

August 31, 2021
by chris
2 Comments

Testing client side map renderers on polygons

I have in the past commented critically on the vector tiles hype that seems to engulf and dominate much of the digital cartography sector these days. What i dread in particular is the really saddening effect this trend seems to have in some cases on cartographic ambition of map producers – the most iconic case example being maybe swisstopo – whose traditional topographic maps are by many considered one of the pinnacles of high quality map design with uncompromising cartographic standards and ambitions – and which now sinks to the level of presenting something as crude as this as a serious contribution to their digital services.

One of the core issues of this trend in my eyes has been that it has been fueled largely by economic factors (in other words: cost cutting of map providers) while practically across the board being executed on the engineering level in an highly superficial and sloppy way. If you look at software development projects in the domain you can almost universally see the short term economic goals in the foreground and a lack of capacity and/or willingness to thoroughly analyze the task from ground up and design and develop a solution with a decent level of genericity and scalability and to invest the time and resources a decent solution calls for.

But while i am confident that this critique is justified i realize that for many people working within the vector tiles hype so to speak it is hard to follow these critical considerations without specific examples. And i also want to better understand myself what my mostly intuitive critical view of the vector tiles trend and the technological developments around it is actually caused by and better identify and where possible quantify the deficits.

Starting at the basics

What i will therefore try to do is exactly what software engineers in the domain seem to do much too rarely: Look at the matter on a very fundamental level. Of course i anticipate that there are those who say: These tools are meant to render real world maps and not abstract tests. Why should we care about such? Well – as the saying goes: You have to crawl before you can learn to walk. To understand how renderers perform in rendering real world maps you need to understand how they work in principle and for that looking at abstract tests is very useful. That you rarely see abstract tests like the ones shown here being used and discussed for map rendering engines in itself is indicative of a lack of rigorous engineering in the field IMO.

And i will start at the end of the data processing and rendering toolchain – the actual rendering – because this is where the rubber meets the road so to speak. We are talking about map rendering here which ultimately always means generating a rasterized visual representation of the map. And despite some people maybe believing that the vector tiles trend removes that part of the equation it of course does not.

In this blog post i will look at the very fundamental basics of what is being rendered by map renderers, that is polygons. The most frequent visual elements in maps that rendering engines have to render are polygons, lines, point symbols and text. And a line of a defined width is nothing different from a long and narrow polygon. Text contains curved shapes but they can with controllable accuracy be converted into polygons. Not all renderers do this in practical implementation, some have distinct low level rendering techniques for non-polygon features and there are of course also desirable features of renderers that do not fit into this scheme. So i am not saying that polygon rendering ability is the only thing that matters for a map renderer but it is definitely the most basic and fundamental feature.

As you can see if you scroll down i only look at simple black polygons on white background examples and ignore the whole domain of color rendering for the moment (color management in map renderers or the lack of it is another sad story to write about another day). Because of the high contrast this kind of test very well brings out the nuances in rendering quality.

The difficulties of actually testing the renderers

I am looking specifically at client side map renderers – that is renderers designed to be used for rendering online maps on the device of the user (the client). This creates several difficulties:

  • They typically run in the web browser which is not exactly a well defined environment. Testing things on both Chromium and Firefox on Linux i observed noticeable differences between the two – which however do not significantly affect my overall conclusions. The samples shown are all from Chromium. I will assume that the variability in results to other browsers and other platforms are similar. Needless to say that if the variability in results due to the client environment is strong that is a problem for practical use of these renderers on its own. After all as a map designer i need predictable results to design for.
  • They are only the final part of an elaborate and specialized chain of data processing designed for cost efficiency which makes it fairly tricky to analyze the renderer performance in isolation without this being distorted by deficits and limitations in the rest of the toolchain.

I managed to address the second point by running the first tests based on GeoJSON files rather than actual vector tiles. All three renderers evaluated support GeoJSON input data and this way you can reliably avoid influences of lossy data compression affecting evaluation of the rendering performance (which is – as i will explain later – a serious problem when practically using these renderers with actual vector tiles).

The test data i created, which you can see in the samples below, is designed to allow evaluating the rendering performance while rendering polygons.

Test patterns used for evaluating polygon rendering

Test patterns used for evaluating polygon rendering

I will quickly explain the different components

  1. Adjacent polygon test with four touching rectangles that are a single multipolygon geometry (which is technically invalid – but all renderers accept this). At two different rotations.
  2. Adjacent polygon test with four touching rectangles that are separate geometries. This is the classic case where AGG produces artefacts (as explained in detail elsewhere).
  3. A Siemens star like test pattern showing resolution, aliasing and color mixing biases.
  4. Dent pattern to show lines of various width both positive (black on white) and negative (white on black) features. Shows any bias between those two cases, artefacts in narrow feature rendering and step response at a strait polygon edge in general.
  5. Inverse Siemens star like pattern with narrow gaps of variable width testing geometric accuracy in rendering and geometry discretization artefacts.
  6. Test for rendering uniformity on circular arcs. Contains both a narrow black line as well as a white gap of same size for checking equal weight of those.
  7. Fine comb gradients testing color mixing and aliasing artefacts.
  8. Fan comb for testing aliasing artefacts.

The test subject

I tested the following client side renderers

  • Maplibre GL – the FOSS fork of Mapbox GL, which was essentially the first browser based vector tiles renderer and can probably be considered the market leader today (though with the fork and development lines diverging and becoming incompatible at some point this could change).
  • Tangram – originally a project of Mapzen this is the other WebGL renderer that has significant use. Having been started by Mapzen its orientation was a bit less narrowly focused on short term commercial needs but it is essentially a ‘me too’ response of Mapzen to Mapbox without much own ambition in innovation (which was a bit of a general problem of Mapzen).
  • Openlayers – the veteran of web map libraries took a very different approach and is now a serious contender in the field of client side map renderers. It uses Canvas rather than WebGL.

For comparison i also show classic Mapnik rendering. This is essentially meant to represent all AGG based renderers (so also Mapserver).

All examples are rendered in a size of 512×512 pixel based on GeoJSON data (in case of the client side renderers) or from a PostGIS database in case of Mapnik.

What serves as reference is – as explained in more detail in my previous discussion of polygon rendering a supersampled rendering – the brute force approach to rendering – performing plain rasterization at a much higher resolution (in this case to represent all the fine details in the test pattern i chose 32767×32768 pixel – that is 64×64 supersampling). When you size that down to 512×512 pixel using plain pixel averaging you get the following.

Supersampled rendering using equal weight pixel averaging (box filter)

Supersampled rendering using equal weight pixel averaging (box filter)

In terms of general precision and edge acuity this is the gold standard. However plain averaging of the pixels is prone to aliasing artefacts as you can observe in the tests for that. Other filter functions can be used to reduce that – though always at some loss in edge acuity and potentially other artefacts. Here the results with the Mitchell filter function – which massively reduces aliasing at a minor cost in edge acuity.


Box filtered reference (left) in comparison to Mitchell filtered rendering

Test results

Here are the results rendering the test geometries in their original form (as GeoJSON data).

Mapnik


reference (left) in comparison to Mapnik rendering

Mapnik is – as you can expect – fairly close to the reference. That includes similar aliasing characteristics. On the fine comb gradient it is even better since it precisely calculates the pixel fractions covered while supersampling does a discrete approximation.

The main flaw is the coincident edge artefacts you can observe on the tests for that on the lower left. Notice however these only occur with coincident edges between different geometries. Within a single geometry (tests on the upper left) this problem does not manifest (though practically using this as a workaround does likely have performance implications). The coincident edge effect also has an influence on the narrow gaps of the inverse Siemens star on the right. In other words: This does not depend on edges to be exactly coincident, it has an influence to some extent anywhere where several polygons partly cover a pixel.

Maplibre GL

Maplibre GL is interesting in the way that it has an antialias option which is off by default and without which the renderer produces results too ridiculous to show here. So what is shown here are the results with antialias: true.


reference (left) in comparison to Maplibre GL rendering

Even with the antialias option the results are bad. Primarily because of a massive bias in rendering enlarging all polygons by a significant fraction of a pixel. This is visible in particular on the fine comb gradients which are turned completely black and the dent pattern where the narrow gaps on the bottom are invisible while the thin dents on top are more or less indistinguishable in width.

In addition to this bias – which turns many of the test patterns non-functional – diagonal edges show significantly stronger aliasing (stair step artefacts), narrow gaps between polygons are non-continuous in some cases and fairly non-uniform on the circle test.

Tangram


reference (left) in comparison to Tangram rendering

Tangram does not have the bias Maplibre GL shows, positive and negative shapes on polygons are rendered in a matching fashion. But it shows similar levels of aliasing which produce a significant amount of high frequency noise even on strait lines and smooth edges like the circle test. Thin lines are rendered proportionally in weight but non-continuously. And the comb gradients produce massive artefacts.

OpenLayers


reference (left) in comparison to Openlayers rendering

OpenLayers is very different in its technical basis of rendering and this shows in the results. In many ways it is the closest to Mapnik and the reference rendering and it shows the least edge aliasing artefacts of all the client side renderers – and accordingly also the clearest representation of the circle test. It however also has its issues, in particular it shows the coincident edge artefacts similar to AGG based renderers (both on the separate and combined geometries), it has issues in proportionally rendering narrow polygon features – including some bias towards the dark (positive) lines compared to the bright (negative) lines.

With actual vector tiles

So far i have shown test results of vector tiles renderers not using actual vector tiles – to look decidedly at the rendering quality and precision and not at anything else. But the question is of course: Do they perform this way, for better or worse, also with actual vector tiles.

The question is a bit tricky to answer because vector tiles are an inherently lossy format of storing geometry data. Coordinates are represented as integers of limited accuracy or in other words: They are rounded to a fixed grid. In addition most vector tile generators seem to inevitably perform a lossy line simplification step before the coordinate discretization and the combination of the two tends to lead to hard to predict results.

The PostGIS ST_AsMVTGeom() function for example (which is meant to be a generic all purpose function to generate geometries for vector tiles from PostGIS data) inevitably performs an ST_Simplify() step without the option to turn that off or adjust its parameters.

So what i did first for testing if there is any difference between how GeoJSON data and vector tile data is rendered is to generate very high resolution vector tiles (extent 65536) as reference. However as it seems only Openlayers seems to actually work with this kind of high resolution vector tiles directly, both Maplibre GL and Tangram seem to reduce the resolution on the client side (the motivation for which eludes me). Hence evaluation of these renderers is limited to normal low resolution tiles.

Client side coordinate discretization in overzoomed display in Maplibre GL (left) and Tangram (middle) in comparison to Openlayers

Client side coordinate discretization in overzoomed display based on the same vector tile data in Maplibre GL (left) and Tangram (middle) in comparison to Openlayers (right)

Here are examples generated with the PostGIS ST_AsMVTGeom() default extent of 4096 – in comparison to the GeoJSON tests shown above.


Maplibre GL rendering from geoJSON data (left) in comparison to vector tiles data (extent 4096)


Tangram rendering from geoJSON data (left) in comparison to vector tiles data (extent 4096)


Openlayers rendering from geoJSON data (left) in comparison to vector tiles data (extent 4096)

As you can see the differences are small and mostly in those areas where aliasing artefacts are strong already. So the principal characteristics of the renderers are the same when used with vector tiles compared to uncompressed geometry data.

What this however also shows is that vector tiles are an inherently lossy format for geometry representation and that there is no such thing as a safe vector tiles resolution beyond which the results are indistinguishable from the original data or even where the difference does not exceed a certain level. Keep in mind that the test patterns used are primarily rendering tests meant to show rendering performance. They are not specifically intended as test patterns to evaluate lossy vector data compression. Also it seems that the resolution of 4096 steps when generating tiles is pretty much on the high side in practical use of vector tiles and the level of lossy compression using ST_Simplify() performed by ST_AsMVTGeom() is less aggressive than what is typically applied.

I will not go into more detail analyzing vector tiles generation here and how rendering quality is affected by lossy compression of data. Doing so would require having a decent way to separate coordinate discretization from line simplification in the process.

Conclusions

Doing this analysis helped me much better understand some of the quality issues i see in vector tile based maps and what contribution the actual renderer plays in those. With both the lossy data compression in the tiles and the limitations of the rendering playing together it is often hard to practically see where the core issue is when you observe problems in such maps. I hope my selective anaylsis of the actual rendering helps making this clearer also for others – and might create an incentive for you for taking a more in depth look at such subjects yourselves.

In terms of the quality criteria i looked at here, that is primarily precision, resolution and aliasing artefacts in polygon rendering, the tested client side renderers perform somewhere between so-and-so and badly. Openlayers shows the overall best performance. Tangram is on the second place, primarily because of much more noisy results due to aliasing artefacts. Maplibre GL makes the bottom end with a massive bias expanding polygons beyond their actual shape essentially rendering many of the tests useless and making any kind of precision rendering impossible – while being subject to similar levels of aliasing as Tangram.

Do i have a recommendation based on these results? Not really. It would be a bit unrealistic to make an overall recommendation based on a very selective analysis like this. Based on my results i would definitely like to see more examples of practical cartography based on the Openlayers vector tiles rendering engine.

If i should really give a recommendation to people looking to start a map design project and wondering what renderer to choose for that it would be a big warning if you have any kind of ambition regarding cartographic and visual quality of the results to think hard about choosing any of the client side renderers discussed here. Because one thing can be clearly concluded from these evaluations: If we compare these renderers with what is the state of the art in computer graphics these days – both in principle (as illustrated by the reference examples above) and practically (as illustrated by the Mapnik rendering – though it is a stretch to call this state of the art probably) – there is simply no comparison. Is this gap between what is possible technically and what is practically available in client side renderers warranted by engineering constraints (like the resources available on the client)? I doubt it, in particular considering what is behind in the web browser and how resource hungry these renderers are in practical use. But i can’t really tell for sure based on my limited insights into the domain.

July 12, 2021
by chris
4 Comments

State of the Map 2021 – observations

The last weekend the State of the Map conference 2021 took place – the second time as a purely virtual conference, you can find my comments on last year on this blog as well.

This commentary is somewhat preliminary because i did not yet view some of the most interesting talks because of real world obligations and the multiple competing tracks. And since the talks were pre-annouced to be recorded and to be made available afterwards while the panels, workshops and self organized sessions were not i focused on the latter a bit more.

The concept of a virtual conference – second iteration

I presented some general musings on the idea and appeal of a virtual conference in last year’s comments most of which still applies of course. Many of my more radical suggestions to depart from a pure 1:1 virtualization of an in-place conference and embracing the fundamentally new chances of the virtual format have not been followed. I have the impression that many of the influential people in the OSMF and around the conference still view the virtual conference as a temporarily necessary stop-gap replacement of what they actually want – a physical meeting of the international OSM jetset – and therefore don’t want to start things that could in any way be incompatible with moving fully back to what they know and enjoy.

Compared to last year where everything had a fairly improvised feel to it, this year the whole conference framework was much more slick and organized. But to me this is not really a positive thing. It is not that last year things technically did not work on a larger scale and this year i saw plenty of cases of (minor) glitches, in particular when you tried to join a room the browser tab often froze for a considerable amount of time. To me this slickness of this year’s conference was most visible in form of the much tighter constraints, less freedom, more builtin distance between the participants of the conference and less freedom for free form interaction. Specifically:

  • I missed the pads that were last year used for comments, questions and discussions of the talks. The free form and non-linear nature of the pads offered so much more freedom for interaction, working together and expressing yourself than this year’s linear chats and isolated fire-and-forget questions.
  • Linear group chats in general pose a structural disadvantage to slow typers and non-native English speakers and incentivize asocial tactics to dominate a discussion. They promote shallowness in discourse and essentially prevent any more elaborate discussion with multiple participants.
  • Last year’s SotM was essentially an open event where everything (streaming of talks, interactive conference rooms, pads for discussion) was openly accessible to everyone without registration. I saw this as a huge benefit and it contributed a lot to the open get-together flair of that event. This year everything (except for maybe streaming of talks) was only open to registered visitors of the conference (and i am not even sure you could still spontaneously register during the conference).
  • There does not seem to be a permanent record of the chats available to follow up on and continue afterwards.

Don’t get me wrong, this was still overall a well made conference and i saw a lot of effort on side of the organizers to make this suitable and inviting for a large variety of people (including in particular the translation efforts – which i did not check out myself). But from my perspective it was in many ways also a step back compared to last year and i kind of fear that is in a way a (potentially mostly subconscious) step to pave the way back to a primarily non-virtual conference again.

Observing filter bubble maintenance in action

The most memorable impression for me was observing the echo chambers most of the panels at this conference turned out to be and essentially observing filter bubble maintenance in action.

The panels i am talking about here are:

First of all there might be a bit of a cultural difference of course. To me a panel discussion on any serious topic would be an event where people with significant differences in their perspective and opinion on a matter meet, each presenting their views and then discussing them so they come to insights that represent an additional value compared to the sum of its parts. In other words – to me a panel discussion is essentially a dialectic format.

The panels from this conference however were more like the panels of a fan convention where actors and producers of popular performing arts praise themselves and each other for their achievements, tell anecdotes and engage in friendly and shallow banter and the main aim is for everyone to have a good time and enjoy themselves.

Now don’t get me wrong – my issue is not that such events took place at SotM. What i found remarkable however is how the participants of all of these panels clearly thought that they were having a meaningful and insightful conversation while what happened was essentially that they reaffirmed each others’ existing preconceptions. The main result of the events was the reinforcement of the existing filter bubbles of the participants and the parts of the audience on the same wavelength by excluding huge parts of what is already a small and relatively homogeneous subset of the OSM community (the conference visitors) from their perception and celebrating a friendly conversation within that bubble to be a discussion with meaning regarding and representative for the larger reality.

All of these panels had participants that could have provided very valuable contributions to a dialectic or argumentative panel discussion on the subjects discussed (and where i would for example definitely say i could learn a lot from having a conversation with them on the subject in question). But without anyone on these panels questioning the preconceptions and assumptions of the participants and opening their eyes to other perspectives on the matter the intellectual value and the gain in insight of these events for listeners on the subject matter was negligible.

In the Paid-Editing/AI-Mapping Tools panel – which was moderated by a professional scientist who should by profession be used to and experienced in questioning and challenging his views and preconceptions i decided to inquire about this with the question:

Why does your panel consist exclusively of people working for large corporations. Do you think others (small businesses, self employed people, hobbyists) have nothing important to contribute on the matter of paid editing and AI mapping tools or did those you have asked decline to participate?

The answer was sobering: They indicated not to believe that those not represented on the panel in their eyes had nothing important to contribute but they still chose to only invite representatives of large corporations.

Some thoughts on the map reading Q&A

When the call for proposals for this conference was announced i was fairly unsure if to submit something. The most natural thing to submit would have been a talk. But the talks were – like last year – to be pre-recorded to then be streamed to the visitors at the specific times of the schedule only. It seems a bit weird to me – not to say disrespectful to the international audience residing in different time zones – to impose such an artificial constraint (since the talks are prerecorded) and to essentially imitate a live event without it actually being live.

Now having talks pre-recorded and not live makes a lot of sense, in particular it lowers the hurdles to have a talk at such a conference massively. Last year i suggested to make use of the natural possibilities coming with that and to make the talks available to be watched by the visitors whenever is convenient for them in advance and to then asynchronously discuss them and to wrap that up with a live discussion similar to what we had now. But this suggestion was not picked up, probably because it was perceived to be a too radical change and it would be incompatible to moving back to a live non-virtual conference without giving that up again of course.

Anyway – my thought was if the conference organizers want to have the conference as a live event only i would want to do something that actually uses the benefits of live interaction with the audience. At the same time i wanted to try introducing the idea of asynchronous participation (like in my suggestion of making the talks available for viewing in advance) into what i do. That is how i developed the idea of an interactive Q&A workshop.

Here are my spontaneous observations and impression of how it went:

For the pre-event asynchronous involvement of the audience i had created a wiki page and a pad to submit topics to discuss. Speaking live at a virtual event of this form is difficult because there is an almost complete lack of real time feedback from the audience. At the same time it is live so in contrast to a pre-recorded talk you can’t just make cuts and re-record certain parts or the whole thing if necessary. Because i kind of anticipated that my original plan was there to be more audience interaction during the workshop.

But while i had advertised it on the diaries and on the OSM-Carto issue tracker there was only one submission (From Jerry (SK53), many thanks for that) before the start of the conference. And Jerry was not present live in the workshop (which was fine – turning up at the workshop was not a prerequisite for submitting case examples) so i could not start off with an interactive approach to the whole thing. And although during the conference a number of more submissions came in i ended up not having any live dialog in audio/video with the audience. That was a bit unfortunate (and i could probably have tried to encourage that more) – it would likely have been an improvement for both me and the audience to connect a voice and a face to the questions and have more symmetric discussion of the case examples.

In the end most of the interaction took place on the chat and the questions. The splitting into chat and questions in the conference interface was something really confusing and IMO quite pointless. You could only show either the chat or the questions and it is easy to miss additions to one of them while concentrating on the other. I see that there is a potential benefit of having anonymous questions but even that would seem possible while still conflating the two in the same display.

Over the duration of the workshop (which ended up to last for 1:30 instead of just one hour) the interactivity improved (probably mostly because i got more used to keeping an eye on the chat and questions and integrating them into my discussion) but for my taste it was still too asymmetric – as said i would have liked more true dialog.

The other important observation i made: For a live event like that involving a screenshare presentation it would be really good to have a triple-screen-setup. You would essentially need:

  • one screen with the conference interface (video, chat, controls etc.)
  • one screen to share
  • one screen for your notes and staging of materials to share

I only had two screens set up connected (had a third one – but that was on a different computer so of limited use) which made things a bit more complicated.

In terms of the actual content of the workshop – it ended up being a bit less basic map reading and more advanced map design questions – like problems of scale dependent rendering and how to deal with the rural-urban divide. I had prepared some material on the matters of road layering and urban landuse, which was more focused on basic explanation of the map but which i ended up not using because there were more than enough questions from the audience to fill the time. But i am a bit afraid that for some of the less experienced listeners the whole workshop was a bit too advanced towards the end.

What i am kind of wondering is if it would make sense to develop something like a regular recurring event out of this outside SotM. Considering how widely the standard map style is used and referred to in the OpenStreetMap world it is quite definitely severely under-explained and under-discussed on a level above the basic “I want feature x to be rendered” and outside the scope of actual style development. Any input on that idea would be welcome.

Navigating the Maze

May 12, 2021
by chris
9 Comments

Navigating the Maze – part 2

This is the second part of a two part post on roads rendering in OpenStreetMap based digital rule based map styles. In the first part i have explained and analyzed the roads rendering system of OSM-Carto and its issues, here i want to explain and discuss the re-designed road rendering system i drafted for the alternative-colors style and the improvements and new features i implemented in it.

A big warning upfront: This road rendering framework is essentially just a sketch. I will explain in the end what i hope in the long term eventually will come out of it. At the moment this is – as said – a bit sketchy and i don’t want to make any claims that it is overall better by any metric than the existing one in OSM-Carto. Or in other words: It is as much a study on what is possible as much as it is one for what is not. Keep that in mind when you think about actually using this. Also if you work with this technique or in general with the AC-style you should be sure to turn off JIT optimization in the Postgresql configuration. Because that does not deal well with the kind of queries used here apparently.

Rethinking layered rendering

The technical limitations of the OSM-Carto road rendering system i discussed in the first part are to some extent caused by specific design choices in OSM-Carto. But for the most part they are the result of principal limitations of the toolchain used by OSM-Carto. The most fundamental limitation of that is that the order in which you query the data from the database and how this is split into a lot of separate queries is directly tied to the order in which things are drawn on the map. There are some limited options to interfere with that on the raster compositioning level (OSM-Carto is using that a bit already and i use this more in the alternative-colors style – a topic for a different post) but in the end you are pretty fundamentally stuck with this paradigm of query order and drawing order being tied to each other.

Most of the software OSM-Carto is using – specifically Carto and Mapnik – are no more actively developed. That means there is not much of a chance that anything will change about this constraints. This also means that the only major component of the OSM-Carto toolchain that is still actively developed is Postgresql and PostGIS.

This essentially suggests the most promising path for rethinking the roads rendering is to work around the limitations of the layered rendering framework as described by combining all road rendering components into a single layer. If you look at the layer list in the first part with 16 layers you can see that this is a pretty radical step. Fortunately there are already quite a few pre-existing changes in the alternative-colors style that make this a bit simpler. Some layers i have dropped (tourism-boundary), others i had moved outside the road layers range (cliffs), which leaves the following layers to deal with:

  • tunnels
  • landuse-overlay
  • line-barriers
  • ferry-routes
  • turning-circle-casing
  • highway-area-casing
  • roads-casing
  • highway-area-fill
  • roads-fill
  • turning-circle-fill
  • aerialways
  • roads-low-zoom
  • waterway-bridges
  • bridges

In addition i moved the aerialways layer above the bridges (because practically most aerialways will probably be physically above bridges they cross) – although this could in the future be changed and integrated back with taking the layering into account of course. That is still 13 layers of course.

So why would i want to combine all of this in a single layer? This has a number of advantages. Since one layer means one SQL query only you don’t have to query stuff several times from the database and you can avoid duplication of SQL code much better. At the same time addressing some of the fundamental limitations of the OSM-Carto road rendering, in particular the lack of proper layering for highway polygons, becomes much easier and does not require a significant volume of additional code.

And most importantly you don’t have to deal with the limitations of the various methods to define the drawing order i discussed in detail in the first part and the complex and difficult to understand interaction between them because you can simply put the features in the order you want to render them in using ORDER BY in SQL. That means you cannot use group-by and attachments any more of course and for every use of attachments you have to essentially duplicate the features to be drawn in SQL (like using a UNION ALL query). But practically that is relatively simple and adding an ORDER BY statement to the wrapping query of the single roads layer is a much more intuitive and better maintainable way to do this in my opinion.

Layering in SQL

So how does the query combining all these 13 layers and the improvements i implemented look like? Before you go to looking through the more than 1700 lines of SQL code you can have a look at the basic structure where i stripped most of the actual queries, in particular the lengthy column lists which make up most of the query. A more elaborate sketch of the structure is available as well. The query makes fairly extensive use of common table expressions to query different sets of features and then use them in different contexts. On the base level of the query (roads_sublayers) we have the CTEs

  • roads_all – which contains all the linear road features from tunnels/bridges/roads-casing/roads-fill as well as things derived from them (for new features – i will discuss those later),
  • road_areas_all – which contains all the road polygon features from highway-area-casing/highway-area-fill
  • tc_all – which contains the stuff from turning-circle-casing and turning-circle-fill

These CTEs currently are independent of each other. roads_all basically contains the UNION ALL from the existing road layers combining highway lines and railway lines – plus the alternative-colors specific features, which use a number of additional CTEs that reference each other.

Based on the three main CTEs there is a stack of 16 sublayer queries combined with UNION ALL. You can find the non-road layers from the above list in there while the road layers do not match the legacy layers directly. Ignoring for the moment those sublayers connected to new alternative-colors specific features we have:

casing
background
fill
area_casing
area_fill
tc_casing
tc_fill
waterway_bridges_casing
waterway_bridges_fill
line_barriers
ferry_routes
landuse_overlay

We don’t need separate sublayers for tunnels and bridges because the sublayers are not what primarily defines the drawing order. The drawing order is defined by the ORDER BY statement that follows the sublayer stack.

That means across all sublayers with a non-null int_bridge attribute (that is everything except the non-roads layers) the features with int_bridge=yes are rendered on top in the order of their layer tag. It does not matter what these are, whether road line bridges, highway polygons, turning circles or waterways. Same for tunnels at the bottom. Apart from that the ordering is similar to the one in OSM-Carto – although i made some adjustments, the effects of which are however mostly very subtle.

Map design in SQL

The most important result of this change is that it allows consistently layering all bridge and tunnel features – no matter if linear roads, highway polygons or waterway bridges – according to the layer attribute as explained in the first part. Here the layering in comparison between OSM-Carto and the alternative-colors style:

Linear roads and waterway bridges layering in OSM-Carto (left) and AC-Style (right)

Highway polygons layering in OSM-Carto (left) and AC-Style (right)

The other fundamental thing i changed and that was also an important prerequisite for the new features i added is to define the line widths for drawing in SQL. This has a two-fold purpose: First it is to make these settings available in SQL for geometry processing and second: To avoid the error prone repetitive way of defining the drawing widths for the different zoom levels as shown in part 1. Parts of this change were already implemented in previous changes where i introduced implicit embankment rendering. This is currently based on a YAML file where you define the different line widths. This file is processed by a python script into alternatively MSS code or an SQL function containing nested CASE statements for selecting the line width based on zoom level and road type. This method is not yet set in stone – an alternative would be to have a separate table in the database containing the line width values. I have now extended approach to also include the casing width values in addition to the overall line width.

Ground unit line width rendering

One thing this change facilitates is allowing to more easily implement ground unit line width rendering at the high zoom levels. The motivation for that is the following: At the low zoom levels roads are universally rendered with a line width larger than their real physical width on the ground. That is universally the case in maps because otherwise the roads would not be visible at smaller scales and even when they would be that would not make for a well readable map. There is however a transit at some point where the drawing width or the roads becomes less than the physical width of the roads. That transit typically happens – depending on latitude, road class and the specific road – at zoom levels between 16 and 18. Drawing the roads more narrow than they are physically is however typically not of benefit for map readability and it also conflicts with the general aim you have when designing large scale maps to move to less geometric abstraction and a depiction closer to the geometric reality on the ground.

Unfortunately you cannot address this simply by increasing the nominal drawing width of the roads in general because not all individual roads of a certain class have the same width of course and what you definitely don’t want at the high zoom levels is to draw the roads wider than their actual width and then obscure other content of the map next to the roads. And fortunately quite a few of the roads in OpenStreetMap have a physical width specified – two million (or about 1 percent) and even more (12 million or 6 percent) have a lanes tag that allows estimating the likely physical width of the road with some accuracy.

To be able to draw lines in ground units on the map you have to be able to convert between map units (also called Mercator meters), ground meters and pixels on the map. The conversion factor between ground units (as tagged with the width tag for example or as estimated from the number of lanes) and map units is called the scale factor of the projection and unfortunately there is no support for directly getting that in PostGIS. I used a generic function that allows approximating that:

ST_DistanceSphere(
  ST_Transform(ST_Translate(geom, 0, 1), 4326),
  ST_Transform(geom, 4326)
)::numeric from (select ST_Centroid(!bbox!) as geom) as p

I do this based on the bounding box of the metatile – not so much for performance reasons but because that ensures geometry processing within a metatile is using the same scale factor – which is essential in some cases. At the zoom levels in question the scale factor will never vary significantly across a metatile. One thing to consider of course in the future is to – for Mercator maps – replace this generic function with the simpler formula specifically for the Mercator projection (because that function is used quite a lot).

The conversion between map units and pixels on the map by the way is the factor !scale_denominator!*0.001*0.28 – so in total you have

pixels = ground_meters/(scale_factor(!bbox!)*!scale_denominator!*0.001*0.28)

This formula is used in carto_highway_line_width_mapped() – which is implemented in roads.sql. Overall the rule for the line width of roads is as follows:

  • If a width is tagged use that as the ground width.
  • If no width is tagged but the number of lanes is tagged then estimate the overall ground width based on the road class and (conservatively estimated) the typical lane width for that type of road.
  • If the ground width determined that way is larger than the nominal line width set by the style than draw it in that width, otherwise in the nominal width.

Currently the style still contains double drawing of the roads in both nominal and ground width (essentially determining the maximum graphically) because i initially had them rendered in different styling. But that should be removed in a future version.

Here is how this looks like at z17 with various different tagged width values – in comparison for different latitudes to show the effect of the variable scale.

Ground unit rendering of roads at z17 at 40 degrees (left) and at 60 degrees latitude (right)

This applies for the roads drawn with the standard casing + fill style. For airport runways the formula is varied to estimate the likely width based on the runway length (using a formula initially suggested for OSM-Carto). For road features with a simple solid/dashed line signature (paths/tracks) the ground width is only visualized at z18 and above – using the styling of footway and track polygons (with some adjustment for tracks to avoid the appearance being too heavy) in addition to the centerline signature. And this ground unit visualization is only used when the drawing width in ground units is large enough to be meaningfully depicted in addition to the centerline.

The ground unit rendering of roads however is not technically as simple as it might seem because of another implicit assumption made by the roads rendering system in OSM-Carto, namely that the drawing order of road classes with the same layer as defined by z_order is in sync with the drawing width progression. That is of course no more universally the case when the tagged width is depicted which would lead to artefacts like the following.

Road junction with artefacts

Solving this requires modifying the road geometries in those cases which is kind of a hassle to do.

Road junction without artefacts

Lanes visualization

Since i already made use of the lanes tagging for estimating the ground unit width of roads i decided to visualize the number of lanes at the high zoom levels as well. As you can see this follows the implicit rules documented which say that trunk roads and motorways are assumed by default to be two lanes. lane_markings=no is indicated with dashed lines instead (click on the image to see).

Lanes visualization at z18+

Lanes visualization at z18+ (click to see also the unmarked version)

In practical use cases ground unit line width rendering and lanes visualization looks like this:

Roads in Strasbourg at z18

Roads in Heidelberg at z18

Roads in Freiburg at z19

This one demonstrating the unfortunate and semantically nonsensical practice of German mappers to tag every road under a bridge as a tunnel.

Of course this style of visualization for both the lanes and the tagged width of the road does not work too well when the number of lanes and the road width changes. This requires mapping and interpretation of additional data, in particular what is mapped with placement=*, which is currently not taken into account.

Turning circles & co.

While we are on the matter of drawing width or road lines – i also extended the existing visualization of turning circles and mini-roundabouts from OSM-Carto with distinct symbolization of both of these as well as turning loops and passing places.

turning circles, turning loops, mini-roundabouts and passing places at z17

turning circles, turning loops, mini-roundabouts and passing places at z17

All of course working in combination with the bridge layering and variable width depiction. The diameter tag for turning circles is also taken into account.

turning circles etc. in combination with specified ground width or roads and tagged diameters, bridges etc. for residential roads ...

turning circles etc. in combination with specified ground width or roads and tagged diameters, bridges etc. for residential roads …

... as well as tracks

… as well as tracks

Sidewalks

The next new feature i looked into visualizing is implicitly mapped sidewalks and cycle tracks. Like in case of embankments sidewalks can be mapped both implicitly as an additional tag on the road (sidewalk=both/left/right) as well as explicitly separately as a way with highway=footway and footway=sidewalk. Both are widely used and there is no clear preference among mappers and it is therefore a bit problematic that OSM-Carto only renders the explicit sidewalks. Like in case of embankments the explicit version is easier to render in a primitive fashion while good quality visualization is much easier with the implicit mapping because you can more easily take the relationship between the road and its sidewalk into account.

What i implemented is a demonstration how visualization of implicit sidewalk mapping as well as that of implicit cycleway tracks with cycleway:right=track/cycleway:left=track can work. This is shown at z18 and above

Rendering of sidewalks and cycleway tracks

Rendering of sidewalks and cycleway tracks

and works also in combination with bridges, fords and implicit embankments. They are not rendered on tunnels to avoid obscuring ground level features.

Rendering of sidewalks in combination with various road variants

Rendering of sidewalks in combination with various road variants

And here is how this looks like in real world situations.

Sidewalks in Ladenburg at z18

Sidewalks in Ladenburg at z18

Sidewalks in Portland at z18

Sidewalks in Portland at z18

Steps details

Rendering of roads in their ground unit width also provides more space to visualize additional details on steps. The normal drawing width of steps does not really provide enough room for showing more information without affecting the recognizably of the steps signature. Here is what is shown with nominal drawing width and when the drawing width is large enough at z18+ for wider steps tagged with their width.

Steps details at z18+

Steps details at z18+

Unpaved roads

The distinct depiction of paved and unpaved roads is one of the oldest open issues in OSM-Carto. The difficulty is that it has to work in combination with a lot of other road properties already visualized. Various forms of dashing have been tried and have been found either to be very noisy in appearance or too similar to existing depictions of tunnels (which use a dashed casing), construction roads (which use a dashed fill) or access restrictions (which use a dashed centerline on the fill) to be intuitively understandable without affecting map readability in general. The most promising idea so far has been the use of a fine grained structure pattern for the road fill. That is highly intuitive in its meaning and can be tuned to be fairly subtle. Lukas Sommer has developed this idea some time ago to be ready for use but unfortunately Mapnik did not support 2d fill patterns for line features and while this was added as a feature some time ago this is not yet available in many production environment and especially not in release versions of Kosmtik. And workarounds for this are a bit ugly to implement in the OSM-Carto road rendering system. I have now added this to the AC-style by generating polygon geometries for the unpaved roads using ST_Buffer() and rendering those instead of the normal line features. That is easily possible with the new setup since the line widths are available on SQL level anyway.

The basic styling of the unpaved road rendering used

The basic styling of the unpaved road rendering used – click to see at z18 with and without tagged width visualization

It is important to consider how this looks like in combination with various other road properties – which you can see in the following illustration. There are some issues with the readability of the access restrictions – which is mostly due to their depiction offering very different levels of contrast on the different fill colors. That needs a closer look at and either tuning the access restriction color or revisiting the principal design of this feature in general.

Unpaved road styling in combination with other road attributes at z17

Unpaved road styling in combination with other road attributes at z17 – click to see more

Also here a few examples of practical situations with this styling.

Unpaved roads

Unpaved roads

Unpaved pedestrian area

Unpaved pedestrian area

Open ends

That’s it for the main new road features and improvements i introduced within the new road rendering framework. There a quire a few other tweaks improving consistency in rendering – like getting tunnels and bridges rendered for raceways. And a subtle centerline on runways and taxiways. And as already hinted at there are quite a few open ends in the whole thing design wise that could use further work. But this blog post is already longer than it should be so i will close for now.

Performance of this new roads rendering approach in terms of computing resources required is not bad by the way. While the roads query is often the slowest query of the style – which is not that astonishing because it contains the data previously processed by a lot of different queries – it is not exceedingly slower than other queries. And it is often faster than the water barriers query, which has been the slowest so far and which is why i added an option to disable that for faster rendering.

Technically one important thing left unfinished is that there is currently a lot of duplication of rendering rules between SQL and MSS code. Like for example i have introduced several zoom level selectors in SQL to limit certain queries to specific zoom level ranges (of the form WHERE z(!scale_denominator!) >= 18) which are unnecessarily mirrored by a matching selector in MSS. This is redundant and prone to errors so it is important to come up with some clear principles which parts of the style logic should be in SQL and which in MSS code.

I updated the sample rendering gallery to the style changes by the way and also added quite a few new samples there.

Conclusions on part 2

The main outcome of the whole endeavor of re-designing the road rendering system this way from my perspective is that

  1. yes, there are much better ways to structure rendering of the roads in an OpenStreetMap based map style without taking shortcuts and as a result failing to consistently representing the road topology than the classic OSM-Carto road rendering setup.
  2. The ad hoc implementation of this in the way presented here based on the Carto+Mapnik toolchain is anything but ideal. In particular the difficulty of debugging the SQL code is a major concern.
  3. The map rendering framework that provides a decent platform for implementing the kind of rendering of roads presented here quite clearly still needs to be developed.

My main hope is that what i show here provides valuable additional insights into what exactly the key factors are to be able to develop map design in an efficient, flexible and scalable way (and scalable here is meant from the perspective of style development, not from the side of rendering efficiency – which is an important matter too of course – though pretty much a separate issue). At least to me it seems after working on this project i now have a much better idea of what the core requirements would be for a future map rendering and design system suitable for sophisticated and high quality rule based cartography with zoom level/map scale and styling specific geometry processing.

In terms of the new features implemented it is too early to draw any definitive conclusions i think. Most of them, in particular the variable width ground unit rendering and the lanes depiction i consider to be in a highly experimental state with a lot of open issues – both technically and design wise. The question of what constitutes the core of meaningful OpenStreetMap based cartography for the very high zoom levels (z19+) that goes beyond simple extrapolation of lower zoom level designs plus functioning as a POI symbol graveyard and that does not depend on mappers essentially directly hand drawing the map is still largely undiscussed.

Navigating the Maze

May 11, 2021
by chris
1 Comment

Navigating the Maze – part 1

This post is part of a longer series on map design questions. I have not written much more recently on my work on the alternative-colors style and there is a bit of a backlog of topics that deserve some discussion. I will make a start here with the most heavy of all – the roads. In this first part i am going to explain the OSM-Carto road rendering system and analyze its issues and limitations.

The challenge of depicting roads

The without doubt most complicated part and for any OSM-Carto map style developer the most serious challange when working on OSM-Carto or one of its derivatives is the roads layers. As Paul some time ago compactly put it the roads rendering in OSM-Carto alone is more complex than many other map styles are in total. Roads (and in context of OSM-Carto roads by the way generally refers to all lines and other features along which land navigation of humans takes place – including roads for automobile navigation, paths for navigation by foot, animal or other vehicle as well as railways and traditionally also runways and taxiways on airports, which however have more recently been moved outside of the roads layer system in OSM-Carto) can be complex in their semantics, connectivity and above all in their vertical layering. Representing that in a well readable and accurate way in a map is a challenge to put it mildly.

Most simpler map styles take shortcuts in this field – which tends to show up in map in non-trivial situations. See for example the following case example:

Road example in Dallas, US in OSM-Carto

Road example in Dallas, US in OSM-Carto

Same example in other maps: left side from top: Bing, Esri, TomTom, right side from top: Maptiler, Mapbox, Thunderforest

Same example in other maps: left side from top: Bing, Esri, TomTom, right side from top: Maptiler, Mapbox, Thunderforest

All maps have their flaws in roads rendering – OSM-Carto in particular with the labeling – which however is a different matter that i am not going to cover here. But beyond that it offers by far the most consistent display of the connectivity and layering of the roads in this intersection.

The way this is done in OSM-Carto i will describe in a moment – but before i want to emphasize that OSM-Carto producing a consistent rendering of how the roads are mapped in OpenStreetMap is of high importance for the quality of data in OSM. It helps ensuring that the information that is necessary for an accurate interpretation of the data is mapped and that the reality is not recorded in a distorting fashion to achieve a certain rendering result. More than in any other field of mapping in OpenStreetMap probably the rules of mapping roads have been developed in connection with how the data is used in rendering while still maintaining the principle that what is mapped and tagged represents verifiable elements of the geography and not just prescriptions how things are supposed to show up in the map. This is an accomplishment that cannot be overstated and that is important to maintain and extend in the future.

The OSM-Carto road layers

To understand the way OSM-Carto renders roads it is important to first understand how the drawing order is defined in the toolchain used by OSM-Carto (Carto and Mapnik). In decreasing priority the drawing order is defined by the following:

  1. The layers of the map style: Data that is rendered in the map is queried from the rendering database in the form of distinct and separate layers and is rendered in exactly the same order. The layer stack is defined in project.mml and starts with the landcover layers (at the very bottom) and ends with labels.
  2. The group-by parameter of the layer in question (in case it is present): The way this works is that the query of the layer is ordered primarily by one column of the results table and by specifying this as group-by parameter you tell carto/mapnik to group the features by that parameter and render them kind of like they were in separate layers.
  3. The attachments in MSS code. That is the ::foo selectors. Those kind of work as a method to accomplish layered drawing within a single layer. But in contrast to the group-by parameter every attachment needs to be coded explicitly, you cannot automatically base attachments creation on some attribute in the data.
  4. The order in which the data is queried in SQL (typically through an ORDER BY statement)
  5. The instances of the drawing rules in MSS code: Those are the prefix with a slash you put in front of the drawing rules. Like a/line-width: 2; b/line-width: 4; Such rules lead to several lines being drawing, one immediately after the other, on a per-feature basis.

OSM-Carto contains exactly two layers that use the group-by feature: The tunnels and the bridges layers for the roads. Overall the roads related layers (and other layers in between) look like this (from bottom to top):

  • tunnels (with group-by layernotnull): with attachments ::casing (high zoom only), ::bridges_and_tunnels_background (high zoom only), ::halo (low zoom only), ::fill
  • landuse-overlay (military landuse overlay)
  • tourism-boundary (theme park and zoo outlines)
  • barriers (line barriers like fences, hedges etc.)
  • cliffs
  • ferry-routes
  • turning-circle-casing
  • highway-area-casing
  • roads-casing: with attachment ::casing
  • highway-area-fill
  • roads-fill: with attachments ::halo (low zoom only), ::fill
  • turning-circle-fill
  • aerialways
  • roads-low-zoom: with attachment ::halo
  • waterway-bridges
  • bridges (with group-by layernotnull): with attachments ::casing (high zoom only), ::bridges_and_tunnels_background (high zoom only), ::halo (low zoom only), ::fill

To better understand the terminology used – here an illustration for how the different parts of the road signature are called.

Road signature elements at high zoom levels (z18)

Road signature elements at high zoom levels (z18)

Road signature elements at low zoom levels (z11)

Road signature elements at low zoom levels (z11)

All the individual road line layers (tunnels, roads-casing, roads-fill and bridges) are ordered in SQL primarily by layernotnull (which is the layer tag with an implicit default value of zero assumed) and by z_order – which represents the hierarchy of road classes.

If we ignore most of the non-road layers for the moment and focus on the high zoom levels we have essentially the following drawing order:

  1. tunnels – ordered by
    • layer
    • casing, background, fill
    • z_order (+ some other criteria in SQL)
  2. casings of normal non-tunnel, non-bridge roads – ordered by
    • first turning circles, then polygons, then linear ways
    • layer
    • z_order (+ some other criteria in SQL)
  3. fills of normal non-tunnel, non-bridge roads – ordered by
    • first polygons, then linear ways, then turning circles
    • layer
    • z_order (+ some other criteria in SQL)
  4. waterway bridges – ordered by
    • layer
  5. bridges – ordered by
    • layer
    • casing, background, fill
    • z_order (+ some other criteria in SQL)

This is how this looks like visually:

Layering of linear roads in OSM-Carto

Layering of linear roads in OSM-Carto

That ordering alone however – despite its complexity – does not ensure a consistent depiction of the roads network. That also relies on the specific way the road lines are drawn. To understand this have a look at the following illustration.

Bridge drawing principles with different layers connecting

Bridge drawing principles with different layers connecting

It shows self intersecting road loops as either tunnels or bridges. On the left how this is rendered when mapped as a single way. To actually represent a road crossing over above itself in form of a bridge or tunnel rather than intersecting with itself on the same level you have to split the way into two and assign different layer values.

That the two ways still visually connect where they meet at the top with their ends depends on the way they are drawn with flat line caps for the casing and round line caps for the fill. This leads – as you can see in the rightmost version – to some smaller artefacts when the bridge/tunnel way ends meet at an angle. That brings me to technical mapping rule number one that is good to consider in mapping:

Bridge or tunnel ways connecting at their ends should always have collinear segments where they meet.

The other thing that i hope this explanation shows is the meaning of the layer tag in OpenStreetMap. The wiki is fairly detailed on this but also a bit confusing. What the layer tag does is describing the relative vertical position of the physical object the feature represents, in particular a road, relative to other elements in the immediate vicinity that it is not connected with. It does not define the drawing order in the map (which depends on how exactly the map shows different features), it does not document the vertical position of a feature relative to ground level. This leads to the second technical mapping rule you can take away from this

Use the layer tag sparsely (both in the extent of where you apply it and regarding what cases you apply it in) where it is necessary to avoid ambiguities in the description of the 3d topology of what is represented by the data.

In all other cases there are other tags that will most likely better describe what you want to document – like bridge=*, tunnel=* location=underground, level=*. Or you are trying to do something you should not do (like – as a mapper – drawing the map yourself).

Limitations of this approach

Some limitations of the described system of road drawing are already visible in the samples shown. Others are not and i will have to explain those in the following. Before i do that however i want to mention one additional aspect of the OSM-Carto road rendering system that i have not mentioned so far and that frequently is an additional source of difficulty for developers.

The actual drawing rules for the roads in CartoCSS for large parts serve the different layers together. In particular for example the styling of the roads fill for tunnels, ground level roads and bridges is using the same CartoCSS code. That has the advantage that the different styling variants (see image below) are consistently applied to all the layers. And changes made to that do not need to be duplicated at several places. But it also makes analyzing and understanding the code overall more difficult because after the road features are split into the different layers for rendering them in the proper order these layers are tied together into a meta-maze again for styling.

Different styling variants on all the road classes - many of these can be cross-combined

Different styling variants on all the road classes – many of these can be cross-combined

Beyond that the main principal technical disadvantages of the OSM-Carto road rendering system are:

  • It requires quite a lot of SQL code being duplicated into the different road layers, in particular the tagging interpretation that is being done on SQL level. Some of this has been reduced more recently using YAML node anchors and references – specifically the roads-casing and roads-fill layers as well as the turning-circle-casing and turning-circle-fill layers. That only works when the SQL code is exactly identical though.
  • This of course also means that it requires unnecessarily querying the same data several times from the database. Practically this is not a big issue performance wise but still it is not a very elegant way to do this.
  • Any non-trivial improvements and extensions to be made to the road rendering system typically require additional layers or at least adding to the SQL code that then needs to get synchronized across several layers and adding to the already significant complexity of the whole system.
  • If you look at roads.mss you can find a lot of fairly mechanical repetitions of code to implement the variation of styling parameters across zoom levels. Like the following:


    [zoom >= 13] { line-width: @secondary-width-z13; }
    [zoom >= 14] { line-width: @secondary-width-z14; }
    [zoom >= 15] { line-width: @secondary-width-z15; }
    [zoom >= 16] { line-width: @secondary-width-z16; }
    [zoom >= 17] { line-width: @secondary-width-z17; }
    [zoom >= 18] { line-width: @secondary-width-z18; }
    [zoom >= 19] { line-width: @secondary-width-z19; }

    Often this is even repeated for other layers or variants – like in this case with:

    [zoom >= 13] { line-width: @secondary-width-z13 - 2 * @secondary-casing-width-z13; }
    [zoom >= 14] { line-width: @secondary-width-z14 - 2 * @secondary-casing-width-z14; }
    [zoom >= 15] { line-width: @secondary-width-z15 - 2 * @secondary-casing-width-z15; }
    [zoom >= 16] { line-width: @secondary-width-z16 - 2 * @secondary-casing-width-z16; }
    [zoom >= 17] { line-width: @secondary-width-z17 - 2 * @secondary-casing-width-z17; }
    [zoom >= 18] { line-width: @secondary-width-z18 - 2 * @secondary-casing-width-z18; }
    [zoom >= 19] { line-width: @secondary-width-z19 - 2 * @secondary-casing-width-z19; }

    [zoom >= 13] { line-width: @secondary-width-z13 - 2 * @major-bridge-casing-width-z13; }
    [zoom >= 14] { line-width: @secondary-width-z14 - 2 * @major-bridge-casing-width-z14; }
    [zoom >= 15] { line-width: @secondary-width-z15 - 2 * @major-bridge-casing-width-z15; }
    [zoom >= 16] { line-width: @secondary-width-z16 - 2 * @major-bridge-casing-width-z16; }
    [zoom >= 17] { line-width: @secondary-width-z17 - 2 * @major-bridge-casing-width-z17; }
    [zoom >= 18] { line-width: @secondary-width-z18 - 2 * @major-bridge-casing-width-z18; }
    [zoom >= 19] { line-width: @secondary-width-z19 - 2 * @major-bridge-casing-width-z19; }

    That adds a lot of code volume and is prone to errors. It also makes styling changes that go beyond simple adjustments of the parameters in the CartoCSS symbols used complicated.

On the level of currently present functionality some of the most significant limitations of the current system are (and note these are only those related to the main road layer stack as described – things like road labeling are a different story):

For the polygon features rendered (that is primarily highway=pedestrian and highway=service polygon features) there is no support for bridges and tunnels. That means the above grid example with highway=pedestrian and highway=service polygon features looks like this:

Layering of highway polygons in OSM-Carto (or the lack of it)

Layering of highway polygons in OSM-Carto

Solving that would be possible but you would have to integrate the polygons into all the linear road layers and that would significantly increase their complexity (and the amount of code that needs to be duplicated between them). Same applies for turning circles obviously – although this is a much less significant limitation practically of course.

Another limitation is that while the waterway bridges are correctly layered among themselves they are not interleaved with the road bridges. So a waterway bridge is always drawn below a road bridge independent of the layer tags.

Road layering with waterway bridges in OSM-Carto

Road layering with waterway bridges in OSM-Carto

Practically this is not a big limitation of course (how many waterway bridges are there on Earth where there is a road bridge below?).

What is more important is that for avoiding complexity OSM-Carto has some features not within the road layers that should be included because they can be above or below road features in reality. Specifically that is bus guideways (which have always been separate creating various issues) and airport runways and taxiways (separated more recently) – which is rarely a problem but there are cases where it is (though in most cases of enclosed pedestrian bridges spanning taxiways the bridge is purely mapped as a building and not as a footway).

What i have not covered are the frequent concerns about overall drawing order of roads, in particular road polygons, above other map features, in particular buildings and landcover features. That is a whole other subject and one connected to much more fundamental question about map design and mapping semantics in OpenStreetMap. And it is not something related to the specific design of road layer rendering used by OSM-Carto.

Conclusions on part 1

I tried above to explain how OSM-Carto renders roads and how this system – while being one of the most sophisticated road rendering systems you can find in OSM data based maps – has its limitations. Anyone who tries to implement improvements to map rendering in the style in any way related to roads struggles with those. In the course of my work on the alternative-colors style i have now decided to try re-designing the roads rendering in a fairly fundamental way. How this looks like and what new features and improvements can be achieved with that will be subject of the second part of this post.

April 15, 2021
by chris
2 Comments

On group communication channels

What i want to discuss here a bit is channels and platforms for group communication (that is communication between more than two people) in the context of an open social project like OpenStreetMap. For many years every now and then people turn up on OSM communication channels and express dissatisfaction with either the particular channel or with communication in the OSM community in general – often connected with the suggestion that everyone should move to [fill in your favorite hype of the day platform]. I want to explain here my personal criteria what aspects qualify a communication channel or platform for productive, constructive and inclusive group communication in context of a global multicultural social project like OpenStreetMap.

A few preliminary remarks: First i want to clarify that this analysis concerns group communication in the sense of symmetric communication between more than two participants on equal levels. For one-to-one communication or asymmetric communication (one-to-many with no symmetric back-channel) different criteria apply. Second, a diverse global community like OpenStreetMap depends on there being diverse communication channels used for communication. Concentrating everything on a few channels and platforms would be an issue on its own. OpenStreetMap can only work in a sustainable way if communication and social interaction within the community does not depend on there being a central communication channel everyone engages on. The very idea of OpenStreetMap is that the main database is the element that holds the project together socially and trying to substitute that with some other means would fundamentally weaken the project.

The criteria

  1. Asynchronous communication: In a global, multilingual community like OSM there is inevitably a significant variation in how fluent people are in the language of a specific communication channel. Asynchronous communication (that is communication where there is no expectation, technical need or social pressure for a close timing between a message and reply to that message) can enormously help to level the field in that regard. Not to mention different time zones. Asynchronous communication also helps accommodating participants with intermittent internet access.
  2. Text based communication: For similar reasons text based communication is generally preferably to audio or video communication. Audio or video group communication is of course also almost universally synchronous anyway. In addition text based communication reduces the bandwidth requirements for the participants, makes any kind of filtering massively easier and especially it reduces the participation barrier because it gives participants better control over the content of their communication and the disclosure of private information.
  3. Structured communication: One fairly essential component of group communication with a larger number of participants is transparent visibility of who communicates in reply to whom. The classic way to visualize that is threaded communication. opinions on this differ a lot – some despise the threaded display of communication while others think it is essential for communication in larger groups to work properly. This has a lot to do with digital socialization of people. Those who have become familiar with digital communication without being exposed to the concept of threading often adopt a dislike of this. And with widespread tools and platform (Microsoft, Google) of email communication over the last decade or more by default hiding threading this applies to quite a lot of people. What is fairly clear however is that if a larger number of people participate actively on a communication channel without the communication being structured into threads it will often turn more and more into a synchronous communication channel because asynchronous replies to older communication is perceived to be disruptive by the most active participants of the channel. And this comes with the issues i mentioned above for the asynchronous communication.
  4. Availability of a linkable and immutable permanent public record of past communication: This is one of the most important points. For group communication to be practically meaningful for a larger community it is absolutely essential that you can refer to past communication in conversation in a way anyone can follow and that you can resolve disagreements on the exact nature of past communication transparently. This is easiest through a complete, reliable, immutable, searchable and linkable record of the full history of communication.
  5. Open platforms and communication protocols: For long term sustainability and to avoid dependency on proprietary technology providers. That means in particular the technical interfaces to participate in the communication should be documented and both legally and technically open to be used by everyone without being tied to using specific software.
  6. Practical diversity in communication clients or interfaces: Connected to the previous point – it is of high value if communication protocols are not only open in principle but if there are also different clients and interfaces available that users can choose between according to their individual needs and preferences. Apart from personal preferences and convenience this is also a matter of accessibility – in particular for people with disabilities.
  7. Practically usable flexible and robust ways of filtering communication by the individual user: This is connected to the previous two points. In case of open communication protocols and a variety of communication clients available, one of the most important features those clients tend to distinguish themselves with is filtering options for the communication. And that is of significance on two levels. First it allows people to focus their attention of what they consider important and of interest for them rather than what others consider important. And as a secondary effect it tends to have an immensely civilizing effect on the participants of a channel. If people know everyone else on a channel has the ability to choose ignoring their messages they tend to weigh more carefully what they write and take more into consideration if what they write is likely to be perceived to be of value by others. To put it bluntly: Never underestimate the pacifying effect of a killfile.
  8. Channel is open to everyone and does not require explicit invitation or other access control or discouragement from active participation. Like the required agreement to terms and rules decided on by others than by those who are actually communicating on the channel. This is again essential for diversity in participation. For many people the psychological barrier to actively contribute in a public group communication channel is quite significant so it is important not to create additional hurdles to that. Of course ultimately most, even non-commercial, operators of communication infrastructure ultimately reserve the right to control access to their infrastructure when they consider it necessary but the key here is if that is a possibility that is practically used or not.
  9. Decentralized protocols. This is the ideal solution for the previous issue but practically there are only very few group communication channels that work in a fully decentralized fashion. Historically Usenet was the iconic example of decentralized group communication and since the demise of Usenet there has so far never been any decentralized protocol for group communication that has gained an in any way comparable significance.

And how do different channels compare?

Here is how the various communication channels commonly used in the OSM community fare regarding the criteria mentioned above. I included Usenet/NNTP although his is not practically relevant both in the OSM community and outside of it because is has historically been very important in forming digital group communication culture on the internet and as you can see it also is outstandingly good at meeting my list of criteria. The reasons for the decline of Usenet and NNTP is a whole different story of course (quite an interesting one though – and one that many people have already written on so there is not that much point in touching that here).

The information on Facebook, Telegram and Whatsapp is based on hearsay and therefore not that reliable – i have never used these platforms so i can’t really provide first hand experience.

OpenStreetMap communication channels and platforms

OpenStreetMap communication channels and platforms

Footnotes from the table:

  1. help.openstreetmap.org has limited support for structured communication.
  2. The OSM wiki in principle allows structured communication and on talk pages it is established practice to thread messages. But you have to do this fully by hand and it only works if everyone does so diligently. No builtin support for that.
  3. On Usenet server operators used to expire messages so there was practically no permanent public record in the system itself. However everyone running a Usenet server could maintain a record and in the late days of Usenet Google practically served as a reference record.
  4. The forum indicates when a post has been modified after initial posting but no record of the original message is retained. Also admins on the forum can (and do) remove messages from the record (with varying frequency, depending on the channel).
  5. The diaries themselves can be edited without the history being visible but the diary comments cannot.
  6. On the wiki the edit history is openly accessible so it serves as an immutable record (though it can be difficult to use as such practically).
  7. Github has a fairly clever system of allowing edits of communication but still allowing users to transparently see if there have been changes and what has been changed. Github however also allows completely removing messages from the public record in some cases (but it is indicated at least that a message has been removed i think).
  8. While there is no alternative interface to posting changeset comments for reading there is the popular tool by Pascal Neis.
  9. Openness in participation applies to most OSM related mailing lists and forums. There are a few exceptions – specifically osmf-talk is only open for posting to OSMF members. There are thematic and regional lists and forums with culture specific rulesets you need to accept for active participation which are usually developed through consensus of those active on the individual channel (like diversity-talk, regional forums).

Conclusion

The bottom line is: Mailing lists still rule by a huge margin compared to everything else that is used today in the OSM community. But of course the criteria i used are my personal preferences and although i have explained why i see it this way there are others who view what i see as advantages as major disadvantages. And that is fine – as i pointed out in the beginning diversity in communication is essential for a diverse global community like OpenStreetMap. There are some in OpenStreetMap who dislike that and who would want to culturally homogenize the OSM community and would like to nudge everyone to adopt the same communication platforms. That is not going to work of course. If there is one field where you can be absolutely certain that people will always vote with their feet it is group communication channels.

Lookout

What is interesting is of course to look at what the future might bring. Will there be platforms and channels that can compete with the old school techniques (NNTP and mailing lists) that came out as winners in my analysis regarding the criteria i looked at? At the moment nothing is visible in that direction i think. Focus in development in group communication technology has for quite a few years been in the real time synchronous domain but as i tried to explain above this is of relatively little importance for highly multicultural use cases like we have them in OpenStreetMap. The trend in asynchronous communication has for quite some time been towards monolithic platforms where the client and its user interface are tightly interwoven with the underlying communication protocols (or in other words: Stuff developed around some fancy web interface meant to serve the fashion of the day in UI design). That is unfortunate because it tends to take many years for a highly non-homogeneous community like OpenStreetMap to truly adopt new communication channels en large. And by the time that happens such fashion of the day frameworks have often already reached the end of their short life and are out of fashion and not maintained any more.

March 22, 2021
by chris
0 comments

Satellite image aquisitions – yearly report 2020

I am a bit late this year but i wanted to continue with my tradition to publish a yearly analysis of the recordings of the two currently operating high resolution open data producing earth observation satellites recording in images in natural color – that is Landsat and Sentinel-2.

Superficially similar analysis is sometimes done by the satellite operators as well of course but in most cases this is not very detailed and in some cases even grossly inaccurate by considering redundancies in published data files as independent recordings (which they obviously are not). Here you can find the most accurate analysis of the spatial distribution of the recordings of Landsat and Sentinel-2 based on the actually published imagery.

Development of total daily published image volume in square kilometers

You can already see from the development of the area covered that for the most part USGS and ESA continued with the same pattern as previous years. The biggest anomaly more recently has been that the USGS apparently for unknown reason stopped off-nadir recordings of Landsat 8 in polar regions for about a year (2019 Northern hemisphere Summer and 2019/2020 southern hemisphere summer) but they have more or less resumed with the previous practice in 2020.

Landsat 8 2020

Sentinel-2 operations continued with the changes made in 2019 (that is extended high latitude coverage in the north at low sun positions with Sentinel-2B, some additional small islands) throughout 2020.

Sentinel-2 2020

More important is what has not changed – and that is in those fields i suggested last year already – so i repeat them here:

For both USGS and ESA:

  • make use of the capacity you have in the northern hemisphere winter to collect more than just sporadic data of the Antarctic.

For the USGS in addition:

  • finally get rid of the last gap in your land area coverage (Iony Island).
  • record off-nadir images of northern Greenland and the central Antarctic on a regular basis (and preferably make use of the flexibility you have here with the chosen path for cloud cover minimization).

And for ESA:

  • stop the non-professional and short sighted catering of special interests and record images based on clear, uniform and transparent criteria.
  • connected to that: rethink your strategy of recording extensive ocean areas around some islands while not recording other larger islands (South Orkney Islands) at all.
  • record with Sentinel-2A and Sentinel-2B the same way or explain clearly why you don’t.

And here as in past years all of the maps showing image coverage together:

year day night day pixel coverage
2014 LS8, LS7 LS8 LS8
2015 LS8, LS7 LS8 LS8
2016 LS8, LS7 LS8 LS8, S2A
2017 LS8, LS7 LS8 LS8, S2A, S2B, S2 (both)
2018 LS8, LS7 LS8 LS8, S2A, S2B, S2 (both)
2019 LS8, LS7 LS8 LS8, S2A, S2B, S2 (both)
2020 LS8, LS7 LS8 LS8, S2A, S2B, S2 (both)

March 16, 2021
by chris
1 Comment

How to invent new tags in 2021

Back in 2010 Jochen wrote a practical guide How to invent new tags providing some useful advise to mappers what to consider when invention new tags. Back then however the world of OpenStreetMap was very different. There were far fewer established tags than today and OpenStreetMap was still very young and most people the guidance was written for were new to OSM and to the whole concept of cartographic data collection.

Today in 2021 OpenStreetMap and the typical situation of people inventing a new tag looks very different. There are so many widely used tags that you can hardly any more invent a tag on the green field so to speak without taking into consideration already existing tags in the semantic vicinity of the tag you consider inventing. And most people who get into the position of inventing a new tag have either already been involved in OSM for quite some time and have developed a firm opinion on how things should be mapped and tagged (not always matching how things are actually mapped and tagged) or come from outside of OpenStreetMap and have a firm opinion on how cartographic data collection should look like shaped outside of OpenStreetMap.

Therefore i decided to formulate a new guidance focusing on things that i consider practically relevant when inventing new tags nowadays. This is mostly based on years of observing and studying tags and tag use as an OSM-Carto maintainer and my observations what decides on if a tag becomes a meaningful tag being adopted and consistently used by mappers and also on observing countless tagging proposals over the years. It is not meant to replace Jochen’s guidance which focuses a lot on formal and technical aspects which i will not cover here but to supplement it with updated guidance on semantic, social, cultural and geographic considerations.

Design tags to be practically verifiable

Verifiability is the core of mapping and tagging in OpenStreetMap. While we have a number of poorly verifiable legacy tags in OpenStreetMap (like tracktype) it is a fairly universal rule that newly invented tags that are not practically verifiable fail to become successful tags. They either fail to become adopted by mappers at all (like view) or worse: they fail to develop a well defined meaning in practical use despite being widely used and therefore lead to mappers wasting time to tag information that is practically useless because it is not consistent.

Practical verifiability means that a mapper needs to be realistically able to determine locally on the ground and independent of other mappers or external data sources if a tag is correct or not. In some cases tagging ideas are inherently subjective (like the usability of a road with a certain class of vehicle). Sometimes non-verifiable tags however also result from being invented as a subjective aggregate of several individually verifiable aspects which could individually be tagged in a verifiable fashion (like to some extent natural=desert).

Another distinct class of non-verifiable tags is data that is defined not through something locally observable but through authoritative information from outside of OSM. This for example applies to ratings or restaurants or other establishments. Such data simply does not belong in OpenStreetMap.

One important aspect of verifiability is the practical ability to localize features (and to delineate them when mapping with linear ways or polygons) in a verifiable fashion. Something might verifiably exist but could still not be practically mappable in OpenStreetMap because it is not practically localizable.

Do not invent tags for storing redundant or computable information

Sometimes tags are invented where mappers are supposed to tag information derived from other data in OSM or outside of OSM or that duplicates existing data in an aggregated fashion. Examples are the tagging of the length of linear features or of the prominence of peaks.

Such tags – as convenient as they might seem to data users – are a bad idea because they take valuable time of mappers to compute and aggregate information that could also be automatically computed and there is nothing to ensure that the information is kept up-to-date so it will never be reliable so data users who want to have this kind of information in good quality will always try to compute it themselves.

Be mindful of existing tags when designing new ones

This is most important when inventing new tags these days because as mentioned above there are already a lot of tags with significant use so that you will hardly be able to invent a new tag without other pre-existing tags being close-by. So how do you proceed here? First of all: Don’t rely too much on the OSM wiki for that. Not all tags with significant use are documented on the wiki and the documentation often does not accurately reflect the actual meaning of tags. Taginfo is usually very helpful to explore what tags exist and how they are used. Using overpass turbo (easiest through the links on taginfo) can be very useful to get more specific insight into how certain tags are used. Be aware that bare use numbers are often misleading due to imports and organized edits sometimes strongly distorting the view and creating the impression that a certain tag is widely used while in fact has only been used by a handful of mappers in an organized or automated fashion. The regional taginfo instances operated by Geofabrik and others can be useful here as well.

Like Jochen already pointed out more than 10 years ago it is not a good idea to invent a new tag that overlaps or re-defines an existing tag with widespread use. This almost always leads to confusion and reduction of data quality of both tags. So what should you do if the idea for a new tag you have in mind overlaps with widely used existing tags?

  • If your tagging idea essentially forms a subclass of an existing tag – like you want to create a tag for vegan restaurants – you should turn that into a subtag for the broader established class. A subtag or secondary tag is a tag that is to be used in combination with another primary tag and develops meaning only in this combination. In this case such a tag already exists of course, it is diet:vegan=yes|only. It can be used in combination with amenity=restaurant but also with other primary tags like amenity=cafe, amenity=fastfood or various types of shops.
  • If your tagging idea overlaps with an existing tag but is also meant to cover things that are not included in said tag you have three options:
    • extend the definition of the existing tag to include what you want to map. This needs to be done with care to avoid devaluing the existing data this tag is used for. If existing data with that tag consistently does not include the kind of thing you want to map and what you want to map is well distinguishable from the existing class of features it might be better to
    • split what you want to map into two separate classes of features – one you map with a subtag to the pre-existing tag and the other with a new primary tag.
    • You can also try to introduce a new tag with the intention to replace existing tagging. This however rarely works if existing tagging is already actively used by a lot of mappers. Most attempts at doing this result in less consistent mapping because they introduce several competing tagging ideas with neither of them universally favored over the others – a development nicely illustrated here. So you need to be very careful with choosing that route. See in particular also the last point below.

Make sure your tag has a positive definition

The success of a new tag also depends on it having a clear positive definition. If you’d for example invent a new tag shop=non_food that lacks a positive definition because it is defined by what it does not sell – namely food. Such tags often turn out to be very broadly and inconsistently used and they tend to discourage the invention of more specific, positively defined tags at the same time. For example someone might tag a museum shop=non_food because it sells tickets to the museum and that is definitely non-food.

Make sure to use a fitting and non-misleading name for the tag

Try to make sure that the key and value you choose for your tag do not have either a more specific or a broader meaning in a native English speaking area. Well known examples where this turned out to be a problem are leisure=park and landuse=forest where the meaning of the tags does not encompass everything that is sometimes referred to in English with the terms park and forest.

This is hard and sometimes poses problems that are not ultimately solvable and you might have to accept an imperfect solution here. But you should try to avoid major problems by evaluating the options for choice of key and value carefully.

Formulate a language independent definition

That might sound like a contradiction because the suggestion to formulate implies use of language and at the same time it should be language independent. How is that meant to work?

The key here is to have a definition that describes the meaning of a tag without referring to culture and language specific terms for that definition. For example you should not define the meaning of natural=beach with a beach. A decent definition for example would be: A landform at the edge of sea or of a lake consisting of unvegetated loose material in grain size between sand and large boulders that is shaped by water waves. Instead of referring to the poorly defined English language term “beach” that many people will associate very different pictures with that definition uses generic terms less open to subjective interpretation (like unvegetated or water waves) to define the use of the tag. While translating the word beach into different languages will result in differences in definition due to semantic differences in the translation of the term translating the more abstract definition will be more accurate and less likely to result in a different interpretation. Being elaborate and possibly even redundant in the definition, delineating the meaning in different terms, can further help with that.

Do not define your tag by referring to Wikipedia

Some mappers try to address the previous point by referring to Wikipedia or copying text from Wikipedia. That is not a good idea. Wikipedia has a completely different perspective on documenting knowledge about the world than OpenStreetMap. Wikipedia documents the commonly accepted meaning of terms based on outside sources. That is (a) bound to change over time and (b) is often self contradicting because different parts of society have different ideas of the meanings of certain terms. We in OpenStreetMap however depend on having a single, globally consistent and locally verifiable definition of tags.

Avoid aggregating unrelated and semantically very different things into a common tag

Tags are meant to semantically structure the geographic reality as we document it in OpenStreetMap. Such a structure is most intuitive if the individual tags are semantically compact or in other words: If in the eyes of the local mapper two features with the same primary tag in the vast majority of cases have more in common than two features with different primary tags. That is not always universally possible. Tagging means classification and classification inevitably requires drawing lines between classes separating things that are semantically similar. A good example are the tags natural=scrub and natural=heath which are distinguished by different heights of scrubs. A tall natural=heath and a not very tall natural=scrub in similar ecological settings might be semantically closer than two very different areas of natural=scrub in different parts of the world. Still natural=scrub is narrow enough in its definition so mappers will intuitively see the semantic similarity of two areas correctly tagged as such no matter how different they are.

I can also formulate this advice in a different way: A newly invented tag should not need to be delineated to too many other tags. Taking again natural=scrub. That needs to and is delineated towards natural=heath and natural=wood regarding the height of the woody plants. It is also delineated towards landuse=orchard for scrubs cultivated for agricultural purposes.

Respect OpenStreetMap’s mapper-centric approach

Tagging in OpenStreetMap is intentionally designed for the ease and clarity from the side of the mapper and not for the usefulness for certain applications of data users. It is therefore not advisable to try designing new tags if you are not using these tags yourself as a mapper or if your main motivation is that you would like mappers to use these tags because of your needs as a data user. Ultimately mapper-centric tagging conventions are in the long term of benefit for both mappers and data users because they ensure high quality data – even if in the short term they sometimes mean additional work for data users to process the data into a form that they need for their application.

Be mindful about geographic diversity

OpenStreetMap in contrast to most other efforts of collecting cartographic data, like for example by public authorities, tries to produce a map of the whole planet. That means dealing with the challenge of an immense geographic diversity around the world. Obviously not every tag is applicable everywhere on the planets so we do and we need to have tags limited to certain settings in application. But it is important to not do so without necessity. Definitions of tags should be practically suitable for all geographic settings. Think for example about the distinction between landuse=farmland and landuse=orchard. That distinction should ideally not be based on a fixed list of products grown (which will inevitably always be incomplete) but on an abstract definition that allows making the distinction for any plant grown on the planet.

Be prepared for failure

This might seem like kind of a pessimistic tune to end this advice with – but it should not be read as such. Everyone – no matter how experienced and knowledgable – should be humble when inventing new tags and be ready to accept that what they suggest might fail. OpenStreetMap’s open tagging system is intentionally designed to deal with failure – with tags being introduced and being used but over time being abandoned in favor of other tagging systems or (more frequently) tags being introduced to replace existing tagging schemes to fix some issues with those but never being broadly adopted because mappers stick to existing tagging. That this happens is not a fault in OpenStreetMap, it is intended.

How tagging in OpenStreetMap looks like is a bit like the geography we map looks like – organically grown, full of flaws, but still largely functional. Yes, you could design a city from scratch on a drawing board to be perfect by some ideal of the perfect city. But it would be really hard to build it and it would be even harder to persuade people to live in such a sterile environment. And even if you solve that problem you would still over time realize that this perfect city is not as robust and adaptable as the organically grown ones. It would be over-optimized to the specific conditions taken as granted by the designer and if those conditions change (for example environmental or cultural factors or demographics) the whole design would be in peril while the organically grown city with all its flaws could organically adjust to changing conditions much better.

January 28, 2021
by chris
0 comments

Snow in Europe

Compared to the previous years this winter seems to turn out to be relatively snow rich in many parts of Europe. On this occasion here a few satellite image impressions of that.

Snow in England on January 25

First a view of snow in the lower parts of much of England – which is not really rare but still not a very common impression either.

Snow in Spain on January 12

Then what is really unusual is the recent extensive snowfall and snowcover over large parts of central and northern Spain including the City of Madrid as shown in the second picture here.

Snow in Greece on January 20

Much more common than in Spain snowfall is in the eastern Mediterranen. I have showed a winter view of the island of Crete before.

Above view shows a recent impression of Greece and the Aegean Sea – featuring snow in many mountain areas including many of the islands, in some cases reaching down to sea level like on the island of Limnos – here a more detailed view of that.

Limnos with snow on January 20

First three images are based on Sentinel-3 data, the last one is created from Sentinel-2 data.

December 16, 2020
by chris
0 comments

Iceberg A-68 meets South Georgia

The large iceberg that has broken off the Antarctic peninsula in 2017 and that has slowly drifted across the Drake Passage during the past year – i showed some pictures from the beginning of that journey already – has now reached the island of South Georgia which results in impressive pictures demonstrating the enormous size of the iceberg. Like this view from Sentinel-3B from December 14:

A-68 meets South Georgia in December 2020

Here the locator map:

The Iceberg might become grounded in shallow water near the island and could this way significantly affect the ecosystem in the area. Note however that while an iceberg of this size is fairly rare, smaller yet large icebergs are very common around South Georgia because currents and winds in the area lead to a large number of icebergs from the Weddell sea area to drift north along a similar route as A-68. So with a long term historical view such a visit of a very large iceberg of more than a hundred kilometers in size near South Georgia is most certainly not unprecedented, even though previous similar events were not recorded in as much detail by earth observation satellites of course.

November 14, 2020
by chris
5 Comments

The OSMF – changes during the past year and what they mean for the coming years – part 2

In the first part of this blog post i summarized the most important developments in the OpenStreetMap Foundation (OSMF) during the past year from my perspective. In this second part i will – based on the observations from the past year and recent trends being visible – give a lookout on how the OSMF could develop in the coming years.

After my previous piece on the OSMF and the perspective for the upcoming general meeting, Severin Menard has published an interesting and likewise critical take on the current situation of the OSMF. I agree with most of what he wrote and his culturally somewhat different perspective on things is very valuable. Definitely recommended for everyone to read.

Corporate takeover – it has already happened

There is one thing however i disagree on with him – the assessment of a corporate takeover as a risk of the future. During the past weeks there have also been some half baked initiatives started (last minute before the general meeting as usual) from within the OSMF board towards resolutions meant to protect the OSMF against external takeover. My view is that these measures and the focus on a defensive strategy against an external attack aimed at controlling the OSMF are meanwhile setting the wrong priorities because the corporate takeover has essentially already happened behind the scenes without that being clearly visible to the outside.

Large corporate OSM data users are not really that interested in staging a coup in the OSMF and run the OSMF themselves at this time. That would be expensive to do and bear a large spectrum of fairly big risks and also it would – if successfully executed – immediately lead to a struggle for control between the major corporate players. The main goal of corporate actors with the OSMF is and has been for some time to prevent meaningful regulation of corporate activity in OSM and of OSM data use based on the OdbL. If that goal is secured, a secondary goal would be for the OSMF to serve as a shared neutral intermediary platform between the different corporations to steer independent volunteer activity in the OSM community in the corporate data users’ collective interests.

The parts of the OSMF board not affiliated with corporate or organized interests seem to have, during the past years, developed the idea that these corporate interests can be negotiated with and that the future of OpenStreetMap lies in compromising between the organized and corporate interests and the core ideas and values of the project. This quite clearly is an illusion – expecting a big corporation like Facebook to make compromises with an insignificant player like the OSMF is at best naive.

If the goal to prevent meaningful regulation of corporate activity through the OSMF has been accomplished permanently, corporations have no reason to oppose effective takeover prevention of the OSMF – because that would help protecting their position in the OSMF against third parties gaining influence. And effective prevention of meaningful regulation does not even depend on there being a pro-corporate majority among the OSMF members because the corporations have other significant channels of influence in the OSMF now (through their financial constributions, through their participation in the working groups and through lobbying of the board on non-public channels).

We will in the near future have a fairly good test case for how robust the ability of corporate interests in the OSMF is in preventing meaningful regulation in form of the attribution guideline that has been worked on during the past years. There are essentially three scenarios of what could happen:

  • The OSMF board decides to adopt a guideline roughly based on the corporate wishlist from the LWG with some minor adjustments for the optics, to maintain the impression that it is not directly what corporate lobbyists have written. That seems the most likely scenario at the moment but it bears the strong risk that the craft mapper community will openly oppose this interpretation of the ODbL which would fundamentally endanger the OSMF’s position in the OSM community.
  • The OSMF board adopts a guideline reflecting the community consensus reading of the ODbL (likely similar to what i drafted) that unconditionally requires attribution that practically makes the user aware of the origin of the data. That would be strongly against the corporate interests and corporations would certainly do everything within their power to prevent that (including withdrawing funds which would leave the OSMF in financial peril because the strongly increased costs make it depend on regular corporate contributions).
  • A decision on the matter is avoided by dragging out the process indefinitely. Although not ideal, because it would be a kind of unstable situation, this would be acceptable for the corporations because it would maintain the status quo of the OSMF not becoming active against data users with insufficient attribution. It would also avoid an open break with the OSM community, although there would be likely increasing pressure from the OSM community on the OSMF to get active in cases of insufficient attribution by the OSMF’s corporate financial contributors.

Some will likely reject my idea that the corporate takeover of the OSMF has already happened. They will argue that if that was true, corporations would push much more aggressively for their interests. I don’t think that is the case though. As explained, the primary interest large corporations have in the OSMF is not positively accomplishing something, it is preventing things negative for their interests from happening. Accomplishing that is much more valuable for them than anything they could proactively try to push for in the OSMF.

What we will certainly see in the coming years is corporations trying to consolidate their influence on the OSMF, in particular by more and more corporate employees being encouraged to volunteer and being paid for work on the OSMF in the working groups, on the board, in committees and other ways. This will happen rather quickly in most of the working groups probably, supported by the move of the OSMF to position itself more like a corporate actor which, as i have explained before, is likely going to have a negative effect on motivating volunteers without career interests in OSM for the OSMF. My estimation is that in 1-2 years a solid majority of the people engaged actively in the OSMF in one way or the other is either employed in some form in an OSM related job or otherwise has a carreer interest at least partly motivating their involvement in the OSMF. Most of them will be employed by corporate OSM data users or organizations around OSM like HOT. Currently in the OSMF board for example we have already at least three members to whom this applies, two corporate employees (Mikel and Paul), one small business employee (Rory).

Centralization of the OSMF

The other big upcoming trend in the OSMF i can observe is an increased centralization of power towards the board. The OSM community overall is highly decentralized and the OSMF has always been kind of an abnormality within that with its hierarchical structure. But traditionally in the OSMF most of the work has been done in the working groups – including development of policies – and the working groups had a high degree of independence, starting from being created by grassroot initiative from within the community, to allowing also non-OSMF members to participate and to by convention allowing board members to contribute, but not allowing them to have a lead role. We more recently, however, see a trend towards more and more policy being either actively influenced by the board or being developed by the board itself from the start. Early manifestations of this trend were the already mentioned Crimea decision (where the board overruled standing policy developed by the working groups) and the organized editing policy (where the board flat out rejected the first draft of the DWG and demanded a more lenient policy). This year we saw a large number of cases where the board created internal policy, often presenting the results as a done deal without having an open discussion – like in case of the diversity statement – further emphasizing this trend.

In addition, as i have discussed in the first part, we saw during the past year the establishment of several committees (a concept that previously did not exist in the OSMF) – put together by and under direct control of the board. And for this year’s general meeting we have an AoA change proposed that allows the establishment of committees consisting of board members and OSMF members and to delegate any powers of the OSMF board to these. In other words: It would allow the OSMF board to recruit volunteers or paid staff (paid by either the OSMF or by third parties) and delegate any kind of function of the board to them. Obviously, such committees would be under immediate and absolute control of the board, people could become members of such committees only by appointment by the board and the board could dissolve or remove powers from such a committee at any time. But, in contrast to the working groups – which don’t have any formal powers and depend for any meaningful decisions on approval by the board – the commitees could be equipped with any of the formal powers and rights the board has within the OSMF.

The effect the establishment of such committees would have is a massive shift in power within the OSMF from the working groups towards the board. Currently the board is essentially limited in what they can do by their numbers. Board members are not able to delegate their formal powers to others. Work requiring more hands can only be done by the working groups, which have a high degree of independence. If the mentioned resolution passes, the board would essentially no more depend in any way on the working groups of the OSMF – they could assign any tasks so far in the remit of the working groups to the committees they appoint and control without any restrictions.

We can already see right now that the board is increasingly trying to take direct influence on what the working groups do and how they work. In public communication they have expressed an interest in reshaping the remit of the CWG and to reactivate the EWG (which – since the EWG is completely inactive right now outside Google summer of code management – amounts to nothing less than bootstrapping a new working group, something the current board at the beginning of the current term still considered inappropriate).

An interesting test case for the tendency of the board to seek direct control over what happens in the OSMF is the establishment of the planned software dispute resolution panel. The DWG has expressed interest in this task but it seems likely that the board will prefer to directly control the composition of this panel.

What is not visible is any indication that this trend in centralization of power in the OSMF is balanced in any way by more independent oversight over decisions and processes within the OSMF. The idea, for example, that the local chapters could gain some kind of meaningful power in the OSMF’s structure and processes, or that the OSMF could share control over key elements of the OSM infrastructure (database rights and the contributor database) with the local chapters, is not in sight. Given how keen the current board seems to be to gain more immediate control over everything within the OSMF, it seems unlikely that the board will be willing to relinquish any power voluntarily any time soon.

Decreasing diversity and brain drain

Another trend that has already been visible for quite a few years is the increasing difficulty for the OSMF to attract competent and committed volunteers. As i have pointed out in context of the plans of the OSMF of paying people for work, this will likely have a significant negative effect on volunteer motivation.

Now, if you see this problem only as a numbers game, it is not unlikely that it is possible to fill this deficit with paid people (paid either by the OSMF or by external interests) or by people which view volunteering in the OSMF as a career builder. As indicated above, this development is already in progress. There are a number of effects this is going to have:

  • Parts of the OSMF (working groups, committees) will become increasingly dominated by culturally narrow circles of people with shared interests (like their careers or shared interests of outside organizations they are affiliated with) – interests which are distinct from the funadamental goals and values of OpenStreetMap. This process is going to be self emphasizing because once this happens the working group or committee in question becomes increasingly unattractive to anyone outside this narrow spectrum who does not share the interests of the group – even if that group of people considers themselves to be open and welcoming to others.
  • It will probably be in particular the deeply committed OSM community members who have many years of experience in the project for whom volunteering in the OSMF will become increasingly unattractive and who will likely seek to contribute in functions outside the OSMF. That means there is likely to be a quite significant brain drain of competency and experience.
  • The de facto goals of the OSMF will increasingly drift away from the goals and values of the OpenStreetMap project to the special interests and culture specific values of those who happen to be dominating the organization. From within the OSMF and its communicative echo chamber it will potentially not be so visible – the prevailing opinion on the OSMF board often already seems to be that that OSMF’s and OpenStreetMap’s goals are neccesarily identical, in an L’état, c’est moi kind of way.

Seeking influence on OpenStreetMap

As a result of what i wrote in the previous sections it seems likely, and there are already indications for it, that the OSMF is going to increasingly try to take influence on the OpenStreetMap project itself, getting in conflict with the basic premise of the OSMF of supporting the project but not controlling it. The fields in which this is likely going to happen are in particular:

  • Community communication channels: One prevailing narrative from within the OSMF more recently has been that “fragmentation of communication” in the OSM community is a big problem that needs to be addressed. That terminology is in itself interesting by the way – the same thing, when considered positively, is called diversity, when deemed negatively it is framed fragmentation. But that just as a side note. We can expect there to be strong pushes from within the OSMF to try taking tighter control over communication channels hosted by the OSMF by imposing behavior regulation. If these attempts will be successful will need to be seen. What can be said with near certainty is that if that happens it would have the opposite effect of what it claims to want, it would more strongly fragment the communication within the OSM community by essentially squeezing out those who do not want to subject themselves up-front to a narrow culture specific communication rule set in their communication and who would move to communication channels outside the control of the OSMF.
  • The OpenStreetMap website: Traditionally the OSMF has no say in the design of the OpenStreetMap website – it is developed in the classic OSM tradition of do-ocracy and consensus. But there have been increasing voices from within the OSMF expressing the desire to less prominently feature the map as the symbol and the main instrument of inter-cultural cooperation in the OSM community and re-design it more like a corporate website and more proactively communicating the OSMFs interests to the visitor. We will likely see a move in that direction in the coming year – if that is going to be successful is not sure in my eyes though.
  • Mapping and tagging: The OSMF board has last year already made a move that could lead towards taking an influence on mapping and tagging in OpenStreetmap, something that is in principle strinctly forbidden by the mission statement of the OSMF, through the plans for a software dispute resolution panel. The primary purpose of the panel is resolving conflicts w.r.t. development decisions of the iD editor which are primarily conflicts about tagging presets and validation rules. Deciding on such conflicts (which is what such a panel would need to do) would inevitably amount to making tagging decisions. The OSMF of course has no means to enforce such decisions currently beyond their influence on the iD development, but still, the potential of attempts in that direction is definitely there. As i have explained above, a secondary interest the financiers of the OSMF have is steering the craft mapper community into a direction beneficial for their data uses. Members of the current OSMF board have already expressed that they deem usefulness of the geodata collected in OSM as the primary goal of the project – in contrast with the traditional goals and values of the project which put in foreground community cohesion. And if you assume usefulness of the data to mean usefulness for the economically important corporate data users’, interests of the corporations and goals of the OSMF board seem to align here. Still, in contrast to the previous two points where clear trends and already visible, on this matter i would not make a clear prediction.

Conclusions

Now this outlook onto the next years is obviously rather bleak and, indeed, my view of the near future of the OSMF is fairly dark. For OpenStreetMap in general i see this, however, as a valuable chance to emancipate itself from the OSMF a bit more – which, no matter where the OSMF is steering, would be a healthy thing. And i also want to point out a few chances i see for the OSMF in the near future. As i will explain, the likeliness of these actually happening is quite small – yet i find it important to show that the OSMF is not inevitably doomed, but that what happens still depends on the people in positions of power making responsible decisions and there is still a possibility for developing into a much more positive direction. Two examples:

  • With the Brexit and the possibility that a move of the OSMF from the UK to the EU is advisable, there is a real chance in the context of such a move to restructure the OSMF from the highly hierarchical and centralized form essentially required by British company law, into a federated organizational structure with checks and balances and meaningful subsidarity rules, as it would be appropriate for an organization within the highly decentralized OSM community. This is, unfortunately, highly unlikely to happen even if a move of the OSMF takes place, because it would essentially require the OSMF board to decide to give away much of its power to more federated authorities in a new organizational structure. But the possibility is there – there are no meaningful principal hurdles against implementing such a change beyond the general difficulty of moving the organization in the first place.
  • The OSMF has the chance, due to the strongly increased public visibility in the last years, to gather the necessary means (and i mean this independent of contibutions of larger corporate financiers with either explicit or implicit strings attached) to return to and to concentrate on positioning itself as a neutral infrastructure provider for the OSM community. With that i mean to – like a state providing roads and other infrastructure to its citizens – provide neutral infrastructure for everyone to use without discrimination. This idea is not the same as desiring a very small OSMF. Infrastructure can be extensive and expensive. If the OSMF would want to offer each local community around the world the infrastructure to design and run their own real time updated map for example, the costs of this would be fairly massive but at the same time it could be highly beneficial for supporting cultural diversity in the OSM community. The key here would be neutrality and non-discrimination. Much of what the OSMF currently does in new money spending is not – the already cited idea to financially support people whose work we know and enjoy is fundamentally incompatible with this idea. Unfortunately, like in the previous point, this vision would require restraint from the OSMF board to resist any urge to govern and actively steer OpernStreetMap in a direction they deem desirable – something i do not see being likely to happen with the current board.

As i wrote in part 1 of this post, i present my prediction for the direction in which the OSMF is headed here to be proven either right or wrong by what will actually happen. And i would be happy to be proven wrong – either by what the future will actually bring or beforehand by convincing arguments brought up by those who see a different development coming. In other words: I openly challenge anyone who disagrees with my analysis to present and argue for their own predictions in public.

One thing the OSMF board seems to have lost over the past year is the willingness to expose their vision and their plans and ideas to an open discussion and try to convince the OSM community of the merits of their ideas in open discourse. Probably at least in parts because with the money the OSMF has, it seems easier to just buy the implementation of your plans instead of engaging with and convincing the community to support you.

And i call to everyone in the OSM community to not accept this. Whether you have the feeling the OSMF is going in the right direction or not, whether you agree with my analysis here or disagree: You should require people in the OSMF to present arguments and reasoning that convince you of the merit and the solidity of their plans and their actions and require them to present a meaningful vision and expectations for the success and outcome of their actions which can be tested against the actual future development just like my predictions. What is best for OpenStreetMap is not the median of all interests articulated, weighed with the amount of money behind them. What is best for the project can only be decided through arguments and reason.