Imagico.de

blog

August 4, 2018
by chris
7 Comments

On the discussion on OSMF supported vector tiles maps

After my initial report on the SotM conference in Milano there are a few things i would like to discuss in more depth. The first one is the vector tiles on osm.org topic.

A bit of background first: The term vector tiles has in the OSM community kind of been a white elephant for the past couple of years. I think over the years vector tiles have been proposed as the magic solution for just about every problem in community map production that exists.

When used with actual meaning and sense and not just as an empty buzzword the term is used for two fairly different things:

  • for the idea of caching the results of queries in a tiled map rendering framework. In a classical rendering system like OSM-Carto the map tiles are generated based on a number of queries of the data from a spatial database (usually postgis) for the area of the tile rendered. The results of these queries are thrown away right after the tile has been rendered. By caching the query result you can much more efficiently render different styling and tile variants – like different labeling languages, different color schemes or different resolution tiles.
  • for the idea of tiled rendering of the map on the client (web browser) instead of the server based on tiled vector data. This has similar advantages as in the first concept but in particular it allows the map provider to outsource rendering and supplying the computational capacity for it to the map user.

As you can probably imagine there are technological similarities between implementations of these two approaches so use of the same term for both of them is not without basis. But it is always important when you talk about vector tiles to make clear which of these two ideas you are talking about.

The first approach described above is fairly non-problematic. It is widely used in maps produced today without this necessarily being visible for the user. Of the maps available on openstreetmap.org several are using these methods and there also exists a port of OSM-Carto that uses server side vector tiles. The second approach however is more tricky because for this to work without serious performance issues the vector tiles transferred to the client must not be much larger than raster tiles. And this is extremely hard and practically it is almost never achieved. For one specific rendering task, the plain color rendering of polygons, i discussed this in more depth some time ago. What map producers currently do to work around this problem is using massive lossy vector data compression. And compared to lossy raster image compression where decades of research went into the methods used and we have have specialized methods for all kinds of specific applications (like raw photo compression) these methods are relatively crude. You can see that in most maps based on client side rendering where the appearance of the map is usually primarily defined by the data volume reduction needs and not by cartographic requirements and considerations and a significant part of the information communicated to the user by the map is compression artefacts rather than geographic information.

So much for general background. What i now want to discuss and comment on a bit here is the proposal from Richard Fairhurst to initiate a project to provide vector tiles for client side rendering (the second approach above) on OSMF infrastructure. There are a few blog posts and messages related to that and there was a breakout session at the SotM conference about the topic.

The alternative-colors style – more eccentric and less purple than OSM-Carto i guess

In general and as i said before i support the initiative to increase the diversity in community created maps available to OpenStreetMap users. But due to the white elephant nature of the topic there is a lot of blind enthusiasm surrounding this that can easily lead to ignoring important things. I want to point out a few of them here – some of them i have already mentioned at the SotM meeting, others are new here.

1) It is currently fundamentally impossible to fulfill all of the functions of the current standard map style with a framework based on vector tiles for client side rendering. This might not seem overly relevant because OSM-Carto at the moment unfortunately all too frequently ignores the requirements of these functions as well (see here) but the fundamental difference is that with OSM-Carto this is a (bad) choice that can easily be changed, with vector tiles for client side rendering this would be much harder.

Most at the meeting at SotM were aware of this limitation but i fear that quite a few people ultimately have a strong desire to push this approach at all costs and just agree to this in an attempt to pacify potential opposition while still believing the white elephant to be the solution to any and all problems. So i will repeat in all clarity here: The client side rendered vector tiles approach is currently fundamentally incapable of generating maps that fulfill all of the core functions of the OSM standard style and there are no technological innovations visible at the horizon that are likely to change that.

2) It would be important IMO to consider and discuss the question if the OSMF should actually become active in this field at all. If you look at the OSMF mission you can see that providing maps for all kinds of purposes is not part of that mission. The fact that the current standard map style fulfills important functions for the OSM community and for the public image of OpenStreetMap, is generally accepted (although you can of course argue how well it actually fulfills these functions). So i think a new, additional map rendering project would – to justify receiving significant support from the OSMF in the form of computer infrastructure – have to state what important functions it aims to fulfill and demonstrate that it is capable to do so within the spirit of the OSMF mission.

There seems to me a significant likelihood that such a project from a user perspective could end up being more or less a knock-off of the OpenMapTiles project or similar and in that case it would IMO be fair to ask if this would warrant the OSMF investing resources in such a project.

3) Paul Norman in the discussion mentioned an important point: The number of developers in the OSM community capable and interested in productively working on map rendering and map design projects in general is fairly limited. This means people will ultimately vote with their feet. This is similar to what i wrote about OSM-Carto last year: The question that will most likely decide on the success of such a community project (no matter if run on OSMF infrastructure or not) is if it can successfully attract competent and committed developers capable of actually achieving whatever goals the project started with.

Not that it matters that much (since i am not very qualified to help kicking off such a project anyway) but i have at the moment rather limited interest in this project myself because my interest in community maps is primarily in those areas for which as said the client side rendering approach is currently unsuitable for. But this is not set in stone and it is quite possible that there are aspects of such a map project that could turn out to be interesting for me as well.

So far for the specific plans for an additional map rendering project on osm.org which as said as an increase in diversity in community maps i would support. But i also want to put a bit of a different perspective on the topic of the future of OSM community maps in general:

Richard’s blog post starts with an accurate analysis of what makes OpenStreetMap different from the big corporate players. The visualization or map rendering part of OpenStreetMap – the standard style – has historically been an important part of OSM becoming a very distinct player in the field. OpenStreetMap would probably not have developed into what it is today if its main visualization platform would have aimed to be a Google Maps lookalike. Instead the standard style as i pointed out before has been pushing the boundaries of what is possible technologically and cartographically in quite a lot of things for a large part of its history and in many aspect in a very different direction than where Google and other commercial map providers are going. And it seems to me that this has a significant part in OSM being able to concentrate on its core values and not following short term trends that seem to promise a quick way to success. That this has diminished more recently in OSM-Carto development is something most people more intensively involved with OSM feel and which certainly is a huge part of what motivates people now betting on and pushing for the white elephant so to speak.

But trying to address this problem now by crawling under the wings of Mapbox or other big corporations and making OSMs public image fully dependent on technology developed and controlled by corporate players would in my opinion be a big mistake. Open Source or not – the past experience with Mapnik and Carto has quite well demonstrated i think that OSM currently lacks the resources and expertise to develop or to organize development of a map rendering framework for the specific needs of the project independent of either the big corporate OSM users or the broader FOSS community. OSM will either need to invest into improving capabilities in this field within the project (which is not that feasible because OSM as a project does not have the level of organization or resources for that) or it would need to reach out more to the FOSS community to foster independent development of map rendering tools for OSMs current and future technological and cartographic needs. Projects in that field already exist (like Mapserver for example) but they are currently mostly developed for smaller commercial users and public institutions. Getting these projects to include OpenStreetMap community maps as an important use case or initiating new projects for OSMs needs together with the independent FOSS development community would be a practical approach that could ensure an independent future for OSM in terms of community maps.

And yes, vector tiles (in the generic sense described above, less in the sense of a specific file format the specifications of which are under control of Mapbox) could likely be a part of such developments. But they are not the white elephant many hope for.

August 3, 2018
by chris
9 Comments

SotM Milano – a summary

I have returned from Milano (from a warm northern Italy to a similarly warm southern Germany) and completed viewing most of the talks i missed at the conference that were recorded and that i did not get around attending. Based on that here a quick summary. I might cover some more specific topics in separate posts later.

First a bit of statistics based on the attendee list – which is not completely reliable because it does not exactly represent who was at the conference and because the Company Name is just a free form field. Note to the SotM WG: Please don’t provide the attendee list as a ridiculously convoluted PDF. This won’t prevent Google from actually harvesting the data in there and it makes this kind of analysis much more difficult. I also would consider it very useful if in the future you would during registration ask people for a bit more information on themselves for statistical purposes which could provide a lot of useful insight into the visitor structure of the conference.

There were 355 per-registered attendees according to the list of which 209 have a company name specified (after removing a few obvious errors interpreting the field incorrectly). That is about 60 percent. As said this is not really a reliable indicator but it is clear from it that the majority of the attendees were visiting SotM as either part of their job or their visit being paid by an organization.

The companies/organizations with the largest numbers of attendees were:

Telenav: 8
Facebook: 7
Mapbox: 7
Microsoft: 5
Grab: 5
Heidelberg Institute for Geoinformation Technology: 5
HOT: 5
Politecnico di Milano: 4
MapAction: 4

The geographic distribution of the attendees was as follows:

Naturally the countries with short travel distance brought in the largest number of non-corporate, non-organization visitors. Of the 66 visitors from Italy only 30 have a company name specified. Of the 58 from Germany it is 25. For the United States on the other hand it is 35 of 47. As said the accuracy of these numbers is not very good but overall it seems quite clear and understandable that when coming to the conference requires a long and expensive journey this significantly reduces the likeliness that a hobbyist community member will come. This seems to be confirmed in conversations because when talking to people from outside Europe most seemed to have either some business connection or are involved in some project that goes beyond a hobby.

The scholarships

There would probably be quite a lot to be said about the scholarship program but so far we seem to have no information on the scholarships beyond what can be found in the program booklet which lists the names of 17 OSMF scholars.

The program

As i already wrote in the pre-conference post the program was not really of particular interest for me. There was no talk i considered a must see and after looking over most of the talk recordings this seems to be confirmed. This absolutely does not mean the talks were bad or that they were not interesting for me – not in the least. But i did not try to watch as many talks as i could but instead spent more time talking to people. This is a bit of a dilemma of course since listening to talks can also be a good starting point for approaching others and starting a conversation.

Since not all the rooms were recorded on video this also meant that i missed a few of the talks without the opportunity to watch them afterwards. I however hope there will be a more or less complete collection of slides available for all the talks – if you gave a talk and have not yet sent the slides to the organizers please do so.

Meeting people

As already indicated meeting and talking to people was my main goal for the conference. There were good opportunities for that although with more than 350 people there were also plenty of cases where you failed to meet someone for the whole three days because you just never really ran across each other. One thing that worked amazingly well was being introduced to others by someone who already knows both people. Christine Karch in particular seemed to be very industrious at that. This is something i can very much recommend to others at such conferences – if you are interested in meeting someone but you are either reluctant to simply walk up to them or you just can’t find them because you don’t know how they look you can just ask someone who knows both of you to make the introduction. Such introductions can also help bridging language barriers by helping out with a bit of translation.

I in particular enjoyed meeting and talking to Dorothea Kazazi, Martin Koppenhoefer, Nicolas Chavent and Rafael Avila Coya all of which i never had met in person before – but of course also many others who i had met before.

The social event

The place of the social event was nice and the food was good but it was not ideal for an OSM conference in several regards:

  • The constraints of entering the place (practically the requirement to wear shoes and that you were not allowed to take larger bags or other things into the place but had to deposit them at the entrance) were something the organizers should have announced in advance. One person from the German community who was routinely walking barefoot and had no shoes with him that evening was not allowed to enter and many were uneasy with leaving their bags with valuable stuff like laptops or cameras.
  • For most of the conference visitors the social event is primarily an opportunity to talk to other visitors of the conference. The music played at the place of the social event that got louder the later it was, made this somewhat unnecessarily difficult.

The awards

Since i somewhat unexpectedly won the award for influential writing (sorry Anonymaps) it seems somewhat ungrateful to criticize them – but i will do it anyway. Apart from the general and hard to solve problem of English language bias which i mentioned previously i also have a problem with the innovation category where none of the nominees would qualify for what i would consider innovative work. This was similar in previous years. I would probably just remove that category from the awards in the future. The way the awards are run they are essentially a popularity contest and popularity and innovation are simply two things that normally do not go hand in hand, innovations if they do at all typically only become popular quite some time after being made and the awards are for stuff made in the previous year.

I would also suggest two further changes:

  • limiting the awards to individuals and small groups of identifiable individuals.
  • adding a ‘none of the above’ option to the voting form and not issuing the award if this option receives more votes than any of the others.

In any case congratulations to the other winners who apart from the wrong categorization in the innovation category i would without reservations all consider deserving winners – without necessarily meaning that the other nominees would all have been less qualified. We all for example had a good laugh about the fact that Simon Poole lost to Richard Fairhurst by one vote after having previously given a recommendation to vote for Richard.

Next year

In my pre-conference post i mentioned that it is unlikely that the SotM is going to take place as close to where i live as this year any time soon – seems i was wrong about that. For me Heidelberg is obviously convenient but this also means there is a clear trend for the SotM being more concentrated on Europe again – with three out of four taking place in Europe. This contrasts with the four years before where three out of four were outside of Europe – kind of in compensation to the first four years which all took place in Europe.

Some general thoughts on the conference

For me personally the SotM visit was a pleasant experience. I however have a seriously uneasy feeling about the fact that the SotM claims to be a conference for the whole OSM community which it from my perspective clearly is not. Given the size and diversity of the OSM community this claim seems unrealistic anyway but maintaining the pretense kind of stands in the way of developing organization and structure of OSM conferences in a direction that is sustainable and productive for the project.

What SotM practically consists of currently seem to be three groups of people:

  • the business visitors who visit the conference as part of their jobs.
  • the international OSM jet set consisting of relatively wealthy active OSM hobbyists who are able and willing to invest the money required to visit the conference from their own pockets.
  • members of the local communities near the place the conference takes place.

Everyone else, in particular local mappers and community members from elsewhere, is not realistically present at the conference – even if scholarships might add a few of those. No one should make the mistake of assuming the visitors of SotM or even the non-business part of them are even remotely representative for the global OSM community.

The main difficulty of planning the SotM conference seems to be balancing the interests of the three groups mentioned. Even before visiting the conference this year my opinion on this has been that emphasizing the weight of the third group and making sure to widely rotate the location of the conference would be the best approach – maybe even to the point of not organizing a separate international conference but instead every year hooking into a different regional conference and giving it special support during that year. But since of the three groups of people mentioned the third one is quite clearly the least influential and least powerful one i don’t have the illusion of this being likely to happen.

July 26, 2018
by chris
0 comments

Milano and SotM

I will be on my way to Milano tomorrow for visiting the SotM 2018 conference.

SotM never had a particular appeal to me in the past years in terms of the program but it is likely not going to take place as close as this year any time soon (the journey to Milano from here via train takes about as long as to Hamburg – and you don’t even have to change trains) so it is a good opportunity to get a first hand impression. And i look forward to the opportunity to meet various people and talk about OpenStreetMap, cartography, geodata etc.

July 24, 2018
by chris
7 Comments

More new colors

I made some more high impact color changes to the alternative-colors style i want to quickly discuss here.

Farmland coloring

First i changed the farmland color. Farmland was is OSM-Carto rendered for a long time with a fairly dark brown tone. This looked odd in particular in contrast to the brighter urban landcovers. Since farmland covers pretty large areas in regions with intensive agricultural use a brighter color makes more sense. The color was therefore changed into something significantly brighter several years ago.

This was a big improvement but maintained the oddity of using a brown color for something vegetation related.

The problem is that in the bright color domain you have relatively little room for multiple distinct colors. Therefore the color had to be pretty strong to be discernible from the other bright colors which was something many people also disliked.

The solution i implemented now is essentially a color swap (plus some tuning) of the farmland color and the education/hospital (societal_amenities) color. This makes quite a lot of sense because of the similarity rule (that features similar in meaning and purpose should use similar colors and those different in meaning and purpose different colors). This is a bit of a disentanglement of area color use in the style.

farmland colors in different OSM map styles

the new farmland color – click to see on larger area

I had contemplated this change for quite a while already but originally was not so satisfied with the result. With some tuning and some time for getting used to it i now however think this works quite well.

Road colors

With the road colors i implemented what is essentially a shift of the road classes by one downwards and extrapolation of a new color for motorways at the top extending the existing scheme.

A bit of background for that as well: Back when the current road color scheme in OSM-Carto was developed there were essentially two major constraints:

  • The colors that could be used were the red-orange-yellow-white progression that had already been used for roads previously (plus the green and blue colors we wanted to stop using for roads). It was not possible to go beyond a red tone in hues since that would have led to confusion with the purple boundaries at low zoom levels.
  • The color differences between the individual classes had to be large enough to be able to reliably distinguish between them.

These constraints meant the number of distinct colors had to be reduced to five (red, dark orange, bright orange, yellow, white) and tertiary roads lost their distinct color.

With purple not being used for boundaries any more in the alternative-colors style i can lift the first constraint and extend the color palette to purple and move back to six road colors.

the road color scheme in OSM-Carto (top) and here (bottom)

Here is how this looks like practically at various zoom levels.

z14 – click for larger area


z13 – click for larger area


z12


z11


z10

I also updated the low zoom rendering demo with the new road color scheme and updated data.

Update: Based on the remark by Ilya in the comments below i adjusted the color calculation script to limit the darkening of the motorway color. This makes motorways somewhat brighter than in the samples above, in particular at the lower zoom levels. You can see this in the samples in the readme and in the low zoom demo.

July 9, 2018
by chris
0 comments

A Midsummer Night’s Dream

Continuing on the late evening Landsat images recorded this year in larger number by the USGS here a few more if these featuring various areas in northern Europe and Asia from late June and early July:

In the northern Ural mountains:

In northern Iceland:

In northern Norway:

In the Verkhoyansk Mountains:

All images can be found in the catalog on services.imagico.de.

July 3, 2018
by chris
0 comments

RadioOSM podcast (in German) about OSMF topics

There is a new episode of the German RadioOSM podcast covering various topics of the OSMF that were also topic in the recent face to face meeting of the OSMF board of directors. I was invited for the podcast as a guest because i had commented in public discussion on several of the topics. I discussed this with Andi and Peda – with Peda as an OSMF board member presenting the topic and providing his inside perspective (as far as he could without violating confidentiality of the board meeting). It was recorded already about a month ago – in case you might wonder about more recent development not being taken into account (though not that much has happened on these topics in between).

It should be emphasized that this is not only a German language discussion, it surely also comes with a selective perspective on the subjects and there are obviously other opinions on these matters that were not present in the discussion. Although Peda described the views and opinions of his fellow board members in addition to his own this is obviously not the same as these opinions being directly presented in the discussion. This is certainly a disadvantage of having a discussion in German but at the same time it allowed us to contemplate things on a level i feel is usually not possible in an English language discussion where naturally British and American views and discussion style often are more dominating.

I apologize for my voice being rather raspy in the beginning, it gets a bit better over time.

I hope listening to this and learning about the various subjects being worked on and being discussed in the OSMF encourages more people to become OSMF members, participate in the discussion and shape the OSMF in the interest of the mappers. And i also hope this discussion in German demonstrates to other non-native English speakers that discourse on OSMF politics does not have to take place in English to be meaningful. Certainly this is easier in German at the moment than in other languages since there are two native German speakers on the OSMF board who can’t help but notice what is being discussed in German but the first step is always having a discussion and developing and articulating opinions. And we have quite a few local OSMF chapters meanwhile which could – if necessary – put quite a lot of additional weight behind the desires and needs of their local communities in the OSMF.

June 29, 2018
by chris
0 comments

Winter and Summer 2018

It is midsummer here and i have for the occasion two satellite image mosaics made from Sentinel-2 data.

The first one is from the north showing spring thawing in northeastern Russia at the Arctic Ocean coast. Contrast between the southern and the northern part of the image is not only due to the latitude but also because the ocean warms up much slower and much less in the summer sun than the land surface. On a smaller scale you can also see that with the lakes where the larger ones show some remaining ice cover even in the south while the smaller ones are ice free.

Thawing of the land however is only superficial. Most of the area shown features permafrost since while summer temperatures can occasionally be quite high summers are short (from mid June to mid September) and the winters are very long and cold.

It takes about a month longer until the Ocean is ice free as well and in Autumn the situation is reversed – continuous snow cover starts forming in late September while the Ocean freezes again in late October.

The second image is a winter view of the Falkland Islands. You can compare that to my normal summer mosaic. The low sun position leads to more dramatic lighting here.

The Falkland Islands also receive some snowfall in winter but it usually does not last very long.

Both images can be found in the catalog on services.imagico.de.

June 14, 2018
by chris
0 comments

The way of the water

This is another post from my series on OpenStreetMap map design – kind of following up on what i wrote about waterbodies and fords some time ago.

Water barriers

The first part is about rendering of water barriers – with this i mean dams, lock gates and weirs as well as waterfalls. Like with fords most of these can be mapped as nodes on a waterway line. Waterfalls can only be mapped as nodes – the waterfall edge as a one-dimensional geometry is to be mapped as a cliff. The standard style renders the node based lock gates and weirs using plain circles in the drawing width of a river starting very late at z17. This is rather crude.

Weirs and lock gates mapped with nodes in OSM-Carto

For waterfalls OSM-Carto recently added rendering as simple POIs – likewise independent of the intersecting waterway.

Like in case of fords it is possible to contruct a line geometry by querying the intersecting waterway. For dams, weirs and lock gates this allows drawing the barriers in the same styling as in cases when they are mapped using a way. For waterfalls it allows a fairly compact yet intuitive display.

I also changed the styling of weirs to indicate the direction of water flow. Node based lock gates and waterfalls are likewise shown with the direction visible. For weirs and waterfalls this is done with a brighter line below indicating the whitewater area below the weir/waterfall. For node based lock gates the line is drawn with a small angle indicating the opening direction.

water barrier rendering at z12-z14, click for z15 version

z17 water barrier rendering

The difficulty here is that the rendering toolchain used for this style (and in fact any commonly used rendering toolchains) make this kind of thing rather complicated to implement. Identifying the intersecting waterway is obviously something you need to do on the database level using SQL. But the geometry to be generated has to be adjusted to the width of the waterway which means you need to do zoom level dependent parametrization of this geometry in SQL code.

Here a few practical examples how this looks like.

Waterfall rendering with flow direction hint at z17

Weir rendering with flow direction hint from line based mapping at z16

Waterfalls on streams at z16

Lock gates mapped with nodes at z15

Weir mapped with node at z17

With weirs and lock gates drawn in the same styling both when mapped as ways and nodes the question arises which method of mapping is better. The problem is that waterways are typically drawn thicker than their real width at small scales, otherwise they would not be well visible, and thinner than their real width or in their real width (in case they are mapped with a riverbank polygon) at larger scales. In the simplest case the line geometries i construct from the nodes are based on the drawing width of the waterway. This means the rendering looks good on small scales but does not work on larger scales with a riverbank polygon if that is not taken into account when generating the line geometry.

Locks in river line drawing width based size

If a weir is mapped with a line across the riverbank polygon this OTOH looks good on high zoom levels but not on the smaller scales since the line geometry is too short there and does not cover the larger-than-real drawing width of the waterway at these scales. I have also implemented solutions of these two other cases but they are significantly more complex than the first case.

Taking riverbank width into account

Here for better understanding the different situations all together – first as rendered in OSM-Carto, second with the techniques i introduce here.

node way
low zoom
high zoom
node way
low zoom
high zoom

So overall mapping water barriers as nodes or linear ways are both fine for the purpose of rendering – as i demonstrate here. Mapping these features with polygons however – no matter how exactly you define the outline of them – is much more difficult to process so much less useful for the purpose of generating high quality maps. For dams polygon mapping is widespread and generally accepted. For the other types of barriers it is – although clearly not suggested by the documentation – not that uncommon due to the widespread misguided idea that mapping things with polygons is universally better than any other option.

One other thing that is important for efficient rendering is to split the mapping of riverbank polygons into relatively small areas and not build polygons extending across hundreds of kilometers.

Another problem with the current toolchain is that for labeling i would have to do all of these queries twice since the labels are drawn in a different layer and you cannot reuse queries across several layers. For the moment i refrained from doing that so labels for water barriers are just point labels and not aligned in direction of the barrier as drawn.

I am not completely sure how this kind of rendering approach fares in terms of mapper feedback. In most relatively simple cases i think producing a rendering based on a reasonable and consistent interpretation of the data like this is fine, even if it goes quite a bit beyond a simple and direct visualization of the data. And in my opinion it is a positive message if mappers see that a relatively simple form of mapping (like a node for indicating a barrier on a waterway) is sufficient information to accurately draw this and the mapper is not required to hand draw the map so to speak. But in particular with the two more sophisticated cases (node based mapping at high zoom with riverbank polygon and way based mapping at low zooms) it is likely that there are some data configurations where the method fails in a non-intuitive way. How problematic this can be is an open question. This is certainly a topic where there is far too little experience to draw reliable conclusions.

Water sources

The other change i worked on is cleaning up the mess with water related point features in OSM-Carto. One of them – the waterfall – is already covered above. The other two water related point features in OSM-Carto at the moment are fountains and springs. Together with waterfalls these three water related features use three different colors which is just awfully bad design.

But before fixing that i had to take care of another problem. The standard style has for a long time used a strong blue tone for transportation and accomodation related symbols – bus stop, parkings, hotels, campsites and things like that. This use of blue – although with a long tradition in OSM – does not really work well for an intuitive understanding of the map and conflicts with any water related features you might want to use a strong blue for. So i did a bit of re-organizing of the symbol colors – expanding use of the violet color previously reserved for air transport for what was previously using blue. This requires some getting used to of course. At the same time i removed the use of green for leisure related point symbols which had recently been introduced and which – just like blue being reserved for water – should be reserved mostly for vegetation related features.

Violet color for transportation and accomodation symbols

This frees the strong dark blue for water related point features.

Spring rendering is something that has been changed in OSM-Carto recently but not in a very advantegous way. The primary motivation for the change was to get rid of the old symbol which was a small ‘s’ shape (standing for spring) which is obviously not ideal for an international map. But it is important to see that apart from this the old design worked quite well as a simple point icon starting at relatively low zoom levels because is was very compact and non-obtrusive and at the same time it was very recognizable and clear.

Rendering of springs in OSM-Carto – old (left) and new (right)

The new design – while well intended – does not really work that well for multiple reasons

  • the generic circle is very non-specific and could likewise indicate all kind of other water related features (or not water related given that strong blue is also used for other purposes).
  • the symbol is bigger and more obtrusive without being clearer to read than the old one. It creates much more noise in the map, especially at low zoom levels.
  • layering places the symbol below waterways which creates all kinds of problems as is but especially since it depends on the water layer ordering (with line features above the polygons) which is a rather limiting contraint.

My approach here is based on using the same base design for different water source point features with variations indicating the specific type of feature. I try to make it very compact so it does not take much more space at the low zoom levels than the legacy ‘s’ symbol. The symbol size is increased a bit for better readability two zoom levels above the starting zoom level. Here is how this looks. Wells start one zoom level later than springs (z15 and z14) and fountains start at z17.

rendering of water sources at z14, z15 and z16

rendering of water sources at z17 (click for z18)

As you can see the base shape is the same for springs connected to waterways and isolated ones so it is recognizable this is the same type of feature. But the connected spring rendering of course has to be adjusted to the waterway rendering so it does not look exactly identical. And i added rendering for hot springs and geysers with a red center dot. Intermittent standalone springs are also differentiated with a symbol variation. And at higher zoom levels you can also see new symbols for amenity=water_point and man_made=water_tap.

The rendering of springs connected to a waterway is implemented by generating the symbol geometry in SQL. Like with the water barriers this requires parametrization of the waterway line width depending on zoom level from within SQL. An alternative approach would be to use SVG markers and rotate them based on the waterway direction but you would need a large number of different SVGs for every line width which in the end would not be any less complicated overall.

Another novelty is the addition of a supplemental beaker symbol that is being rendered next to the main symbols at higher zoom levels for water sources with a supplemental amenity=drinking_water or drinking_water=yes tag. This is another case where the limitations of the rendering toolchain are visible – ideally placement of the supplemental symbol would be whereever there is room for it but i don’t think this is possible with the tools used by the style. So i just went with a simple constant northwest position. What would be possible is to place the symbol for connected springs so it does not cover the waterway.

The small symbols for water sources are rendered with a thin bright halo for better visibility on dark and structured background. Here a few practical examples how this looks like in the map.

Springs connected to streams at z15


Wells at z16


Standalone spring at z16


Spring connected to stream at z18


Fountains and well at z17


Fountains and well at z18

Summary

What i have shown here are a number of cartographic design techniques that can help creating better maps:

  • using interrelationships between different map elements (specifically water related point features and the waterway they are located on) to create a more harmonic and less noisy map image and provide additional information (like waterway direction) to the map reader.
  • using subtle symbol variations to visualize similar features allowing for more detailed information in the map without adding a lot of complexity to interpretation, using compound symbols to transport additional information (here: indicate availability of drinking water at a water source).
  • using simplified version of symbols at smaller scales for a more compact representation of features.
  • unifying different forms of mapping the same type of feature (here: water barriers mapped with nodes and ways) into a unified design for different map scales.

As always you can find the changes discussed here for studying and testing in the alternative-colors style.

June 8, 2018
by chris
0 comments

Another late evening image

In the previous post i showed an example for the extended high latitude ascending orbit coverage recorded this year by Landsat 8. Here another example of the Island of Jan Mayen with the unusual late evening sun position and a nice cloud framing.

Landsat evening view of Jan Mayen – May 2018

I wrote more in depth about the nature of high latitude evening images of satellites in sun synchroneous orbit last year.

For comparison see my Jan Mayen mosaic based on normal morning images from later in summer.

June 5, 2018
by chris
0 comments

Satellite image news

Quite a few things have happened in the world of open data satellite images since i last reported on satellite image news.

The most significant change is that Sentinel-2 is now recording significantly more images in most parts of the world. Here is the updated coverage volume plot:

Open data satellite image coverage development

You can also find updated satellite image coverage visualizations.

The increased recording volume (approximately 1.5 times the recording area compared to before) is possible though data transfer via optical links to relay satellites in geostationary orbit. As already indicated in the previous report this increased recording volume is used primarily for giving up the preferential treatment of Europe and Africa in recording and now covering all major land masses except the Antarctic with a ten days interval per satellite, five days combined. In addition as also mentioned a few additional islands previously not covered are included now. To see that here the recording number maps for the 2017 calendar year as well as the incomplete 2018 numbers.


The newly included islands are mostly British Overseas Territories. And in addition to the islands themselves we also see a significant ocean extent around them being included. This varies a bit between Sentinel-2A and Sentinel-2B and it seems they stopped doing that again after some time.

Separate maps for Sentinel-2A 2017, Sentinel-2B 2017, Sentinel-2A 2018 and Sentinel-2B 2018 are available as well.

With the increased recording capacity i have to say i find the continued catering for special interests rather annoying. They would clearly have the capacity now to record all land surfaces at a five days interval – likely including the Antarctic, depending a bit on the recording limit due to low sun position in winter. They would slightly have to cut down on the coverage of high northern latitude for that but you can’t seriously defend having a more than daily coverage of northern Greenland at the expense of not recording various smaller islands at all.

The largest islands completely unrecorded by Sentinel-2 at the moment are the South Orkney Islands.

Let’s get to other news items:

  • There is a new report on the Copernicus program data access. Last year i discussed this in more depth. The new report again has a fairly bad signal to noise ration but there are also a few interesting facts you can find in there:
    • Availability of the data access – which was very bad the previous year as most Sentinel data users have seen first hand – has improved quite a bit.
    • Publication timeliness is still quite bad for Sentinel-2 data with 16 percent of the packages being published more than 48 hours after recording. And this probably does not even include the unpublished packages i mentioned previously (i.e. packages that are never published). And you need to see this also in relation to the fact that more than half of the Sentinel-2 data products downloaded are less than two days old. Predictable availability of data is one of the key criteria for routine data users.
    • In terms of trends it seems the data use via open access channels is growing faster than via all the other channels. And data use is growing much faster world wide than in Europe.
    • The report also mentions for the first time AFAIK plans for long term availability of the data. So far all recorded images are available for immediate download which requires a quite large and continuously growing direct access storage capacity. This is supposed to be changed in the future in a similar way as it is done by the USGS, i.e. that you will have to request older products which are then made available with some delay. The plans for that as sketched are pretty specific and quite reasonable – delays are supposed to be at maximum 24 hours and requests are meant to be scriptable.
  • ESA has announced that they are planning to re-package the larger multiple tile Sentinel-2 data packages from the first year of operation into single tile packages. Actual reprocessing of older data is not planned before 2019.
  • Sentinel-3B, the last of the dedicated satellites of the Copernicus program, was launched with much fanfare in April. No data is available yet and from past experience with Sentinel-3A it is unlikely this will happen any time soon. Even for Sentinel-3A they are still keeping back some of the higher level data products from the general public.
  • The USGS has crippled the Landsat bulk metadata download options. You can still download bulk metadata packages but this is not completely up-to-date and at least for Landsat 7 it is very incomplete. Other metadata you can now only get through EarthExplorer in a manual, not scriptable fashion. This makes analysis of image recordings like i show above much more difficult. Seems like USGS and ESA are converging here.
  • Otherwise Landsat operations are pretty much business as usual. It seems the USGS has significantly increased the number of night time (ascending orbit) acquisitions at high latitude in 2018. See the plot for all of 2017 in comparison to 2018 so far. An example can be found below.
  • Plans for launching Landsat 9 in late 2020 seem to be pretty definite now – not really much room for delays there with Landsat 7 being bound to reach the end of useful life soon afterwards.

Bennett Island late evening in spring

Another topic i want to mention: There has been some fuzz about the idea of reverting the decision of opening access to Landsat data being contemplated by the US government. This started with an article on the matter. To me it seems the complaints and cry-out about the mere idea of this kind of stood in the way of seeing what the real background of this is. Most people who have voiced their objection to this argue that it does not make sense from an overall economic standpoint and that both private sector and science profit so much from open Landsat data that this more than compensates for the costs.

The problem is that this is not how political decisions are made. Political decisions are a matter of interests and the power that stands behind them. Which interests have the strongest influence on the decision making process here is kind of hard to predict here but in any case I don’t consider the science lobby to be of significant power here, in particular not with the current US administration.

In any case the most likely effect such a discussion on the political level will have is on the planning of the future of the Landsat program in terms of financial scope and capabilities. Landsat 9 (as mentioned above to be launched in late 2020) will be a fairly direct copy of Landsat 8. Discussion on the capabilities of future Landsat 10 is currently in progress and it seems the timing of the political discussion could be directed at influencing that.

And what some people also forget is that you can’t really close open data you have already published for free and unrestricted use. So the Landsat archive data will remain free to use (though not necessarily available through the USGS) in the future even if the US government decides to stop open access to new images at some point.

May 25, 2018
by chris
0 comments

Why OpenStreetMap needs open satellite image data

Yes, i have emphasized this before but here is a great example why OpenStreetMap – in its endeavor to create the best map of the world – cannot rely on commercial services with proprietary images offered for mapping in OSM to provide images for mapping wherever they are needed.

Here an area at the northeast shore of the Caspian Sea where pretty big structures have been built recently – about 50 Kilometer long in total and most likely in connection to the nearby Kashagan oil field which you can also find in the image further to the northwest.

Neither of these (the shore structures and the oil field) you can find in any of the various global image layers available for mapping in OSM. One features a construction site at the shore in an early stage. And neither of these places is accurately mapped in OpenStreetMap.

Apart from this i also added a number of other images to the OSMIM – a 2018 update for the North Sea tidalflats, an image of the recently completed bridge over the Kerch Strait – the bridge is well mapped in OSM but many of the coastal changes, access road contructions and similar related changes in the surrounding could still use updates. And finally i still have not given up on encouraging OSM mappers to work on mapping the Antarctic – with an other interesting area at the Antarctic coast being added based on recent imagery. I should probably also point out – since not everyone is probably aware of this – that nowhere on Earth is it so easy and effortless to at least locally turn OpenStreetMap into the best map of the world. If you map in this area based on the provided images (or in most of the other image areas for the Antarctic you can find in the OSMIM) you can be almost certain that what you create is the best map in the world for this particular area. And at the same time you learn quite a lot about a fairly interesting but at the same time very little known part of the world.

May 21, 2018
by chris
0 comments

Differentiated rendering of woodland in maps

This post is in continuation of my attempts started at the end of last year with path and waterbody rendering to write a bit more about OpenStreetMap map design. What i here want to cover is differentiated rendering of woodlands.

First a bit of history on the tagging and rendering of woodland in the OpenStreetMap world. Apart from the ill-defined distinction between landuse=forest and natural=wood – on which there most likely will never be a consensus on what it means – mappers in OpenStreetMap very early started differentiating types of woodland with the wood key. Primary values were deciduous, coniferous and mixed.

This was one of the most ostentatious cases of geographical bias in OpenStreetMap. The mappers who introduced this concept (from lowland areas in Central Europe) were used to coniferous trees being evergreen and broadleaved trees being deciduous. On a global level however we have all combinations of this so over time mappers realized that wood=deciduous/coniferous/mixed is not a very sensible way to classify woodlands and new keys leaf_type and leaf_cycle were introduced. These are now generally accepted.

Historic development of use of wood and leaf_type/leaf_cycle

For further context on woodland depiction in maps in general Jerry wrote a nice history of woodland cartography in 2014.

What i implemented in the alternative-colors style now goes back to the first major modification of woodland styling in OSM-Carto. Back then the main differentiation was still landuse=forest and natural=wood – which, even under the assumption of a consistent difference between the two tags, was too strong relative to other differences in the style.

Legacy wood and forest rendering in the standard style before 2015

This was changed in 2015 to a unified rendering using tree pair symbols – discussion here.

Unified woodland rendering after 2015

This was envisioned to be a provisonal solution until leaf_type could be differentiated – which was being done here, continuing to use the symbols from the pairs and maintaining use of the pairs for areas with no leaf_type specified.

Current OSM-Carto rendering since August 2017

Now this design is in my eyes rather sub-optimal. The abstraction in the symbols borders to being non-intuitive (you could also see a tennis racket and cake shovel). Pattern symbols are very different from individual point feature icons, they work in very different ways for the map reader. Recognition of the individual symbol is not required to be able to read the map, it is the pattern as a whole that needs to work. This means the lack of reliable recognizability of the individual symbol does not necessarily kill this concept but on the other hand there is also no good reason for extreme geometric simplicity to ensure a very clear and sharp rendering. Instead the pattern should be harmonic with the rest of the map while being – as far as possible – intuitive to understand.

Back in 2015 when woodland rendering was discussed i made a number of concept drafts for different styling ideas for woodland symbolization.




The first uses the same symbol idea that was later chosen for OSM-Carto, the second is a design with a somewhat more sophisticated shape, the third is oriented at the styling used in German topographic maps and the last one is something completely different – based on the idea of a top view depiction in contrast to the profile views of trees that are used in most woodland symbols.

The advantage of this is that by not using figurative symbols but a mere structure pattern – similar to the ones used for bare ground types – you in a way better avoid distracting from the other distinct signatures of point and line features in the map. The disadvantage is that you have much less possibilities to transport distinct information with the pattern. The three types – broadleaved, needleleaved and a mixture of both are about the maximum in differentiation this allows while a figurative symbol pattern would in principle allow a much larger number of different symbol types.

Circles and dots – which can be considered an abstract depiction of the top view of trees – are relatively common for woodland representation. Older French topographic maps for example used this universally, newer ones only for broadleaved forests while the coniferous tree symbols are profile depictions. You can find this as well as the newer variants on the IGN Géoportail.

Here a rendering borrowing from the newer French design.

More abstract top view rendering of needleleaved trees can be found in Norwegian maps – here two examples – more can be found in the Kartverket historic map collection.

This concept is regarding perspective somewhat similar to the one i showed above and can also be adopted in digital map rendering of course.

All of this is based on the idea of two woodland types – coniferous (which as said is usually implicitly evergreen) and broadleaved (which is implicitly deciduous). But this synchronicity between leaf_type and leaf_cycle is actually not that common on a global level and leaf_cycle, that is the distinction between evergreen and deciduous woodland, is locally often the more meaningful distinction. Boreal forests are dominated by coniferous trees and the main distinction is between evergreen (spruce, fir and pine) and deciduous forest (larch). At low latitudes (tropics and subtropics) woodland habitats are dominated by broadleaved trees which are either evergreen or deciduous – loosing their leaves during dry season. In the southern hemisphere temperate zone evergreen broadleaved trees are much more common than on the northern hemisphere so the situation is different there as well.

Depiction of leaf_cycle independent of leaf_type or species of tree is something very uncommon in classical cartography – largely because as already indicated separation of these two dimensions of woodland typology is not significant in many of the countries which were historically leading in cartography. The additional problem with OpenStreetMap based maps that serve as mapper feedback is that you would not only need three classes of leaf_cycle – evergreen, deciduous and mixed, you also need a fourth unknown class.

I went with using different colors for the symbols to illustrate leaf_cycle for the moment. I combined this with new, relatively traditional shapes – but this is not set in stone, i am still contemplating other ideas. Here how this looks like.

leaf_cycle differentiation by symbol color

The new tree symbols

This is implemented for natural=wood/landuse=forest, natural=scrub and natural=wetland + wetland=swamp as well as natural=wetland + wetland=mangrove – the latter without differentiation since mangroves are always broadleaved and evergreen.




As you can see i render unknown leaf_type for wood and swamp without symbols – not using the tree pair concept as is OSM-Carto. This means leaf_cycle is not shown if leaf_type is not tagged which is faily rare in reality. The symbols for scrub are trunk-less simplified and smaller versions of the tree symbols. I maintained the legacy scrub symbol for areas with unspecified leaf_type. Tagging of leaf_type and especially leaf_cycle for scrub is much less common than for wood/forest and next to non-existent for swamp. Here are the numbers.

  • natural=wood: 4.6M
    • leaf_type: 258k
      • leaf_type=broadleaved: 144k
      • leaf_type=mixed: 83k
      • leaf_type=needleleaved: 30k
    • leaf_cycle: 111k
      • leaf_cycle=deciduous: 84k
      • leaf_cycle=mixed: 14k
      • leaf_cycle=evergreen: 12k
  • landuse=forest: 3.5M
    • leaf_type: 329k
      • leaf_type=broadleaved: 165k
      • leaf_type=mixed: 85k
      • leaf_type=needleleaved: 78k
    • leaf_cycle: 115k
      • leaf_cycle=deciduous: 68k
      • leaf_cycle=mixed: 24k
      • leaf_cycle=evergreen: 18k
      • leaf_cycle=semi_deciduous: 2k
  • natural=scrub: 1.9M
    • leaf_type: 48k
      • leaf_type=broadleaved: 15k
      • leaf_type=mixed: 4.1k
      • leaf_type=needleleaved: 29k
    • leaf_cycle: 15k
      • leaf_cycle=deciduous: 11k
      • leaf_cycle=mixed: 2.6k
      • leaf_cycle=evergreen: 1.8k
  • wetland=swamp: 80k
    • less than 1k with leaf_type/leaf_cycle

This also means further differentiation in rendering – for example showing distinct species of trees beyond the broad classification of leaf_type, is currently limited by there not being any significant volume of data that could build on. This is an interesting cartographic topic and you can find in classical paper based cartography various attempts in that direction. Likewise for the distinction between dense, closed canopy forests and open woodlands. Something for another time.

The symbols used are going to be in the next version of jsdotpattern.

Deciduous broadleaved forest at z14

Mixed/mixed forest and unspecified scrub + forest at z16