Imagico.de

blog

The way of the water

June 14, 2018
by chris
0 comments

The way of the water

This is another post from my series on OpenStreetMap map design – kind of following up on what i wrote about waterbodies and fords some time ago.

Water barriers

The first part is about rendering of water barriers – with this i mean dams, lock gates and weirs as well as waterfalls. Like with fords most of these can be mapped as nodes on a waterway line. Waterfalls can only be mapped as nodes – the waterfall edge as a one-dimensional geometry is to be mapped as a cliff. The standard style renders the node based lock gates and weirs using plain circles in the drawing width of a river starting very late at z17. This is rather crude.

Weirs and lock gates mapped with nodes in OSM-Carto

For waterfalls OSM-Carto recently added rendering as simple POIs – likewise independent of the intersecting waterway.

Like in case of fords it is possible to contruct a line geometry by querying the intersecting waterway. For dams, weirs and lock gates this allows drawing the barriers in the same styling as in cases when they are mapped using a way. For waterfalls it allows a fairly compact yet intuitive display.

I also changed the styling of weirs to indicate the direction of water flow. Node based lock gates and waterfalls are likewise shown with the direction visible. For weirs and waterfalls this is done with a brighter line below indicating the whitewater area below the weir/waterfall. For node based lock gates the line is drawn with a small angle indicating the opening direction.

water barrier rendering at z12-z14, click for z15 version

z17 water barrier rendering

The difficulty here is that the rendering toolchain used for this style (and in fact any commonly used rendering toolchains) make this kind of thing rather complicated to implement. Identifying the intersecting waterway is obviously something you need to do on the database level using SQL. But the geometry to be generated has to be adjusted to the width of the waterway which means you need to do zoom level dependent parametrization of this geometry in SQL code.

Here a few practical examples how this looks like.

Waterfall rendering with flow direction hint at z17

Weir rendering with flow direction hint from line based mapping at z16

Waterfalls on streams at z16

Lock gates mapped with nodes at z15

Weir mapped with node at z17

With weirs and lock gates drawn in the same styling both when mapped as ways and nodes the question arises which method of mapping is better. The problem is that waterways are typically drawn thicker than their real width at small scales, otherwise they would not be well visible, and thinner than their real width or in their real width (in case they are mapped with a riverbank polygon) at larger scales. In the simplest case the line geometries i construct from the nodes are based on the drawing width of the waterway. This means the rendering looks good on small scales but does not work on larger scales with a riverbank polygon if that is not taken into account when generating the line geometry.

Locks in river line drawing width based size

If a weir is mapped with a line across the riverbank polygon this OTOH looks good on high zoom levels but not on the smaller scales since the line geometry is too short there and does not cover the larger-than-real drawing width of the waterway at these scales. I have also implemented solutions of these two other cases but they are significantly more complex than the first case.

Taking riverbank width into account

Here for better understanding the different situations all together – first as rendered in OSM-Carto, second with the techniques i introduce here.

node way
low zoom
high zoom
node way
low zoom
high zoom

So overall mapping water barriers as nodes or linear ways are both fine for the purpose of rendering – as i demonstrate here. Mapping these features with polygons however – no matter how exactly you define the outline of them – is much more difficult to process so much less useful for the purpose of generating high quality maps. For dams polygon mapping is widespread and generally accepted. For the other types of barriers it is – although clearly not suggested by the documentation – not that uncommon due to the widespread misguided idea that mapping things with polygons is universally better than any other option.

One other thing that is important for efficient rendering is to split the mapping of riverbank polygons into relatively small areas and not build polygons extending across hundreds of kilometers.

Another problem with the current toolchain is that for labeling i would have to do all of these queries twice since the labels are drawn in a different layer and you cannot reuse queries across several layers. For the moment i refrained from doing that so labels for water barriers are just point labels and not aligned in direction of the barrier as drawn.

I am not completely sure how this kind of rendering approach fares in terms of mapper feedback. In most relatively simple cases i think producing a rendering based on a reasonable and consistent interpretation of the data like this is fine, even if it goes quite a bit beyond a simple and direct visualization of the data. And in my opinion it is a positive message if mappers see that a relatively simple form of mapping (like a node for indicating a barrier on a waterway) is sufficient information to accurately draw this and the mapper is not required to hand draw the map so to speak. But in particular with the two more sophisticated cases (node based mapping at high zoom with riverbank polygon and way based mapping at low zooms) it is likely that there are some data configurations where the method fails in a non-intuitive way. How problematic this can be is an open question. This is certainly a topic where there is far too little experience to draw reliable conclusions.

Water sources

The other change i worked on is cleaning up the mess with water related point features in OSM-Carto. One of them – the waterfall – is already covered above. The other two water related point features in OSM-Carto at the moment are fountains and springs. Together with waterfalls these three water related features use three different colors which is just awfully bad design.

But before fixing that i had to take care of another problem. The standard style has for a long time used a strong blue tone for transportation and accomodation related symbols – bus stop, parkings, hotels, campsites and things like that. This use of blue – although with a long tradition in OSM – does not really work well for an intuitive understanding of the map and conflicts with any water related features you might want to use a strong blue for. So i did a bit of re-organizing of the symbol colors – expanding use of the violet color previously reserved for air transport for what was previously using blue. This requires some getting used to of course. At the same time i removed the use of green for leisure related point symbols which had recently been introduced and which – just like blue being reserved for water – should be reserved mostly for vegetation related features.

Violet color for transportation and accomodation symbols

This frees the strong dark blue for water related point features.

Spring rendering is something that has been changed in OSM-Carto recently but not in a very advantegous way. The primary motivation for the change was to get rid of the old symbol which was a small ‘s’ shape (standing for spring) which is obviously not ideal for an international map. But it is important to see that apart from this the old design worked quite well as a simple point icon starting at relatively low zoom levels because is was very compact and non-obtrusive and at the same time it was very recognizable and clear.

Rendering of springs in OSM-Carto – old (left) and new (right)

The new design – while well intended – does not really work that well for multiple reasons

  • the generic circle is very non-specific and could likewise indicate all kind of other water related features (or not water related given that strong blue is also used for other purposes).
  • the symbol is bigger and more obtrusive without being clearer to read than the old one. It creates much more noise in the map, especially at low zoom levels.
  • layering places the symbol below waterways which creates all kinds of problems as is but especially since it depends on the water layer ordering (with line features above the polygons) which is a rather limiting contraint.

My approach here is based on using the same base design for different water source point features with variations indicating the specific type of feature. I try to make it very compact so it does not take much more space at the low zoom levels than the legacy ‘s’ symbol. The symbol size is increased a bit for better readability two zoom levels above the starting zoom level. Here is how this looks. Wells start one zoom level later than springs (z15 and z14) and fountains start at z17.

rendering of water sources at z14, z15 and z16

rendering of water sources at z17 (click for z18)

As you can see the base shape is the same for springs connected to waterways and isolated ones so it is recognizable this is the same type of feature. But the connected spring rendering of course has to be adjusted to the waterway rendering so it does not look exactly identical. And i added rendering for hot springs and geysers with a red center dot. Intermittent standalone springs are also differentiated with a symbol variation. And at higher zoom levels you can also see new symbols for amenity=water_point and man_made=water_tap.

The rendering of springs connected to a waterway is implemented by generating the symbol geometry in SQL. Like with the water barriers this requires parametrization of the waterway line width depending on zoom level from within SQL. An alternative approach would be to use SVG markers and rotate them based on the waterway direction but you would need a large number of different SVGs for every line width which in the end would not be any less complicated overall.

Another novelty is the addition of a supplemental beaker symbol that is being rendered next to the main symbols at higher zoom levels for water sources with a supplemental amenity=drinking_water or drinking_water=yes tag. This is another case where the limitations of the rendering toolchain are visible – ideally placement of the supplemental symbol would be whereever there is room for it but i don’t think this is possible with the tools used by the style. So i just went with a simple constant northwest position. What would be possible is to place the symbol for connected springs so it does not cover the waterway.

The small symbols for water sources are rendered with a thin bright halo for better visibility on dark and structured background. Here a few practical examples how this looks like in the map.

Springs connected to streams at z15


Wells at z16


Standalone spring at z16


Spring connected to stream at z18


Fountains and well at z17


Fountains and well at z18

Summary

What i have shown here are a number of cartographic design techniques that can help creating better maps:

  • using interrelationships between different map elements (specifically water related point features and the waterway they are located on) to create a more harmonic and less noisy map image and provide additional information (like waterway direction) to the map reader.
  • using subtle symbol variations to visualize similar features allowing for more detailed information in the map without adding a lot of complexity to interpretation, using compound symbols to transport additional information (here: indicate availability of drinking water at a water source).
  • using simplified version of symbols at smaller scales for a more compact representation of features.
  • unifying different forms of mapping the same type of feature (here: water barriers mapped with nodes and ways) into a unified design for different map scales.

As always you can find the changes discussed here for studying and testing in the alternative-colors style.

Landsat evening view of Jan Mayen - May 2018

June 8, 2018
by chris
0 comments

Another late evening image

In the previous post i showed an example for the extended high latitude ascending orbit coverage recorded this year by Landsat 8. Here another example of the Island of Jan Mayen with the unusual late evening sun position and a nice cloud framing.

Landsat evening view of Jan Mayen – May 2018

I wrote more in depth about the nature of high latitude evening images of satellites in sun synchroneous orbit last year.

For comparison see my Jan Mayen mosaic based on normal morning images from later in summer.

LC81782382018115LGN00_expose.ann

June 5, 2018
by chris
0 comments

Satellite image news

Quite a few things have happened in the world of open data satellite images since i last reported on satellite image news.

The most significant change is that Sentinel-2 is now recording significantly more images in most parts of the world. Here is the updated coverage volume plot:

Open data satellite image coverage development

You can also find updated satellite image coverage visualizations.

The increased recording volume (approximately 1.5 times the recording area compared to before) is possible though data transfer via optical links to relay satellites in geostationary orbit. As already indicated in the previous report this increased recording volume is used primarily for giving up the preferential treatment of Europe and Africa in recording and now covering all major land masses except the Antarctic with a ten days interval per satellite, five days combined. In addition as also mentioned a few additional islands previously not covered are included now. To see that here the recording number maps for the 2017 calendar year as well as the incomplete 2018 numbers.


The newly included islands are mostly British Overseas Territories. And in addition to the islands themselves we also see a significant ocean extent around them being included. This varies a bit between Sentinel-2A and Sentinel-2B and it seems they stopped doing that again after some time.

Separate maps for Sentinel-2A 2017, Sentinel-2B 2017, Sentinel-2A 2018 and Sentinel-2B 2018 are available as well.

With the increased recording capacity i have to say i find the continued catering for special interests rather annoying. They would clearly have the capacity now to record all land surfaces at a five days interval – likely including the Antarctic, depending a bit on the recording limit due to low sun position in winter. They would slightly have to cut down on the coverage of high northern latitude for that but you can’t seriously defend having a more than daily coverage of northern Greenland at the expense of not recording various smaller islands at all.

The largest islands completely unrecorded by Sentinel-2 at the moment are the South Orkney Islands.

Let’s get to other news items:

  • There is a new report on the Copernicus program data access. Last year i discussed this in more depth. The new report again has a fairly bad signal to noise ration but there are also a few interesting facts you can find in there:
    • Availability of the data access – which was very bad the previous year as most Sentinel data users have seen first hand – has improved quite a bit.
    • Publication timeliness is still quite bad for Sentinel-2 data with 16 percent of the packages being published more than 48 hours after recording. And this probably does not even include the unpublished packages i mentioned previously (i.e. packages that are never published). And you need to see this also in relation to the fact that more than half of the Sentinel-2 data products downloaded are less than two days old. Predictable availability of data is one of the key criteria for routine data users.
    • In terms of trends it seems the data use via open access channels is growing faster than via all the other channels. And data use is growing much faster world wide than in Europe.
    • The report also mentions for the first time AFAIK plans for long term availability of the data. So far all recorded images are available for immediate download which requires a quite large and continuously growing direct access storage capacity. This is supposed to be changed in the future in a similar way as it is done by the USGS, i.e. that you will have to request older products which are then made available with some delay. The plans for that as sketched are pretty specific and quite reasonable – delays are supposed to be at maximum 24 hours and requests are meant to be scriptable.
  • ESA has announced that they are planning to re-package the larger multiple tile Sentinel-2 data packages from the first year of operation into single tile packages. Actual reprocessing of older data is not planned before 2019.
  • Sentinel-3B, the last of the dedicated satellites of the Copernicus program, was launched with much fanfare in April. No data is available yet and from past experience with Sentinel-3A it is unlikely this will happen any time soon. Even for Sentinel-3A they are still keeping back some of the higher level data products from the general public.
  • The USGS has crippled the Landsat bulk metadata download options. You can still download bulk metadata packages but this is not completely up-to-date and at least for Landsat 7 it is very incomplete. Other metadata you can now only get through EarthExplorer in a manual, not scriptable fashion. This makes analysis of image recordings like i show above much more difficult. Seems like USGS and ESA are converging here.
  • Otherwise Landsat operations are pretty much business as usual. It seems the USGS has significantly increased the number of night time (ascending orbit) acquisitions at high latitude in 2018. See the plot for all of 2017 in comparison to 2018 so far. An example can be found below.
  • Plans for launching Landsat 9 in late 2020 seem to be pretty definite now – not really much room for delays there with Landsat 7 being bound to reach the end of useful life soon afterwards.

Bennett Island late evening in spring

Another topic i want to mention: There has been some fuzz about the idea of reverting the decision of opening access to Landsat data being contemplated by the US government. This started with an article on the matter. To me it seems the complaints and cry-out about the mere idea of this kind of stood in the way of seeing what the real background of this is. Most people who have voiced their objection to this argue that it does not make sense from an overall economic standpoint and that both private sector and science profit so much from open Landsat data that this more than compensates for the costs.

The problem is that this is not how political decisions are made. Political decisions are a matter of interests and the power that stands behind them. Which interests have the strongest influence on the decision making process here is kind of hard to predict here but in any case I don’t consider the science lobby to be of significant power here, in particular not with the current US administration.

In any case the most likely effect such a discussion on the political level will have is on the planning of the future of the Landsat program in terms of financial scope and capabilities. Landsat 9 (as mentioned above to be launched in late 2020) will be a fairly direct copy of Landsat 8. Discussion on the capabilities of future Landsat 10 is currently in progress and it seems the timing of the political discussion could be directed at influencing that.

And what some people also forget is that you can’t really close open data you have already published for free and unrestricted use. So the Landsat archive data will remain free to use (though not necessarily available through the USGS) in the future even if the US government decides to stop open access to new images at some point.

Lützow-Holm Bay

May 25, 2018
by chris
0 comments

Why OpenStreetMap needs open satellite image data

Yes, i have emphasized this before but here is a great example why OpenStreetMap – in its endeavor to create the best map of the world – cannot rely on commercial services with proprietary images offered for mapping in OSM to provide images for mapping wherever they are needed.

Here an area at the northeast shore of the Caspian Sea where pretty big structures have been built recently – about 50 Kilometer long in total and most likely in connection to the nearby Kashagan oil field which you can also find in the image further to the northwest.

Neither of these (the shore structures and the oil field) you can find in any of the various global image layers available for mapping in OSM. One features a construction site at the shore in an early stage. And neither of these places is accurately mapped in OpenStreetMap.

Apart from this i also added a number of other images to the OSMIM – a 2018 update for the North Sea tidalflats, an image of the recently completed bridge over the Kerch Strait – the bridge is well mapped in OSM but many of the coastal changes, access road contructions and similar related changes in the surrounding could still use updates. And finally i still have not given up on encouraging OSM mappers to work on mapping the Antarctic – with an other interesting area at the Antarctic coast being added based on recent imagery. I should probably also point out – since not everyone is probably aware of this – that nowhere on Earth is it so easy and effortless to at least locally turn OpenStreetMap into the best map of the world. If you map in this area based on the provided images (or in most of the other image areas for the Antarctic you can find in the OSMIM) you can be almost certain that what you create is the best map in the world for this particular area. And at the same time you learn quite a lot about a fairly interesting but at the same time very little known part of the world.

Woodland rendering in maps

May 21, 2018
by chris
0 comments

Differentiated rendering of woodland in maps

This post is in continuation of my attempts started at the end of last year with path and waterbody rendering to write a bit more about OpenStreetMap map design. What i here want to cover is differentiated rendering of woodlands.

First a bit of history on the tagging and rendering of woodland in the OpenStreetMap world. Apart from the ill-defined distinction between landuse=forest and natural=wood – on which there most likely will never be a consensus on what it means – mappers in OpenStreetMap very early started differentiating types of woodland with the wood key. Primary values were deciduous, coniferous and mixed.

This was one of the most ostentatious cases of geographical bias in OpenStreetMap. The mappers who introduced this concept (from lowland areas in Central Europe) were used to coniferous trees being evergreen and broadleaved trees being deciduous. On a global level however we have all combinations of this so over time mappers realized that wood=deciduous/coniferous/mixed is not a very sensible way to classify woodlands and new keys leaf_type and leaf_cycle were introduced. These are now generally accepted.

Historic development of use of wood and leaf_type/leaf_cycle

For further context on woodland depiction in maps in general Jerry wrote a nice history of woodland cartography in 2014.

What i implemented in the alternative-colors style now goes back to the first major modification of woodland styling in OSM-Carto. Back then the main differentiation was still landuse=forest and natural=wood – which, even under the assumption of a consistent difference between the two tags, was too strong relative to other differences in the style.

Legacy wood and forest rendering in the standard style before 2015

This was changed in 2015 to a unified rendering using tree pair symbols – discussion here.

Unified woodland rendering after 2015

This was envisioned to be a provisonal solution until leaf_type could be differentiated – which was being done here, continuing to use the symbols from the pairs and maintaining use of the pairs for areas with no leaf_type specified.

Current OSM-Carto rendering since August 2017

Now this design is in my eyes rather sub-optimal. The abstraction in the symbols borders to being non-intuitive (you could also see a tennis racket and cake shovel). Pattern symbols are very different from individual point feature icons, they work in very different ways for the map reader. Recognition of the individual symbol is not required to be able to read the map, it is the pattern as a whole that needs to work. This means the lack of reliable recognizability of the individual symbol does not necessarily kill this concept but on the other hand there is also no good reason for extreme geometric simplicity to ensure a very clear and sharp rendering. Instead the pattern should be harmonic with the rest of the map while being – as far as possible – intuitive to understand.

Back in 2015 when woodland rendering was discussed i made a number of concept drafts for different styling ideas for woodland symbolization.




The first uses the same symbol idea that was later chosen for OSM-Carto, the second is a design with a somewhat more sophisticated shape, the third is oriented at the styling used in German topographic maps and the last one is something completely different – based on the idea of a top view depiction in contrast to the profile views of trees that are used in most woodland symbols.

The advantage of this is that by not using figurative symbols but a mere structure pattern – similar to the ones used for bare ground types – you in a way better avoid distracting from the other distinct signatures of point and line features in the map. The disadvantage is that you have much less possibilities to transport distinct information with the pattern. The three types – broadleaved, needleleaved and a mixture of both are about the maximum in differentiation this allows while a figurative symbol pattern would in principle allow a much larger number of different symbol types.

Circles and dots – which can be considered an abstract depiction of the top view of trees – are relatively common for woodland representation. Older French topographic maps for example used this universally, newer ones only for broadleaved forests while the coniferous tree symbols are profile depictions. You can find this as well as the newer variants on the IGN Géoportail.

Here a rendering borrowing from the newer French design.

More abstract top view rendering of needleleaved trees can be found in Norwegian maps – here two examples – more can be found in the Kartverket historic map collection.

This concept is regarding perspective somewhat similar to the one i showed above and can also be adopted in digital map rendering of course.

All of this is based on the idea of two woodland types – coniferous (which as said is usually implicitly evergreen) and broadleaved (which is implicitly deciduous). But this synchronicity between leaf_type and leaf_cycle is actually not that common on a global level and leaf_cycle, that is the distinction between evergreen and deciduous woodland, is locally often the more meaningful distinction. Boreal forests are dominated by coniferous trees and the main distinction is between evergreen (spruce, fir and pine) and deciduous forest (larch). At low latitudes (tropics and subtropics) woodland habitats are dominated by broadleaved trees which are either evergreen or deciduous – loosing their leaves during dry season. In the southern hemisphere temperate zone evergreen broadleaved trees are much more common than on the northern hemisphere so the situation is different there as well.

Depiction of leaf_cycle independent of leaf_type or species of tree is something very uncommon in classical cartography – largely because as already indicated separation of these two dimensions of woodland typology is not significant in many of the countries which were historically leading in cartography. The additional problem with OpenStreetMap based maps that serve as mapper feedback is that you would not only need three classes of leaf_cycle – evergreen, deciduous and mixed, you also need a fourth unknown class.

I went with using different colors for the symbols to illustrate leaf_cycle for the moment. I combined this with new, relatively traditional shapes – but this is not set in stone, i am still contemplating other ideas. Here how this looks like.

leaf_cycle differentiation by symbol color

The new tree symbols

This is implemented for natural=wood/landuse=forest, natural=scrub and natural=wetland + wetland=swamp as well as natural=wetland + wetland=mangrove – the latter without differentiation since mangroves are always broadleaved and evergreen.




As you can see i render unknown leaf_type for wood and swamp without symbols – not using the tree pair concept as is OSM-Carto. This means leaf_cycle is not shown if leaf_type is not tagged which is faily rare in reality. The symbols for scrub are trunk-less simplified and smaller versions of the tree symbols. I maintained the legacy scrub symbol for areas with unspecified leaf_type. Tagging of leaf_type and especially leaf_cycle for scrub is much less common than for wood/forest and next to non-existent for swamp. Here are the numbers.

  • natural=wood: 4.6M
    • leaf_type: 258k
      • leaf_type=broadleaved: 144k
      • leaf_type=mixed: 83k
      • leaf_type=needleleaved: 30k
    • leaf_cycle: 111k
      • leaf_cycle=deciduous: 84k
      • leaf_cycle=mixed: 14k
      • leaf_cycle=evergreen: 12k
  • landuse=forest: 3.5M
    • leaf_type: 329k
      • leaf_type=broadleaved: 165k
      • leaf_type=mixed: 85k
      • leaf_type=needleleaved: 78k
    • leaf_cycle: 115k
      • leaf_cycle=deciduous: 68k
      • leaf_cycle=mixed: 24k
      • leaf_cycle=evergreen: 18k
      • leaf_cycle=semi_deciduous: 2k
  • natural=scrub: 1.9M
    • leaf_type: 48k
      • leaf_type=broadleaved: 15k
      • leaf_type=mixed: 4.1k
      • leaf_type=needleleaved: 29k
    • leaf_cycle: 15k
      • leaf_cycle=deciduous: 11k
      • leaf_cycle=mixed: 2.6k
      • leaf_cycle=evergreen: 1.8k
  • wetland=swamp: 80k
    • less than 1k with leaf_type/leaf_cycle

This also means further differentiation in rendering – for example showing distinct species of trees beyond the broad classification of leaf_type, is currently limited by there not being any significant volume of data that could build on. This is an interesting cartographic topic and you can find in classical paper based cartography various attempts in that direction. Likewise for the distinction between dense, closed canopy forests and open woodlands. Something for another time.

The symbols used are going to be in the next version of jsdotpattern.

Deciduous broadleaved forest at z14

Mixed/mixed forest and unspecified scrub + forest at z16

Landsat/Sentinel-2 mosaics of the Subantarctic islands

May 8, 2018
by chris
0 comments

Images of the Subantarctic Islands

Kind of balancing the recently shown imagery of the northern polar region i here introduce a number of new satellite image mosaics from the south of all the Subantarctic Islands. These islands – similar to some parts of the Arctic – are often neglected in satellite image layers in map services – either by being fully missing or being only covered in poor quality. Some of these are also rather difficult because satellite recording schedules likewise skip them. Sentinel-2 for example has not recorded the South Sandwich Islands and Gough Island for a long time and has only recently started covering them and is still not recording Bouvet Island, Amsterdam Island, Saint Paul Island, the Antipodes Islands, the Snares Islands and the Bounty Islands.

Here the illustrated list of islands i produced new images from:

Kerguelen Islands

The image of the Kerguelen Islands is the largest mosaic and also one of the most difficult in terms of clouds.

Heard Island and the McDonald Islands

Heard Island is located southeast of the Kerguelen Islands and more heavily glaciated and with less vegetation. The McDonald Islands are smaller and slightly west of Heard.

Tristan da Cunha

Tristan da Cunha in the South Atlantic Ocean is the northmost of the islands covered here and apart from the Falkland Islands also the most populated.

Gough Island

Gough Island is located southeast of Tristan da Cunha. Sentinel-2 has only started covering it recently in early 2018 so this image is combining Landsat and Sentinel-2 data.

Bouvet Island

Bouvet Island is located further south and almost completely glaciated. This is not currently covered by Sentinel-2 but the cloud situation here is difficult so more frequent image recordings would be very useful.

Crozet Islands

The Crozet Islands feature a very wet and windy climate and are located in the southwest Indian Ocean.

Prince Edward Islands

Prince Edward Island and Marion Island (the larger one at the bottom) are located somewhat further to the west. The difficulty here is less producing a cloud free than producing a snow free image.

Amsterdam Island

Saint Paul Island

Further north in the Indian Ocean there are two small islands, Amsterdam Island and Saint Paul Island, which are not included in the Sentinel-2 recording plan and are therefore rendered based on Landsat data here.

South Sandwich Islands

The southmost of the Subantarctic Islands are the South Sandwich Islands. These are not particularly difficult per se regarding clouds since the area often has good weather in late winter but producing a good cloud free summer image is another story. The recent inclusion in Sentinel-2 recordings helped a lot here.

Macquarie Island

Moving further to the east we have various islands south of New Zealand – starting with Macquarie Island.

Auckland Islands

Campbell Islands

Antipodes Islands

Snares Islands

Bounty Islands

And finally we have the New Zealand Subantarctic Islands – in various sizes and forms. Only the Auckland Islands and the Campbell Islands are included in Sentinel-2 recordings, the others are only covered by Landsat.

I also produced updated images of South Georgia and the Falkland Islands:

South Georgia

Falkland Islands

And finally also a mosaic of the Balleny Islands in the Antarctic:

Balleny Islands

All mosaics are available as usual on services.imagico.de.

Landsat mosaic based on pixel statistics

April 25, 2018
by chris
0 comments

Arctic mosaic and colors revisited

Continuing from the previous post here as promised more examples for application of pixel statistics methods to Landsat and Sentinel-2 data and how this can help to produce more accurate colors.

I have mentioned before that based on the spectral characteristics for accurate natural colors Landsat offers a significantly better basis than current low resolution systems like MODIS and Sentinel-3 and that from Landsat 7 and EO-1 via Landsat 8 to Sentinel-2 there is a notable trend to less suitability for accurate color reproduction.

Visible light spectral bands of common open data earth observation satellite sensors

This assessment which is based on the spectral characteristics is something that is hard to demonstrate practically based on individual images because the differences in viewing conditions are usually large compared to the differences in colors due to different spectral characteristics.

I have now produced a larger area image mosaic based on pixel statistics methods (which i discussed in the previous post) and by comparing with the MODIS based Green Marble mosaic i can point out the effects of different spectral characteristics much better. The image uses data from Landsat 7, Landsat 8 and Sentinel-2 for the land surfaces and the Green Marble as background for water areas. The data basis is not that broad so there are also significant color differences due to incomplete convergence. But you can still pretty prominently see the color differences compared to the MODIS mosaic.

Arctic mosaic based on Landsat and Sentinel-2 data

Green Marble for comparison

The most striking difference is that the MODIS based Green Marble rarely features true gray tones. Most areas that are gray in the Landsat/Sentinel-2 mosaic show up in red and brown colors in the Green Marble. This is a result of the more narrow spectral bands in the MODIS instrument, in particular the green band. Gray colors mean reflection is more or less equal in all three spectral bands of the human eye but this does not necessarily mean it is completely uniform across the visible range. If it is not a narrow spectral band will usually result in a non-neutral color being registered for a surface that would appear to be of neutral color in direct view by the human eye. The opposite is possible as well but practically much less likely.

Moscow based on Landsat pixel statistics

Moscow based on Green Marble

Verkhoyansk Mountains based on Landsat pixel statistics

Verkhoyansk Mountains based on Green Marble

This Arctic mosaic shown by the way is to my knowledge the first complete natural color mosaic of the Arctic in a better-than-MODIS resolution. I don’t want to specify an actual resolution because of the limitations of pixel statistics method described in the previous post. It was processed in a 30m grid (based on the multispectral Landsat resolution). Obviously my regional mosaics like those of Greenland and Scandinavia offer a significantly better resolution but are also more costly to produce.

Greenland sample form the Artic mosaic based on Landsat and Sentinel-2 data

Same area based on the Landsat mosaic of Greenland

Same area based on Green Marble

I also have full coverage of Europe but so far not beyond. If you are interested in other areas let me know.

Europe mosaic based on Landsat and Sentinel-2 data

pixel statistics based on Landsat - Pyrenees

April 21, 2018
by chris
0 comments

Satellite image pixel statistics

As regular readers of this blog know i have produced quite a few satellite image mosaic products over the past years and reviewed the work of others in that field as well. New products of this type have been introduced quite frequently by various companies over the past 2-3 years but amazingly it seems quality of those has largely plateaued – most of the stuff being produced these days i don’t really find innovative enough to warrant a closer look.

And this is despite the fact that we have a lot of factors that could be considered to provide good conditions for quality improvements:

  • newer satellites are producing higher quality data.
  • we have a quickly growing body of image data that can be used as data source for image mosaic production.
  • computers to process the data are getting more powerful and cheaper.

The question i want to shed some light on is why despite these promising circumstances we don’t see a widespread improvement in quality of end use satellite image mosaic products being available in services.

There are economic factors involved here of course but one of the most important reasons is technological. Focus in satellite image mosaic production over the past 5-10 years has almost exclusively been on what i call pixel statistics techniques. The Mapbox cloudless atlas has been the iconic example for this. It was not the first – the Blue Marble Next Generation from 2005 was technically also based on pixel statistics and techniques like this date back at least to early AVHRR data products in the 1980s. But the Mapbox mosaic was the first such product commercially produced on a global level for direct visual application.

Pixel statistics techniques means that for every pixel in your processing grid you take all the source image data you have from different images taken at different times and you run some statistics over it to estimate an ideal, representative pixel value to be used in the image mosaic to be created.

The statistics used can be very simple – which can under the right circumstances still lead to pretty good results, this is what the Mapbox mosaic nicely demonstrated. But they can also be more complex and even take into account other secondary data sources not derived from the image data. My Green Marble mosaic demonstrates this quite well. The main point all these techniques have in common is that every pixel is treated independently. This makes it a very attractive approach because it is (a) relatively simple to formulate an algorithm for this and (b) it is very efficient to run on a large scale. These advantages have led to almost everyone who wanted to do something in the field of satellite image mosaic production in recent years to choose a pixel statistics based approach.

What people apparently often have not realized when making this decision is that this strategy does not work equally well at different spatial resolutions. Satellite image mosaicing is not a scale independent problem. I hinted this already back when reviewing the Google Landsat mosaic.

pixel statistics at 250m resolution – the Green Marble

The thing is at a pixel size of 250m or more you can reasonably treat every pixel of the image independently. As long as you have a sufficiently broad data basis for your statistics to converge leading to low uncorrelated noise levels and no significant banding artefacts and your statistical method is well chosen you can achieve well readable results. But if you move to significantly higher resolutions this does not work any more because our ability to read and understand a higher resolution image of the earth surface depends increasingly on fairly delicate spatial relationships within the image. And this is frequently lost if you treat every pixel independently.


15m resolution Landsat images

Many pixel statistics based mosaics you can find made from Landsat or Sentinel-2 data do not actually get to the point where you can see that because the mentioned requirements are not met (broad enough data basis and suitable statistical approach) but when they do the results usually still appear confusing and are way behind an individual high quality image in terms of acuity and readability.

In short: Pixel statistics are a highly attractive method for image mosaic production that can work very nicely and efficiently at coarse resolutions. But they are practically unsuitable for much higher resolutions. I have never seen anyone attempting to apply pixel statistics methods to very high resolution images with pixel sizes in the sub meter range and it would probably look pretty horrible (though it might nicely demonstrate the point i am trying to make here). In the intermediate resolution range you see an effect of diminishing returns, i.e. when all other things are equal you would see that increasing source image and processing grid resolution would at some point not significantly improve the quality of the resulting image any more.


Sub-meter resolution images (from IGN Spain)

Practically this effect overlaps with other influences like the limitations in positional accuracy of the source images and the typically lower number of images available at higher resolutions so it can be practically difficult to separate different effects. But while the latter problems can be overcome with technological improvements the main problem is a principal one that will always pose a hard limit to methods based on pixel statistics.

And ultimately this is why quality of satellite image mosaic products has not much improved in the last years.

Because of these limitations i have – for higher resolution mosaics in the Landsat/Sentinel-2 resolution range (10-15m) – always concentrated on methods not based on pixel statistics. But pixel statistics have their charm – economically but also because you can produce very accurate colors. Colors in an individual satellite image are always subject to the specific conditions under which the image was taken. You can put a lot of effort into counteracting that with atmosphere and BRDF compensation but such methods also inevitably introduce variance due to inaccurate simplifying assumptions being made. With a sufficiently broad data basis pixel statistics can help reducing this variance and give you more precise colors.

pixel statistics based on Landsat images

This is why i looked into pixel statistics based methods for Landsat and Sentinel-2 images recently. Not so much because of the higher resolution (which is nice to have but as explained comes with diminishing returns). More because what you can achieve in terms of colors.

More on that in the next post. Until then here a quick teaser:

April 21, 2018
by chris
0 comments

It is called progress

Came across this remarkable map comparison today – click on the image to get to the source:

Note i re-touched the image to make it look a bit more like it is meant to look – see the description of the image after following the link. This however is not completely accurate – you need to ignore the different label languages.

I find this a fairly educative and thought provoking example on several levels. You have the general concept of the map (static vs. dynamic in user interaction), you have the underlying technology to produce the map and you have the map design. And above all of this you have the purpose of the map (being a locator map in a Wikipedia article). One obvious thing you for example could ask yourself is why the map on the left looks like it does and why does the map on the right look like it does – in other words: what are the reasons and motives for the design used here? Since both maps are produced for the same purpose you will probably agree it is somewhat odd they differ that much. Does this have reasons in the static vs. dynamic interaction? Does it have reasons in technology? Is it a matter of changing map design fashion? Or is it something entirely different?

Note i wrote about the economic side of this matter, incidentally also in the context of Wikimedia maps, several years ago. I also wrote about the sociological side of map design in context of OpenStreetMap-Carto more recently. But i still find this a rather intriguing topic with many open questions. If you have additional thoughts and perspectives on this matter i would be curious to read about it in the comments.

Northeastern Alps by Landsat 1 in 1972

April 20, 2018
by chris
0 comments

Happy Anniversary Landsat Open Data

Today ten years ago the USGS started opening the full Landsat image archive as open data. Although this was not the first release of satellite imagery as open data – a selection of Landsat images was already opened before and MODIS data was likewise already usable by everyone years before it can today still be without reservations called the historic decision that most strongly shaped the satellite image landscape of today.

Here one of the early Landsat images from the Landsat 1 Multispectral Scanner System from 1972. Since MSS did not record a blue spectral band this uses a synthetic blue channel estimated from the other spectral bands (i wrote about this in the context of ASTER images before).


April 11, 2018
by chris
0 comments

Codification of contact

This blog post discusses the idea of Codes of Conduct, these are documents regulating social interaction, in the OpenStreetMap project. I in particular want to focus on Codes of Conduct for non-virtual meetings, i.e. for events where people meet in person.

First a bit of background: Social interaction in a society is normally regulated by two different sets of rules

  • The social conventions – the non-codified standards of social interaction of a society, largely defined by tradition (we do things certain ways because our parents and grandparents have already done so) and often fairly specific in the details to the social class, subculture or even family.
  • The local justice system.

Normally following these two sets of rules in everyday life is something we can manage without much effort. But as indicated these are local rules. With cross cultural and international social interaction things become much more difficult. You are very likely to break social conventions in international social interaction and depend on tolerance and generosity of others in such cases. Because of this cross cultural international social interaction is usually characterized by careful and considerate actions and reactions in the attempt to find a common ground in terms of common social conventions. A significant part of this is also successfully managing failure of a working social interaction – the organized and respectful retreat from a failed attempt at such.

These mechanisms of cross cultural social interaction have developed over thousands of years. The world’s travel literature is full of stories and anecdotes about positive examples of such with eye level interaction and cultural exchange and negative examples with catastrophic failures sometimes leading to violent results as well as many cases of arrogance and narrow-mindedness leading to a peaceful but completely unbalanced interaction. If you know people well who have significant experience with eye level cross cultural interaction you can usually observe a distinct change in habitus and body language when they meet a person and realize a significant difference in social conventions. In cultures where balanced, peaceful interaction with other cultures is common (through trade and travel for example) these mechanisms often have found a place in the culture’s social conventions in form of certain rituals and procedures.

In my experience how well people can handle this also depends a lot on past experience in dealing with people following very different social conventions. For example people who have grown up in a rural area are often better at this because they tend to get exposed early in their life to the significantly different and contrasting social conventions of life in towns and cities while people growing up in a city – while they might routinely experience a larger variety in social conventions in their immediate environment (though usually through the anonymity of city life) they often never experience a similarly harsh contrast in those conventions until they have grown up.

And while today in a way we have more frequent cross cultural interaction than at any time in history due to real time international communication and relatively inexpensive travel opportunities most of this interaction tends to be highly asymmetrical and truly balanced eye level cross cultural social interaction has probably – relatively speaking – become a rare exception.

The corporate code of conduct

Codes of conduct were invented as an additional set of rules of social interaction in the context of corporations regulating interaction of corporations with their employees and among employees, being created by the corporate management and being contractually agreed upon by people.

Reasons for creating those are in particular

  • to avoid the sometimes unreliable nature of uncodified social conventions.
  • adjusting the social conventions of the ordinary employees (who might come from a significantly different social and cultural background) to those of the management for their convenience.
  • limiting some of the freedoms offered by the justice system and social conventions because they are not considered good for productivity.
  • avoiding conflicts due to differences in laws and social conventions of employees by imposing a uniform set of rules above them. This is in particular important for international corporations.

Practically a corporate code of conduct is typically meant to supersede social conventions and local laws. It can not normally contradict local laws (though there are quite a few cases where internal corporate rules are actually in conflict with legal requirements) but since code of conduct rules are usually more restrictive than general laws they practically form the relevant limits of accepted behavior.

If we now have the idea of creating a code of conduct for an international OpenStreetMap meeting – like the SOTM conference – we could have two potential goals with that based on what i explained above:

  • helping and supporting to engage in eye level cross cultural social interaction in the way i described above (i.e. careful and considerate interaction to try establishing a common ground in social conventions).
  • managing the event like a corporate event under a corporate code of conduct.

Now the SOTM CoC actually does neither of these. It does not provide any significant guidance how to perform cross cultural social interaction and it also lacks the clarity of rules and the goal orientation of a corporate code of conduct. Instead it comes closer to a third type which i would call a political code of conduct.

The political code of conduct

The political code of conduct is the result of the idea of a corporate code of conduct being adapted by and for social reform and social justice movements and organizations. The idea here is to – just like with a corporate code of conduct – essentially replace existing social conventions and laws (because they are considered injust) with a set of rules – in this case not designed to optimize productivity but to achieve certain political goals.

The political goals are not immediately obvious in the SOTM CoC since it has been toned down compared to the document it has been derived from.

Now i don’t want to judge the political ideas behind this but no matter what you think of them it should be clear that the resulting rules will primarily have the goal of implementing the political ideas (just like the corporate CoC wants to increase productivity). Most political CoCs are created in a culturally fairly homogeneous environment (like an organization of people with common social background and political goals). While there are occasionally translations of such documents in different languages i have never seen a CoC that has been designed with multilingual input and discussion.

All of this is highly problematic in the way that it does not allow people to freely seek and find an individual common ground in social conventions between them but imposes a certain set of social conventions from the top. No matter what the political motives for creating such rules are they always come from a specific cultural background and are imposed on the rest of a global and culturally diverse community in an act of cultural dominance.

What remains to be discussed is how a code of conduct could look like that is meant to help and support people to engage in cross cultural social interaction on eye level in the traditional way without a culturally biased rule set being imposed.

A culturally neutral code of conduct

Here an attempt at this. Since this is formulated in a certain language you can of course argue that it is not neutral anyway but i put quite a lot of effort into this not relying on a specific interpretation of language and the meaning of certain words but being based on general thoughts and ideas that just happen to be communicated here in a certain language.

Some might also think calling this a code of conduct is incorrect because it is so very different from most documents you see titled this way. I would use a quote from the CoC of the Chaos Communication Congress which pretty well describes the idea behind this draft as well:

This is not a CoC in the anglo-american sense of the word. It appeals to morality rather than trying to instill it.

The event you are participating in is visited by a large variety of people with very different cultural and social backgrounds as well as personal views, ideas and abilities. Experiencing this, getting to know such a large variety of very different people can be a very educative and enjoyable experience but also requires tolerance, curiosity and open-mindedness from the participants. If you are able and willing to bring this with you, you are very welcome to participate in the event. This document is meant to help you do that in a way that makes it a positive experience for all participants.

As guests and visitors of the event you are expected to conform with the local laws. You are also encouraged to familiarize yourself with the local customs and social conventions before and during the visit. This will help you during as well as outside the event.

When interacting with others at the event you need to expect and accept that other guests and visitors might have views, ideas and expectations very different to those you are familiar with. You are expected to be open-minded and tolerant towards such differences. We encourage you to reach out to, communicate and interact with others but when doing that you should be sensitive to them and to the possibility that your behaviour might make them uncomfortable.

We expect you to always treat others at the event with at least the same level of respect, tolerance and generosity as you expect and depend on others to extend to you. To accomplish this you should try to always put the goal of a friendly and open-minded interaction and the comfort of others with this interaction above your specific goals in it – like for example an argument or a discussion you are having. As a participant of the event you are required to be willing to adjust your behaviour in the interest of others and at the same time should as much as you can avoid requiring others to adjust their behaviour to you.

The above rules and suggestions should avoid misunderstandings and conflicts and help resolve smaller issues amicably in most situations. In case of more difficult conflicts when interacting with others you are encouraged to approach other participants of the event to mediate the conflict. If others approach you to help with conflicts try to mediate by attempting to help people find a common ground without actually engaging in the conflict yourself. If you are unable to do so or in cases of more serious conflicts you should approach the organizers of the event. Our aim in such a situation will be to help the parties and if necessary give specific instructions to them which you are required to follow. Such intervention will always try as much as possible to stay neutral and not take sides in the conflict.

If you are a proponent of corporate or political CoCs you most certainly will not like this because it follows a very different approach to the problem of cross cultural social interaction. In my opinion this approach is the only way to organize cross cultural interaction in a non-judgemental way that allows all people of a globally diverse community opportunity to express themselves and have the chance for cross cultural exchange. You can still argue that you do not actually need such a document or you might want to shorten it further compared to the above.

The most common argument of proponents of political CoCs is that the rules are meant to protect the weak from the strong, the marginalized from the dominant and are therefore justifiable. But that is in itself based on putting specific social conventions which lead to the perception of weak and strong, of marginalized and dominant above others and is therefore culturally biased. The language and the words used by the CoCs themselves to set the limits of acceptable behavior already imply the dominance of certain social conventions – which is why my draft above is mostly limited to suggestions meant to help people in their social interaction (i.e. being educative rather than normative) and explaining fundamental ethical principles instead of imposing specific rules that require familiarity with the language and the underlying social conventions to follow them.

Another thing that should be kept in mind is that obviously the local justice system has a special role in the whole thing. This is not that different if you have no or a different kind of CoC obviously. So the question where to have an international meeting is a question where the justice system of the place in question has quite an impact.

How about virtual places?

Now how about CoCs for digital communication channels and platforms? If you have a truly global international channel the same as above applies naturally. But most digital channels are at least language specific and in case of OpenStreetMap often further specific to certain countries or regions. Then you can think about documenting certain common social conventions for those. But you need to keep in mind that you cannot claim the channel or platform in question is open for or be representing the whole global OSM community any more.

TLDR: Engaging in eye level cross cultural social interaction in a careful and considerate way guided by basic and universal moral principles and dominated by tolerance and respect for the other side and a willingness to accept differences in social conventions even if they are inconvenient – like essentially countless generations before us have for thousands of years managed peaceful cross cultural contact in the past – is not the best way to do this, it is just the only way.

Design oriented generalization of open geodata

March 20, 2018
by chris
0 comments

FOSSGIS 2018 and talk announcement

Tomorrow once again the annual FOSSGIS conference starts – this year in Bonn. At the moment it looks like it is going to be pretty damn cold…

I am going to present a talk on Thursday afternoon about design oriented generalization of open geodata.

Here a preview of two sample map renderings i am going to show:

Update: Video of the talk is available on media.ccc.de and on youtube.