I have been contemplating writing an update on this for more than a year now, but i have been hesitant because the conclusions i am going to draw in the end are pretty dire and i had still hoped for a turn to the better. And writing a blog post here with fairly negative conclusions seemed to me to be something that could further contribute to seal the fate of the project.
The history of the project
I will start by roughly sketching the history of OpenStreetMap-Carto. For further illustration of that i also produced a comparison gallery of how different versions of OSM-Carto render the current OSM data in various places around the world. The project began in 2012 with Andy Allan porting the old Mapnik XML based style, that had powered the main map of OpenStreetMap so far, to CartoCSS. CartoCSS was one of the most significant innovations in rule-based digital map design as it was the first (and – depending on your perspective – might still be the only) format for describing rule-based map design that was developed with primarily the map designer in mind and not the comfort and liking of the tool developers (as it was the case for Mapnik XML and is for example for JSON based formats that became popular later).
Andy’s contribution was not just the hand-work of formally porting the style, he also did the important ground work to determine how the CartoCSS based framework would need to be used to scale to the level of complexity the style already had back then.
The port to CartoCSS and the availability of TileMill as a fairly easy to use tool to develop CartoCSS styles encouraged quite a few people to newly get into map style development. Most of the initial work was technical restructuring. Technically the style gained its current shape, in particular with the structure of the roads layers, during these first years. Feature additions were also made to some extent, but the overall appearance and design initially stayed very close to the Mapnik XML style.
Originally Andy Allan managed everything on the project himself, but after some time Matthijs Melissen, Paul Norman and Mateusz Konieczny were added as maintainers who could also merge changes and the system of consensus decisions of the maintainers was established. Later, in 2016, Daniel Koć, myself and Michael Glanznig were appointed as additional maintainers, in 2017 also Lukas Sommer and in 2019 Joseph Eisenberg. Of these only Michael Glanznig has formally retired since, but both Andy Allan and Matthijs Melissen have completely withdrawn from active participation in the project now – leaving five more or less active maintainers.
2015 and 2016 were kind of the golden years of the project. With the interest raised and the better accessibility of style development through CartoCSS, quite a few people by that time had gained enough experience to tackle bigger design challenges the project’s development and the needs of the OSM community created. This in particular refers to the style OpenStreetMap-Carto was based on being strongly connected design wise to the British origin of the project. During the time from 2015 to 2016 the style essentially emancipated itself from this relatively narrow cultural origin and became a truly international project. Components of this emancipation were in particular:
- Refactoring of urban landuse and building coloring as the first step into systematic color design in OSM-Carto.
- The roads rendering redesign developed by Mateusz as part of a GSOC project – departing from the traditional British color scheme and adjusting road rendering for good readability in a wide variety of settings world wide.
- Moving to a standardized design of POI symbols for most features rendered with point symbols – work that has in particular been pursued by Michael.
- Introducing algorithm generated relaxed random periodic patterns for depicting a larger variety of landcover types beyond what is possible with just fill colors – paired with systematic and holistic design of the area fill color scheme in the style. This in particular facilitated the introduction of a significant number of new landcover types that are rare in areas where historically OpenStreetMap first became popular but widespread in other parts of the world – bare ground landcovers, various wetland types and reefs.
These accomplishments in making the style more international and serving the potential global community in a more balanced fashion were made possible through various technical and design innovations. For this short period OpenStreetMap-Carto was truly avant-garde in the development of rule-based map design.
What i especially think we have reason to be proud of is that we agreed back then on a document outlining uniquely progressive fundamental goals of the project. In particular the idea that we aim to create a map style for the potential global map user and no special consideration is given to the current geographic distribution of actual map use is a fairly revolutionary concept that has, to my knowledge, never be formulated in such clarity anywhere else. And it was not only a goal on paper, we seriously tried to work towards this goal as i hope the list of changes above illustrates somewhat.
But this quite amazing development came at a price:
- we were not perfect and while we introduced many significant improvements, there were also things that got lost on the way. Noteworthy is in particular that the depiction of paths for non-automobile transport was significantly better overall in the old Mapnik XML style. The look back at what was lost after changes have been made is something that was and still is done much too rarely.
- we outgrew the possibilities of the technical framework we were using. While initially what could be and what was done in OSM-Carto was mostly limited by our collective limits in knowledge and experience of how the possibilities our toolchain offered could be efficiently used to create a better map, developers soon mastered these possibilities and reached the limit of what was easily possible. At the same time active development of Mapnik and Carto slowed and ultimately mostly stopped. The options offered by CartoCSS in combination with PostGIS are still manifold but with no higher level abstractions in the tools using those amounts to adding significant levels of complexity on the style level or in form of additional tools and working with script generated CartoCSS/SQL code to integrate those into the rendering framework.
- the move to become a more global and internationally focused map style was not really that popular. Quite a few people in the OSM community who – coming from regions traditionally influential in OSM, in particular urban mappers from the UK, Europe and North America – were naturally expecting OSM-Carto to primarily cater their interests. Some of them were disappointed by OSM-Carto having a different focus and not quickly supporting the latest trends in urban mapping when they started becoming popular in these regions. Others disliked the fact that OSM-Carto was so distinctly and ostentatiously different from mainstream commercial maps like those from Google and Mapbox and would have very much preferred us to align map design more to what they were used to from these maps.
- most importantly: This development meant that the skills required to productively work on improving the style were gradually increasing. That in particular meant that for new contributors it became more and more difficult to make substantial changes to the style without first becoming familiar with the design principles of the project.
These developments led to increasing divergence in the maintainer team about the future direction of the project. I discussed this part of the history of OSM-Carto in more depth in my previous blog posts on the project here. In April 2017 the decision was made to give the individual maintainers more autonomy and we maintained this modus of operations until late 2018. Joseph summarized this nicely here.
The hope was that by giving maintainers autonomy in making decisions we could avoid the impasse we had reached on many strategic questions and get to a sustainable dynamic development of the style again. Unfortunately that did not happen. What happened instead was that development got dominated by a relatively small group of people who developed primarily feature additions and design adjustments of interest from an urban European perspective that were often at odds with the overall design paradigm and the goals of the project as they had been pursued before. At the same time, none of the big challenges the style faced, many of which were visible already in 2016, were actually addressed. Many of the developers and maintainers active before who would have been qualified to work on bigger changes, withdrew from active development of OSM-Carto. We have discussed these dynamics on the issue tracker and those were among the main reasons why we moved back to the consensus principle after about 1.5 years.
During this time the style moved significantly away from a consensus position that all active maintainers could support. This created the massive challenge to find a way back to a consensus among the project maintainers about the strategic direction of the style once the consensus principle was re-established. And despite significant efforts, in particular by Joseph, we did not manage to do so. To this date there is no strategic consensus among the maintainers and design wise development is mostly stalled. We have managed to make a number of technical changes – including big ones again but design wise no agreement can be found on most larger changes and even reverting changes that were made without consensus does not find agreement.
This continues to be the situation to this date. Some maintainers (including myself) continue to review changes that are being suggested but releases have become very sporadic and usually contain little of substance in terms of visible changes. I try to – even when i disapprove of a change – indicate that i would be fine with the other maintainers accepting the change none the less if they agree on it, this way actively supporting a more robust form of consensus. But that is no substitute for an actual common goal and a strategic direction we agree on. And most capable style developers have stopped contributing to the project because the lack of a clear overall direction does not provide for a supportive environment to work on good map design.
What went wrong?
It would, of course, be easy to now point back to the decision to abolish the consensus principle in 2017 and say: This is the wrong decision from which on everything went wrong. And i could even say i warned about the potential side effect of the project loosing the common sense for the strategic direction back then. But i don’t think this would be a complete assessment of the situation.
From early in its history OSM-Carto was burdened with the fact that it was the only community map design project in the OSM community with a global scope and that did not have a specialized thematic focus. As a result, a lot of people projected their ideas, interests and needs onto the project. If you try to find some structure in these diverse influences you can in my eyes identify three main strategic directions that the style has been pushed in:
- The populist direction. During the 1.5 years when consensus requirements were lifted and maintainers could make decisions autonomously in OSM-Carto, we pretty much followed this approach in purity. It means focus in development is on small, atomic changes, made and decided on without a larger overall design strategy and essentially allowing anyone who would like to change something to do so, at least as long as it has substantial popular support from people with a loud voice. This has the tendency to lead towards what i, in my previous blog post about OSM-Carto, called a naive art like style. We have already seen that in the long term this is non-sustainable because it fails to consider the need for larger and systematic changes to maintain the technical viability of the project. And it also fails to acknowledge that the usability of a map depends on an overall design concept and strategy that is being consistently followed. But that does not mean it is not an attractive short term model for developing a map style. In case of an OSM map style influential on mappers, there is a strong risk that over time mappers will start and will need to start compensating for the deficits of such a direction by essentially hand painting the map through their mapping work. However, in moderation (meaning in particular to mitigate the imbalance resulting from the loud voices appearing to represent the popular opinion) popular involvement is an essential component both for evaluating design and for recruiting talented and qualified people in development, in particular in case of a cooperative community project like OSM-Carto.
- The technocratic direction. In purity this means essentially treating style development as software development. The style is treated as a software to process the map data into a graphical representation. You try to do so efficiently – both in terms of run time performance and w.r.t. code complexity. But you pay no significant attention to the non-technical requirements there are for the resulting map. Many digital map design projects work close to this model, they are managed by software engineers with little background and little ambition in actual map design. This model has one major flaw: Map style development is not software development. As a result, such maps at best play a bit clumsily with design possibilities. And the resulting map is usually imbalanced, overly abstract and detached from the geographic reality it tries to represent. On the other hand, since digital maps are powered by technology, the technical expertise of software developers is of course a crucial component for the long term sustainability of any map style.
- The artistic direction. This direction means focusing on the design component of the map and being guided by the pursuit of excellence in this field. Artistic here is not meant in the sense of pure art but to contrast map design work based on an understanding of map viewing situation and a competent use of colors and visual structures in light of this with the purely technical approach. There are very few map styles in the field of automated rule-based digital cartography so far that put a strong and sustained emphasis on this direction, largely because map designers with artistic ambition still predominantly design maps through manual processing of concrete data. For me this has always been the most interesting direction, exactly because it is a field in its nascence with lots of unexplored territory. In purity the pursuit of this direction would likely lead to highly sophisticated maps which are technically inefficient, difficult to maintain and which constantly struggle with the technical limitations of the map design frameworks they use. But this is also the direction where there is clearly the most unused potential, both in terms of actual map design, as well as in terms of talented and capable designers who could be recruited.
These three paradigms are fundamentally contradicting each other and it seems that map development projects have the tendency to concentrate on one or two of these focus directions and neglect the other(s), because balancing all three directions is difficult and takes a lot of effort. At the same time, as i tried to explain, neglecting one or two of these directions will, in the long term, not be sustainable because all three components are essential.
The core of the problem
Until 2016 we essentially managed to unite these three principal directions in a common style, because we made use of the significant options the technical framework we used and our own expanding capabilities and skills in map design gave us. This reservoir of capabilities made it possible to bridge the gaps between these, otherwise, largely incompatible ideas. But we were already reaching the limits of this development in 2016 as hinted above – both in terms of the technical framework, the capabilities of which we started to outgrow, and in terms of the human design skills as a factor (which made it increasingly difficult to find and recruit talented and capable people as contributors and to educate them in the specific needs of the style).
What we would have needed back in 2016 and also now (though with the baggage of the history of the past years this seems rather unrealistic) is a new effort of consolidation similar to what has happened at the beginning of OSM-Carto. That would in particular mean tapping and developing new technical possibilities, because – as mentioned above – we already outgrew the possibilities of the technical framework we were using back in 2016. This is challenging because most of the components of the framework OSM-Carto uses are not actively maintained any more and there is no open source rendering system on the horizon that could replace it. Therefore i have suggested for quite some time that it is essential for the OSM community to strategically invest into map rendering systems that can produce high rendering quality. Rendering systems suitable in terms of the map styling interfaces, to be used in cooperative community projects, and that scale well towards higher sophistication in map design. This would be paramount to ensure the project’s ability to actively work on and substantially participate in the discourse on graphical representation of OpenStreetMap data in maps.
Where things might go in the future
Back in 2017 when OSM-Carto moved away from consensus based decision making i started the alternative colors style as a fork of OSM-Carto. Initially it was meant purely to demonstrate how the design principles we had developed in OSM-Carto until 2016 could continue to serve as a basis for a balanced map design for a truly global audience. But over the years it became increasingly clear to me that OSM-Carto would likely not find its way back to a balance between the three main strategic directions. And it also became clear that beyond OSM-Carto, community cartography in OpenStreetMap would most likely focus on the technocratic and populist directions. Businesses and non-commercial organizations around OSM produce maps with a predominantly technocratic direction. The OpenStreetmap Foundation has, for some time now, indicated an interest to actively bootstrap new community map design projects – and the OSMF is dominated by people with technical backgrounds. The OSMF board as the main decision making body even consists exclusively of people with a technical IT background. To mitigate this deficit they will likely try to give strong voices from the mapper community a bit of a say in these projects. But no one active in the OSMF has in the past ever indicated to value competence and experience in the artistic aspects of map design.
Long story short – in light of it being highly likely that future OSM community map design will predominantly be negotiated between the populist and technocratic direction, i increasingly started to try to explore the potential of the artistic direction. I don’t have the capacity or the skills to really consolidate the technical basis of map development and to develop map rendering technology that would give map design truly the options to move to the next level as i sketched it to be the desirable future for OSM-Carto above. Hence, much of what i implemented in the alternative colors style is technically somewhat awkward – most ostentatiously visible in the >1500 lines long SQL query for the new roads rendering system. But working on such case studies in advanced map design is an important and necessary step i think to determine what capabilities a future map design framework would need to have and how it would best be structured to truly provide a suitable environment for a balanced map style to flourish in a similar way as CartoCSS and Mapnik did for OSM-Carto in 2015 and 2016.
If the OSM community will be in any way actively involved in shaping a future of map design along these lines and in a balance of all the three main directions i described above, is an open question, but as already indicated i consider the chances for this to be quite small. OSM-Carto is still the most likely basis on which this might happen because it already provides a huge amount of preexisting ground work for this. Starting a new project from scratch would have its benefits as well, but the amount of basic, yet careful and considerate design work that would be necessary to bring such an endeavor off the ground would be quite massive.
The other question is if the future of cooperative digital rule-based map design will be based on free open source software or not. Considering how dominant open source is in automatically rendered digital map production these days, this might seem a silly question. But as mentioned above, map design with significant artistic ambition is still predominantly based on manual processing of concrete data and not designing rules for processing generic data these days and, in that field, open source software only plays a marginal role so far. If the FOSS community and the OSM community do not strategically invest in this direction, it is very likely that commercial software companies and service providers will – based on the good standing they already have with map designers with a more artistic and less technical orientation – successfully try to dominate that field.
Reflections on the social limits of community map design
In my previous blog posts on OSM-Carto i discussed quite a bit about the social dynamics of cooperative community map design and pointed out that OSM-Carto is also a social experiment testing if cooperative map design on this scale is possible and sustainable. It becomes increasingly clear now that this experiment has failed. But does this mean that a cooperative map design endeavor that is not either based on a centralized authoritarian leadership or on a small, culturally homogeneous team of people with common interests cannot work? That is a possibility. But not the only one i think. Above i listed what i identified as a number of factors that had led in 2016 to an increasing divergence in the maintainer team about the future direction of the project. None of these factors is however inevitably making such a cooperative effort unfeasible.
The key points that i would identify to be essential for a future experiment in a similar direction, to create a balanced general purpose OpenStreetMap map style for a truly global audience through open community cooperation, to work would be:
- Bootstraping such a project is a step of paramount importance. In OSM-Carto we had a reasonably diverse group of developers at least roughly covering all the three main directions i consider essential for a well balanced and sustainable map. We managed to codify the key ideas and values of the project into a guiding document but the agreement on these common ideas and values did not last long enough for the project to assemble a developer community invested enough into these values under the more adverse conditions the project was going to meet in the coming years.
- Such a project needs long term strategic support from an active software development community invested in supporting map design not only as a technocratic endeavor but that also values and supports specifically the artistic direction. If a map design project outgrows the technical framework it is based on, like OSM-Carto increasingly did in 2016, this is a huge problem. Not only because it technically makes it harder to implement design wise necessary or desired changes, but also because it makes contributing increasingly unattractive for the most qualified designers who strive for excellence in design. A map design project where the appearance of the map is not primarily determined by the conscious decision of designers about how they want the map to look, but by the limitations of the software used, is destined for mediocrity and is never going to be a balanced map well serving its design goals.
- Significant effort needs to be put into writing down practical guidance and providing educative materials and training regarding the practical design rules and principles followed by the project. This is especially important as the field of automated rule based map design is so young and so little material discussing this topic is available at all.
I would consider it likely that if these points can be addressed and followed diligently from the start, a map design project like OSM-Carto can work in the long term. But it is definitely hard and failure is likely. One of the key challenges OSM-Carto struggled with and that any project with similar ambitions will struggle with in the near future, is that it is essentially in uncharted waters not only as a social endeavor, like i discussed previously, but also in terms of map design itself. This is both a huge chance and a burden.