You are browsing the archive for Rufus Pollock.

Ubernomics: Platform Monopolies & How to Fix Them

- September 6, 2018 in Open Economics, Open/Closed

First version: Dec 2016, updated Feb 2018. This blog is a summary of the full article at http://rufuspollock.com/ubernomics Around the world countries have struggled to work out how to deal with Uber, AirBnB and their like. Are these new apps something to be welcomed or something to be stopped? But how we treat Uber-like companies is largely dependent on how we see and understand them. There is, in fact, nothing especially digital about the underlying economics of Uber’s business model. In fact, Uber’s economics are very similar to businesses that have been with us for hundreds or even thousands of years: marketplaces / platforms. Because of the positive feedback between buyers and sellers, the initial free competition between marketplaces tends towards “one” – one dominant marketplace. And, hence, to monopoly if that one marketplace is exclusively owned and controlled. Monopoly is not inevitable though: we can have open platforms/marketplaces – just as we have free and open markets. To do this requires us to open the digital infrastructure and, where appropriate, to open the order book too. Note: we’ll use the term platform and marketplace interchangeably. Today, “platform” is the more common term in the digital world and in economics (“two-sided platforms” etc). However, here we often prefer “marketplace” because it connects this with something both familiar and ancient.

Market Day in Stockport in 1910s. Link

1. Uber-like companies are marketplace or platform companies

Stripped to their essence Uber, AirBnB and the like resemble a very old economic structure: the marketplace. Marketplaces are where buyers and sellers come together to exchange. On Uber this is riders and drivers. On AirBnB owners and renters etc. Marketplaces have existed for thousands of years, practically since civilization first began. More broadly, we have platforms. This includes companies like Facebook, Google, Microsoft and eBay. Facebook is a platform mediating between users – and advertisers. Google is a platform mediating between users and content – and advertisers. Microsoft’s Windows is an operating system platform mediating between apps and users. Strictly platforms as broader than marketplaces. For example, an operating system or social network is a platform but not, strictly, a marketplace. However, many of the same ideas apply and so the distinction does not matter much here.

2. Platforms tend to one because of positive feedback between buyers and sellers

Like a snowball down a mountain, marketplaces, once past a critical size, have the potential to grow rapidly thanks to positive feedback where buyers and sellers both value size because it offers:
  • liquidity: you will be able to trade e.g. book a taxi, rent an apartment etc
  • diversity: they have the product you want e.g. this particular fish is available, that stock has a market maker, there is a taxi in your area (not just central London)
Furthermore, buyers and sellers usually don’t want to have to participate in lots of different marketplaces (“multi-homing” is a pain). Combined with the positive feedback effects this creates a strong pressure for there to be just one marketplaces.

3. Marketplaces tend to monopoly (unless made open)

Because of snowball economics over time you converge on just one marketplaces – “marketplaces tend to one”. You don’t have ten fish markets in a town, you have one. You don’t have fifty stock exchanges, you have one. The question then is: is that marketplace “closed”: exclusively owned and controlled by one entity. If so it becomes a monopoly. Or is an “open” marketplace where anyone can participate on fair and equitable terms? Note: there can be substantial competition to become the monopolist. There may also be some competition between regional monopolies when there is enough geographic or preference diversity: if you live in Scotland you won’t go to London to buy your fish so several local fish markets can exist (with limited competition between them at the fringes).

4. These marketplaces are not “contestable”

It is not easy to build a new one and compete against the old one. Why? Because buyers are numerous and independent coordination between them is very hard. The same is true for sellers (though to a lesser extent because sellers are usually less numerous and diverse than buyers – fifty fishmongers at a market might supply thousands of fish-buyers). This makes coordinated action – such as switching to a different competing market – very hard: as a buyer I don’t want to head over to the new fish market only to discover all the fish sellers are still at the old marketplace. Similarly, no fish-seller wants to risk moving their stall to the new marketplace until they know the buyers will all be there – its a chicken and egg problem with thousands and chickens and eggs who all need to act simultaneously!

5. Thus, the monopoly marketplace owner has a lot of power

Thus, the owner of marketplace has a lot of power – once the marketplace is established. At an early stage marketplace industries will often be highly dynamic and competitive as firms fight to get critical mass and dominate the market. This can mislead policy-makers into believing the market is competitive which in turn prevents them from acting at a crucial early stage when it would be relatively easy to put in place long-term pro-competitive policies (e.g. establishing a neutral exchange, or regulated marketplace access rates).

6. That power is inevitably abused to the detriment of buyers and consumers

When an organization has a lot of power it will use it to its advantage. In the case of the marketplace, the obvious thing is for the owner to start aggressively charging the users of the marketplace for access. Depending exactly on how the marketplace works it can charge buyers, sellers or both. Often charging sellers is preferred because they are easier to identify, contract with and track (and they have a larger and better sense of the value of the marketplace per entity). Thus, it is Uber’s drivers who get charged the 20-25% fee by Uber (this fee, of course, gets passed on to consumers but they don’t directly see it). Note: a side benefit of charging sellers is that it makes the fees largely hidden to buyers which is good both for PR and politically: if buyers got upset they might start pushing politicians to regulate the marketplace. For example, most people think that Google is just wonderful because it provides them with a valuable service for “free”. They don’t see, of course, that they do pay – just indirectly through the sellers (advertisers and content providers) who have to pay Google or supply Google with free content.

7. The solution is to open the marketplace

The solution to marketplace monopolies is to make the marketplace open: accessible to all buyers and sellers on equitable and non-discriminatory terms. This involves two parts:
  • Opening the software, protocols and non-personal data that power the marketplace.
  • Universal, equitable access to the order book database with pricing set to cover the cost of maintenance. Preferably this would involve the order book being run and managed by an independent third-party with governance in place to ensure a transparent and equitable pricing and access policy.
It is worth emphasizing that competition between proprietary, closed, marketplaces is not sufficient. Openness is essential.

8. Remuneration rights can pay for open

It costs money to create the software, protocols and (non-personal) data that power a marketplace. Traditionally, entrepreneurs and investors fund the creation of these based on the hope of becoming a marketplace monopolist and making it rich. Without the monopoly why would they invest? One option would be farsighted funding by the state – as with the Internet. However, this is problematic: how can the state know exactly which entrepreneurs should be backed with what ideas? Instead, we can use remuneration rights. These provide a free-market-like but open-compatible way to fund innovators. In essence, remuneration rights combine a common subscription payment from citizens organized by the government combined with a market-based payment of those monies to innovators based on whose innovations get used. You can read more about these ideas in my book The Open Revolution. Finally, actually running the platform itself costs money even if the protocols and software are free and open to use – you still need data centers and sysadmins to keep the servers running. With the software and protocols being free, service providers can freely compete and users will have a choice of who they use – just as we choose today between different “Internet Service Providers” who operate the Internet and provide us with access to it.

9. Pro-activity is essential

Policymakers and stakeholders need to take a pro-active approach. It is far easier to shape a marketplace towards openness early in its development than it is to handle an entrenched and powerful monopolist marketplace once in place. Read on  

Solving the Internet Monopolies Problem – Facebook, Google et al

- May 9, 2018 in open, Open Knowledge Definition, Open/Closed

The good news is that an increasing number of people seem to agree that:
  1. Facebook, Google etc are monopolies
  2. That is a problem
Agreeing we have a problem is always a crucial first step. But to go further we need to:
  1. Correctly diagnose the disease — in particular, avoid confusing the symptoms with the root cause
  2. Identify a cure that actually works
On point one, the answer is that he root cause is costless copying (plus platform effects) combined with monopoly rights. Costless copying (and platform effects) would always lead to some kind of standardization — witness the Internet. But that is not a problem — in fact it is advantage to have a single standard. The problem arises when the standard platform is owned by one entity and becomes a monopoly as we have today with Google in search, Facebook in social networking etc. The solution flows from the diagnosis: make these platforms open and have an open-compatible, market-oriented way to pay innovators (i.e. have remuneration rights). By make a platform open I mean make its protocols, algorithms, software and know-how open, free to all to use, build on and share. More details on all of this in my upcoming book: http://rufuspollock.com/book

Photo by Jonas Lee on Unsplash

An example of a mis-diagnosis: their control of our personal data

One prevalent misdiagnosis is that the issue with Facebook and Google and the source of (much) of their monopoly power is to do with their control of our personal data. See, for example, the Economist’s cover in May 2017 showing Internet monopolies as oil rigs mining personal data – an allusion to the common assertion that “personal data is the new oil”.

From this mis-diagnosis flows a proposed solution: limit Facebook and Google’s access to our personal data and/or ensure others have access to that personal data on equal terms (“data portability”). Data portability means almost nothing in a world where you have a dominant network. So what if I can get my data out of Facebook if no other network has a critical mass of participants. What is needed is that Facebook has a live, open read/write API that allows other platforms to connect if authorized by the user. In fact, personal data is a practical irrelevancies to the monopoly issue. Focusing on it serves only to distract us from the real solutions. Limiting Facebook’s and Google’s access to our personal data or making it more portable would make very little difference to their monopoly power, or reduce the deleterious effects of that power on innovation and freedom — the key freedoms of enterprise, choice and thought. It make little difference because their monopoly just doesn’t arise from their access to our personal data. Instead it comes from massive economies of scale (costless copying) plus platform effects. If you removed Google’s and Facebook’s ability to use personal data to target ads tomorrow it would make very little difference to their top or bottom lines because their monopoly on our attention would be little changed and their ad targeting would be little diminished — in Google’s case the fact you type in a specific search from a particular location is already enough to target effectively and similar Facebook’s knowledge of your broad demographic characteristics would be enough given the lock-hold they have on our online attention. What is needed in Google’s case is openness of the platform and in Facebook’s openness combined with guaranteed interoperability (“data portability” means little if everyone is on Facebook!). Worse, focusing on privacy actually reinforces their monopoly position. It does so because privacy concerns:

  • Increase compliance costs which burden less wealthy competitors disproportionately. In particular, increased compliance costs make it harder for new firms to enter the market. A classic example is the “right to be forgotten” which actually makes it harder for alternative search firms to compete with Google.
  • Make it harder to get (permitted) access to user data on the platform and it is precisely (user-permitted) read/write access to a platform’s data that is the best chance for competition. In fact, it now gives monopolists the perfect excuse to deny such access: Facebook can now deny other competing firms (user-permitted) access to user data citing “privacy concerns”.

An example of a misguided solution: build a new open-source decentralized social network

Similarly, the idea sometimes put forward that we just need another open-source decentralized social network is completely implausible (even if run by Tim Berners-Lee*). Platforms/networks like Facebook tend to standardize: witness phone networks, postal networks, electricity networks and even the Internet. We don’t want lots of incompatible social networks. We want one open one — just like we have one open Internet. In addition, the idea that some open-source decentralized effort is going to take on an entrenched highly resourced monopoly on its own is ludicrous (the only hope would be if there was serious state assistance and regulation — just in the way that China got its own social networks by effectively excluding Facebook). Instead, in the case of Facebook we need to address the monopoly at root: networks like this will always tend to standardization. The solution is ensure that we get an open rather than closed, proprietary global social network — just like we got with the open Internet. Right now that would mean enforcing equal access rights to facebook API for competitors or, enforcing full open sourcing of key parts of the software and tech stack plus getting guarantees ongoing non-discriminatory API access. Even more importantly we need to prevent these kind of monopolies in future — we want to stop shutting the door after the horse has bolted! This means systematic funding of open protocols and platforms. By open i mean the software, algorithms and non-personal data are open. And we need to fund the innovators who create and develop these and the way to do that is replacing patents/copyright with remuneration rights.

Fake news: confusing which symptoms are related to which disease

We must also be careful not to confuse which symptoms are related to which disease. For example, fake news *is* a problem but it is only tangentially related to the disease of monopoly. The causes of fake news are many and various and much more complex than simple monopoly. In fact, one could argue that more diversity in media actually makes echo chambers worse. Reducing monopoly at Facebook and Google level may bring *some* improvement but it is probably secondary. For more, see https://rufuspollock.com/2016/11/26/fake-news-post-truth—is-it-news-and-what-can-we-do/
* see as an example of just this kind of proposal https://meanjin.com.au/essays/the-last-days-of-reality/

Requiem for an Internet Dream

- December 12, 2017 in Internet, open, Open/Closed

The dream of the Internet is dying. Killed by its children. We have barely noticed its demise and done even less to save it. It was a dream of openness, of unprecedented technological and social freedom to connect and innovate. Whilst expressed in technology, it was a dream that was, in essence, political and social. A dream of equality of opportunity, of equality of standing, and of liberty. A world where anyone could connect and almost everyone did. No-one controlled or owned the Internet; no one person or group decided who got on it or who didn’t. It was open to all. But that dream is dying. Whilst the Internet will continue in its literal, physical sense, its spirit is disappearing. In its place, we are getting a technological infrastructure dominated by a handful of platforms which are proprietary, centralized and monopolized. Slowly, subtly, we no longer directly access the Net. Instead, we live within the cocoons created by the Internet’s biggest children. No longer do you go online: you go on Facebook or you Google something. In those cocoons we seem happy, endlessly-scrolling through our carefully curated feeds, barely, if ever, needing to venture beyond those safe blue walls to the Net beyond. And if not on Facebook, we’ll be on Google, our friendly guide to the overwhelming, unruly hinterlands of the untamed Net. Like Facebook, Google is helpfully ensuring that we need never leave, that everything is right there on its pages. They are hoovering up more and more websites into the vastness that is the Googleplex. Chopping them up and giving them back to us in the bite-sized morsels we need. Soon we will never need to go elsewhere, not even to Wikipedia, because Google will have helpfully integrated whatever it was we needed; the only things left will be the advertisers who have something to sell (and who Google need to pay them). As the famous Microsoft mantra went: embrace, extend, extinguish. Facebook, Google, Apple and the like have done this beautifully, aided by our transition back from the browser to the walled garden of mobile. And this achievement is all the more ironic for its unintended nature; if questioned, Facebook and Google would honestly protest their innocence. Let me be clear, this is not a requiem for some half-warm libertarianism. The Internet is not a new domain, and it must play by laws and jurisdictions of the states in which it lives. I am no subscriber to independence declarations or visions of brave new worlds. What I mourn is something both smaller and bigger. The disappearance of something rare and special: proof that digital was different, that platforms at a planetary scale could be open, and that from that magical combination of tech and openness something special flowed. Not only speech and freedom of speech, but also innovation and creativity in all its wondrous fecundity and generous, organized chaos on a scale previously unimagined. And we must understand that the death of this dream was not inevitable. It is why I hesitate to use the word dream. Dreams always fade in the morning; we always wake up. This was not so much a dream as possibility. A delicate one, and a rare one. After all, the history of technology and innovation is full of proprietary platforms and exclusive control — of domination by the one or the few. The Internet was different. It was like language: available to all, almost as a birthright. And in the intoxicating rush of discovery we neglected to realise how rare it was. What a strange and wonderful set of circumstances had caused its birth: massive, far-sighted government investment at DARPA, an incubation in an open-oriented academia, maturity before anyone realised its commercial importance, and its lucky escape in the 1990s from control by the likes of AOL or MSN. And then, as the web took off, it was free, so clearly, unarguably, and powerfully valuable for its openness that none could directly touch it. The Internet’s power was not a result of technology but of a social and political choice. The choice of openness. The fact that every single major specification of how the Internet worked was open and free for anyone to use. That production grade implementations of those specifications were available as open software — thanks to government support. That a rich Internet culture grew that acknowledged and valued that openness, along with the bottom-up, informal innovation that went with it. We must see this, because even if it is too late to save the Internet dream, we can use our grief to inspire a renewed commitment to the openness that was its essence, to open information and open platforms. And so, even as we take off our hats to watch the Internet pass in all its funereal splendour, in our hearts we can have hope that its dream will live again.

An interview with Rufus Pollock – Why I am Excited about MyData 2016 in Finland

- August 17, 2016 in Events, network, OK Finland, Open Knowledge

A few weeks ago I sat down for a virtual interview with Molly Schwartz from Open Knowledge Finland about my thoughts on open data and mydata and why I am so excited about the MyData 2016 conference taking place from August 31 to September 2 in Helsinki which is being organized by Open Knowledge Finland in partnership with Aalto University and Fing. You can register for MyData 2016 here. Discount price for the members of the Open Knowledge Network is just 220 eur / 3 day conference. Ask for the discount code from (jogi@okf.fi) before registering at MyData 2016 Holvi store. You can also apply to be a volunteer for the conference. This event shares many of the same organizers as the 2012 Open Knowledge Festival in Helsinki so you can expect the same spirit of fun, creativity and quality that made that such an incredible experience.

Transcript:

Molly Schwartz: “So hi everybody, this is Molly Schwartz here, one of the team members helping to put on the MyData conference in Helsinki from August 31 to September 2. And I’m sitting here with one of our plenary speakers, Dr. Rufus Pollock, who is one of founders and the president of Open Knowledge, a worldwide network working to provide access to more open and broad datasets. So we’re very excited to have him here. So, Rufus, something that not a lot of people know is that MyData is actually an initiative that was born out the Finnish chapter of Open Knowledge (OKFFI), how do you feel about things that were kind of started by your idea spring up of their own accord?” Rufus Pollock: Well, it’s inspirational and obviously really satisfying. And no just in a personal way: it’s just wonderful to see how things flourish. Open Knowledge Finland have been an incredibly active chapter. I first went to Finland in I think it was 2010, and I was inspired then. Finland is just a place where you have a feeling you are in in a wise societ. The way they approach things, they’re very engaged but they have that non-attachment, a rigor of looking at things, and also trying things out. Somehow there’s not a lot of ego, people are very curious to learn and also to try things out, and I think deep down are incredibly innovative. And I think this event is really in that tradition. I think the area of personal and MyData is a huge issue, and one with a lot of connections to open data, even if it’s distinct. So I think it’s a very natural thing for a chapter from Open Knowledge to be taking on and looking at because it’s central to how we look at the information society, the knowledge society, of the 21st Century. MS: Definitely. I totally agree. I like that you brought up that this concept of personal data is somewhat distinct, but it’s inevitably tied to this concept of opening data. Oftentimes opening datasets, you’re dealing with personal datasets as well. So, what are the kind of things they’re you’re planning to speak about, loosely, at the conference, and what do you look forward to hearing from other people who will be at the MyData? RP: Yes, that’s a great question. So, what am I looking to talk about and engage with and what am I looking forward to hearing about? Well, maybe I’ll take the second first. What I am looking forward to I think one of the reasons I’m really excited to participate and come is it’s the area where – even though I obviously know a lot about data and open data – this area of personal data is one where I am not as much an expert – by a long way. So I’m really curious to hear about it and especially about things like: what is the policy landscape? What do people think are the big things that are coming up? I’m really interested to see what the business is looking at. There’s been quite a lot of discussion about how one could innovate in this space in a way that is both a wider opportunity for people to use data, personal data in usable ways, maybe in health care, maybe in giving people credit, I mean in all kinds of areas. But how do you do that in a way that respects and preserves people’s privacy, and so on. So, I think that’s really interesting as well, and again I’m not so up on that space. I’m looking forward to meeting and hearing from some of the people in that area. my data 2016 And similarly on the policy, on the business side, and also on the civil society side and on the research side. I’ve heard about things like differential privacy andsome of the breakthroughs we’ve had over the last years about how one might be able to allow people like researchers to analyse information, like genetics, like healthcare without getting direct access to the individual data and creating privacy issues. And there’s a clearly a lot of value one could have from researchers being able to look at, for example, at genomic data from individuals across a bunch of them. But it’s also crucial to be able to preserve privacy there, and what are the kind of things going on there? And the research side I think would also touch on the policy side of matters as well. What I would like to contribute That brings to what, for my part, I would like to contribute. I think Open Knowledge and we generally are on a journey at a policy level. We’ve got this incredible information revolution, this digital revolution, which means we’re living in a world of bits, and we need to make sure that world works for everyone. And that it works, in the sense that, rather than delivering more inequality, which is could easily do so, and more exploitation, it give us fairness and empowerment, it brings freedom rather than manipulation or oppression. And I think openness is just key there. And this vision of openness isn’t limited to just government we can do it for all public datasets. By public datasets I don’t just mean government datasets, I mean datasets that you can legitimately share with anyone. Now private, personal data that you can’t legitimately give to anyone, or share with anyone — or you shouldn’t be able to! So I think an interesting question is how those two things go together — the public datasets and the private, personal data. How they go together both in overall policy, but also in the mind of the public and of citizens and so on — how are they linked? And this issue of how we manage information in the 21st century doesn’t just stop at some line where it’s like, oh, it’s public data, you know, and therefore we can look at it this way. Those of us working to make a world of open information have to look at private data to. At Open Knowledge we have always had this metaphor of a coin. And one side of this coin is public data, e.g. government data. Now that you can open to everyone, everyone is empowered to have access. Now the flip side of that coin is YOUR data, your personal data. And your data is yours: you should get to choose how it’s shared and how it’s used. Now while Open Knowledge is generally focused on, if you like, the public side, and will continue to do so, overall I think across the network this issue of personal data is just huge, it has this huge linkage. And I think the same principles can be applied. Just as for open data what we say is that people have freedom to access, share, use, government, whatever data is being opened, so with YOUR data, YOU should be empowered to access, share, and use that, as YOU see fit. And right now that is just not the case. And that’s what leads to the abuses we get concerned about, but it’s also what stops some of the innovation and stops people from being empowered and able to understand and take action on their own lives — what might you learn from having my last five years of, say, your shopping receipts or mobile phone location data. Ultimately what happens to public data and what happens to personal data, they’re interconnected, both in people’s minds and, in a sense, they don’t just care about one thing or another, they care about, how is digital information going to work, how’s my data going to be managed, how’s the world’s data going to be managed. I also think MyData is some of the most relevant issues for ordinary people. For example, just recently I had to check if someone paid me and it was just a nightmare. I had to scroll back through endless screens on my online banking account to find ways to download different files to piece it all together. Why didn’t they let me download all the data in a convenient way rather than having to dig forever and then only get the last three months. They’ve got that data on their servers, why can’t I have it? And, you know, maybe not only do I want it, but maybe there’s some part I would share anonymized, it could be aggregated and we could discover patterns that might be important — just, as one example we might be able to estimate inflation better. Or take energy use: I would happily share my house’s energy use data with people, even if it does tell you when I go to bed, I’d be happy to share if that let’s us discover how to make things environmentally better. The word I think at the heart of it is empowerment. We at Open Knowledge want to see people empowered in the information age to understand, to make choices, to hold power to account, and one of the fundamental things is you being empowered with the information about you that companies or governments have, and we think you should be given access to that, and you should be choosing who else has access to it, and not the company, and not the government, per se. MS: Yes. And that’s exactly the principle of why MyData came out of Open Knowledge, as you mentioned earlier, the idea of why can not these principles of Open Knowledge, of the datasets we want to be receiving, also apply to our data that we would like to be open in the same way back to us? RP: Absolutely correct Molly, I mean just yes, absolutely. MS: And that’s why it’s also so interesting, so many people have been talking about kind of this inherent tension between openness and privacy, and kind of, changing how we’re thinking about that, and seeing it actually as the same principles just being applied to individual people. RP: Exactly, back in 2013 I wrote a post with my co-CEO Laura James about this idea and even used the term MyData. There’s an underlying unity that you’re pointing out that actually is a deep principle. Remember openness isn’t an end in itself, right, it’s a means to an end – like money! And the purpose of having information and opening it up is to empower human beings to do something, to understand, to innovate, to learn, to discover, to earn a living, whatever it is. And that idea of empowerment, fundamentally, is common in both threads, both to MyData and personal data and access to that, and the access to public data for everyone. So I think you are totally right. MS: Yes. So, thank you so much Rufus for joining us today, we are so looking forward to having you at the conference. You mention that you’ve been to Finland before. How long ago was that? RP: I was t there in 2012 for Open Knowledge Festival which was amazing. And then in 2010. Finland is an amazing place, Helsinki is an amazing place, and it will be an amazing event, so I really invite you to come along to the conference. MS: I second that, and it’s many of the same people who are involved in organizing the Open Knowledge Festival who are involved in organizing MyData, so we can expect much of the same. RP: A brilliant programme, high quality people. An incredible kind of combination of kind of joy and reliability, so you’ll have an amazing time, come join us. MS: Yes. Ok, so thank you Rufus, and we will see you in August! RP: See you in August!

Why Open Source Software Matters for Government and Civic Tech – and How to Support It

- July 13, 2016 in Featured, Open Data, Open Software, Open/Closed, Policy, research

Today we’re publishing a new research paper looking at whether free/open source software matters for government and civic tech. Matters in the sense that it should have a deep and strategic role in government IT and policy rather than just being a “nice to have” or something “we use when we can”. As the paper shows the answer is a strong yes: open source software does matter for government and civic tech — and, conversely, government matters for open source. The paper covers:
  • Why open software is especially important for government and civic tech
  • Why open software needs special support and treatment by government (and funders)
  • What specific actions can be taken to provide this support for open software by government (and funders)
We also discuss how software is different from other things that government traditionally buy or fund. This difference is why government cannot buy software like it buys office furniture or procures the building of bridges — and why buying open matters so much. The paper is authored by our President and Founder Dr Rufus Pollock.

Why Open Software

We begin with four facts about software and government which form a basis for the conclusions and recommendations that follow.
  1. The economics of software: software has high fixed costs and low (zero) marginal costs and it is also incremental in that new code builds on old. The cost structure creates a fundamental dilemma between finding ways to fund the fixed cost, e.g. by having proprietary software and raising prices; and promoting optimal access by setting the price at the marginal cost level of zero. In resolving this dilemma, proprietary software models favour the funding of fixed costs but at the price of inefficiently raised pricing and hampering future development, whilst open source models favour efficient pricing and access but face the challenge of funding the fixed costs to create high quality software in the first place. The incremental nature of software sharpens this dilemma and contributes to technological and vendor lock-in.

  2. Switching costs are significant: it is (increasingly) costly to switch off a given piece of software once you start using it. This is because you make “asset (software) specific investments”: in learning how to use the software, integrating the software with your systems, extending and customizing the software, etc. These all mean there are often substantial costs associated with switching to an alternative later.
  3. The future matters and is difficult to know: software is used for a long time — whether in its original or upgraded form. Knowing the future is therefore especially important in purchasing software. Predictions about the future in relation to software are especially hard because of its complex nature and adaptability; behavioural biases mean the level of uncertainty and likely future change are underestimated. Together these mean lock-in is under-estimated.
  4. Governments are bad at negotiating, especially in this environment, and hence the lock-in problem is especially acute for Government. Government are generally poor decision-makers and bargainers due to the incentives faced by government as a whole and by individuals within government. They are especially weak when having to make trade-offs between the near-term and the more distant future. They are even weaker when the future is complex, uncertain and hard to specify contractually up front. Software procurement has all of these characteristics, making it particularly prone to error compared to other government procurement areas.

The Logic of Support

Note: numbers in brackets e.g. (1) refer to one of the four observations of the previous section.
A. Lock-in to Proprietary Software is a Problem Incremental Nature of Software (1) + Switching Costs (2)
imply …
Lock-in happens for a software technology, and, if it is proprietary, to a vendor Zero Marginal Cost of Software (1) + Uncertainty about the Future in user needs and technologies (3) + Governments are Poor Bargainers (4)
imply …
Lock-in to proprietary software is a problem
Lock-in has high costs and is under-estimated – especially so for government B. Open Source is a Solution Lock-in is a problem
imply …
Strategies that reduce lock-in are valuable Economics of Software (1)
imply …
Open-source is a strategy for government (and others) to reduce future lock-in
Why? Because it requires the software provider to make an up-front commitment to making the essential technology available both to users and other technologists at zero cost, both now and in the future Together these two points
imply …
Open source is a solution
And a specific commitment to open source in government / civic tech is important and valuable C. Open Source Needs Support
And Government / Civic Tech is an area where it can be provided effectively Software has high fixed costs and a challenge for open source is to secure sufficient support investment to cover these fixed costs (1 – Economics)
+
Governments are large spenders on IT and are bureaucratic: they can make rules to pre-commit up front (e.g. in procurement) and can feasibly coordinate whether at local, national or, even, international levels on buying and investment decisions related to software. imply … Government is especially well situated to support open source
AND
Government has the tools to provide systematic support
AND
Government should provide systematic support

How to Promote Open Software

We have established in the previous section that there is a strong basis for promoting open software. This section provides specific strategic and tactical suggestions for how to do that. There are five proposals that we summarize here. Each of these is covered in more detail in the main section below. We especially emphasize the potential of the last three options as it does not require up-front participation by government and can be boot-strapped with philanthropic funding. 1. Recognize and reward open source in IT procurement. Give open source explicit recognition and beneficial treatment in procurement. Specifically, introduce into government tenders: EITHER an explicit requirement for an open source solution OR a significant points value for open source in the scoring for solutions (more than 30% of the points on offer). 2. Make government IT procurement more agile and lightweight. Current methodologies follow a “spec and deliver” model in which government attempts to define a full spec up front and then seeks solutions that deliver against this. The spec and deliver model greatly diminishes the value of open source – which allows for rapid iteration in the open, and more rapid switching of provider – and implicitly builds lock-in to the selected provider whose solution is a black-box to the buyer. In addition, whilst theoretically shifting risk to the supplier of the software, given the difficulty of specifying software up front it really just inflates upfront costs (since the supplier has to price in risk) and sets the scene for complex and cumbersome later negotiations about under-specified elements. 3. Develop a marketing and business development support organization for open source in key markets (e.g. US and Europe). The organization would be small, at least initially, and focused on three closely related activity areas (in rough order of importance):
  1. General marketing of open source to government at both local and national level: getting in front of CIOs, explaining open source, demystifying and derisking it, making the case etc. This is not specific to any specific product or solution.

  2. Supporting open source businesses, especially those at an early-stage, in initial business development activities including: connecting startups to potential customers (“opening the rolodex”) and guidance in navigating the bureaucracy of government procurement including discovering and responding to RFPs.
  3. Promoting commercialization of open source by providing advice, training and support for open source startups and developers in commercializing and marketing their technology. Open source developers and startups are often strong on technology and weak on marketing and selling their solutions and this support would help address these deficiencies.
4. Open Offsets: establish target levels of open source financing combined with a “offsets” style scheme to discharge these obligations. An “Open Offsets” program would combine three components:
  1. Establish target commitments for funding open source for participants in the program who could include government, philanthropists and private sector. Targets would be a specific measurable figure like 20% of all IT spending or $5m.

  2. Participants discharge their funding commitment either through direct spending such as procurement or sponsorship or via purchase of open source “offsets”. “Offsets” enable organizations to discharge their open source funding obligation in an analogous manner to the way carbon offsets allow groups to deliver on their climate change commitments.
  3. Administrators of the open offset fund distribute the funds to relevant open source projects and communities in a transparent manner, likely using some combination of expert advice, community voting and value generated (this latter based on an estimate of the usage and value of created by given pieces of open software).
5. “Choose Open”: a grass-roots oriented campaign to promote open software in government and government run activities such as education. “Choose Open” would be modelled on recent initiatives in online political organizing such as “Move On” in the 2004 US Presidential election as well as online initiatives like Avaaz. It would combine central provision of message, materials and policy with localized community participation to drive change.

Rufus Pollock: I am now Non-Executive at Open Knowledge International

- February 26, 2016 in News

Today, I am pleased to announce the completion of my move to a non-executive role at Open Knowledge International. When Pavel Richter took over as CEO last year, my plan was to move to the role of President and transition to a non-executive position on a timeline that allowed me to support Pavel as he took over and drove Open Knowledge International forward. I’m delighted to say that this plan has progressed well. As a result, today I am happy to announce my move to a fully non-executive role is complete. Going forward, I will no longer have day to day responsibilities or involvement at Open Knowledge International — though I will, of course, continue to discharge any outstanding consulting or management responsibilities to particular projects and clients. I will also continue to provide ongoing advice and support to Pavel and the Leadership Team. Rufus_Pollock_1 This is a special moment for me: Open Knowledge has been a major passion of mine since I founded it over a decade over. It has also often been an all-consuming one, especially in the last five years as full-time CEO. Now, thanks to Pavel and the great team we have in place, I have the privilege of being to step back and watch it develop and grow whilst personally having the space to explore new interests and opportunities. Lastly, I want to emphasize that I am deeply committed to the ongoing success both of Open Knowledge International and the wider Open Knowledge community. In addition I remain passionate about openness and open knowledge. We have only just begun on our journey to create an open information age and I plan to remain active promoting openness around the world. I will continue to be an active community member and volunteer for Open Knowledge and the open knowledge movement. Finally, with my non-executive transition the best way to get in touch or keep up to date will be changing. Going forward please use my Open Knowledge Directory page or see my personal website at: http://rufuspollock.org/about/.

Announcement – Open Definition 2.1

- November 10, 2015 in Open Definition, Open Knowledge

Today Open Knowledge and the Open Definition Advisory Council are pleased to announce the release of version 2.1 of the Open Definition. The definition “sets out principles that define openness in relation to data and content” and continues to play a key role in supporting the growing open ecosystem. The Open Definition was first published in 2005 by Open Knowledge and is maintained today by an expert Advisory Council. This new version is a refinement of version 2.0, which was the most significant revision in the Definition’s nearly eleven-year history. This version is a result of over one year of discussion and consultation with the community including input from experts involved in open data, open access, open culture, open education, open government, and open source. This version continues to adhere to the core principles while strengthening and clarifying the Definition in three main areas.

What’s New

Version 2.1 incorporates the following changes:
  • Section 1.1 Open License or Status, formerly named Open License, has been changed to more clearly and explicitly include works that while not released under a license per se, are still considered open, such as works in the public domain.
  • In Version 2.0, section 1.3 specified the requirement for both machine readability and open formats. In Version 2.1 these requirements are now separated into their own sections 1.3 Machine Readability and 1.4 Open Format.
  • The new 1.4 Open Format section has been strengthened such that in order to be considered open, the work has to be able to be both in a format which places no restrictions, monetary or otherwise and it has to be able to be fully processed by at least one free/libre/open-source software tool. In version 2.0 only one of these conditions was needed to satisfy the requirement.
  • An attribution addendum has been added to recognize the work that the definition is based on.
Version 2.1 also includes several other less significant changes to enhance clarity and better convey the requirements and acceptable conditions.

More Information

Authors

This post was written by Herb Lainchbury, Chair of the Open Definition Advisory Council and Rufus Pollock, President and Founder of Open Knowledge.

Putting Open at the Heart of the Digital Age

- June 5, 2015 in Open Knowledge

Introduction

I’m Rufus Pollock. In 2004 I founded a non-profit called Open Knowledge The mission we set ourselves was to open up all public interest information – and see it used to create insight that drives change. What sort of public interest information? In short, all of it. From big issues like how our government spends our taxes or how fast climate change is happening to simple, everyday, things like when the next bus is arriving or the exact address of that coffee shop down the street. For the last decade, we have been pioneers and leaders in the open data and open knowledge movement. We wrote the original definition of open data in 2005, we’ve helped unlock thousands of datasets. And we’ve built tools like CKAN, that powers dozens of open data portals, like data.gov in the US and data.gov.uk in the UK. We’ve created a network of individuals and organizations in more than 30 countries, who are all working to make information open, because they want to drive insight and change. But today I’m not here to talk specifically about Open Knowledge or what we do. Instead, I want to step back and talk about the bigger picture. I want to talk to you about digital age, where all that glitters is bits, and why we need to put openness at its heart.

Gutenberg and Tyndale

To do that I first want to tell you a story. Its a true story and it happened a while ago – nearly 500 years ago. It involves two people. The first one is Johannes Gutenberg. In 1450 Gutenberg invented this: the printing press. Like the Internet in our own time, it was revolutionary. It is estimated that before the printing press was invented, there were just 30,000 books in all of Europe. 50 years later, there were more than 10 million. Revolutionary, then, though it moved at the pace of the fifteenth century, a pace of decades not years. Over the next five hundred years, Gutenberg’s invention would transform our ability to share knowledge and help create the modern world. The second is William Tyndale. He was born in England around 1494, so he grew up in world of Gutenberg’s invention. Tyndale followed the classic path of a scholar at the time and was ordained as a priest. In the 1510s, when he was still a young man, the Reformation still hadn’t happened and the Pope was supreme ruler of a united church across Europe. The Church – and the papacy – guarded its power over knowledge, forbidding the translation of the bible from Latin so that only its official priests could understand and interpret it. Tyndale had an independent mind. There’s a story that he got into an argument with a local priest. The priest told him: “We are better to be without God’s laws than the Pope’s.” Tyndale replied: “If God spare my life ere many years, I will cause the boy that drives the plow to know more of the scriptures than you!” What Tyndale meant was that he would open up the Bible to everyone. Tyndale made good on his promise. Having fled abroad to avoid persecution, between 1524 and 1527 he produced the first printed English translation of the Bible which was secretly shipped back to England hidden in the barrels of merchant ships. Despite being banned and publicly burnt, his translation spread rapidly, giving ordinary people access to the Bible and sowing the seeds of the Reformation in England. However, Tyndale did not live to see it. In hiding because of his efforts to liberate knowledge, he was betrayed and captured in 1534. Convicted of heresy for his work, on the 6th October 1536, he was strangled then burnt at the stake in a prison yard at Vilvoorden castle just north of modern day Brussels. He was just over 40 years old.

Internet

So let’s fast forward now back to today, or not quite today – the late 1990s. I go to college and I discover the Internet. It just hit me: wow! I remember days spent just surfing around. I’d always been an information junkie, and I felt like I’d found this incredible, never-ending information funfair. And I got that I was going to grow up in a special moment, at the transition to an information age. We’d be living in this magical world, where the the main thing we create and use – information – could be instantaneously and freely shared with everyone on the whole planet.

But … why Openness

So, OK the Internet’s awesome … Bet you haven’t heard that before! BUT … – and this is the big but. The Internet is NOT my religion. The Internet – and digital technology – are not enough. I’m not sure I have a religion at all, but if I believe in something in this digital age, I believe in openness. This talk is not about technology. It’s about how putting openness at the heart of the digital age is essential if we really want to make a difference, really create change, really challenge inequity and injustice. Which brings me back to Tyndale and Gutenberg.

Tyndale revisited

Because, you see, the person that inspired me wasn’t Gutenberg. It was Tyndale. Gutenberg created the technology that laid the groundwork for change. But the printing press could very well have been used to pump out more Latin bibles, which would then only have made it easier for local priests to be in charge of telling their congregations the word of God every Sunday. More of the same, basically. Tyndale did something different. Something so threatening to the powers that be that he was executed for it. What did he do? He translated the Bible into English. Of course, he needed the printing press. In a world of hand-copying by scribes or painstaking woodcut printing, it wouldn’t make much difference if the Bible was in English or not because so few people could get their hands on a copy. But, the printing press was just the means: it was Tyndale’s work putting the Bible in everyday language that actually opened it up. And he did this with the express purpose of empowering and liberating ordinary people – giving them the opportunity to understand, think and decide for themselves. This was open knowledge as freedom, open knowledge as systematic change. Now I’m not religious, but when I talk about opening up knowledge I am coming from a similar place: I want anyone and everyone to be able to access, build on and share that knowledge for themselves and for any purpose. I want everyone to have the power and freedom to use, create and share knowledge. Knowledge power in the 16th century was controlling the Bible. Today, in our data driven world it’s much broader: it’s about everything from maps to medicines, sonnets to statistics. Its about opening up all the essential information and building insight and knowledge together. This isn’t just dreaming – we have inspiring, concrete examples of what this means. Right now I’ll highlight just two: medicines and maps.

Example: Medicines

Everyday, millions of people around the world take billions of pills, of medicines. Whether those drugs actually do you good – and what side effects they have – is obviously essential information for researchers, for doctors, for patients, for regulators – pretty much everyone. We have a great way of assessing the effectiveness of drugs: randomized control trials in which a drug is compared to its next best alternative. So all we need is all the data on all those trials (this would be non-personal information only – any information that could identify individuals would be removed). In an Internet age you’d imagine that that this would be a simple matter – we just need all the data openly available and maybe some way to search it. You’d be wrong. Many studies, especially negative ones, are never published – the vast majority of studies are funded by industry who use restrictive contracts to control what gets published. Even where pharmaceutical companies are required to report on the clinical trials they perform, the regulator often keeps the information secret or publishes it as 8,000 page PDFs each page hand-scanned and unreadable by a computer. If you think I’m joking I’ll give just one very quick example which comes straight from Ben Goldacre’s Bad Pharma. In 2007 researchers in Europe wanted to review the evidence on a diet drug called rimonabant. They asked the European regulator for access to the original clinical trials information submitted when the drug was approved. For three years they were refused access on a variety of grounds. When they did get access this is what they got initially – that’s right 60 pages of blacked out PDF. We might think this was funny if it weren’t so deadly serious: in 2009, just before the researchers finally got access to the data, rimonabant was removed from the market on the grounds that it increased the risk of serious psychiatric problems and suicide. This situation needs to change. And I’m happy to say something is happening. Working with Ben Goldacre, author of Bad Pharma, we’ve just started the OpenTrials project. This will bring together all the data, on all the trials and link it together and make it open so that everyone from researchers to regulators, doctors to patients can find it, access it and use it.

Example: Maps

Our second example is maps. If you were looking for the “scriptures” of this age of digital data, you might well pick maps, or, more specifically the geographic data on which they are built. Geodata is everywhere: from every online purchase to the response to the recent earthquakes in Nepal. Though you may not realize it, most maps are closed and proprietary – you can’t get the raw data that underpins the map, you can’t alter it or adapt it yourself. But since 2004 a project called OpenStreetMap has been creating a completely open map of the planet – raw geodata and all. Not only is it open for access and reuse use the database itself is collaboratively built by hundreds of thousands of contributors from all over the world. What does this mean? Just one example. Because of its openness OpenStreetMap is perfect for rapid updating when disaster strikes – showing which bridges are out, which roads are still passable, what buildings are still standing. For example, when a disastrous earthquake struck Nepal in April this year, volunteers updated 13,199 miles of roads and 110,681 buildings in under 48 hours providing crucial support to relief efforts.

The Message not the Medium

To repeat then: technology is NOT teleology. The medium is NOT the message – and it’s the message that matters. The printing press made possible an “open” bible but it was Tyndale who made it open – and it was the openness that mattered. Digital technology gives us unprecedented potential for creativity, sharing, for freedom. But they are possible not inevitable. Technology alone does not make a choice for us. Remember that we’ve been here before: the printing press was revolutionary but we still ended up with a print media that was often dominated by the few and the powerful. Think of radio. If you read about how people talked about it in the 1910s and 1920s, it sounds like the way we used to talk about the Internet today. The radio was going to revolutionize human communications and society. It was going to enable a peer to peer world where everyone can broadcast, it was going to allow new forms of democracy and politics, etc. What happened? We got a one way medium, controlled by the state and a few huge corporations. Look around you today. The Internet’s costless transmission can – and is – just as easily creating information empires and information robber barons as it can creating digital democracy and information equality. We already know that this technology offers unprecedented opportunities for surveillance, for monitoring, for tracking. It can just as easily exploit us as empower us. We need to put openness at the heart of this information age, and at the heart of the Net, if we are really to realize its possibilities for freedom, empowerment, and connection. The fight then is on the soul of this information age and we have a choice. A choice of open versus closed. Of collaboration versus control. Of empowerment versus exploitation. Its a long road ahead – longer perhaps than our lifetimes. But we can walk it together. In this 21st century knowledge revolution, William Tyndale isn’t one person. It’s all of us, making small and big choices: from getting governments and private companies to release their data, to building open databases and infrastructures together, from choosing apps on your phone that are built on open to using social networks that give you control of your data rather than taking it from you. Let’s choose openness, let’s choose freedom, let’s choose the infinite possibilities of this digital age by putting openness at its heart. Thank you.

Open Knowledge appoints Pavel Richter as new CEO

- April 29, 2015 in Featured, News, Open Knowledge, Open Knowledge Foundation, Press

I am delighted to announce we have found the newest member of the Open Knowledge team: Pavel Richter joins us as our new CEO! Pavel Richter Pavel’s appointment marks a new chapter in the development of Open Knowledge, which, over the last ten years, has grown into one of the leading global organisations working on open data and open knowledge in government, research, and culture. Pavel has a rich and varied background including extensive time both in business and in the non-profit sector. In particular, Pavel brings his experience from over five years as the Executive Director of Wikimedia Deutschland: under his leadership, it grew to more than 70 staff, an annual budget of nearly 5 million Euros, and initiated major new projects such as Wikidata. Pavel’s engagement follows an extensive international search, led by a team including members of the Board of Directors as well as a Community Representative. Personally, I am delighted and excited to welcome Pavel as CEO. This appointment represents an important step in the development of Open Knowledge as an organisation and community. Over the last decade, and especially in the last five years, we have achieved an immense amount. Going forward one of our most important opportunities – and challenges – will be to forge and catalyse a truly global movement to put openness at the heart of the information age. Pavel’s experience, insight and passion make him more than equal to this task and I am thrilled to be able to work with him, and support him, as he takes on this role.

We’re Hiring at Open Knowledge: Project Managers, Developers and Data Wranglers

- March 19, 2015 in Featured, Jobs, Open Knowledge Foundation Local Groups

Open Knowledge are delighted to advertise several new open positions:
  • Project Manager – Open Data
  • Python Developer
  • Frontend Developer
  • Data Wrangler
A brief summary on each post can be found below. Full details and application forms can be found on http://okfn.org/about/jobs/.

 Project Manager – Open Data

We are looking for a professional and dynamic hands-on project manager to manage a portfolio of projects at Open Knowledge – an international, world-leading non-profit working on open data. The project management style will need to fit within our creative and innovative atmosphere, and should help us retain organisational flexibility and agility. The projects requiring management will vary, but in general will range from £25k/several months/1-2 team members, to £500k+/several years/4-6 team members. Some projects will involve substantial software design and delivery and require good technical understanding and the ability to manage a technology delivery project. In general the project teams are made up of specialists who have a good sense of the area of work and may be able to be the public face of the project; the key role of the project manager is to ensure planning, delivery, tracking and reporting occurs reliably and to a good standard. Open Knowledge’s partners and clients include national government bodies, NGOs, and organisations such as the World Bank. Projects funded by grants are delivered for philanthropic foundations, the European Commission (eg. through the FP7/H2020 programme) and others. Find out more or apply »

Python Developer

Working in the fast-growing area of open data, we build open source tools to drive transparency, accountability and data-driven insight. Our flagship product CKAN runs official national data portals from the UK to Brazil, US to Australia and hundreds more around the world. We also build a variety of other open source software to help people access information and turn data into insight. We’re looking for a python developer to join our team who is smart, quick-to-learn and interested in contributing to a fast-growing and exciting area. You can be based anywhere – we operate as a virtual, online team – and can offer a flexible working structure – we care about delivery, not being 9-5 at your desk! Find out more or apply »

Data Wrangler

Work on cutting edge data-driven, high impact open knowledge projects with a world-leading non-profit in areas ranging from government finances to health-care. We are looking for someone with good experience in “small-to-medium” data wrangling (e.g. you’ve been scraping in python for a while, have a deep love for CSV … ). You must be a self-starter, capable of working remotely and taking initiative as well as working effectively in a team with others. Find out more or apply »

Frontend Developer

We are looking for talented front-end developers to work with us on an ongoing, freelance basis on a variety of open-source data-driven projects ranging from healthcare to illegal logging. Find out more or apply »