You are browsing the archive for Open Data.

New Report: Avoiding data use silos – How governments can simplify the open licensing landscape

Danny Lämmerhirt - December 14, 2017 in licence, Open Data, Policy, research

Licence proliferation continues to be a major challenge for open data. When licensors decide to create custom licences instead of using standard open licences, it creates a number of problems. Users of open data may find it difficult and cumbersome to understand all legal arrangements. More importantly though, legal uncertainties and compatibility issues with many different licenses can have chilling effects on the reuse of data. This can create ‘data use silos’, a situation where users are legally allowed to only combine some data with one another, as most data would be legally impossible to use under the same terms. This counteracts efforts such as the European Digital Single Market strategy, prevents the free flow of (public sector) information and impedes the growth of data economies. Standardised licences can smoothen this process by clearly stating usage rights. Our latest report  ‘Avoiding data use silos – How governments can simplify the open licensing landscape’ explains why reusable standard licences, or putting the data in the public domain are the best options for governments. While the report has a focus on government, many of the recommendations can also apply to public sector bodies as well as publishers of works more broadly. The lack of centralised coordination within governments is a key driver of licence proliferation. Different phases along the licensing process influence government choices what open licences to apply – including clearance of copyright, policy development, and the development and application of individual licences. Our report also outlines how governments can harmonise the decision-making around open licences and ensure their compatibility. We aim to provide the ground for a renewed discussion around what good open licensing means – and inspire follow-up research on specific blockages of open licensing. We propose following best practices and recommendations for governments who wish to make their public sector information as reusable as possible:
  1. Publish clear notices that concisely inform users about their rights to reuse, combine and distribute information, in case data is exempt from copyright or similar rights.
  2. Align licence policies via inter-ministerial committees and collaborations with representative bodies for lower administrative levels. Consider appointing an agency overseeing and reviewing licensing decisions.
  3. Precisely define reusable standard licences in your policy tools. Clearly define a small number of highly compatible legal solutions. We recommend putting data into the public domain using Creative Commons Zero, or applying a standard open license like Creative Commons BY 4.0.
  4. If you still opt to use custom licences, carefully verify if provisions cause incompatibilities with other licences. Add compatibility statements explicitly naming the licences and licence versions compatible with a custom licence, and keep the licence text short, simple, and reader-friendly.

Custom licences used across a sample of 20 governments

Czech Open Data Challenge: a showcase of amazing transparency apps

Michaela Rybičková - December 7, 2017 in competition, czech republic, network, network updates, OK Czech, Open Data, Open Knowledge Network

This blog post was written by the Czech Republic Open Knowledge team as part of our blog series of Open Knowledge Network updates.  In the fifth edition of Czech open data challenge, interested parties from the ranks of the public, non-profit organizations and companies were invited to submit applications that use or generate open data. Applications developed between November 2016 and October 2017 could compete. This year, the competition was dominated by transparency apps. Many of the 24 contestants focused on improving the efficiency of public spending or parliamentary watchdog. Others chose to provide convenient access to information about pharmacies or publishing stats about lawyers. In this blog you can find more information about some of this year’s winners. The winner, Hlídač státu (http://www.hlidacstatu.cz, State watchdog), is a strong tool of control over public spending. It connects a registry of contracts with data about donations to political parties and presents it in a comprehensible manner. Michal Blaha, author of Hlídač státu, said that he takes his victory as a commitment. „Open data are making the public administration more democratic and transparent, as they balance the relationship between citizen and officer.” he explained. The second place was awarded to the civic initiative KohoVolit.eu for their Inventura hlasování (Inventory of voting in the Chamber of Deputies in 2013-2017, https://volebnikalkulacka.cz/cs/inventura-hlasovani-2017/). It is a user-friendly way to compare one’s opinions with voting of individual MPs. More than 400.000 people used the app ahead of the latest election. The third place was taken by Databáze prázdných domů (Database of Empty Houses, www.prazdnedomy.cz), which aggregates information about abandoned and decrepit buildings in Czechia. The project aims to save remarkable houses and find new uses for vacant real estate.

Screenshot of prazdnedomy.cz, visualising vacant real estate in Brno, Czech Republic

For the first time in the contest history, the Student prize was awarded to a middle school, Střední škola zemědělská a potravinářská Klatovy. A group of five youngsters led by an enthusiastic teacher spent one weekend at school to create interesting visualisations over real time data of the Czech parliamentary election: http://volby.maleskoly.info/ The Otakar Motejl Fund award for projects which increase government transparency, was given to CityVizor (https://cityvizor.cz/), a joint effort of Ministry of Finance and an alliance of cities. It is a unique example of the government cooperating with local administrations and helps to present city budgets and spending to citizens and allows the municipalities to share IT expenses. A special award went to the Czech Open Street Maps community, for their tireless effort of providing detailed and up-to-date map data. Being openly available, they are an invaluable resource for many successful businesses as well as civic initiatives. Another positive sign is, that political parties themselves start to leverage the power of open data and civic apps. The political party STAN for example built a mobile app which tracks votes and attendance of their MPs. The winners were awarded with prize money (up to 20 000 CZK), security software or trainings in online marketing.

Pin it in the Parks: Crowdsourcing park facilities information in Dublin

RouteToPA - November 24, 2017 in Open Data, Route to PA

Since 2015 Open Knowledge International has been part of the consortium of RouteToPA, a European innovation project aimed at improving citizen engagement by enabling meaningful interaction between open data users, open data publishers and open data. This month, the project is running the Pin it in the Parks competition together with Smart Dublin to encourage, inform and engage citizens and residents on services and issues in their local areas. This blog has been reposted from the RouteToPA website Ever wondered if there are tennis courts or exercise machines in the parks near you? What about playgrounds or skate parks or even historical monuments? A four weeks competition, Pin it in the Parks, encourages citizens to share information on the facilities available in parks near them in the city of Dublin. By using the RouteToPA android app, the user will have the ability to take photos of facilities they encounter and provide its exact location, hence making this information available to everyone. Here is what we know about some of Dublin’s park facilities, through information collected by SmartDublin & RouteToPA project:

This leafletjs-datalet was created by RTPA in his newsfeed from this dataset

Why is this competition important?

Local authorities are coming together all over Ireland to work together to improve access to information and to make data more accessible and easy to use. Participating in this competition will strengthen the role played by citizens and residents firstly to push towards more open data about issues that touch on their daily lives and secondly to raise awareness about the current state of the parks. Citizens and residents that are aware of their needs and their surrounding form a stronger pressure group. The competition encourages teamwork, as participants are strongly encouraged to apply in teams which will increase their chances of getting more points.

The information collected throughout this competition will push citizens to start discussions around their needs and the current states of the parks amongst other topics. RouteToPA, through its SPOD platform, allows participants to start and participate in the conversation around the data collected. Obtaining credible data is the first step towards finding solutions to the challenges a society might face.

Finally, keep an eye on the Pin it in the Parks website where the competition updates and final results will be made available.
 

Pin it in the Parks: Crowdsourcing park facilities information in Dublin

RouteToPA - November 24, 2017 in Open Data, Route to PA

Since 2015 Open Knowledge International has been part of the consortium of RouteToPA, a European innovation project aimed at improving citizen engagement by enabling meaningful interaction between open data users, open data publishers and open data. This month, the project is running the Pin it in the Parks competition together with Smart Dublin to encourage, inform and engage citizens and residents on services and issues in their local areas. This blog has been reposted from the RouteToPA website Ever wondered if there are tennis courts or exercise machines in the parks near you? What about playgrounds or skate parks or even historical monuments? A four weeks competition, Pin it in the Parks, encourages citizens to share information on the facilities available in parks near them in the city of Dublin. By using the RouteToPA android app, the user will have the ability to take photos of facilities they encounter and provide its exact location, hence making this information available to everyone. Here is what we know about some of Dublin’s park facilities, through information collected by SmartDublin & RouteToPA project:

This leafletjs-datalet was created by RTPA in his newsfeed from this dataset

Why is this competition important?

Local authorities are coming together all over Ireland to work together to improve access to information and to make data more accessible and easy to use. Participating in this competition will strengthen the role played by citizens and residents firstly to push towards more open data about issues that touch on their daily lives and secondly to raise awareness about the current state of the parks. Citizens and residents that are aware of their needs and their surrounding form a stronger pressure group. The competition encourages teamwork, as participants are strongly encouraged to apply in teams which will increase their chances of getting more points.

The information collected throughout this competition will push citizens to start discussions around their needs and the current states of the parks amongst other topics. RouteToPA, through its SPOD platform, allows participants to start and participate in the conversation around the data collected. Obtaining credible data is the first step towards finding solutions to the challenges a society might face.

Finally, keep an eye on the Pin it in the Parks website where the competition updates and final results will be made available.
 

「これからどうする、官民データ活用法」開催報告

okfj - November 16, 2017 in Events, Featured, News, Open Data, オープンデータ, 官民データ活用推進基本法

当日の様子 2017/11/14の夜に、オプンデータ・トークシリーズの第21回となる「これからどうする、官民データ活用法」を開催しました。当日参加いただいた皆様、ありがとうございました。 冒頭、内閣官房情報通信技術(IT)総合戦略室山路参事官より「政府におけるオープンデータの取組」についてご紹介頂きました。
都道府県の単位でみると40箇所がすでにオープンデータに取り組んでおり、残り7つ。基礎自治体では約260とまだまだこれからの状況です。政府としてさらに推進するために「自治体ガイドライン」「地方公共団体向けパッケージ」「オープンデータ100」「フォーマット標準例」「オープンデータ伝道師」など、モノとヒトの両面から支援の枠組みを用意しています。ぜひ周知、活用を進めて行きたいものです。
以下資料の見出しのみ列挙します。
1. 官民データ活用推進基本法関連の動き
2. 行政保有データの棚卸し
3. 官民データ相談窓口の設置
4. 官民ラウンドテーブルの開催(案)
5.地方公共団体によるオープンデータの取り組み 次に、OKJP副理事長の川島より「これからどうする 官民データ活用推進基本法」と題して、官民データ活用推進基本計画はきっかけであり、皆で誰もが情報の価値を最大限享受できる社会(知らなくて困ったのない社会)を創っていこう、という呼びかけが行われました。
以下見出しのみ列挙します。
1. データによる共創(Data Collaboratives)
2. ユースケースから考えよう
3. Glocal by Default
4. 一人一人が情報のリスクを判断できる社会へ
5. サービスとしてのコミュニティCommunity as a Service (CaaS) 最後に、参加者からの質問をその場で受け付け、いいねの多い順に答えていけるツールSli.doによる質疑応答が行われ、50件余りの質問が寄せられました。
以下、質問の一部をご紹介します。要望ばかりでなく自分たちでできることの提案があったり、また会場内から回答のフォローもあり、まさに官民連携した動きを作り出していこうという意気込みが感じられるイベントとなりました。()内の数字は会場のいいね数。
  • (9)自治体関係者が不在の中で、自治体の課題を議論しても限界がある気がします。国のデータの更なる公開に向けた取組についてお聞きできればと思います。
  • (7)地方公共団体のオープンデータへの取り組みに関して、自治体数でのみ評価されているが、実際は一度データを公開したっきり、その後の継続的な活動が行われていないような自治体もあるように思います。もっと質的な評価も必要ではないでしょうか?
  • (7)「官民データ活用推進計画策定の手引」にコミュニティ側からフィードバックをしたいです。Google Docs で原文が公開されたりしたら、提案しやすいなーなんて思います。
  • (7)地番や住居表示の位置情報のように公開されるのが当然と思われるデータでも公開できないと考えられているものもある。こういう考えて方を行政として議論して整理してほしい。
  • (7)自治体間でアイデアの共有、試行錯誤の共有をする場が必要だと思っています。そのような場を設定しませんか?
  • (7)地方公共団体のオープンデータへの取り組みに関して、自治体数でのみ評価されているが、実際は一度データを公開したっきり、その後の継続的な活動が行われていないような自治体もあるように思います。もっと質的な評価も必要ではないでしょうか?
  • (6)データとAPIはセットだと思います。 APIがあれば利用もしやすいです。
  • (6)小規模な自治体であれば、自治体内の事業者も限られていて、オープンデータ活用の需要が感じられない面もあるかと思います。例えば、人口10万人未満の小規模自治体でオープンデータ活用が進んだ事例があれば、教えてもらえませんか?
  • (6)行政の中で切り離しできるフォーマットの見直しは大賛成です!
  • (6)最近海外では、openwashingという単語がしばしば聞かれるようになりました。ショボいデータだけ公開するなど、クリティカルなデータを公開せず、「やったことにするオープンデータ」が次の課題のひとつと言われています。事例ベースだと、そのようなデータが公開の中心になる可能性が高いのではないでしょうか。
  • (6)先日の衆議院選挙はオープンデータになっていますか?
  • (6)本当に小さい自治体では自前のオープンデータサーバを持つのはしんどいのではないか。国が無償のサーバを提供できないか。
  • (6)オープンデータは可視化アプローチだけでなく、データ連携を対象とした場合、データフュージョンなど解析的なアプローチも必要かと考えています。こうした研究との連携について、自治体と民間と研究機関(データ分析企業等)との連携事例を作りませんか。

Annonce job : Développeur.se d’application mobile

Samuel Goeta - November 6, 2017 in biodiversité, Communiqué, job, OKFN France, Open Data, open knowledge France, opendata, Wikidata

Contexte

Le Pôle biodiversité forêts océans du Ministère de l’Europe et des Affaires étrangères et Open Knowledge France annoncent un partenariat afin de valoriser les données ouvertes et biens communs numériques dans la lutte contre le trafic d’espèces menacées. L’objectif du partenariat est de développer une application tout public pour identifier plus facilement les espèces menacées, déterminer leur statut juridique en fonction du pays, et informer sur les sanctions encourues. Le partenariat vise également à promouvoir l’accès, l’usage et la contribution aux données ouvertes sur les espèces menacées, notamment à travers les biens communs numériques tels que wikicommons ou wikidata. Ce partenariat, pour lequel Open Knowledge France a reçu un financement d’amorçage de 10 000 euros, est le résultat d’une collaboration entamée entre le Ministère et l’association lors du hackathon #Diplonum du Quai d’Orsay qui s’est tenu les 25 et 26 janvier dernier. Ce projet s’appuiera également sur des acteurs spécialisés dans la lutte contre le trafic d’espèces menacées tels que le Muséum national d’Histoire Naturelle, le secrétariat CITES qui gère la convention internationale sur les espèces menacées, le Centre de surveillance de la conservation de la nature (UNEP-WCMC) qui édite la base de données Species+, les Douanes ainsi que des spécialistes des projets wikidata et wikicommons.

La lutte contre le trafic d’espèces menacées

Le trafic d’espèces menacées, animales ou végétales, est aujourd’hui le quatrième commerce illégal dans le monde en terme de montant. Il permet de financer le terrorisme et contribue à l’extinction des espèces. En réponse, un cadre international de collaboration a été mis en place, la CITES, pour Convention sur le commerce international des espèces de faune et de flore sauvages menacées d’extinction. Ce cadre définit une liste des espèces menacées par pays, plus de 35 000, ainsi que les mesures de protection associées à appliquer, comme l’interdiction de la vente. En France, c’est le Ministère des Affaires Étrangères qui coordonne la lutte contre le trafic illégal d’espèces à l’international, à travers le Pôle biodiversité forêts océans et en lien avec d’autres institutions telles que les Douanes, le Ministère du Développement Durable mais également le Muséum d’Histoire Naturelle et le secrétariat international CITES basé à Genève au sein de l’UNEP, l’Agence des Nations Unis pour l’Environnement.

Les enjeux de l’accès aux données

L’une des principales difficultés de la mise en œuvre de l’accord CITES est de pouvoir identifier précisément quelles sont les espèces menacées et les sanctions à appliquer en fonction du pays. L’enjeu est à la fois de pouvoir mieux informer les différents intervenants dans la lutte contre le trafic, et en premier lieu, les agents des douanes et les autres forces de contrôle, mais également de mieux sensibiliser les potentiels acteurs du trafic (vendeurs, acheteurs, touristes, etc.) sur les sanctions encourues, et ce en fonction du pays. Il existe déjà un grand nombre d’informations sur la protection des espèces menacées, notamment le registre Species+ qui répertorie les espèces et leur statut juridique par pays. Cependant, ce registre est plutôt destiné aux experts, moins aux agents de douane ou au touristes, et n’a pas pour fonction l’identification d’une espèce. La plupart des informations sont par ailleurs uniquement disponibles en anglais.

Objectifs du partenariat

Ce partenariat propose donc de développer une application web mobile permettant de facilement identifier une espèce, déterminer son statut, et informer sur les sanctions. Une telle application pourra être utilisée à la fois pour sensibiliser et éduquer les touristes, mais également pour renforcer le pouvoir des forces de contrôle et acteurs du cadre CITES. Le second axe de ce partenariat est d’explorer les possibilités offertes par les communs numériques pour mieux gérer, promouvoir et utiliser les contenus en ligne relatifs aux espèces menacées tels que base de données, images, et textes de lois.

Votre mission

En tant que développeur d’application mobile, vous aurez la responsabilité de développer un prototype pour une application mobile (IOS/Android) qui aidera les utilisateurs à identifier les espèces menacées et informer sur leur statut juridique et les peines encourues. En tant que membre de l’équipe projet chez Open Knowledge France, vous serez impliqué dans ĺ’ensemble des phases de développement de l’application, du design au lancement officiel, et vous collaborez étroitement avec nos autres partenaires, notamment le Ministère des Affaires Étrangères et Européennes, le Muséum d’Histoire Naturelle, le secrétariat CITES et l’UNEP-WCMC. Vous pourrez également contribuer et apporter vos idées à d’autres tâches, telles que la gestion des données, l’identification des espèces, et l’organisation d’événements et de campagne de sensibilisation.. Enfin, vous aurez un intérêt direct dans la réussite de ce premier prototype qui doit nous permettre de sécuriser des financements et des partenariats additionnels pour la suite du projet.

Votre profil

  • Expert en Développement Applications Mobiles (iOS et Android)
  • Bonnes compétences en UX/UI
  • Compétences de base en développement Web (HTML 5, CSS 3, Javascript) et back-end
  • Maîtrise Français et Anglais
  • Expérience de développement avec des données et contenus Wikidata et Wikicommons serait un plus.
  • Intérêt pour l’open data et les communs numériques
  • Intérêt pour les questions environnementales

L’application

L’application utilisera les données de la base Species+ en combinaison avec d’autres sources de référence. Le projet souhaite également explorer l’utilisation des données des communs numériques, tels que wikidata ou wikicommons, notamment pour les images et autres informations manquantes dans la base Species+. Les spécifications techniques de l’application feront l’objet de discussion et validation durant la phase d’inception.

Pourquoi rejoindre ce projet ?

  • Pour contribuer à la préservation des espèces
  • Aider des institutions nationales et internationales dans leur lutte contre le trafic d’espèces
  • Promouvoir et sensibiliser à l’open data et aux communs numériques
  • Apprendre et collaborer avec des scientifiques et spécialistes de la biodiversité
  • Rejoindre le réseau Open Knowledge et la communauté d’acteurs des communs numériques et de l’open data.

Détail de la prestation

Début : dès que possible. De préférence dès Décembre 2017. Impératif Janvier 2018.
Montant : 5000 €
Type de contrat/prestation : à discuter en fonction de votre statut (auto-entrepreneur, stage, etc.)
Lieu : home office ou co-working de préférence près de Paris afin de pouvoir participer aux réunions avec les différents partenaires.
Possibles voyages en Europe (Genève, Cambridge, Bruxelles)

Comment postuler

Envoyer un CV, votre profil github si vous en avez un, portfolio et type de contrat/prestation à contrat@okfn.fr

Welcome to Open Knowledge Awards 2017

Serdar Temiz - October 17, 2017 in Open Data

okawards_logoStockholm, 2017/10/17 – Open Knowledge Sweden is pleased to announce the Open Knowledge Awards 2017. Open knowledge is knowledge, often in the form of data, available for everyone to freely use, reuse and redistribute without any sorts of restrictions. Open Knowledge Sweden envisions enlightened societies where everyone has access to key information in order to make daily or life changing choices with the help of relevant information, where authorities, institutions, businesses and other entities are fully transparent and at the same time, using open knowledge to increase innovation and efficiency on finding solutions. Open Knowledge should be a mainstream concept and a natural part of our everyday lives. The Open Knowledge Awards will help raise awareness on how society, including authorities, businesses, organizations and other entities can benefit from open knowledge. OK Awards covers categories such as transparency, entrepreneurship, open science, region/municipality and business initiative. The award winners will set an example of how businesses and organizations have best used open knowledge for innovative solutions, how authorities have been more transparent with the use of open knowledge and how public figures have used their influence for change in that direction, both cultural and legal. “We had our first OK Awards in collaboration with KTH and with support of Wikimedia Sweden last year and it was a great success. This year, in addition to KTH Industrial Marketing and Entrepreneurship Division, we also collaborate with Dataföreningen; expect to have more nominations and more guests at our event with support from the Open Knowledge Community. We believe that OKA is providing recognition to change makers that push for innovation as well as transparent and accountable democracy. It also raises the bar every year for all open knowledge stakeholders in Sweden.” – Serdar Temiz, Chairman of Open Knowledge Sweden To nominate entities/people and for more information about the OK Awards 2017 event: Visit our website www.okawards.org You can read more about previous years winners here Feel free to contact us regarding press, sponsorship or volunteer contribution Best regards,
Johanna de Vera Marketing/Project Manager
E-mail: johanna@okfn.se
Phone: +46(0)723237388

AbreLatam / Condatos: after the first 5 years

Oscar Montiel - October 12, 2017 in abrelatam, condatos, Events, Latin America, Open Data

This is a somewhat belated entry about the Abrelatam and Condatos, the regional open data conference of Latin America. It comes more than a month after the conference took place in San José, Costa Rica, but the questions raised there are still relevant and super important for advancing open data in Latin America and working towards truly open states. After five years, the discussions have shifted. We don’t only talk about open data and how to make it happen but about, for example: privacy and how we can make sure our governments will guarantee this the right to privacy in open data work; data standards and how to make them interoperable; and business models and how to be a sustainable organization that can last beyond project funding. These discussions are crucial in the current context in Latin America, with cases of corruption like Lava Jato or #GobiernoEspía in Mexico. They are particularly important if we want open data to not only be a bunch of good intentions, but rather infrastructure that is there for and because of citizens. Still, we have a big challenge ahead. As it was often commented in various sessions, we need to systematize all the knowledge we have gathered in these 5 years. We also need to be able to share it with the newcomers and open it up to organizations that aren’t traditionally in the open data sphere. This will help us avoid the echo chamber and keep the work focused on important matters and make open data a valuable asset in the construction of open states. At the same time, we need to learn from our mistakes, understand what has worked and what hasn’t, continue improving the work, not only go to conferences and speak about the amazing work we do, but also talk about where we make mistakes and help other avoid them. This won’t be an easy task, but I think we have the right ingredients to make it happen: we have a mature community that is eager to share its experiences and learnings. We’re ready to take on the next five years and construct an open region.  

Re-publica Thessaloniki: Ενώνοντας την Ευρώπη από το Βορρά έως το Νότο

Spyridoula Markou - September 17, 2017 in Featured, Featured @en, News, Open Data, re:publica, ανοικτά δεδομένα, δήμος, Εκδηλώσεις, Νέα, φεστιβάλ

Η ιδέα για το πρότζεκτ re:connecting Europe γεννήθηκε το Μάιο του 2017 στο Βερολίνο και το ταξίδι που θα ένωνε το Βορρά με το Νότο έφτασε στο τέλος του μετά την ολοκλήρωση του re:publica Thessaloniki. Οι εκδηλώσεις, που διήρκησαν τρεις ημέρες (11-13 Σεπτεμβρίου), περιελάμβαναν παρουσιάσεις και συζητήσεις με άτομα από τον πολιτικό χώρο, τις επιστήμες […]

Frictionless Data v1.0

Paul Walsh - September 5, 2017 in Frictionless Data, Open Data

  Data Containerisation hits v1.0! Announcing a major milestone in the Frictionless Data initiative. Today, we’re announcing a major milestone in the Frictionless Data initiative with the official v1.0 release of the Frictionless Data specifications, including Table Schema and Data Package, along with a robust set of pre-built tooling in Python, R, Javascript, Java, PHP and Go. Frictionless Data is a collection of lightweight specifications and tooling for effortless collection, sharing, and validation of data. After close to 10 years of iterative work on the specifications themselves, and the last 6 months of fine-tuning v1.0 release candidates, we are delighted to announce the availability of the following: We want to thank our funder, the Sloan Foundation, for making this release possible.

What’s inside

A brief overview of the main specifications follows. Further information is available on the specifications website.
  • Table Schema: Provides a schema for tabular data. Table Schema is well suited for use cases around handling and validating tabular data in plain text formats, and use cases that benefit from a portable, language agnostic schema format.
  • CSV Dialect: Provides a way to declare a dialect for CSV files.
  • Data Resource: Provides metadata for a data source in a consistent and machine-readable manner.
  • Data Package: Provide metadata for a collection of data sources in a consistent and machine-readable manner.
The specifications, and the code libraries that implement them, compose to form building blocks for working with data, as illustrated with the following diagram. This component based approach lends itself well to the type of data processing work we often encounter in working with open data. It has also enabled us to build higher-level applications that specifically target common open data workflows, such as our goodtables library for data validation, and our pipelines library for declarative ETL.

v1.0 work

In iterating towards a v1 of the specifications, we tried to sharpen our focus on the design philosophy of this work, and not be afraid to make significant, breaking changes in the name of increased simplicity and utility. What is the design philosophy behind this work, exactly?
  • Requirements that are driven by simplicity
  • Extensibility and customisation by design
  • Metadata that is human-editable and machine-usable
  • Reuse of existing standard formats for data
  • Language-, technology- and infrastructure-agnostic
In striving for these goals, we removed much ambiguity from the specifications, cut features that were under-defined, removed and reduced various types of optionality in the way things could be specified, and even made some implicit patterns explicit by way of creating two new specifications: Data Resource and Tabular Data Resource. See the specifications website for full information.

Next steps

We are preparing to submit Table Schema, Data Resource, Data Package, Tabular Data Resource and Tabular Data Package as IETF RFCs as soon as possible. Lastly, we’ve recently produced a video to explain our work on Frictionless Data. Here, you can get a high-level overview of the concepts and philosophy behind this work, presented by our President and Co-Founder Rufus Pollock.