You are browsing the archive for Open Fiscal Data.

Introducing Version 1 of the Fiscal Data Package specification

- May 28, 2018 in fiscal data package, Fiscal transparency, Open Fiscal Data, Open Spending

The Fiscal Data Package is a lightweight and user-oriented format for publishing and consuming fiscal data. Fiscal Data Packages are made of simple and universal components, are extremely flexible, can be produced from ordinary spreadsheet software and used in any environment. This specification started about five years ago with a first version (then known as the “Budget Data Package”). Since then we’ve made quite a few iterations, until a fairly stable version was reached, which we name ‘version 0.3’. This version was field-tested in various use cases and scenarios – most prominent among them is the Government of Mexico, who adopted Fiscal Data Package to be used for publishing their official budget data. For the past six months we’ve been hard at work in reshaping this specification to make it simpler to use and easier to adopt, while improving its flexibility and extensibility – thus making it relevant for more users. In many ways, this new version is the result of the collected experience and lessons learned in the past few years, working with partners and understanding what works and what doesn’t.

So what is the Fiscal Data Package philosophy?

The basic motivation behind Fiscal Data Package is to create a specification which is open by nature – based on other open standards, supported by open tools and software, modular, extensible and promoted transparently by a large community. The Fiscal Data Package is designed to be lightweight and simple to use – providing a small but flexible set of features, based on real-world requirements and not theoretical ones. All the while, the built-in extensibility allows this spec to adapt to many different use cases and domains. It is also possible to gradually use more and more part of this specification – progressive enhancement – thus making it easier to implement with existing data while slowly improving the data quality. A main concern we wanted to tackle was the ability to work with data as it currently exists, without forcing publishers to modify the contents or structure of their current data files in order to “adapt” them to the specification. This is a big deal, as publishers often publish data that’s the output of existing internal information systems, and requiring them to do any sort of data cleaning or wrangling on the data prior to uploading in a major source of friction for adoption.

And what is it not?

With that in mind, it’s also important to understand what this specification doesn’t handle. This specification is, by design, non-opinionated about which data should be published by publishers – which datasets, which fields and and the internal processes these reflect. The only things Fiscal Data Package is concerned with are how fiscal data should be packaged and providing means for publishers to best convey the meaning of the data – so it can be optimally used by consumers. In addition to that, it provides details regarding file-formats, data-types, metadata and structuring the data in files.

What we learned

As previously mentioned, via a wide range of technical implementations, partner piloting, and fiscal data projects with other civic tech and data journalist partners, we’ve learned a lot about what works in Fiscal Data Package v0.3, and what does not. We want to take these learnings and make a more robust and future proof v1.0 of the specification. One of the first thing we noticed wasn’t working was fiscal modelling. Version 0.3 of the specification contained an elaborate system for the modelling of fiscal data. In practice, this system turned out to be too complicated for normal users and error prone (inconsistent models could be created). To add to that, modelling was not versatile enough to account for the very different source files existing with real users, nor was it expressive enough to convey the specific semantics required by these users. A few examples of this strictness include:
  • The predefined set of classifications for dimensions. This hard-coded list did not capture the richness of fiscal data ‘in the wild’, as it contained too few and too broad options.
  • Measure columns were assumed to be of a specific currency, disregarding datasets in which the currency is provided in a separate column (or non monetary measures).
  • Measure columns were assumed to be of a specific budgeting phase (out of 4 options) and of a single direction (income/expenditure), ignoring data sets which have different phases, or that the phase or direction are provided in a separate column – or data sets which are not related to budgets altogether…
Another lesson learned is about file formats. Contrary to what its name might suggest, the world of fiscal data files is a wild jungle – every sort and form of file exists there (if you just look hard enough). Now, while machines will always prefer to read data files in their denormalised (or unpivoted) form – as it’s the most verbose and straightforward one – publishers will often choose a more compact, pivoted form – and as the proverb goes, there is more than one way to pivot a table. Other publishers would take out from the file some of the data, and append it as a separate code list file, or split large files based on year, budget direction or department. Version 0.3 of the specification assumed data files would only be provided in a very specific pivoted form – which might apply to some cases, but practically failed on many other variations that we’ve encountered.

Many different variations

What new features does Fiscal Data Package v1.0 provide?

First of all, it introduces a novel and simple way for supporting a wide variation of data file structures – pivoted and unpivoted, with code-lists and without them, provided in a single file or spanning across multiple files. To do that we’ve added 3 different devices:
  • We added the concept of ‘constant fields’: while still supporting any form of metadata added to the Fiscal Data Package descriptor, adding a field with some constant data is often a cleaner and more complete way for adding missing information to the dataset.
  • Added built-in facility for ‘unpivoting’ (or de-normalising) the source data: data is no longer expected to be provided in a very specific pivoted form – any structure of the data is now supported.
  • Use of Foreign Keys for allowing use of code-lists as part of the specification.
When we know the structure of the data, it allows us to bring all datasets to a single structure. This is crucial for comparisons – how can we compare two datasets when their structure is different? When the structure is known, it’s easier to ask questions about the data and easily refer to a single data point in the data (e.g. “what was the allocated budget for this contract in 2016?”).

Denormalisation

The second big feature of Version 1 is the introduction of ColumnTypes. ColumnTypes are a lightweight taxonomy for describing the columns of a fiscal data file – that is, not the concepts but their representations. For example, these types are not concerned with ‘Deficit’, ‘Supplier’ or ‘Economic classification’ – these are fiscal concepts. However, when put into a data file, columns such as ‘Supplier last name’ or ‘Title of 2nd level of func. class. in Dutch’ might be used. ColumnTypes are concerned with the data files themselves – and provide a way to extract the concept out of the columns. ColumnTypes can be combined into taxonomies of similarly-themed types. In these taxonomies, it’s possible to define some relationships between different types – for example, indicate a few ColumnTypes are parts of a more abstract concept. It’s also possible to assign data types and validation rules to a ColumnType, and more. Alongside this specification we’re also releasing two fiscal taxonomies which serve as standards for publishing budget files and spending files. These can be found here:

What’s next?

This announcement is of a release candidate – we’re looking forward for getting feedback and collaborating with the open-data and fiscal-standard communities. We’re planning to update existing tools (such as OpenSpending) and to build new tools to support this specification and provide integrations for other systems. Lastly – all this work wouldn’t have been available without the support and collaboration with our partners – chief among them are GIFT – Global Initiative for Fiscal Transparency, as well as the International Budget Partnership, Omidyar Network, google.org, The World Bank, the Government of Mexico and many other pilot governments. We thank them all for generous support in making this work possible. We really believe that Fiscal Data Package is an opportunity for governments and organisations that see the benefit in publishing budgets to foster transparency as part of a liberal democracy. You are invited to join us on this journey, which many government partners such as Croatia, Guatemala, Burkina Faso and Mexico have already started.
It is needed more than ever.

Why public country-by-country reporting should be introduced in the UK and across Europe

- November 22, 2017 in cbcr, OD4TJ, Open Fiscal Data, paradise papers

The release of the Paradise Papers has drawn attention to international calls for greater tax transparency to tackle the issues raised by the leak of millions of documents detailing the offshore behaviour of some of the world’s richest people and corporations. Along with a renewed push for the creation of public beneficial ownership registers to stop the use of anonymous companies for illicit purposes, debate has focused on how a transparency measure called public country-by-country reporting – or public CBCR – may help dissuade multinational corporations from shifting their profits to tax havens or low tax jurisdictions.

Countries where country leaders, politicians, public officials, or their close family/associates are implicated in the Paradise Papers. Image: JayCoop

Calls for action on this issue from campaigners including the Tax Justice Network, Tax Justice UK, Oxfam and Transparency International have been echoed by politicians from Labour leader Jeremy Corbyn and a cross-party selection of UK MPs to MEPs and the European Commission’s Taxation and Customs Union. Some large companies and investors have also spoken out in favour. 78% of British voters would be in favour of public CBCR for multinationals present in UK, according to a new YouGov poll conducted for Oxfam. So what is public CBCR and what steps could politicians take to use it to shine a light on tax avoidance behaviour?

What is country-by-country reporting?

CBCR requires that corporations publish information about their economic activities in all of the countries where they operate. This includes information on the taxes they pay, the number of people they employ and the profits they report. In February, as part of our Open Data for Tax Justice project, Open Knowledge International published a white paper co-authored by Alex Cobham, Jonathan Gray and Richard Murphy which examined the prospects for creating a global public database on the tax contributions and economic activities of multinationals as measured by CBCR. This found that such a public database was possible and that a pilot database could be created by bringing together the best existing source of public CBCR information – disclosures made by European Union banking institutions in line with the Capital Requirements Directive IV (CRD IV) passed in 2013. In July, we took the first steps towards the creation of this pilot.

What should be done now?

While our white paper found that some sources of public CBCR are available now, debates on expanding this measure to apply to a wider range of corporations have been slow to progress in the UK and across Europe.    The UK took the lead by legislating in favour of public country-by-country reporting in the 2016 Finance Act. But the final cross-party amendment from Caroline Flint MP did not include an implementation date and the UK government now says it will only “support the development of a public country-by-country reporting model that operates on a multilateral basis”. In July 2017, the European Parliament voted in favour of requiring all large multinational corporations to publish public CBCR information as open data. But this issue has now been passed over to the Council of the European Union for them to consider how to proceed and it is unclear how or when a decision by EU member states will be taken. The issue will continue to be debated in coming weeks. On 22nd November, Nigel Mills MP will lead a debate on public CBCR at the Houses of Parliament in the UK. Oxfam have called on the UK government to enforce comprehensive public CBCR for UK companies by the end of 2019. Shortly afterwards, a special Paradise Papers hearing will be held at the European Parliament on 28th November. And in early December, the final recommendations from the Panama Papers committee at the European Parliament which call for “the need of an ambitious public country-by-country reporting (CBCR) in order to enhance tax transparency and public scrutiny of multinational enterprises (MNEs)” are due to be voted on by MEPs. To tackle the issues exposed by the Paradise Papers, we urge the UK government to take the lead by putting their 2016 powers into action and encourage the European Parliament and EU Council to endorse swift moves towards public CBCR to help expose and tackle profit shifting by multinational corporations. Please email contact@datafortaxjustice.net if you’d like to be added to the Open Data for Tax Justice mailing list or want to join the Open Data for Tax Justice network. You can also follow the #OD4TJ hashtag on Twitter for updates.

Openbudgets.eu: the new platform for financial transparency in Europe

- September 7, 2017 in open budget, open budget data, open budgets, Open Fiscal Data, Open Spending

Today, OpenBudgets officially launches its fiscal transparency platform. Using
OpenBudgets.eu journalists, civil servants, and data scientists can process, analyse, and explore the nature and relevance of fiscal data.
The platform offers a toolbox to everyone who wants to upload, visualise and analyse fiscal data. From easy to use visualisations and high level analytics to fun games and accessible explanations of public budgeting and corruption practices along with
participatory budgeting tools, it caters to the needs of journalists, activists, policy makers and civil servants alike. The first successful implementations and projects have been developed in Thessaloniki, Paris, and Bonn, where civil society organisations and civil servants have together built budget visualisation for the general public.The cooperation between IT and administration resulted in 3 local instances of OpenBudgets.eu, setting the example for future implementations around Europe. On the EU level, the project has campaigned for transparency in MEP expenses and better quality data on the European subsidies. The OpenBudgets.eu project
subsidystories has uncovered how almost 300 billion in EU subsidies is spent. The MEP expenses campaign has led to the President of the European Parliament committing to introduce concrete proposals for reform of the MEPs’ allowance scheme by the end of the year. Finally, the project has created tailor-made tools for journalists as our research has shown that there was a lack of contextual knowledge and knowledge on the basics of accounting. ‘Cooking budgets’presents the basics of accounting in a satirical website, and the successful game ‘The good, the bad and the accountant’ simulates the struggle of a civil servant to retain its integrity. The three approaches and audiences to public budgeting have resulted in a holistic platform which tailors to the wider public who wants to have more insights in their local, regional, national and even EU budgets. With the launch of OpenBudgets.eu the field of financial transparency in Europe is enriched by new tools, data, games and research for journalists, civil society organisations and civil servants alike, resulting in a valuable resource for a broad target audience. OpenBudgets.eu has received funding from the European Union’s H2020 EU research and innovation programme under grant agreement No 645833 and is implemented by an international consortium of nine partners (including Open Knowledge International and Open Knowledge Foundation Germany) under the coordination of Fraunhofer IAIS.

Open Data by default: Lorca City Council is using OpenSpending to increase transparency and promote urban mobility.

- February 7, 2017 in Fiscal transparency, Open Fiscal Data, Open Spending, OpenSpending, smart city, Smart Region

Castillo de Lorca. Torre Alfonsina (Public Domain)

Lorca, a city located in the South of Spain with currently 92,000 inhabitants, launched its open data initiative on January 9th 2014. Initially it offered 23 datasets containing transport, mobility, statistical and economic information. From the very beginning, OpenSpending was the tool selected by Lorca City Council because of its capabilities and incredible visualization abilities. The first upload of datasets was done in 2013, on the previous version of OpenSpending. With the OpenSpending relaunch last year, Lorca City Council continued to make use of the OpenSpending datastore, while the TreeMap view of the expenditure budget was embedded on the council’s open data website. In December 2016, the council’s open data website was redesigned, including budget datasets built with the new version at next.openspending.org. The accounting management software of Lorca allows the automatic conversion of data files to csv. format, so these datasets are compatible with the requested formats established by OpenSpending. Towards more transparency and becoming a smart city In 2015, when the City of Lorca transparency website was launched, the council decided to continue with the same strategy focused on visualization tools to engage citizens with an intuitive approach to the budget data. Lorca is a city pioneer in the Region of Murcia in terms of open data and transparency. So far, 125 datasets have been released and much information is available along with the raw data. It deserves to be highlighted that there are pilot project initiatives to bring open data to schools, which was carried out during the past year. In 2017, we will resume to teach the culture of open data to school children with the main goal to demonstrate how to work with data by using open data. In the close future the council plans to open more data directly from the sources, i.e. achieve policy of open data by default. And of course Lorca intends to continue exploring other possibilities that Open Spending offers us to provide all this data to the citizenry. In addition, Lorca is working to become a smart city (article in Spanish only) – open data is a key element in this goal. Therefore, Lorca’s open data initiative will be a part of the Smart Social City strategy from the very beginning. 

Just Released: “Where Does Europe’s Money Go? A Guide to EU Budget Data Sources”

- July 2, 2015 in Data Journalism, eu, European Union, Featured, financial transparency, Follow the Money, open budget data, Open Fiscal Data, Open Knowledge, Open Spending, Policy, research, Where Does My Money Go

The EU has committed to spending €959,988 billion between 2014 and 2020. This money is disbursed through over 80 funds and programmes that are managed by over 100 different authorities. Where does this money come from? How is it allocated? And how is it spent? Today we are delighted to announce the release of “Where Does Europe’s Money Go? A Guide to EU Budget Data Sources”, which aims to help civil society groups, journalists and others to navigate the vast landscape of documents and datasets in order to “follow the money” in the EU. The guide also suggests steps that institutions should take in order to enable greater democratic oversight of EU public finances. It was undertaken by Open Knowledge with support from the Adessium Foundation.
Where Does Europe's Money Go?
As we have seen from projects like Farm Subsidy and journalistic collaborations around the EU Structural Funds it can be very difficult and time-consuming to put together all of the different pieces needed to understand flows of EU money. Groups of journalists on these projects have spent many months requesting, scraping, cleaning and assembling data to get an overview of just a handful of the many different funds and programmes through which EU money is spent. The analysis of this data has led to many dozens of news stories, and in some cases even criminal investigations. Better data, documentation, advocacy and journalism around EU public money is vital to addressing the “democratic deficit” in EU fiscal policy. To this end, we make the following recommendations to EU institutions and civil society organisations:
  1. Establish a single central point of reference for data and documents about EU revenue, budgeting and expenditure and ensure all the information is up to date at this domain (e.g. at a website such as ec.europa.eu/budget). At the same time, ensure all EU budget data are available from the EU open data portal as open data.
  2. Create an open dataset with key details about each EU fund, including name of the fund, heading, policy, type of management, implementing authorities, link to information on beneficiaries, link to legal basis in Eur-Lex and link to regulation in Eur-Lex.
  3. Extend the Financial Transparency System to all EU funds by integrating or federating detailed data expenditures from Members States, non-EU Members and international organisations. Data on beneficiaries should include, when relevant, a unique European identifier of company, and when the project is co-financed, the exact amount of EU funding received and the total amount of the project.
  4. Clarify and harmonise the legal framework regarding transparency rules for the beneficiaries of EU funds.
  5. Support and strengthen funding for civil society groups and journalists working on EU public finances.
  6. Conduct a more detailed assessment of beneficiary data availability for all EU funds and for all implementing authorities – e.g., through a dedicated “open data audit”.
  7. Build a stronger central base of evidence about the uses and users of EU fiscal data – including data projects, investigative journalism projects and data users in the media and civil society.
Our intention is that the material in this report will become a living resource that we can continue to expand and update. If you have any comments or suggestions, we’d love to hear from you. If you are interested in learning more about Open Knowledge’s other initiatives around open data and financial transparency you can explore the Where Does My Money Go? project, the OpenSpending project, read our other previous guides and reports or join the Follow the Money network. Where Does Europe’s Money Go - A Guide to EU Budget Data Sources