An Open Knowledge Foundation Site
logo Planet Open Knowledge

You are browsing the archive for Serah Rono.

Frictionless Data and FAIR Research Principles

- August 14, 2018 in Data Package, Frictionless Data

In August 2018, Serah Rono will be running a Frictionless Data workshop in CopenHagen, congregated by the Danish National Research Data Management Forum as part of the FAIR Across project. In October 2018, she will also run a Frictionless Data workshop at FORCE11 in Montreal, Canada. Ahead of the two workshops, and other events before the close of 2018, this blog post discusses how the Frictionless Data initiative aligns with FAIR research principles. An integral part of evidence-based research is gathering and analysing data, which takes time and often requires skill and specialized tools to aid the process. Once the work is done, reproducibility requires that research reports be shared with the data and software from which insights are derived and conclusions are drawn, if at all.  Widely lauded as a key measure of research credibility, reproducibility also makes a bold demand for openness by default in research, which in turn fosters collaboration. FAIR (findability, accessibility, interoperability and reusability) research principles are central to the open access and open research movements.
“FAIR Guiding Principles precede implementation choices, and do not suggest any specific technology, standard, or implementation-solution; moreover, the Principles are not, themselves, a standard or a specification. They act as a guide to data publishers and stewards to assist them in evaluating whether their particular implementation choices are rendering their digital research artefacts Findable, Accessible, Interoperable, and Reusable.”

Wilkinson, M. D. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data3:160018 doi: 10.1038/sdata.2016.18 (2016)

Data Packages in Frictionless Data as an example of FAIRness

Our Frictionless Data project aims to make it effortless to transport high quality data among different tools & platforms for further analysis. The Data Package format is at the core of Frictionless Data, and it makes it possible to package data and attach contextual information to it before sharing it.

An example data package

Data packages are nothing without the descriptor file. This descriptor file is made available in a machine readable format, JSON, and holds metadata for your collection of resources, and a schema for your tabular data.

Findability

In Data Packages, pieces of information are called resources. Each resource is referred to by name and has a globally unique identifier, with the provision to reference remote resources by URLs. Resource names and identifiers are held alongside other metadata in the descriptor file.

Accessibility

Since metadata is held in the descriptor file, it can be accessed separately from associated data. Where resources are available online – in an archive or data platform – sharing the descriptor file only is sufficient and data provenance is guaranteed for all associated resources.

Interoperability

The descriptor file is saved as a JSON file, a machine-readable format that can be processed with great ease by many different tools during data analysis. The descriptor file uses accessible and shared language, and has provision to add descriptions, and information on sources and contributors for each resource, which makes it possible to link to other existing metadata and guarantee data provenance. It is also very extensible, and can be expanded to accommodate additional information as needed.

Reusability

Part of the metadata held in a data package includes licensing and author information, and has a requirement to link back to original sources thus ensuring data provenance. This serves as a great guide for users interested in your resources. Where licensing allows for resources to be archived on different platforms, this means that regardless of where users access this data from, they will be able to trace back to original sources of the data as needed. For example, all countries of the world have unique codes attached to them. See how the Country Codes data package is represented on two different platforms:  GitHub, and on DataHub. With thanks to SLOAN Foundation for the new Frictionless Data For Reproducible Research grant, we will be running deep dive workshops to expound on these concepts and identify areas for improvement and collaboration in open access and open research. We have exciting opportunities in store, which we will announce in our community channels over time.

Bonus readings

Here are some of the ways researchers have adopted Frictionless Data software in different domains over the last two years:
  • The Cell Migration and Standardisation Organisation (CMSO) uses Frictionless Data specs to package cell migration data and load it into Pandas for data analysis and creation of visualizations. Read more.
  • We collaborated with Data Management for TEDDINET project (DM4T) on a proof-of-concept pilot in which we used Frictionless Data software to address some of the data management challenges faced by DM4T. Read more.
  • Open Power System Data uses Frictionless Data specifications to avail energy data for analysis and modeling. Read more.
  • We collaborated with Pacific Northwest National Laboratory – Active Data Biology and explored use of Frictionless Data software to generate schema for tabular data and check validity of metadata stored as part of a biological application on GitHub. Read more.
  • We collaborated with the UK Data service and used Frictionless Data software to assess and report on data quality, and made a case for generating visualisations with ensuing data and metadata. Read more.
Our team is also scheduled to run Frictionless Data workshops in the coming months:
  • In CopenHagen, congregated by the Danish National Research Data Management Forum as part of the FAIR Across project, in August 2018.
  • In Montreal, Canada, at FORCE11 between October 10 and 12, 2018. See the full program here and sign up here to attend the Frictionless Data workshop.

Comments Off on Frictionless Data and FAIR Research Principles

Improving your data publishing workflow with the Frictionless Data Field Guide

- March 27, 2018 in data infrastructures, Data Quality, Frictionless Data

The Frictionless Data Field Guide provides step-by-step instructions for improving data publishing workflows. The field guide introduces new ways of working informed by the Frictionless Data suite of software that data publishers can use independently, or adapt into existing personal and organisational workflows. Data quality and automation of data processing are essential in creating useful and effective data publication workflows. Speed of publication, and lowering costs of publication, are two areas that are directly enhanced by having better tooling and workflows to address quality and automation. At Open Knowledge International, we think that it is important for everybody involved in the publication of data to have access to tools that help automate and improve the quality of data, so this field guide details open data publication approaches with a focus on user-facing tools for anyone interested in publishing data. All of the Frictionless Data tools that are included in this field guide are built with open data publication workflows in mind, with a focus on tabular data, and there is a high degree of flexibility for extended use cases, handling different types of open data. The software featured in this field guide are all open source, maintained by Open Knowledge International under the Frictionless Data umbrella and designed to be modular. The preparation and delivery of the Frictionless Data Field Guide  has been made possible by the Open Data Institute, who received funding from Innovate UK to build “data infrastructure, improve data literacy, stimulate data innovation and build trust in the use of data” under the pubtools programme. Feel free to engage the Frictionless Data team and community on Gitter. The Frictionless Data project is a set of simple specifications to address common data description and data transport issues. The overall aim is to reduce friction in working with data and to do this by making it as easy as possible to transport data between different tools and platforms for further analysis. At the heart of Frictionless Data is the Data Package, which is a simple format for packaging data collections together with a schema and descriptive metadata. For over ten years, the Frictionless Data community has iterated extensively on tools and libraries that address various causes of friction in working with data, and this work culminated in the release of v1 specifications in September 2017.  

Comments Off on Improving your data publishing workflow with the Frictionless Data Field Guide

Frictionless Data: Introducing our Tool Fund Grantees

- July 3, 2017 in Frictionless Data

Frictionless Data is an Open Knowledge International project which started over 10 years ago as a community-driven effort of Open Knowledge Labs. Over the last 3 years, with funding from partners like the Sloan Foundation and Google, the Frictionless Data team has worked tirelessly to remove ‘friction’ from working with data. A well-defined set of specifications and standards has been published and used by organizations in the data space, our list of data packages has grown and our tools are evolving to cater to some of the issues that people encounter while working with data. Owing to our growing community of users and the need to extend implementation of version 1.0 specifications to additional programming languages, we launched the Frictionless Data Tool Fund in March 2017. The one-time $5,000 grant per implementation language has so far attracted 70+ applications from our community of mostly exceptional developers from all over the world. At this time, we have awarded four grants for the implementation of Frictionless Data libraries in R, Go, Java and PHP. We recently asked the grantees to tell us about themselves, their work with data and what they will be doing with their grants. These posts are part of the grantee profile series, written to shine a light on Frictionless Data’s Tool Fund grantees, their work and to let our technical community know how they can get involved.  

Ori Hoch: Frictionless Data Tool Fund PHP grantee

“I would also love to have PHP developers use the core libraries to write some more high-level tools. With the availability of the PHP libraries for Frictionless Data the task of developing such plugins will be greatly simplified”

From juggling fire clubs to working with data at The Museum of The Jewish People, to how he envisions use of the Frictionless Data PHP libraries to develop high level tools, read Ori Hoch’s profile to find out what he will be working on and how you can be a part of it.

Daniel Fireman: Frictionless Data Tool Fund Go grantee

“I hope to use the Tool Fund grant I received to bring Go’s performance and concurrency capabilities to data processing and to have a set of tools distributed as standalone and multi-platform binaries”
Read Daniel Fireman’s profile and find out more about his work on social network platforms, how he’s used GO to improve data transparency in Brazil, the challenges he has encountered while working with data and how he intends to alleviate these things  using his Go grant.

Open Knowledge Greece: Frictionless Data Tool Fund R grantee

“ We are going to implement two Frictionless Data libraries in R – Table Schema for publishing and sharing tabular-style data and Data Package for describing a coherent collection of data in a single package, keeping the frictionless data specifications.”
Read Open Knowledge Greece’s profile and find out more about Open Knowledge festival that they will be hosting in Thessaloniki, Greece in 2018, plus, why they are excited to be implementing Frictionless data libraries in R and how you can contribute to their efforts.

Georges Labrèche: Frictionless Data Tool Fund Java Grantee

“Data is messy and, for developers, cleaning and processing data from one project to another can quickly turn an awesome end-product idea into a burdensome chore. Data packages and Frictionless Data tools and libraries are important because they allow developers to focus more on the end-product itself without having to worry about heavy lifting in the data processing pipeline.”
From travel, to physics and astronautics, read Georges Labrèche’s profile and find out more about his interests and  work at Open Data Kosovo as well as how you can follow and contribute to his work around Frictionless Data’s Java libraries.   Frictionless Data is a project of Open Knowledge International. Interested in knowing more? Read about the standards, data and tools, and find out how to get involved. All code developed in support of this project is hosted on the frictionlessdata organization on GitHub. Contributions are welcome. Have a question or comment?  Let us know on our Discuss forum or on our Gitter chat.

Comments Off on Frictionless Data: Introducing our Tool Fund Grantees

ROUTETOPA User Stories

- June 28, 2017 in Route to PA

ROUTETOPA is a European innovation project aimed at improving citizen engagement by enabling meaningful interaction between open data users, open data publishers and open data. Open Knowledge International is one of 12 partners working on the project and our main mandate is to build genuine and active communities around the ROUTETOPA tools. In this blogpost, we share more information on the ROUTETOPA user stories. ROUTETOPA is an acronym that stands for Raising Open and User-friendly, Transparency Enabling Technologies for Public Administrations. It is a three-year, multidisciplinary,  European Union Horizon 2020 innovation project aimed at improving citizen engagement by enabling meaningful interaction between open data users, open data publishers and open data. ROUTETOPA’s seeks to do this in 4 ways:
  1. Through the Social Platform for Open Data (SPOD) which seeks to enable social interactions between open data users and local Governments
  2. Through the Transparency Enabling Toolset (TET), built on CKAN and conglomerating data from existing local government open data platforms
  3. Through SIM, which seeks to provide Public Administrations with statistical analysis on user behaviour and generalized feedback from users on SPOD and TET so public administrations can understand what citizens are interested in
  4. Through GUIDE, a set of recommendations for open data publishers, extrapolated from SIM, aimed at promoting higher transparency levels through open data
Open Knowledge International works on building genuine and active communities around the ROUTETOPA tools. One of the key ways we intend to do this is by finding meaningful ways for communities to interact with data and with each other on the ROUTETOPA platforms. We have now defined the following user stories:

USER STORIES

Open Data Enthusiasts

  1. I am a citizen interested in knowing what ROUTETOPA is and why I and others should care
  2. I am a citizen wondering what open data is and how transparency benefits me
  3. I am a resident in City X interested in knowing what data exists for my city.
  4. I am a resident in city X interested in tracking an ongoing project in my area.
  5. I am a graduate/post-graduate student interested in using open data for my research/ thesis/ paper
  6. I am a teacher / lecturer /professor looking to introduce / teach my class about open data
  7. I am a journalist looking to tell a data story on a City X
  8. I am a journalist looking to write an article about H2020 and the ROUTETOPA project as a beneficiary
  9. I am a media house representative looking for know more about ROUTETOPA tools
  10. I am a policy maker interested in learning how ROUTETOPA tools work so I can see if it makes sense for my local authority to take up the use of these tools.
  11. I am an activist interested in discussing issues in the community I live in with my local authority
  12. I am a business owner interested in opportunities for business with public administrations
  13. I am a data scientist looking for big, quality data from Area X for use in my work
  14. I am a developer interested in the underlying code for ROUTETOPA tools
  15. I am a developer with additional feature suggestions for ROUTETOPA tools
  16. I am a designer with UI/UX suggestions for the ROUTETOPA platform
  17. I am a non-EU open data user wondering whether I should interest myself in ROUTETOPA
  18. I am a non-EU open data user interested in replicating ROUTETOPA tools for my continent
  19. I am part of an open data community lead looking to discuss open data topics with my community

Open data publishers

  1. I am a public official wondering what ROUTETOPA is and why I should care / be involved
  2. I am a public official looking to get citizen feedback on the data our local authority has opened up
  3. I am a public administrator keen on answering questions the community has raised regarding the data my administration has published and for which I am responsible
  4. I am a public administrator looking at solutions that other public administrations have employed in publishing open data to determine what suits us best
  5. I am a public administrator keen on involving citizens in Area X in decision making that affects our community
  6. I am a public administrator and I’m new to open data and would like to get an introduction to this form of open government
  7. I am a public administrator instructed to find open data cases in other municipalities that are easy to duplicate  (low hanging fruits cases)
  8. I am a public administrator and I would like to implement a long term open government data strategy in my municipality. And I need an action plan
  9. I am a public administrator looking for easy ways to update citizens about the open government data project and progress
  10. I am a public administrator who maintains an open data platform based on CKAN and is looking for economical and easy extensions that add value to our open government data program
  11. I am a public official who is looking for ways to update and communicate with involved colleagues from other units of my administration
More information on the project and its outcomes is available from http://routetopa.eu.

Comments Off on ROUTETOPA User Stories

Social Platform for Open Data – enabling interactions at Accountability Hack 2017

- May 31, 2017 in Route to PA

Since 2015 Open Knowledge International has been part of the consortium of ROUTETOPA (Raising Open, User-friendly, Transparency-Enabling Technologies for Public Administrations, routetopa.eu), an EU-funded project that is working round the clock to provide platforms for open data enthusiasts to interact on the subject of open data with data publishers and with other open data users.
The upcoming Accountability Hack 2017 event has adopted ROUTETOPA’s  Social Platform for Open Data for use before, during and after the hackathon. In this blog we share more information about the event and about how you can use this platform.  This blog has been crossposted from the ROUTETOPA blog (English) and the Accountability Hack blog (Dutch).
On June 9, at Accountability Hack 2017 in The Hague, teams will have eight hours to brainstorm, design and prototype solutions to some of the Netherlands’ most pertinent issues concerning accountability that open data can address. The Open State Foundation is organizing this hackathon for the Court of Audit, Statistics Netherlands and several ministries.
More than 100 developers, data-analysts and journalists are competing to make the best app that promotes accountability of government spending and performance. In total there is a prize amount of €7.500 euros. As many citizens as are available are encouraged to save the date and attend the day-long event which will take place in the building of the Dutch House of Representatives. Hackathon participants are free to work on projects alone, but are encouraged to form teams with people from different disciplines. Developers, data scientists, policy makers, academics, designers, researchers, librarians, et al will be in attendance on the day. But how will a librarian, for example, find a designer and developer to work on a project with during the Hackathon? Accountability Hack 2017 event has adopted ROUTETOPA’s  Social Platform for Open Data for use before, during and after the hackathon. On SPOD, participants will be able to discover open data, create and join groups, hold data-driven discussions, share documents and images with team members and create data visualizations very easily from the same platform. Pretty cool!

The Social Platform for Open Data (SPOD)

To get started, go to nl-spod.routetopa.eu and sign up with your email address, or using your Facebook credentials. Once logged in, you should have access to the entire Social Platform for Open Data. In it, you can:
  • Edit your profile to reflect what you would like others to know, key among them your field of expertise so teams that need you can reach out to you
  • Add friends in order to directly message them
  • Write updates that will be aggregated with other updates and visible to all in the NewsFeed area
  • Use Agora, named after Greek public spaces, to create and work in project groups.
  • Use co-creation for the more sensitive,  invite-only discussions teams wish to hold at the beginning of the hackathon. Ultimately, teams are required to share as much information on their hackathon projects as possible in Agora.
Why are we encouraging teams to use the Social platform for Open Data during Accountability Hack 2017? Eight hours is not nearly enough to identify a problem the team should work on, brainstorm through plausible solutions, find data, design and prototype the solution. In fact, without documentation detailing what teams are hoping to achieve and without a common space for teams to work collaboratively, it is virtually impossible to build on the idea presented at a hackathon beyond the day. Additionally, people might be interested in contributing datasets, helpful examples or just subscribing for updates on your project. Having all of these things happening in one place would be more than ideal.

Submit your app via SPOD

We ask all teams to register to SPOD before or during Accountability Hack. We ask all participants to hand in their working prototype at the end of the day via the platform. In the public post you need to state the name of the application, screenshots, link (if available), description of the application, used datasets, further plans and finally what was not possible due to lack of time or data (quality). Only apps that are submitted using SPOD will be eligible for the prizes. Besides submitting the apps we encourage Accountability Hack attendees to use the platform before, during and after the hackathon to work, explore and collaborate data. Please send any hackathon-specific questions via email to Open State Foundation.

How does SPOD work?

To better illustrate how you can use the Social Platform for Open Data during the hackathon, here are four examples, with videos embedded:

Team updates throughout hackathon via SPOD

Professor X and two Computer students have signed up for accountabilityhack.nl and are set to start work on a open elections project. Here’s what this team can do:
  1. The team lead should create a room for the team to work collaboratively from by going to Agora, and clicking on the blue + button, such as the one in the image below.

    A good Agora room is one where the description is as detailed and clearly written as possible.
  2. All other Team members can become part of the project group in Agora by simply clicking on an Agora room, then clicking the ENTER button. It is worth noting that all activities in Agora are public and can be viewed by all SPOD users.
  3. Teams can then hold project specific discussions within their Agora room. Beyond this, here are some great things you can do in your Agora rooms
    1. Search for datasets relevant to your project theme. The Social Platform for Open Data is connected to data.openstate.eu
    2. Create visualizations, called datalets, with the data that you find within SPOD i.e. create maps, charts or tables and embed this in your responses to team members
If team needs an additional member, copying the URL at the top of the screen and pasting it in a status update in the NewsFeed area ensures everyone logged into SPOD can see it. See example below:


Using SPOD’s What’s New feature

Alex is a nurse looking to attend accountabilityhack.nl. Having recently learnt about open data online, Alex has an idea for an open health project, would like to to form a team and build on it, but knows no one else at the Hackathon. Here’s what Alex can do:
  1. Use SPOD’s What’s new feature to check for any updates from teams looking for additional members.
  2. Post an update in Newsfeed detailing skills and themes of interest i.e. Health. This update will appear in What’s new. Alex should then look out for comments against her update or inbox messages

Finding and sharing datasets in Agora

Team Z is all set for the hackathon and would like to know how to utilize SPOD maximally during the hackathon for their benefit. Here’s a great video that shows you all the neat things you can achieve in Agora:

 Closed team collaborations? Create a CoCreation room

Having worked together on a different project before, Team Y is all set and would like to start discussions about their work away from the public eye at first. Although Team Y will need to create a team space in Agora and share at least some information about their project in the course of the day before the hackathon ends, here is how CoCreation rooms in SPOD can be used to serve this purpose:
  1. Teams can create a room for invite-only discussions (known as a knowledge room) ,
  2. or an invite only room to work on a spreadsheet collaboratively (known as a data room). We highly recommend using a data room for invite-only team collaboration.
Watch the CoCreation walkthrough video below:
The walkthrough above is enough to get you started, but there’s lot’s more you can do on the Social Platform for Open Data. If you feel lost, the ? on the top right hand corner of your screen has a detailed guide to using SPOD, be sure to check it out. For a complete overview, the video below gives a more visual walkthrough of the entire SPOD platform:
You can also ask your questions in this Agora room dedicated to answering your platform questions throughout the day so we, or any other person at the hackathon, can answer you. In case you run into great difficulty that needs our immediate attention,update your status with the word HELP at the beginning and we will respond, write us an email,  tweet at us or Facebook us and we will respond promptly. Happy exploring!

Comments Off on Social Platform for Open Data – enabling interactions at Accountability Hack 2017

Three ways ROUTETOPA promotes Transparency

- March 14, 2017 in ckan, Open Data, Open Knowledge, Route to PA

Data sharing has come a long way over the years. With open source tools, improvements and new features are always quickly on the rise. Serah Rono looks at how ROUTETOPA, a Horizon2020 project advocate for transparency.

From as far back as the age of enlightenment, the human race has worked hard to keep authorities accountable. Long term advocates of open data agree that governments are custodians, rather than owners, of data in their keep and should, therefore, avail the information they are charged with safekeeping for public scrutiny and use. Privacy and national security concerns are some of the most common barriers to absolute openness in governments and institutions in general around the world.

As more governments and organisations embrace the idea of open data, some end up, inadvertently, holding back on releasing data they believe is not ready for the public eye, a phenomenon known as ‘data-hugging’. In other instances, governments and organisations end up misleading the general public about the actual quantity and quality of information they have made public. This is usually a play at politics – a phenomenon referred to as ‘open-washing’ and is very frustrating to the open data community. It does not always stop here – some organisations are known to notoriously exaggerate the impact of their open data work  – a phenomenon Andy Nickinson refers to as ‘open-wishing’.

The  Horizon2020 project, Raising Open and User-Friendly Transparency Enabling Technologies for Public Administrations (ROUTETOPA), works to bridge the gap between open data users and open data publishers. You can read the project overview in this post and find more information on the project here.

In an age of open-washing and data-hugging, how does ROUTETOPA advocate for transparency

  1. ROUTETOPA leads by example!

The source code for ROUTETOPA tools is open source and lives in this repository. ROUTETOPA also used CKAN, a renowned data portal platform, as the basis for its Transparency Enabling Toolkit (TET). TET provides public administrators in ROUTETOPA’s pilot cities with a platform to publish and open up their data to the public. You can read more about it here. 

       2. Data publishers as pilot leads

ROUTETOPA pilots are led by public administrators. This ensures that public administrators are publishing new data regularly and that they are also at hand to answer community questions, respond to community concerns and spearhead community discussions around open data in the five pilot cities.

3.Use of online and offline communication channels

Not only does ROUTETOPA have an active social media presence on Facebook, Twitter and Youtube, it also has its own social media platform, the Social Platform for Open Data (SPOD) that provides a much needed avenue for open data discourse between data publishers and users.  The pilots in Prato, Groningen, Dublin, Issy and Den Haag also hold regular workshops, focus groups and tool test parties. Offline engagement is more relatable, and creates rapport between public administrations and citizens and is also a great avenue for making data requests.

The ROUTETOPA consortium also runs an active blog that features project updates and lessons learnt along the way. Workshops and focus groups are a key part of the success of this project, as user feedback informs the development process of ROUTETOPA tools.

ROUTETOPA partners also attend and spread the work in open data conferences and seminars, to keep the open data community across Europe in the know, and as an avenue to invite the community to test the tools, give feedback, and if it suites, adapt the tools for use in their organizations, institutions and public administrations.

Need clarification, or want to plug in and be a part of ROUTETOPA’s progress? Write to serah.rono@okfn.org. Stay open!

Comments Off on Three ways ROUTETOPA promotes Transparency

7 ways the ROUTE-TO-PA project has improved data sharing through CKAN

- February 27, 2017 in ckan, Route to PA

Data sharing has come a long way over the years. With open source tools, improvements and new features are always quickly on the horizon. Serah Rono looks at the improvements that have been made to open source data management system CKAN through the course of the ROUTE-TO-PA project.  In the present day, 5MB worth of data would probably be a decent photo, a three-minute song, or a spreadsheet. Nothing worth writing home about, let alone splashing across front pages of mainstream media. This was not the case in 1956 though –  in September of that year, IBM made the news by creating a 5MB hard drive. It was so big, a crane was used to lift it onto a plane. Two years later, in 1958, the World Data Centre was established to allow users open access to scientific data. Over the years, data storage and sharing options have evolved to be more portable, secure, and with the blossoming of the Internet, virtual, too. One such virtual data sharing platform, CKAN, has been up and running for ten years now. CKAN is a powerful data management system that makes data accessible – by providing tools to streamline publishing, sharing, finding and using data. CKAN is aimed at data publishers (national and regional governments, companies and organizations) wanting to make their data open and available. It is no wonder then that ROUTE-TO-PA, a Horizon2020 project pushing for transparency in public administrations across the EU, chose CKAN as a foundation for its Transparency Enhancing Toolset (TET). As one of ROUTE-TO-PA’s tools, the Transparency Enhancing Toolset provides data publishers with a platform on which they can open up data in their custody to the general public. So, what improvements have been made to the CKAN base code to constitute the Transparency Enhancing Toolset? Below is a brief list:

1. Content management system support

CKAN Integration with a content management system enables publishers to publish content related to datasets and publish updates related to the portal in an easy way. TET WordPress plugin seamlessly integrates TET enabled CKAN and provides rich content publishing features to publishers and an elegantly organized entry point to data portal. 

2. PivotTable

CKAN platform has limited data analysis capabilities, essential for working with data. ROUTE-TO-PA added a PivotTable feature to allow users to view, summarize and visualize data. From the data explorer in this example, users can easily create pivot tables and even run SQL queries.  See source code here.

3. OpenID

ROUTE-TO-PA created an OpenID plugin for CKAN which enabled OpenID authentication on CKAN. See source code here.

4. Recommendation for related datasets

With this feature, the application recommends related datasets a user can look at based on the current selection and other contextual information. The feature guides users to find potentially useful and relevant datasets. See example in this search result for datasets on bins in Dublin, Ireland.

5. Combine Datasets Feature

This feature allows users to combine related datasets in their search results within TET into one ‘wholesome’ dataset. Along with the Refine Results feature, the Combined Datasets feature is found in the top right corner of the search results page, as in this example. Please note, that only datasets with the same structure can be combined at this point. Once combined, the resulting dataset can be downloaded for use.

6. Personalized search and recommendations

Personalized search feature allows logged-in users to get personalized search based on details provided in their profile. In addition logged-in users are provided with personalized recommendations based on their profile details.

7. Metadata quality check/validation

Extra validations to dataset entry form are added to prevent data entry errors and to ensure consistency. You can find, borrow from and contribute to CKAN and TET code repositories on Github, join CKAN’s global user group or email serah.rono@okfn.org with any/all of your questions. Viva el open source!

Comments Off on 7 ways the ROUTE-TO-PA project has improved data sharing through CKAN

ROUTETOPA – An Introduction

- November 7, 2016 in ckan, Open Government Data, Open Knowledge, Route to PA

Transparency, at best, makes for unity and positive development in democracies around the world. There are elaborate commentaries on what transparency entails, but for me, transparency translates to open. Think open doors. In any given building in a public space, an open door often turns to an invitation to come in and look around, while a closed one would often need a ‘we are open’ sign to convey the same message. It goes without saying that accessibility should, therefore, be at the heart of transparency. This is, however, not always the case. In a majority of democracies around the world, accessing and effectively using public data often requires humans to jump over incalculable legislative hoops, difficult-to-use portals or just incomprehensible published data. Where humans survive these, then outdated data, incomplete datasets, lack of follow-up and proper and closely monitored feedback loops with data publishers often make for a complete web of frustration. Unfortunately, while many administrations and organizations claim to be open, the fact remains that the information they are charged with safe-keeping cannot be accessed by most, easily and in a timely fashion. Thus, their efforts around open data equate to glass walls so clear that people often mistake them for entry spaces. Ever walked into a glass wall? It is as laughable as it is painful, regrettable, often embarrassing and irreversible.  And ouch! Everyone gets a headache from bumping into glass walls. Enter ROUTETOPA Raising Open and User-friendly, Transparency-Enabling Technologies for Public Administrations (ROUTETOPA) is a Horizon2020 project, funded by the European Union, that seeks to bring down the glass walls, so to speak, by providing online platforms for data publishers to open up data and for open data users to not only access, but converse over, make sense out of and act on the information released. Through ROUTETOPA’s Transparency Enhancing Toolset (TET), public administrators can publish all public data in their possession, in bulk at first, followed by regular updates ‘as the data happens’. TET is built on the world’s leading open-source data portal platform, CKAN and is designed to help data publishers to deal with the ‘no platform to publish on’ conundrum. TET encourages data publishers, and especially local authorities in this case, to avail data that is of public interest in central locations accessible by all and everyone else to look at the data, ask questions about it, share it with all interested parties for discussion and engage the data publishers in this regard.   ROUTETOPA’s Social Platform for Open Data (SPOD) allows for meaningful social interactions between open data publishers and open data users over open data. Open data enthusiasts can engage with data publishers over data that is availed on public domains, like the Transparency Enhancing Toolset, pushing for clarification and answers, providing feedback and use-cases for the published data, informing and contributing to new policies, among other things. But, do we really need another social media platform, one may ask. True, there’s many out there, each unique in its own way, and as some shutdown, more come up. I think we can all attest to the fact that, if you needed to have an open data conversation online today, or to find out about, say, budgeting information for a country you have not been to (like Kenya, where I am from), your audiences and sources and go-to places would be as scattered as mine. The Social Platform for Open Data tries to conglomerate all these conversations, deliberations, debates, et al around open data. Public administrators can involve citizens to find out what they input is before creating policy, citizens can reach out and ask for clarifications on various datasets released, or to send out dataset requests, or … the possibilities around this focused effort are vast. Map with the locations of Route-to-PA Currently in its piloting phase in 5 cities and 4 countries in the European Union, the ROUTETOPA project seeks to engage open data enthusiasts in testing and shaping its tools, through workshops, hackathons and data expeditions. Interested? Send an email to Serah Rono (serah.rono@okfn.org) and we will get you started. Stay open! Stay in touch! You can write to Serah – serah.rono@okfn.org

Comments Off on ROUTETOPA – An Introduction