You are browsing the archive for Ivan Begtin.

BudgetApps: The First All-Russia Contest on Open Finance Data

- January 16, 2015 in Budget Data, historical data, OKF Russia, Open Data

This is a guest post by Ivan Begtin, Ambassador for Open Knowledge in Russia and co-founder of the Russian Local Group. budgetapps2 Dear friends, the end of 2014 and the beginning of 2015 have been marked by an event, which is terrific for all those who are interested in working with open data, participating in challenges for apps developers and generally for all people who are into the Open Data Movement. I’m also sure, by the way, that people who are fond of history will find it particularly fascinating to be involved in this event. On 23 December 2014, the Russian Ministry of Finance together with NGO Infoculture launched an apps developers’ challenge BudgetApps based on the open data, which have been published by the Ministry of Finance over the past several years. There is a number of various datasets, including budget data, audit organisations registries, public debt, national reserve and many other kinds of data. Now, it happened so that I have joined the jury. So I won’t be able to participate, but let me provide some details regarding this initiative. All the published data can be found at the Ministry website. Lots of budget datasets are also available at The Single Web Portal of the Russian Federation Budget System. That includes the budget structure in CSV format, the data itself, reference books and many other instructive details. Data regarding all official institutions are placed here. This resource is particularly interesting, because it contains indicators, budgets, statutes and numerous other characteristics regarding each state organisation or municipal institution in Russia. Such data would be invaluable for anyone who considers creating a regional data-based project. One of the challenge requirements is that the submitted projects should be based on the data published by the Ministry of Finance. However, it does not mean that participants cannot use data from other sources alongside with the Ministry data. It is actually expected that the apps developers will combine several data sources in their projects. To my mind, one should not even restrict themselves to machine-readable data, because there are also available human-readable data that can be converted to open data formats by participants. Many potential participants know how to write parsers on their own. For those who have never had such an experience there are great reference resources, e.g. ScraperWiki that can be helpful for scraping web pages. There are also various libraries for analysing Excel files or extracting spreadsheets from PDF documents (for instance, PDFtables, Abbyy Finereader software or other Abbyy services ). Moreover, at other web resources of the Ministry of Finance there is a lot of interesting information that can be converted to data, including news items that recently have become especially relevant for the Russian audience.

Historical budgets

There is a huge and powerful direction in the general process of opening data, which has long been missing in Russia. What I mean here is publishing open historical data that are kept in archives as large paper volumes of reference books containing myriads of tables with data. These are virtually necessary when we turn to history referring to facts and creating projects devoted to a certain event. The time has come at last. Any day now the first scanned budgets of the Russian Empire and the Soviet Union will be openly published. A bit later, but also in the near future, the rest of the existing budgets of the Russian Empire, the Soviet Union, and the Russian Soviet Federated Socialist Republic will be published as well. These scanned copies are being gradually converted to machine-readable formats, such as Excel and CSV data reconstructed from these reference books – both as raw data and as initially processed and ordered data. We created these ordered normalised versions to make it easier for developers to use them in further visualisations and projects. A number of such datasets have already been openly published. It is also worth mentioning that a considerable number of scanned copies of budget reference books (from both the Russian Empire and USSR) have already been published online by Historical Materials, a Russian-language grass-root project launched by a group of statisticians, historians and other enthusiasts. Here are the historical machine-readable datasets published so far: I find this part of the challenge particularly inspiring. If I were not part of the jury, I would create my own project based on historical budgets data. Actually, I may well do something like that after the challenge is over (unless somebody does it earlier).

More data?

There is a greater stock of data sources that might be used alongside with the Ministry data. Here are some of them: These are just a few examples of numerous available data sources. I know that many people also use data from Wikipedia and DBPedia.

What can be done?

First and foremost, there are great opportunities for creating projects aimed at enhancing the understandability of public finance. Among all, these could be visual demos of how the budget (or public debt, or some particular area of finance) is structured. Second, lots of projects could be launched based on the data on official institutions at bus.gov.ru. For instance, it could be a comparative registry of all hospitals in Russia. Or a project comparing all state universities. Or a map of available public services. Or a visualisation of budgets of Moscow State University (or any other Russian state university for that matter). As to the historical data, for starters it could be a simple visualisation comparing the current situation to the past. This might be a challenging and fascinating problem to solve.

Why is this important?

BudgetApps is a great way of promoting open data among apps developers, as well as data journalists. There are good reasons for participating. First off, there are many sources of data that provide a good opportunity for talented and creative developers to implement their ambitious ideas. Second, the winners will receive considerable cash prizes. And last, but not least, the most interesting and perspective projects will get a reference at the Ministry of Finance website, which is a good promotion for any worthy project. Considerable amounts of data have become available. It’s time now for a wider audience to become aware of what they are good for.

On the way to the new market of information in Russia

- June 29, 2012 in External, linked-open-data, Open Government Data, Open Standards, WG EU Open Data, WG Open Government Data

On June 5th at the Higher School of Economics in Moscow a round table conference took place, devoted to the opening of state-collected datasets. It was convened by the Higher School of Economics (HSE) together with the Russian Office of the World Wide Web Consortium (W3C). Open data is the new trend in the state practices of the developed countries, and Russia also acknowledges the importance of this trend. The Presidential Decree of May 7th 2012, “About key measures for the improvement of the state governmental system” states that it is important to publish open government data by July 15th, 2013. Oleg Pak, from the Ministry of Economic Development, told the round-table that his department is currently developing the standards and the concept of open data in Russia. Within the framework of this concept, they will develop a comprehensive strategy for open data usage in Russia. This concept should become a roadmap for the work of all authorities engaged in the realization of this vision. As a rule, the realization of national projects for open data has two goals. The first is socio-political – the State should open the data for its citizens. This goal can be easily achieved with the existing level of technology. The main issue at the discussion in June was the achievement of the second goal: the transformation of state data arrays into a product suitable for cost-effective use. This would allow businesses to form a new structure of services, and offer previously non-existent things on the market. In many countries, this is already happening, as Victor Klintsov (W3C, HSE) pointed out. “The USA Administration has already published over a million data sets. This has been published not for “readers”, but for computers and services which use this data for development of new data, products and services”, he said. Pavel Pugachev (Ministry of Communications and Mass Communications) cited the example of an IT-company in the US. Its programmers use anonymised medical data about outbreaks and numbers of patients, process it and supply large pharmaceutical companies with the results. This allows those companies to develop their demand and supply tactics. Pugachev suggested we ought to determine the open data priorities according to which data types will be most interesting to the market, and concentrate our efforts on opening them first and foremost. A key issue in the data that is being opened in Russia is that of interoperability. Releases so far have been based on the idea of human consumption – it is largely unsuitable for computer “consumption”, being unmatched and in different formats. This massively limits its business potential. Meanwhile, in Moscow alone there are 4,000 portals state-owned portals and organizations without consistent principles of data delivery. Common publishing standards need to be established as a matter of priority. Nonetheless, as Daniel Hladky (W3C) pointed out, we cannot simply wait for the development of all the regulations that will allow perfect publication: “Publish, what you have. As you can and by any means. Good or bad, with mistakes, unattractively, even if 90% of this data will be badly structured. Maybe it lacks metadata. I would like to say that it is necessary to pick up speed. If 5% of the information is useful, it will be a start and a push for the development of business”. In developed countries the open data market started not from acts of government, but from the activity of individuals who collected information and published it on their portals, bringing it up to a machine-processable state. This opinion was supported by Maksim Dubinin (OpenStreetMap, GIS-Lab projects): “The community of users and the culture of usage will not appear until open data is presented in large enough quantities”. He shared his experience in the area of geodata. “When it became clear that it was impossible to wait for governmental steps in this field, projects started to appear in which users contributed geodata by themselves. Over 600,000 people around the world have taken part in the OpenStreetMap project. As a result, some governmental organizations have started using data created by users.” Undoubtedly, this needs to come from both ends at once. Progress with Russian open data projects will be presented by Daniel Hladky during the European commission workshop Using Open Data: policy modeling, citizen empowerment, data journalism, which is going to take place in Brussels this week. Victor Klintsov promised that the next meeting of the round table participants will be held this autumn. W3C office is planning to invite the leaders of open data projects from the USA and Great Britain. The shorthand transcript and presentation graphics of the round table conference will be published on the site of W3C Russian office www.w3c.org.ru

Infokultura and Apps4Russia

- May 21, 2012 in External, Open Government Data, WG Open Government Data

During recent years, the Russian Federation has undertaken a number of developments in its open data legislation strategy. This trend inspired a team of professionals to get together and start a non-profit organization, “INFOKULTURA”. Understanding that data availability is crucial for an information society and the development of an information culture, we emphasise the establishment and promotion of Open Data concept though a number of activities, for example the Apps4Russia Contest, conferences, seminars, research and expertise provision. It is worth mentioning that not only technical capacities and access to the Internet are needed to promote the idea of informational culture as an important social topic: the lack of administrative burden, technological, legal, time and other constraints on the data availability are essential. These problems can be solved with the help of open licenses, open standards and open data sources. warfly Warfly – one of last year’s Apps4Russia winners Here in Russia, there is growing interest among society and State authorities in Open Data expertise, and in responding to it “INFOKULTURA” has developed a set of proposals in support of the development, implementation and promotion of effective tools for efficient and successful information interaction between government and society. During discussions and team meetings it was agreed that the main goals of the organization should be focused on promoting the following ideas:
  • Information Culture – human activity that is associated with self-actualization and the individual initiatives manifested within the information society. Information culture is based on new data production, communication, environment change, integration of humanity into a common information space and the development of digital tools in order to create more varieties of information.
  • Open Government – new modes of interaction between the government authorities and other public institutions in which the bodies that compose the state government provide a broad disclosure of its activities, provide feedback mechanisms for citizens and public institutions, and an operational set of tools for citizens and public institutions to influence the decision-making process.
  • Civilian Control – the way the society can have influence with government bodies and institutions, develop the public good, and help to solve social problems.
  • Plain Language – the idea that the society deserves clear communications from the government authorities – clear and concise language designed to ensure the recipient understands as quickly and completely as possible. Plain language seeks to be easy to read, understand, and use.
  • Understandable Government – a new platform to broaden and ease official communications between government and society via Infographic tools, Clear visualized guidance and other comprehensive interaction mechanisms that designed to help people find out new ways to engage with State agencies, and take action to improve the way they work.

Apps4Russia Contest

In April 2012 the second Apps4Russia contest was announced at RIF+CIB 2012, the Internet industry’s main event in Russia. The Apps4Russia contest was initiated to promote the idea of Open Data. The main goal is to encourage Russian developers to create projects based on Open Government Data, aiming to increase public benefit and improve government transparency. In 2011 the Apps4Russia was held in the private initiative format with a prize fund of EUR 4,000. This year the prize fund has been significantly increased since it is expected that more developers from various levels, inlcuding school students, will submit to the contest. In 2011 the following Submissions were nominated and awarded:
  • “Fire tracking concept” – public monitoring of the natural resources;
  • “DataPult” – interactive infographics data based on downloaded statistics;
  • “Warfly: World War II aerial photography” – aerial photographs of Russian and West European cities during the period 1939-1945. All photographs are synchronized with modern landscape.
datapult Datapult – one of last year’s Apps4Russia winners This year all submissions will be reviewed under the following categories:
  • Great — main competition for all developers. (Examples topics: anti-corruption, clear and plain statistics, environmental monitoring, Russia’s role in the global world etc.)
  • Junior — school students competition (individual and group entries)
  • Visualization — interactive and poster infographics competition based on Open Data resources;
  • Cities and urban life environment — web-sites and applications created to improve urban life environment, through citizens and community initiatives
Call for Proposals timeframe: April – September
Summing up date – 12 September 2012 (Day of the Programmer). Official web-site: http://www.apps4russia.ru