Planet Open Knowledge 
O que faremos com os 40 trilhões de gigabytes de dados disponíveis em 2020?
Elza Maria Albuquerque - September 29, 2017 in Big Data, Dados Abertos, Inovação
Pessoa segura e olha para um celular. Foto: Pixabay / Creative Commons CC0.
Figura 1 – Estimativa de crescimento do volume de dados digitais de 2010 a 2020 [2].
- Melhorar a eficiência da gestão pública e a qualidade das políticas públicas;
- Agregar valor às informações e decisões governamentais;
- Fomentar a inovação mediante a utilização de dados abertos no desenvolvimento de aplicações e serviços inovadores;
- Promover o crescimento econômico através de informações ofertadas de forma massiva, permanente e confiável, a ser utilizada ou transformada para a criação de novos negócios e melhoria dos serviços de governo.
- Poderá haver uma escassez de talentos necessários para que as organizações possam aproveitar o potencial do Big Data. Em 2018, somente nos Estados Unidos da América, está previsto um gap de 140 a 190 mil profissionais com habilidades para análise de grandes bases de dados diante da demanda existente, e na camada gerencial, a previsão é que o gap seja de cerca de 1,5 milhões de gestores e analistas com o know-how necessário para usar o Big Data como subsídio para a tomada de decisão eficaz; [2]
- Algumas questões devem ser superadas para capturar o potencial do Big Data, como o estabelecimento de políticas para tratamento da privacidade, segurança da informação e propriedade intelectual, bem como a reorganização dos fluxos produtivos para incorporar este novo ativo [2];
- Para a tomada de decisão eficaz, as empresas poderão organizar não apenas as suas informações, mas também consumir cada vez mais as informações de terceiros (como fornecedores, governo, etc.) o que vai resultar em um esforço ainda maior para a melhoria da oferta de dados considerando o caráter cada vez mais descentralizado destes recursos de dados [2];
- No que tange a Dados Abertos Governamentais, em 2012, já existiam cerca de 115 catálogos de dados governamentais disponíveis, ofertando cerca de 710.000 conjuntos de dados [4]. Atualmente, em 2015, segundo o DataPortals.org, existem 417 catálogos de dados governamentais abertos disponíveis em todos os continentes, o que comprova a rápida ascensão e distribuição geográfica desta oferta de dados;
- Segundo a IBM[5], 80% dos dados produzidos nas empresas são desestruturados, ou seja, requerem um esforço muito maior para ser aproveitado para subsidiar a tomada de decisão, e certamente, parte destes dados não serão úteis para tal finalidade;
- Quanto ao potencial de uso dos dados digitais do mundo, o estudo da EMC aponta um dado preocupante: em 2012, apenas 23% da informação digital do mundo é útil para gerar novas informações e conhecimento e apoiar a tomada de decisão no âmbito do Big Data, e deste total, apenas 3% destas informações são úteis para uso imediato (os demais 20% ainda precisam ser tratados para estar aptas ao uso) [1];
- No cenário tecnológico atual, o volume de dados aptos a serem explorados para tomada de decisão (valor analítico) deve alcançar apenas 33% do volume total de 40 Zettabytes [1].
Thiago Ávila é conselheiro consultivo da Open Knowledge Brasil. * Estes artigos contam são oriundos de pesquisas científicas desenvolvidas no Núcleo de Excelência em Tecnologias Sociais (NEES), do Instituto de Computação da Universidade Federal de Alagoas (UFAL) e contam com a contribuição direta dos pesquisadores Dr. Ig Ibert Bittencourt (UFAL), Dr. Seiji Isotani (USP), e Armando Barbosa, Danila Oliveira, Judson Bandeira, Thiago Ávila e Williams Alcântara (UFAL). [1] Gantz, John and Reinsel. (2012). David. The Digital Universe In 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East. EMC Corporation. Acesso em: jul. 2015. Disponível em: http://www.emc.com/collateral/analyst-reports/idc-the-digital-universe-in-2020.pdf [2] Manyika, James; Chui, Michael; Brown, Brad; Bughin, Jacques; Dobbs, Richard; Roxburgh, Charles & Byers, Angela Hung. (2011). Big data: The Next Frontier For Innovation, Competition, And Productivity. McKinsey Global Institute. Disponível em: http://www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_innovation. Acesso em: jul. 2015 [3] Norma Técnica para Publicación de Datos Abiertos en Chile (2013). Gobierno de Chile. Unidad de Modernización y Gobierno Digital. Disponível em: http://instituciones.gobiernoabierto.cl/NormaTecnicaPublicacionDatosChile_v2-1.pdf. Acesso em: maio. 2015. [4] Hendler, James and Holm, Jeanne and Musialek, Chris and Thomas, George. (2012). US Government Linked Open Data: Semantic.data.gov. IEEE Intelligent Systems, p. 25-31, vol. 27. doi: 10.1109/MIS.2012.27. [5] IBM. (n.d).Apply New Analytics Tools To Reveal New Opportunities. IBM. Acesso em: jul. 2015. Disponível em: http://www.ibm.com/smarterplanet/us/en/business_analytics/article/it_business_intelligence.html Texto publicado no site Thiago Ávila. Ele faz parte da série de artigos Dados abertos conectados.
2. Introduction to Open Data – Alwenyi Catherine Cassidy
participants share their ideas on how to approach different datasets and how to feed this into an official open data portal that can carry all these datasets together. One of the participants, Aymen Latrach, discussed the problems his team faces when it comes to data transparency about extractives in 
program at AUB. The Data team at Data Aurora were really happy sharing this experience with students from different academic backgrounds, including media studies, engineering or business.
The workshop was mainly led by 
On the first day, the students were introduced to data journalism, from a theoretical approach, in particular, the data pipeline which outlined the different phases in any data visualization project: find, get, verify, clean, analyze and present. After that, students were being technically involved in scraping and cleaning data using tools such as open refine and Tabula.
Day two was all about mapping, from mapping best practices to mapping formats and shapes. Students were first exposed to different types of maps and design styles that served the purpose of each map. Moreover, best mappings techniques and visualizations were emphasized to explain their relative serving purpose. Eventually, participants became able to differentiate between the dot maps and the choropleth maps as well as many others. Then they used twitter data that contained geolocations to contrast varying tweeting zones by placing these tweets at their origins on cartodb. Similarly, they created other maps using QGIS and Tilemill. The mapping exercises were really fun and students were very happy to create their own maps without a single line of code.
On the third day, Bahia gave a lecture on network analysis, some important mathematical notions needed for working with graphs as well as possible uses and case studies related to this field. Meanwhile, Ali was unveiling different open data portals to provide the students with more resources and data sets. After these topics were emphasized, a technical demonstration on the use of Gephi to analyze two topics was
performed. Students were analyzing climate change and later, the AUB media group on Facebook was also analyzed and we had its graph drawn. It was very cool to find out that one of the top influencers in that network was among the students taking the training. Students were also taught to do the same analysis for their own friends’ lists. Facebook data was being collected through Netviz and the visualizations were being drawn using Gephi.
After completing the interactive types of visualizations, the fourth day was about static ones, mainly, infographics. Each student had the chance to extract the information needed for an interesting topic to transform it into a visual piece. Bahia was working around with students, teaching them how to refine the data so that it becomes simple and short, thus usable for building the infographic design. Later, Yousif, a senior creative designer at Data Aurora, trained the students on the use of Photoshop and illustrator, two of the tools commonly used by infographic designers. At the end of the session, each student submitted a well done infographic of which some are posted below.
After the workshop Zayna had small talks with the students to get their feedback and here she quoted some of their opinions:
“It should be a full course, the performance and content was good but at some point, some data journalism tools need to be more mature and
user-friendly to reduce the time needed to create a story,” said Jad Melki, Director of media studies program at AUB, “it was great overall.”
“It’s really good but the technical parts need a lot of time. We learned about new apps. Mapping, definitely I will try to learn more about it,” said Carla Sertin, a media student.
“It was great we got introduced to new stuff. Mapping, I loved it and found it very useful for me,” said Ellen Francis, civil engineering student. “The workshop was a motivation for me to work more on this,” she added, “it would work as a one semester long course.”
Azza El Masri, a media student, is interested in doing MA in data journalism. “I like it I expected it to be a bit harder, I would prefer more advanced stuff in scraping,” she added.
After the step above, I refined the collected tweets and uploaded the data into CartoDB. It is one of my favorite open source mapping tools and I will make sure to write a CartoDB tutorial in future posts. I used the Bubble or proportional symbol map which is usually better for displaying raw data. Different areas had different tweeting rates and this reflected how different countries reacted. Countries like South Africa, UK, Spain, and Indonesia had higher tweeting rates. The diameter of the circles represents the number of retweets. With respect to colors, the darker they appeared, the higher the intensity of tweets is.
That’s not the whole story! Basically, it is easy to notice that some areas have high tweeting rates such as Indonesia and Spain. After researching about this topic, it was quite interesting to know that Mandela had a unique connection with Spain, one forged during 
