Video and Audio versions
Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.
The tremendous growth of computing power, especially in the form of mobile devices, means that more and more people can analyze data themselves, or have applications do it locally, reducing the need for large central systems to perform all the calculations. Indeed some of the value from Hadoop and other cluster computing systems comes from using lots of small, relatively powerful systems to do more work than one single computer could.
However we can not only analyze, but also gather data. There is a rise in the citizen scientist, in which individuals can help scientists by gathering data and sharing it with others. This can reduce the costs of gathering data, but more importantly, gather much more, and varied, data than might otherwise be possible. There are a number of projects available if you’re an amateur scientist that wants to join in.
Most of us are data professionals and might be interested in working with more data. Perhaps we ant a set for demo purposes, but perhaps we want to work on a project that interests us and could use our technical talents. Many of the projects out there make their data available, and you should consider using such a set if it strikes your fancy.
More importantly, I would hope that more people would analyze the raw data and look to verify, or dispute, the conclusions of the project owners. As data professionals, we can help people learn how they can analyze and manipulate data better with the tools we have available. With more transparency, we can have healthier debates on the causes, the effects, and even the potential future implications of how to build a better world in the future.