Data Democratisation Energy Australia
First I wanted to thank Instituto Cervantes in Sydney for inviting me to be part of an awesome conversation about data democratisation in their Research Bites series.
I thought I share my presentation and what I hope attendees got out of the session.
Data Democratisation is simply put: making digital information readily available for non-technical users.
I have observed three critical groups that need to be coordinated and orchestrated to achieve true Data Democratisation:
- Policies, Legislation and Governance; Can we get it, from Where, Who can see it.
- Technology, Infrastructure and Visualisations; How much data, How fast, and How does it look like.
- Education and Continous Improvement; Best Practices, More People, User Feedback
To be frank, I don't know enough about the first group to express my opinion about it publicly, I often find when people rush into opinions tends to be wrong and very irresponsible.
However, I do a thing or two about the technology that has allowed my business to compete with much bigger corporations.
We truly believe in the power of Data Democratisation and today we operate as a middleware between the raw sources and the non-technical user, our vision is to democratise energy data to everyone, we started with Corporate, Retailers and New Renewable Generation project.
Technology
The journey and influence that technology has in the data landscape has been outstanding. Companies like Facebook, Airbnb, Google, Apache and many others are fully supporting and funding amazing opensource projects that give new Startups the tools to really compete with anyone. The traditional plutocracy of the old outdated Data Giants it's giving in to the nimble and cost-effective approach of Data Startups.
In Australia however, the Energy industry still lags behind for a number of reasons. And mostly operates on the old premise “That's for the IT department to solve”.
Side note: People in the innovation areas in energy are becoming less technical and more “Creative Focus”, trying to emulate that image of the Steve Jobs (RIP) myth, of the non-technical creative that can shape the future. Please Stop.
Infrastructure
Is Storage a real problem anymore? NAH and So Analysing this data must take forever, wrong again.
If you put the work and learn (not hard) how to balance Serverless, with Microservices, SQL Engines (not DB’s) and a bit of orchestration you can run almost supercomputer type calculations and analysis with almost no system administration and very affordable. We do it, and we are happy to share it if you want to learn more.
Visualisations
Off course we need great visualisation tools, remember, this is for the non-technical user, is for the expert in the field. The user that can make sense of tables, charts and timelines. The tools today are easy to use and very complete, even if you want o build your own, some of the opensource options out there are incredible.
All sounds good huh?
Well, it is, but to be completely honest, not everyone asks the end-non-technical user what they want, and most time they are giving incomplete or even worse not relevant data to carry important research.
This is the three critical group disconnect.
Technology leads the way and today is perfectly capable to achieve complete Data Democratisation, but technology is not the gatekeeper, people are.
We need to train people and businesses to use the right tools and software, educate them into ethical usage of the data and the wonderful world of objective exploration.
And we need Governments, Executives and Organisations to catch up with the rest of the world.
Because as per today Data is not quite Democratic, is mostly Meritocratic and in some cases still Plutocratic.
The first step is to make it a goal, not a nice thing to aspire, but an actual goal with Milestones and due dates. This is engineering and scientific it does not need to be so free-spirited.
I do hope I’ve sparked a tiny little bit of curiosity to learn and share, evangelise Data Democratisation within your organisation, it’s a great goal to have, and will make a difference in the future.
