Building a Culture of
Accessible and
Accountable
Data Sharing

Timothy Martin Chief Analytics Officer
NYC Buildings

Organizations all share one specific challenge...

Sharing Information

What are the challenges with data sharing initiatives?

Technical

  • Working with legacy systems
  • Poor data quality
  • Limited metadata documentation
  • Obscure business logic


						CASE WHEN "- Key Job Information (Job Number and Status)".
	"Job Type" = 'NB' THEN "- 8 - Additional Information".
	"Total Construction Floor Area"  *148 ELSE  "
	- Job Filing Facts"."Estimated Job Cost" END

Culture

  • Staff are not comfortable with the concept
  • Executive participation is critical
  • Must push the message that...
    • More sharing = More insights

Exposure

  • Unintended consequences of exposing data
  • Scrutiny on established methods and quality control

Multiple versions of the truth

What are the benefits of data sharing initiatives?

Discovery

  • Accelerate discovery of new insights
  • Empower end users to analyze data in different ways
  • Modern BI tools make analytics easier to federate
  • Discovery/Visualization with little programming

Scale

  • Empower non-tech staff with self-service analytics
  • Free up analytics staff to work on advanced projects

Consensus

  • Build consensus around challenging questions
  • Force organization to develop common methods
  • Make analysis and reporting more authoritative
  • Improve overall transparency

Automation

  • Data sharing forces automation
  • Ensure reproducibility through scripting algorithms

Transition from Siloes
to Federated Analytics

Initial Setbacks

  • Sharing data technically versus making sense of it
  • Analytics insights not connecting with business issues
  • Business experts may have technical gaps

Training

  • Growing pains with getting analytics staff up to speed with the business side
  • Requires veteran analysts to train other analysts on data elements and reporting methods

Data Dictionaries

  • Critical to have; however, often do not provide total insight into business operations

User Guides

  • Train staff how to use data
  • Create conversation around methodology

What about sharing data externally?

Privacy

  • Address PII concerns
  • Balance customer protections with transparency goals
  • Ensure that anonymization can’t be easily reversed

Open Data

  • Requirement to publish all government data
  • Used by data aggregators and civic technologists
  • Presents challenges and opportunities

Open Algorithms

  • Next big initiative in government data sharing
  • Driven by increased scrutiny on bias in algorithms
  • Benefit: External groups can evaluate assumptions

Messaging

  • Visibility into the information you want to share

Building a Culture of Accessible and Accountable Data Sharing