Indicators and Data Effectiveness Assessment to Foster Impactful Innovation Hackathon

Nicolas Sacchetti 2022-09-26

Statistics Canada (StatCan) partnered with Policies, Processes and Practices for Performance of Innovation Ecosystems (P4-IE) Conference to run a hackathon May 4th to 6th, 2022 on innovation data indicators and assessment. The objective of the hackathon was to explore how data and indicators could be better leveraged to help government foster impactful innovation.

The event centred on the following question(s) to examine how data and indicators could be used or captured to better inform government initiatives:

  1. Design – How could the design of business innovation support programs be modified to include data capture to facilitate delivery and assessment?
  2. Delivery – How might we use available data to improve the effectiveness of government supports available?
  • Assessment – How might we use available methods or indicators that address the challenges of evaluating a business support program, such as the existence of multiple or successive supports used, or multiple targeted outcomes?

Participants

11 of the 26 participants came from Statistics Canada. There were 3 other participants that withdrew during the event for a variety of reasons. See below for a complete list of organizations that were represented, number of participants in parentheses:

  • Statistics Canada (11)
  • Polytechnique Montréal (5)
  • Canada Revenue Agency (3)
  • Innovation, Science, and Economic Development Canada (2)
  • Global Affairs Canada (1)
  • National Research Council Canada (1)
  • Toronto Metropolitan University (1)
  • Treasury Board Secretariat of Canada (1)
  • Vrije Universiteit Brussel (1)

Team Projects

Participants worked in teams and generated the ideas described below. Each entry is accompanied by commentary from the executive panel that reviewed the videos. To view the videos, please email StatCan’s Innovation Coordinator.

Team 1

Proposal: The formation of a “Precision BIGS” overseen by the new Canadian Innovation and Investment Agency (CIIA) to provide support and personal recommendations to firms. The recommendations would be informed by Statistics Canada, who would collect data in the same way it did with the original Business Innovation and Growth Support (BIGS) database. The recommendations would additionally create personalised benchmarking reports to the various firms that have answered. It is a free consultation/benchmark structure to funnel firms to the right institutions for funding/support that doubles as an incentive for them to provide data to Statistics Canada. 

Executive Panel: The panel thought that the idea to encourage firms to share their data with StatCan so that the CIIA can help them is a great one, as focusing on reducing burden on businesses while finding ways to return value to respondents is a worthwhile angle. It was clear how each stakeholder group would benefit from this idea, and the panel was unanimous in their thinking that this is an idea to pursue further. One panellist noted its similarities to the Canadian business productivity benchmarking tool, but thought that linking businesses to services the rest of the government could provide (not just BDC) would be an improvement.

Team 2

Proposal: Measure environmental performance metrics for all government initiatives using existing data to evaluate the combined action of all federal programs.

Executive Panel: The idea to use existing data to measure progress through a climate lens was extremely compelling and the panel remarked on the timeliness of this idea as it is a priority area for the Government of Canada. One panellist noted when StatCan recently launched the Experimental environmental, social and governance dashboard, there was an appetite from stakeholders to push it down further to the firm-level, which this idea would achieve. Although the panel would have liked to see an example or prototype of the indicator and how it would be shared with users, they noted that there is definitely a need to better measure the undesirable outputs that are created as a consequence of the production of desirable outputs. Overall, the panel was excited about this idea, and one panellist also noted that they would pass this idea along to the group working on ESG indicators at StatCan.

Team 3          

Proposal: The creation of a centralized data dictionary that houses all terminology and definitions, in addition to variance across different organizations. The accessible data dictionary would enable standardization across existing systems while complying to citizens’ data ownership and ensuring compliance with new systems.

Executive Panel: Overall, the panel thought that a government-wide approach to managing information that Canadians share with us would be a huge step forward in a key challenge in the GoC. Although there would need to be consultations with many partners to determine how the data dictionary would work to actually produce standards, there is no denying that coordinating on naming conventions, definitions and formats, and having a portal that documents who has what information would be extremely useful. One challenge to implementing this idea will be to balance the needs of coordination while leaving flexibility to adapt to specific circumstances (for example, regional development agencies are part of the federal government portfolio and the definition of a large firm may vary from region to region). The panel commended the team for acknowledging the obstacles to making this happen while also proposing a concrete first step that could be implemented to demonstrate the benefits of this approach.

Team 4

Proposal: A linked database that relates BIGS data to COVID-19 support programs, to determine how many BIGS recipients were affected by the pandemic and how receiving BIGS support affected the health of their businesses during the pandemic. The team proposed focusing on immigrant-owned businesses.

Executive Panel: While one panellist commented that the team made a strong use case for choosing to focus on immigrant-led businesses, another noted that there are many non-immigrant-owned businesses that innovate and focusing on this subsection would eliminate important information. As an alternative, they suggested another potential way to limit the scope of the initial study could be to focus on high-growth firms. That said, overall the panel was in support of this idea and thought that the proposal touched on a very important question that should have been addressed when the adequacy of COVID-support programs was being examined. 

Team 5

Proposal: A centralized customer relationship management (CRM) portal/database that shares basic information (including company data, program/service data and departmental data) about the recipient company of a given government program/service between departments. Public servants working with recipient companies can use the CRM portal to determine which departments have already served this company and therefore coordinate delivery of other services. Specific program deliverers can also use the portal to seek out potential new recipients/clients.

Executive Panel: The panel thought this proposal was promising and had them envisioning ways in which it could be useful to businesses across Canada. Rules would need to be developed and made clear to businesses on what is shared and not shared, as some may prefer to have their interactions compartmentalized, but the panel overall liked the focus on reusability of existing GoC resources. One panellist noted that within StatCan alone there is an appetite to improve our CRM system, but at the government-wide level there are even more advantages. Additionally, business-to-business and business-to-academic connections are also important to the innovation ecosystem, so this proposal had one panellist thinking about how this registry could facilitate those connections as well.

Team 6

Proposal: An interactive map intended to improve the way Canadians access business-related data for a particular geographic region of interest (province/city-level).

Executive Panel: The value of the idea was clearly demonstrated by the team’s proposal, especially given their use of visuals to demonstrate achievability. The panel thought that a mapping tool was a strong idea, and that the aggregate view coupled with an ability to drill down the information by areas of interest (i.e. mixing business data with general population data for a selected geography) was very intuitive. The proposal also had one panellist thinking about how this tool could be used to provide information that helps identify the addressable market for program deliverers, including a comparison of what areas are being served and what areas are not being served. 

Key Takeaways

All six ideas pitched by the teams were clearly delivered, engaging and had us convinced of their feasibility.

Team 1’s fitness trainer analogy was extremely effective. The way the duties, roles and responsibilities were broken out seems realistic and promising.

Team 2’s idea to bring an environmental lens to all government initiatives and measure progress in a priority area for government and Canadians was extremely compelling, and it is worth investigating if StatCan has the data available to do this on our own.

We appreciated Teams 4 and 5’s proposals to test their ideas on a small group/subset first to determine achievability before moving into a more complex system. Although it is too late for Team 4’s proposed assessment of programs to inform current COVID-support programs that are winding down, many lessons can be learned from the analysis they proposed. There may also be an opportunity to integrate Team 5’s CRM system into Team 1’s “precision BIGS” consultation/benchmark structure.

With more thought into how the government can use Team 3’s data dictionary and Team 6’s mapping application to improve the design, delivery and evaluation of business support policies, these tools have the potential to address some key challenges facing the GoC while also leveraging StatCan’s strengths.

We would like to see all the ideas moved forward towards implementation with a pilot. As a starting point, one panellist suggested all teams comprised of public servants should submit their ideas to the Public Service Data Challenge to gain traction from the Treasury Board Secretariat (TBS) and other government agencies.

P4IE Conference Panel Discussion

Following the hackathon, a panel discussion during the Policies, Processes and Practices for Performance of Innovation Ecosystems Conference discussed lessons learned from the hackathon. Although the majority of the discussion did not center on the hackathon projects individually, the panel noted that throughout the proposals there was an overall focus on the need for centralization of data and how we can create value with this data in general. They commented that despite the difficulties to measure if one business support program is impacting another, we can leverage already existing available data to find some of these insights, as many teams had suggested. The panel also discussed the need for more granular data and the immense benefit for programs supporting business innovation to have access to information by region, and thought that StatCan could help with moving this forward.

Overall, the panel was impressed by all the hackathon team’s work, and gave brief shout-outs to the work of Team’s 1, 4 and 5 in particular, noting that they hoped these projects would move forward.

Special Thanks

A special thank you goes out to the Partnership for the Organisation of Innovation (4POINT0) who led the organization of the P4IE conference, for the opportunity to facilitate this hackathon and add value to the conference and business innovation in Canada. We would also like to thank our mentors from StatCan and TBS – Julio Rosa, Huju Liu and Rashid Nikzad – for their time and dedication during the hackathon check-ins where they provided guidance and support to the teams. These mentors were an integral part of the hackathon, and their detailed and relevant feedback was highly valued by the participants.

This content has been updated on 2022-09-27 at 23 h 35 min.