Generating input for a publicly available mobility insights dashboard
Create a scalable and future-proof data architecture based on publicly available requirements
Challenge
Create a scalable and future-proof data architecture based on publicly available requirements
Mobility is a crucial challenge for our society. It contributes to economic prosperity and also plays an essential role in the climate objectives. But do we have all the necessary tools to analyze our mobility? For example: How much does traffic congestion cost on an annual basis? Or which rail routes have the most delays? For a new platform developed by one of our clients, we helped to turn those questions into practical insights based on publicly accessible data sources from various mobility and (public) transport organizations.
Approach
Working together with our client’s internal data scientist to assess data sources
We started off by analyzing all publicly accessible data sources and selected which data was fit and relevant for captation. A swift captation was essential because most data could not be registered retroactively as these were daily-based data, such as the amount of traffic congestion or train delays. We therefore decided to immediately start with the registration of as many data sources as possible. By working closely together with our client’s internal data scientist, we determined the most interesting data for further analysis and comparison. This was an important step, as being able to draw insightful conclusions from the data was the goal of this project. We then developed a technical back end to map and centralize these data sources. We made sure to create a scalable and future-proof data architecture that our client can use as a foundation for potential future expansions of the product.
During the creation of the data architecture we constantly assessed which features would be essential for the final product while considering the technical feasibility of these features. This resulted in a constant dialogue between our technical team and the data science team to balance the essentials and nice-to-haves.
Impact
Cloud-based technologies guarantee secure data availability
Datashift often fulfills a more consultancy-based role. Here, we really helped to develop the project from the ground up which means we also took care of the security aspect. This is an especially important part of the product as it will be publicly available. In order to guarantee a continuous availability of the platform even when certain data sources are unavailable, we opted for a technological framework that is cloud-based. We also implemented a quality monitoring and error reporting system that immediately sends out a notification via cell phone and email in case of calamities.
Based on our selected data sources, users of the interactive platform are now able to compare mobility data and trends over a longer period of time. This information is now freely accessible in a clear and digital dashboard created by an external partner. The interactive ways to access these data sources is one of the main innovations of this project. By making optimal use of cloud-based technologies and event-driven ETL processes, we guaranteed a swift import of all selected data sources. Moreover, cloud storage makes for an affordable storage solution that is remotely accessible. These features really highlight the advantages of utilizing a cloud-based data architecture, something we always strive for when applicable.