A user’s journey through the Destination Earth Data Lake

DestinE is implemented through a strategic partnership between three entrusted entities, with each responsible for a specific component. ECMWFEuropean Centre for Medium-Range Weather Forecasts More and ESAEuropean Space Agency oversee the digital twins and DestinE PlatformSelf-standing DestinE system component, interfacing with the More respectively, while the DestinE Data Lake (DEDL) is implemented by EUMETSATEuropean Organisation for the Exploitation of Meteorological.

The Data Lake provides access to data of unprecedented scale and diversity. Its structure and how it fits into DestinE as a whole are perhaps best understood by viewing the journey of a user – a broad term that embraces a wide variety of stakeholders, such as public authorities, small and medium-sized enterprises (SMEs) and startups, academia and research, and the general public.

Video infographic of the user journey through the Data Lake

An example might be a national agency wishing to understand the impact of global warming on one of its regions by answering ‘what-if’ questions. Entering via the DestinE PlatformSelf-standing DestinE system component, interfacing with the More, inside the Data Lake the agency’s analyst seamlessly retrieves past data from the Climate Change Adaptation Digital Twin, along with other relevant data for the area. The analyst develops an application, hosted within the Data Lake, that processes the information and produces AI-powered historical variable maps of the chosen region. They then interact with the Climate Adaptation Digital Twin again to generate a future scenario. Finally, the results of the prediction, together with the historical data, are pulled into a technical report and shared with the DestinE community.

Both the video and the analyst’s query illustrate what a user’s journey through the Data Lake may look like, involving a number of features and services:

• Discover data: via Harmonised Data Access (HDA), users can discover a vast volume of products in the DEDL Data Portfolio. These comprise the datasets produced by the digital twins and federated access to the common European Green Deal data spaces, along with user-generated DestinE data that will be available in the future. HDA offers a straightforward route to data, and the service’s application programming interface (API) simplifies connectivity
• Digital twin data: with the Climate Change Adaptation Digital Twin, users can explore a range of climate-related scenarios from the past, present or future. The Weather-Induced Extremes Digital Twin yields predictions of the near-short term in high resolution
• AI/ML: many datasets can be converted to be AI-ready, enriching processing using machine learning algorithms
• User workflows: DEDL users define tasks for retrieving and processing data by creating scripts of programming code, known as workflows, using languages like Python or R
• Hosted applications: users create their own applications, stored and executed within the DEDL, typically for responding to what-if questions on scenarios within the Earth system. User tools are provided through various Edge Services, including the STACK service for Jupyter-based workflow creation, HOOK for pre-defined workflow functions, and ISLET Storage for users’ data and results
• Virtual machines and clusters: DEDL data bridges, next to European high-performance supercomputers, allow processing near to where the data is produced. Rather than on their own PC, users run workflows on a virtual machine in the DEDL. Again, Edge ServicesNear-data processing services on the DEDL. More are key to facilitating access, and ISLET Compute is one such service that enables users to manage these virtual machines for deploying their workflows

An overview of the Data Lake services and Data Portfolio

As of June 2025, several new DEDL features have been implemented to enhance the user experience:

• My Data Lake Services is a self-service portal that gives users a one-stop-shop for requesting and managing Edge ServicesNear-data processing services on the DEDL. More
• The HDA API has been improved with new functions
• Additional datasets are available through the HDA, including statistics from Eurostat datasets and products from EUMETSAT’s second and third-generation Meteosat satellites

The incorporation of Eurostat and EU-JRC collections highlights the power of data fusion. By combining data from heterogeneous sources, users can derive insight that is multidimensional and more meaningful.

Further improvements towards the end of the year will focus on the generation of AI-ready data, more new HDA features and better system resource management capabilities. Details on DEDL services, along with comprehensive technical documentation, can be found on the Data Lake web pages.