Welcome to ECAS’s documentation!

The ENES Climate Analytics Service (ECAS) will enable scientific end-users to perform data analysis experiments on large volumes of climate data, by exploiting a PID-enabled, server-side, and parallel approach.

It aims at providing a paradigm shift for the ENES community with a strong focus on data intensive analysis, provenance management, and server-side approaches as opposed to the current ones mostly client-based, sequential and with limited/missing end-to-end analytics workflow/provenance capabilities.

To login into Jupyter, enter the username and password provided to you after registration at DKRZ. If you would like an account or need assistance, please email ecas-support@dkrz.de.

Note

ECAS is part of the EOSC-HUB service catalogue

Contents:

Overview

The ENES Climate Analytics Service (ECAS) will enable scientific end-users to perform data analysis experiments on large volumes of climate data, by exploiting a PID-enabled, server-side, and parallel approach.

It aims at providing a paradigm shift for the ENES community with a strong focus on data intensive analysis, provenance management, and server-side approaches as opposed to the current ones mostly client-based, sequential and with limited/missing end-to-end analytics workflow/provenance capabilities.

To login into Jupyter, enter the username and password provided to you after registration at DKRZ. If you would like an account or need assistance, please email ecas-support.

What you can do with ECAS?

  • Data analysis

  • Visualization

  • Interactive programming

Who is providing ECAS?

ECAS is being developed and deployed at DKRZ and CMCC. We are trying to keep both instances synchronized as much as possible. However, in some cases, usage of the service can slightly differ between the two instances. More or less steps might be required from the users. This documentation is principally for ECASLab at DKRZ. For specific information about Ophidia framework, we suggest that you use the official documentation.

ECAS Users

  • Climate community

ECAS Components

Architecture

_images/new_ecas_architecture.png

Involved services/frameworks

  • Ophidia: is a research project on big data analytics for eScience. It provides a framework for parallel I/O and data analysis, an array-based storage model and a hierarchical storage organization to partition and distribute multidimensional scientific datasets.

  • B2Handle: the b2handle Python library is a client library for interaction with a Handle System server. The role of B2Handle is to enable PID-provenance support

  • B2DROP and B2SHARE: are EUDAT services that provide a flexible and open data sharing. ECAS users will be able to access/share data and notebooks.

  • EOSCHUB AAI: besides LDAP, B2ACCESS, EGI Check in and Indigo IAM can be used to login to the ECASLab.

  • ESGF: provides a robust software stack covering different areas. The most significant for ECAS is Data node. Actually, CMIP5 and soon CMIP6 data are available from ESGF data pool at CMCC and DKRZ.

How to register/Log in

There are two ways to register/log in to ECAS. The current release of ECAS allows authentication/authorization using LDAP. For this, an account at DKRZ is required. However, since ECAS is related to the EOSCHUB project, the authentication/authorization process will be through one the EOSCHUB AAI providers (INDIGO IAM, B2ACCESS and EGI Check IN).

Login/Register with DKRZ account

The current registration and Log-in process is based on LDAP.

Important

You need an account at DKRZ in order to use ECAS.

The registration at DKRZ is straightforward and is available at this link. After you get an account, there is one more important step, which requires to join the ECAS project.

Note

To join a project, simply open your new account and click on Join existing project and select ECAS.

More information on how to register can be found here. If you encounter any troubles during the registration process, please don’t hesitate send an E-Mail to ecas-support@dkrz.de.

After you get an account and join the ECAS project, you can log in to the JupyterHub.

Login/Register with EOSC HUB AAI

Currently, we are working on a solution that allows you to log into ECASLab through one of EOSCHUB AAI providers: EGI Check in, B2ACCESS or Indigo IAM. A second JupyterHub instance is provided for this purpose and can be found here.

When you are at the JupyterHub log in page just click on EOSCHUB AAI button and you will be redirected to our AAI broker.

_images/JupyterHub_login.png

At this stage, you can either user your favorite AAI provider or simply create a local account.

_images/registration_login_page.png

If you are using the service for the first time, you will be asked to accept the terms of use and to confirm your E-Mail and update the profile if necessary. In case you do not want to use the available AAI providers, you can register and get a local account.

Note

Please note that it takes some moment to validate your account and add you to the corresponding ECAS group. Do not be surprised to see the forbidden message the first time you log in to JH.

Sharing with ECAS

Sharing your results with your team members or a broader community researcher enhances the promotion of your research. In ECAS, there are various ways to share and your computation results (datasets, code and Jupyter notebooks).

Sharing is achieved mainly through the following EUDAT services:

  • B2DROP is a DropBox-like service to store and share your data/workflows. A 20GB storage is allocated for each ECAS user.

  • B2SHARE is a user-friendly, reliable and trustworthy way for researchers, scientific communities and citizen scientists to store and share small-scale research data from diverse contexts. Persistent identifiers are assigned to any shared data or workflow, which improve the its findability.

For more details, please consult the corresponding documentation on how to use these services. ECAS is integrated within the software stack. There should be no extra effort to use them within your workflows. Only mandatory step is to provide your credentials in order to access “your” private workspace.

ECAS File System

This sections lists the directories, which are available in the Jupyter notebook. This important to know what are operations allowed on each of them.

User workspace

This space is only for the user opening a session at ECAS. The data and workflows have read-write access and are stored in a specific docker volume at disk. Even if you close your session or you restart the notebook, data will not be lost.

Shared with others

The following directories are read-only directories and you are not allowed to create or save notebooks/data.

  • data,

  • notebooks,

  • quickstart and

  • workflows

The content of these directories is avalable for all users logged into ECASLab. The conf is also a shared directory and contains information on how to mount your private B2DROP into the Jupyter notebook. This is not recommended to store other files in this directory.

ESGF Data Pool

… Already available but documentation coming soon.

Ophidia File System

Please go here for more information about the Ophidia filesystem.

ECAS and WPS

Note

ongoing work!

Objectives

The integration with Birdhouse enables running/access ECAS processes through the web.

Birdhouse

Get in Touch!

If you need any kind of support related to ECAS, feel free to use one of these channels to communicate with us.

Indices and tables