CDM

CDM - CKAN Data Manager - is your tool to move large datasets to and from CKAN instances. CDM is specially designed to move big datasets in Tb range efficiently and reliably. It compresses and decompresses data on the fly, splits data into smaller chunks and uploads/downloads them in parallel.

The tool provides both a TUI (Text-based User Interface) and a CLI (Command-line Interface). The TUI is designed for interactive use, allowing you to explore options and run processes step by step in a user-friendly way. The CLI is intended for automation and scripting, which is especially useful on HPC systems or for batch processing.

Download and extract the archive directly into ~/bin. Make sure the directory exists before running the command.

Linux

macOS (AMD64)

macOS (ARM64)

Ensure that ~/bin is included in your PATH (e.g. by adding the following line to ~/.bashrc or ~/.zshrc):

Verify the installation:

On our HPC system, CDM is provided as a module. You do not need to download or extract anything manually. Simply load the module in your terminal before using the software:

Open a terminal (PowerShell) and run the following commands. All files will be installed to %USERPROFILE%\bin.

Download the latest CDM archive:

Extract the archive into the bin directory:

Add %USERPROFILE%\bin to your PATH environment variable so you can run cdm from any terminal:

Close and reopen your terminal, then verify the installation:

Before you can use CDM, you need to configure it with the URL of your CKAN instance and an API key.

The URL of your CKAN instance is https://ckan.fdm.uni-greifswald.de/. You can generate your API key in your user profile settings. To do this, log in to CKAN and click on your user name in the top-right corner. Then navigate to the "API Token" tab to generate your API key.

Once you have your API key, start the TUI with cdm and navigate to setup. There, you can enter the CKAN URL and your API key. In the CLI, you can run the following command: