Federal Science DataHub

Migrating Databricks

This document provides information about:

  • migrating your Databricks resources from the proof-of-concept (PoC) environment to your new FSDH workspace in the production (prod) environment
  • offboarding your Databricks resources from the PoC environment should you not be shifting your research to the prod environment

Please note: Active analysis and experimentation in PoC can continue until Oct. 31, 2025. All databases, storage, web apps, and Databricks resources must be migrated or offboarded from PoC by Nov. 30, 2025.

A. Migrating from PoC to prod

To migrate your Databricks resources, you must have a new workspace in prod. Once we launch at the end of June, you can proceed with the workspace intake process through the GC Hosting Services Portal. Guidance is available on the FSDH GCXchange site and FSDH product guide (opens new window).

Once your workspace has been provisioned in prod, you can migrate resources from PoC to prod. The Federal Science DataHub support team will assist you with migrating your Databricks resources to your new workspace in the PROD environment. Please submit a Support Request to get started.

Please note, only the following Databricks resources can be migrated from the POC environment to the PROD environment:

  • Notebooks
  • Repositories
  • Jobs
  • Secrets
  • Table ACLs
  • ML models

If you have Databricks resources not listed above (e.g., clusters, including custom conda clusters, or mounted storage), they will need to be recreated in your new workspace. Please refer to Databricks documentation (opens new window) or reach out to the FSDH team for more information.

B. Offboarding from POC

If you are not opening a workspace in the PROD environment, you may want to export your Databricks resources (e.g., notebooks, repositories, etc.) from the POC environment and import them into your local environment or another Databricks workspace.

← Migrating PostgreSQL →