How to Access a Mount Point on Databricks
Today I learned how to mount an Azure blob on Databricks.
Databricks offers a variety of ways to attach storage points onto your notebooks. To do that, the platform configures each cluster with a FUSE mount, which they call Databricks File System (DBFS).
Let’s use an Azure storage blob as example. If you want to mount it somewhere in your cluster, use the method in the built-in module dbutils
inside your notebook:
CONTAINER_NAME = "<container name>"
STORAGE_ACCOUNT = "<storage account>"
MOUNT_POINT = "/mnt/mount_point"
SAS_TOKEN = "<your SAS token>"
extra_configs = {
f"fs.azure.sas.{CONTAINER_NAME}.{STORAGE_ACCOUNT}.blob.core.windows.net": SAS_TOKEN
}
dbutils.fs.mount(
source=f"wasbs://{CONTAINER_NAME}@{STORAGE_ACCOUNT}.blob.core.windows.net",
mount_point=MOUNT_POINT,
extra_configs=extra_configs,
)
Once mounted, you can access your mount point by prepending the path with /dbfs
. E.g.:
import os
res = os.listdir("/dbfs" + MOUNT_PATH)
print(res)