Create a cluster

Spinning up a cluster in easy, whether you want to try ML Studio locally or use it in the cloud.
Cloud
Create a GKE k8s cluster
You need to have Google Cloud SDK for this step, follow the instructions to install it here.
1
export cluster_name=mlstudio-cluster
2
export cluster_zone=us-central1-a
3
​
4
gcloud container clusters create $cluster_name \
5
    --machine-type=n1-standard-4 \
6
    --num-nodes 1 \
7
    --enable-autoscaling --min-nodes 0 --max-nodes 6 \
8
    --zone $cluster_zone
9
    
10
# Chnage kubectl current context
11
gcloud container clusters get-credentials $cluster_name --zone $cluster_zone
Copied!
For cost savings you can use --preemptible nodes. 
They offer the same machine types and options as regular compute instances and last for up to 24 hours.
1
export cluster_name=mlstudio-cluster
2
export cluster_zone=us-central1-a
3
​
4
gcloud container clusters create $cluster_name \
5
    --machine-type=n1-standard-4 \
6
    --preemptible \
7
    --num-nodes 1 \
8
    --enable-autoscaling --min-nodes 0 --max-nodes 6 \
9
    --zone $cluster_zone
10
    
11
# Chnage kubectl current context
12
gcloud container clusters get-credentials $cluster_name --zone $cluster_zone
Copied!
Create a GPU accelerated cluster
You can instead create a GPU accelerated cluster by appending --accelerator type=nvidia-tesla-t4,count=1 to the previous command. And then creating a DaemonSet to instal Nvidia drivers.
1
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/master/nvidia-driver-installer/cos/daemonset-preloaded.yaml
Copied!
So the final commands to create a GPU accelerated cluster would look like:
1
export cluster_name=mlstudio-cluster
2
export cluster_zone=us-central1-a
3
​
4
gcloud container clusters create $cluster_name \
5
     --machine-type=n1-standard-4 \
6
     --accelerator type=nvidia-tesla-t4,count=1 \
7
     --num-nodes 1 \
8
     --enable-autoscaling --min-nodes 0 --max-nodes 6 \
9
     --zone $cluster_zone
10
​
11
# Install NVIDIA GPU device drivers
12
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/master/nvidia-driver-installer/cos/daemonset-preloaded.yaml
13
​
14
# Chnage kubectl current context
15
gcloud container clusters get-credentials $cluster_name --zone $cluster_zone
Copied!
Local
Create a Minikube k8s cluster
1
minikube start --cpus 5 --memory 10096
Copied!
Docker Desktop
Download Docker Desktop application and follow the instructions on enabling Kuberentes.
You need a minimum of:
5 CPUs
8 Gb of RAM
15 Gb of fee desk space

Prerequisites

Next - Install

HELM

Last modified 1yr ago

Copy link

Contents

Cloud

Create a GPU accelerated cluster

Local