Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 50 additions & 19 deletions docs/quick-start/installation.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,101 @@
# Quick Start

In this quick start, we will use Kubedoop's hive-operator as an example to show how to deploy a Hive Metastore in a Kubernetes cluster.
In this quick start, we will use Kubedoop's hive-operator as an example to show
how to deploy a Hive Metastore in a Kubernetes cluster.

## Prerequisites

To start using Kubedoop, you need to meet the following requirements:

- Install [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)
- Install [Helm](https://helm.sh/docs/intro/install/) v3+
- Prepare a Kubernetes cluster

## Install OLM
## Install the Operator

Kubedoop relies on the Operator Lifecycle Manager (OLM) to manage Operators. If OLM is not installed in your cluster, you can install it with the following command:
Kubedoop uses Helm charts to deploy and manage Operators.
Container images are published to [quay.io](https://quay.io/kubedoopcharts) and
charts are distributed via a Helm repository.

### Add the Helm Repository

```bash
curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.25.0/install.sh | bash -s v0.25.0
helm repo add kubedoop https://zncdatadev.github.io/kubedoop-helm-charts/
helm repo update
```

## Install hive-operator
### Install Built-in Operators

OLM manages the installation of Operators through [Subscription](https://olm.operatorframework.io/docs/concepts/subscriptions/). We can install the hive-metastore-operator by creating a Subscription:
The commons-operator, listener-operator, and secret-operator are required
dependencies for all product Operators. Install them first:

```yaml
kubectl apply -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/olm-subscriptions.yaml
```bash
helm install commons-operator kubedoop/commons-operator -n operators --create-namespace
helm install listener-operator kubedoop/listener-operator -n operators
helm install secret-operator kubedoop/secret-operator -n operators
```

Verify that the operator's pod is running normally:
### Install the Hive Operator

```bash
kubectl get pods -n operator
helm install hive-operator kubedoop/hive-operator -n operators
```

Verify that the Operator pod is running normally:

```bash
kubectl get pods -n operators
```

## Create Namespace

Create a namespace for hive to deploy the hive cluster:
Create a namespace for Hive to deploy the Hive cluster:

```bash
kubectl create ns hive
```

## Deploy a Hive Cluster

The Hive cluster is managed by the hive-operator. We can deploy a Hive Metastore by creating a HiveCluster object:
The Hive cluster is managed by the hive-operator. You can deploy a Hive
Metastore by creating a HiveMetastore custom resource:

```yaml
kubectl apply -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/hive-metastore.yaml
```bash
kubectl apply -f - <<EOF
apiVersion: hive.kubedoop.dev/v1alpha1
kind: HiveMetastore
metadata:
name: hive-metastore
namespace: hive
spec:
roleGroups:
default:
replicas: 1
EOF
```

## Access Hive Metastore

After the Hive cluster is deployed, we can access the Hive Metastore with the following command:
After the Hive cluster is deployed, you can access the Hive Metastore with
the following command:

```bash
kubectl exec -it hive-metastore-0 -n hive -- bash
```

## Clean Up Resources

Run the following command to clean up the hive cluster:
Run the following command to clean up the Hive cluster:

```bash
kubectl delete -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/hive-cluster.yaml
kubectl delete hivemetastore hive-metastore -n hive
```

Run the following command to clean up the operator:
Run the following command to uninstall the Operators:

```bash
kubectl delete -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/olm-subscriptions.yaml
helm uninstall hive-operator -n operators
helm uninstall secret-operator -n operators
helm uninstall listener-operator -n operators
helm uninstall commons-operator -n operators
```
Original file line number Diff line number Diff line change
@@ -1,73 +1,105 @@

# 快速开始

Kubedoop 是一个基于 Kubernetes 的数据平台,它提供了一套完整的数据处理工具,包括数据采集、数据存储、数据处理、数据分析等功能。Kubedoop 通过 Operator 的方式部署在 Kubernetes 集群中,用户可以通过简单的配置文件来部署和管理数据处理任务。
Kubedoop 是一个基于 Kubernetes 的数据平台,它提供了一套完整的数据处理工具,
包括数据采集、数据存储、数据处理、数据分析等功能。Kubedoop 通过 Operator
的方式部署在 Kubernetes 集群中,用户可以通过简单的配置文件来部署和管理
数据处理任务。

在这篇快速开始中,我们以 Kubedoop 的 hive-operator 为例,介绍如何在 Kubernetes 集群中部署一个 Hive Metastore。
在这篇快速开始中,我们以 Kubedoop 的 hive-operator 为例,介绍如何在
Kubernetes 集群中部署一个 Hive Metastore。

## 环境准备

要开始使用 datastack ,需要满足一下条件
要开始使用 Kubedoop,需要满足以下条件

- 安装 [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)
- 安装 [Helm](https://helm.sh/docs/intro/install/) v3+
- 准备一个 Kubernetes 集群

## 安装 OLM
## 安装 Operator

Kubedoop 使用 Helm Chart 来部署和管理 Operator。
容器镜像发布在 [quay.io](https://quay.io/kubedoopcharts),
Chart 通过 Helm 仓库分发。

datastack 依赖于 Operator Lifecycle Manager (OLM) 来管理 Operator。如果你的集群中没有安装 OLM,你可以通过以下命令安装:
### 添加 Helm 仓库

```bash
curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.25.0/install.sh | bash -s v0.25.0
helm repo add kubedoop https://zncdatadev.github.io/kubedoop-helm-charts/
helm repo update
```

## 安装 hive-operator
### 安装内置 Operator

commons-operator、listener-operator 和 secret-operator 是所有产品 Operator
的必需依赖,需要先安装:

```bash
helm install commons-operator kubedoop/commons-operator -n operators --create-namespace
helm install listener-operator kubedoop/listener-operator -n operators
helm install secret-operator kubedoop/secret-operator -n operators
```

OLM 通过 [Subscription](https://olm.operatorframework.io/docs/concepts/subscriptions/) 来管理 Operator 的安装。我们可以通过创建一个 Subscription 来安装 hive-metastore-operator:
### 安装 Hive Operator

```yaml
kubectl apply -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/olm-subscriptions.yaml
```bash
helm install hive-operator kubedoop/hive-operator -n operators
```

验证 operatorpod 是否正常运行:
验证 OperatorPod 是否正常运行:

```bash
kubectl get pods -n operator
kubectl get pods -n operators
```

## 创建命名空间

创建一个 hive 的命名空间,用于部署 hive 集群:
创建一个 Hive 的命名空间,用于部署 Hive 集群:

```bash
kubectl create ns hive
```

## 部署一个 Hive 集群
## 部署 Hive 集群

Hive 集群是通过 hive-operator 来管理的,我们可以通过创建一个 HiveCluster 对象来部署一个 Hive Metastore:
Hive 集群通过 hive-operator 来管理。你可以通过创建 HiveMetastore 自定义
资源来部署一个 Hive Metastore:

```yaml
kubectl apply -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/hive-metastore.yaml
```bash
kubectl apply -f - <<EOF
apiVersion: hive.kubedoop.dev/v1alpha1
kind: HiveMetastore
metadata:
name: hive-metastore
namespace: hive
spec:
roleGroups:
default:
replicas: 1
EOF
```

## 查看 Hive Metastore

Hive 集群部署完成后,我们可以通过以下命令来访问 Hive Metastore:
Hive 集群部署完成后,你可以通过以下命令来访问 Hive Metastore:

```bash
kubectl exec -it hive-metastore-0 -n hive -- bash
```

## 清理资源

运行下面命令,清理 hive 集群:
运行下面命令,清理 Hive 集群:

```bash
kubectl delete -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/hive-cluster.yaml
kubectl delete hivemetastore hive-metastore -n hive
```

运行下面命令,清理 operator
运行下面命令,卸载 Operator

```bash
kubectl delete -f https://raw.githubusercontent.com/kubedoop.dev/kubedatastack/main/examples/hive/olm-subscriptions.yaml
helm uninstall hive-operator -n operators
helm uninstall secret-operator -n operators
helm uninstall listener-operator -n operators
helm uninstall commons-operator -n operators
```
Loading