Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/zh/05-features/05-auto-tagging/08-k8s-crd.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ inputs:

由于一些用户的 Kubernetes 环境可能具有特殊的配置或安全要求,使得标准化的提取工作负载类型和工作负载名称的方式无法按预期工作,或者用户可能希望根据自己的逻辑来定制工作负载类型和工作负载名称。基此,DeepFlow 支持用户通过添加自定义的 lua 插件提取工作负载类型和工作负载名称。Lua 插件系统通过在固定的地方调用 Lua Function 获取一些用户自定义的工作负载类型和名称,提高 K8s 资源对接的灵活性与普适性。

## lua 插件编写示例
## Lua 插件编写示例

```lua
-- 固定写法,以导入 json 解析的包
Expand Down
5 changes: 0 additions & 5 deletions docs/zh/08-integration/01-process/02-lua-plugin.md

This file was deleted.

4 changes: 2 additions & 2 deletions docs/zh/10-release-notes/05-ce-6.6-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ permalink: /release-notes/release-6.6-ce
- AutoTracing
- 调用链追踪火焰图自动校正不同机器的细微时钟偏差。
- AutoTagging
- 支持利用 Lua Plugin 自定义 K8s 工作负载抽象规则,[文档](../integration/process/lua-plugin/)。
- 支持利用 Lua Plugin 自定义 K8s 工作负载抽象规则,[文档](../features/auto-tagging/k8s-crd/)。
- 支持同步 LoadBalancer 类型的容器服务。
- Server
- 支持使用 OceanBase 替换 MySQL。
Expand All @@ -171,7 +171,7 @@ permalink: /release-notes/release-6.6-ce
- 当协议头中存在 TraceID 时,支持关闭 eBPF syscall_trace_id 计算(通过配置 `syscall_trace_id_disabled`),以降低对业务性能的影响。
- 调用链追踪火焰图自动校正不同机器的细微时钟偏差。
- AutoTagging
- 支持利用 Lua Plugin 自定义 K8s 工作负载抽象规则,[文档](../integration/process/lua-plugin/)。
- 支持利用 Lua Plugin 自定义 K8s 工作负载抽象规则,[文档](../features/auto-tagging/k8s-crd/)。
- Agent
- 支持完全关闭 cBPF 数据采集(通过配置 `tap_interface_regex` 为空字符串),以降低内存开销。
- 支持 deepflow-agent 使用一个 Socket 传输所有观测数据,并可通过 `multiple_sockets_to_ingester` 配置项关闭此特性从而使用多个 Socket 以提升传输性能。
Expand Down
2 changes: 1 addition & 1 deletion docs/zh/10-release-notes/06-ee-6.6-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ N/A
- 默认采集容器中的进程信息。
- 管理能力
- 录入 K8s 集群时支持指定 ClusterID,使得集群重新录入时可复用旧 ClusterID。
- 支持利用 Lua Plugin 自定义 K8s 工作负载抽象规则,[文档](../integration/process/lua-plugin/)。
- 支持利用 Lua Plugin 自定义 K8s 工作负载抽象规则,[文档](../features/auto-tagging/k8s-crd/)。
- 限制同一组织同一区域内仅支持一个`采集器同步`类型的云平台。
- 支持通过普通账号的 AK/SK、携带 ResourceGroupId 完成阿里云资源同步。
- 预置资源同步滞后、资源关联关系异常的系统告警。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -129,99 +129,6 @@ The only thing to note is that the tap_interface_regex of the collector only nee

The only thing to note is that the tap_interface_regex of the collector only needs to be configured as the Node NIC list.

# Special K8s Resources or CRD

In these scenarios, the following operations are required:

- Enable and disable the corresponding resources in the Agent advanced configuration
- Configure Kubernetes API permissions

## OpenShift

In this scenario, the default `Ingress` resource acquisition needs to be disabled, and the `Route` resource acquisition needs to be enabled.

Agent advanced configuration is as follows:

```yaml
static_config:
kubernetes-resources:
- name: ingresses
disabled: true
- name: routes
```

ClusterRole configuration addition:

```yaml
rules:
- apiGroups:
- route.openshift.io
resources:
- routes
verbs:
- get
- list
- watch
```

## OpenKruise

In this scenario, the `CloneSet` and `apps.kruise.io/StatefulSet` resources need to be obtained from the API.

Agent advanced configuration is as follows:

```yaml
static_config:
kubernetes-resources:
- name: clonesets
group: apps.kruise.io
- name: statefulsets
group: apps
- name: statefulsets
group: apps.kruise.io
```

Note that Kubernetes's `apps/StatefulSet` needs to be added here.

ClusterRole configuration addition:

```yaml
- apiGroups:
- apps.kruise.io
resources:
- clonesets
- statefulsets
verbs:
- get
- list
- watch
```

## OpenGauss

In this scenario, the `OpenGaussCluster` resource needs to be obtained from the API.

Agent advanced configuration is as follows:

```yaml
static_config:
kubernetes-resources:
- name: opengaussclusters
```

ClusterRole configuration addition:

```yaml
- apiGroups:
- opengauss.sig
resources:
- opengaussclusters
verbs:
- get
- list
- watch
```

# Limited Permissions for Running Agent in K8s

## No Permission to Deploy K8s Daemonset
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,113 @@
---
title: Lua plugin
permalink: /integration/process/lua-plugin
title: K8s CRD Labels
permalink: /features/auto-tagging/k8s-crd
---

> This document was translated by ChatGPT

# About Lua Plugin
# Common special K8s resources or CRDs

When an unsynchronized (workload-unassociated) container Pod is detected:
- If the value of Pod's `metadata.ownerReferences[].apiVersion = apps.kruise.io/v1beta1`, then the corresponding K8s platform should be OpenKruise.
- If the value of Pod's `metadata.ownerReferences[].apiVersion = opengauss.sig/v1`, then the corresponding K8s platform should be OpenGauss.

In these scenarios, the following operations are required:

- Enable and disable the corresponding resources in the Agent advanced configuration
- Configure Kubernetes API permissions

## OpenShift

In this scenario, the default `Ingress` resource acquisition needs to be disabled, and the `Route` resource acquisition needs to be enabled.

Agent advanced configuration is as follows:

```yaml
static_config:
kubernetes-resources:
- name: ingresses
disabled: true
- name: routes
```

ClusterRole configuration addition:

```yaml
rules:
- apiGroups:
- route.openshift.io
resources:
- routes
verbs:
- get
- list
- watch
```

## OpenKruise

In this scenario, the `CloneSet` and `apps.kruise.io/StatefulSet` resources need to be obtained from the API.

Agent advanced configuration is as follows:

```yaml
static_config:
kubernetes-resources:
- name: clonesets
group: apps.kruise.io
- name: statefulsets
group: apps
- name: statefulsets
group: apps.kruise.io
```

Note that Kubernetes's `apps/StatefulSet` needs to be added here.

ClusterRole configuration addition:

```yaml
- apiGroups:
- apps.kruise.io
resources:
- clonesets
- statefulsets
verbs:
- get
- list
- watch
```

## OpenGauss

In this scenario, the `OpenGaussCluster` resource needs to be obtained from the API.

Agent advanced configuration is as follows:

```yaml
static_config:
kubernetes-resources:
- name: opengaussclusters
```

ClusterRole configuration addition:

```yaml
- apiGroups:
- opengauss.sig
resources:
- opengaussclusters
verbs:
- get
- list
- watch
```

# Other K8s CRD
## About Lua Plugin on the Server

Due to some users' Kubernetes environments possibly having special configurations or security requirements, the standardized way of extracting workload types and workload names may not work as expected. Alternatively, users might want to customize workload types and workload names based on their own logic. Therefore, DeepFlow supports users in extracting workload types and workload names by adding custom Lua plugins. The Lua plugin system enhances the flexibility and universality of K8s resource integration by calling Lua Functions at fixed points to obtain some user-defined workload types and names.

# Lua Plugin Writing Example
## Lua Plugin Writing Example

```lua
-- Fixed syntax to import the JSON parsing package
Expand Down Expand Up @@ -62,7 +160,7 @@ function GetWorkloadTypeAndName(metaJsonStr)
end
```

# Upload Plugin
## Upload Plugin

Lua plugins support runtime loading. After uploading the plugin using the deepflow-ctl tool in the DeepFlow runtime environment, it will be automatically loaded. Execute the following command in the environment:

Expand All @@ -83,7 +181,7 @@ You can delete a specific plugin by its name with the following command:
deepflow-ctl plugin delete <name>
```

# Example
## Example

For instance, if the metadata of a Pod in the current k8s environment is as follows:

Expand Down Expand Up @@ -139,4 +237,4 @@ function GetWorkloadTypeAndName(metaJsonStr)
end
```

After uploading this plugin, you can extract the corresponding workload type and workload name for this Pod.
After uploading this plugin, you can extract the corresponding workload type and workload name for this Pod.
2 changes: 1 addition & 1 deletion translate/translated/10-release-notes/03-ce-6.6-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ permalink: /release-notes/release-6.6-ce
- When the TraceID is present in the protocol header, support disabling eBPF syscall_trace_id calculation (via configuration `syscall_trace_id_disabled`) to reduce the impact on business performance.
- Automatically correct minor clock deviations between different machines in the distributed tracing flame graph.
- AutoTagging
- Support using Lua Plugin to customize K8s workload abstraction rules, [documentation](../integration/process/lua-plugin/).
- Support using Lua Plugin to customize K8s workload abstraction rules, [documentation](../features/auto-tagging/k8s-crd/).
- Agent
- Support completely disabling cBPF data collection (via configuration `tap_interface_regex` as an empty string) to reduce memory overhead.
- Support deepflow-agent to use a single socket to transmit all observability data, and this feature can be disabled via the `multiple_sockets_to_ingester` configuration item to use multiple sockets to improve transmission performance.
Expand Down
Loading