Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 27 additions & 17 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ on:
inputs:
ref:
description: "Why trigger?"
required: true
required: true
type: string

env:
IMAGE: "docs"

Expand All @@ -27,9 +27,9 @@ jobs:

strategy:
matrix:
node-version:
- 12.x
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
node-version:
- 12.x
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
- name: Checkout
Expand Down Expand Up @@ -57,7 +57,7 @@ jobs:
key: docs-${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
docs-${{ runner.os }}-pnpm-store-

- name: Get current branch
id: get_branch
run: |
Expand All @@ -69,19 +69,29 @@ jobs:

- name: Node build
shell: bash
env:
NODE_OPTIONS: "--max-old-space-size=4096"
run: |
# 清理缓存以释放内存
pnpm store prune || true
# 安装依赖
pnpm install gray-matter json2yaml
node addTime.js
node addTime.js ./translate/translated
node copy-file.js ./docs/zh ./translate/translated false
# 显示内存使用情况
free -h
# 执行构建步骤
node --max-old-space-size=4096 addTime.js
node --max-old-space-size=4096 addTime.js ./translate/translated
node --max-old-space-size=4096 copy-file.js ./docs/zh ./translate/translated false
rm -rf translate/translated/README.md
mv translate/translated/* ./docs
rm -rf ./docs/map.txt
pnpm install
node downloadFile.js ${{ env.BRANCH }}
node downloadCSV.js ${{ env.BRANCH }}
node --max-old-space-size=4096 downloadFile.js ${{ env.BRANCH }}
node --max-old-space-size=4096 downloadCSV.js ${{ env.BRANCH }}
# 清理内存
node --max-old-space-size=4096 -e "if (global.gc) global.gc()"
pnpm run build
node copy-file.js
node --max-old-space-size=4096 copy-file.js

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
Expand All @@ -91,14 +101,14 @@ jobs:
uses: docker/login-action@v2
with:
registry: "${{ secrets.REGISTRY_ALIYUN_ADDR }}"
username: "${{ secrets.REGISTRY_ALIYUN_USER }}"
username: "${{ secrets.REGISTRY_ALIYUN_USER }}"
password: "${{ secrets.ALIYUN_DF_CLOUD_REGISTRY_PASSWORD }}"

# - name: debug
# shell: bash
# run: |
# more $GITHUB_EVENT_PATH

- name: set env
if: ${{ github.event_name == 'pull_request' && github.repository == 'deepflowio/docs' && github.event.pull_request.base.ref == 'main' }}
run: |
Expand All @@ -108,7 +118,7 @@ jobs:
if: ${{ github.event_name == 'push' && github.repository == 'deepflowio/docs'}}
run: |
echo "IMAGE_TAG_PREFIX=${{ github.ref_name }}"|sed 's|main|latest|' >> $GITHUB_ENV

- name: set env
if: ${{ github.event_name == 'workflow_dispatch' && github.repository == 'deepflowio/docs'}}
run: |
Expand Down Expand Up @@ -137,7 +147,7 @@ jobs:
shell: bash
run: |
echo "TEMPLATE=green" >> $GITHUB_ENV

- name: info
shell: bash
if: ${{ github.repository == 'deepflowio/docs'}}
Expand Down Expand Up @@ -178,7 +188,7 @@ jobs:
if: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.ref == 'main' && github.repository == 'deepflowio/docs' && github.event.pull_request.merged == true }}
shell: bash
run: |
kubectl set image -n deepservice-test deployment deepservice-docs deepservice-docs=${{ secrets.REGISTRY_ALIYUN_ADDR }}/deepservice/${{ env.IMAGE }}:${{ env.IMAGE_TAG_PREFIX }}-${{ github.run_id }}-${{ github.run_attempt }}
kubectl set image -n deepservice-test deployment deepservice-docs deepservice-docs=${{ secrets.REGISTRY_ALIYUN_ADDR }}/deepservice/${{ env.IMAGE }}:${{ env.IMAGE_TAG_PREFIX }}-${{ github.run_id }}-${{ github.run_attempt }}

- name: update
if: ${{ github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/main' && github.repository == 'deepflowio/docs' }}
Expand Down
84 changes: 42 additions & 42 deletions docs/zh/01-about/06-users.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/zh/02-ce-install/01-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ permalink: /ce-install/overview

DeepFlow 中的 eBPF 能力(AutoTracing、AutoProfiling)对内核版本的要求如下:
| 体系架构 | 发行版 | 内核版本 | kprobe [1] | Golang uprobe | OpenSSL uprobe | perf |
| -------- | ------ | ------- | ---------- | ------------- | -------------- | ---- |
| -------- | ------ | ------- | ---------- | ------------- | -------------- | ---- |
| X86 | CentOS 7.9 | 3.10.0-940+ **[2]** | Y | Y **[3]** | Y **[3]** | Y |
| | RedHat 7.6 | 3.10.0-940+ **[2]** | Y | Y **[3]** | Y **[3]** | Y |
| | \* | 4.14 **[4]** | Y | Y **[3]** | | Y |
Expand Down
3 changes: 2 additions & 1 deletion docs/zh/02-ce-install/02-all-in-one.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Grafana auth: admin:deepflow

我们不推荐使用 Docker 部署 DeepFlow Server 端,具体原因如下:

1. Server 端依赖 K8s 的 [lease](https://kubernetes.io/zh-cn/docs/concepts/architecture/leases/) 进行选主,通过多副本实现高可用性。而 Docker 环境缺乏 K8s 的这一机制,导致 Server 端仅能以单副本模式运行。在 Agent 节点数量较多或数据采集量较大的场景下,单副本实例可能因资源瓶颈而无法承载高并发的数据量。
1. Server 端依赖 K8s 的 [lease](https://kubernetes.io/zh-cn/docs/concepts/architecture/leases/) 进行选主,通过多副本实现高可用性。而 Docker 环境缺乏 K8s 的这一机制,导致 Server 端仅能以单副本模式运行。在 Agent 节点数量较多或数据采集量较大的场景下,单副本实例可能因资源瓶颈而无法承载高并发的数据量。
2. 当 Server 端以单副本形式部署时,配套的 ClickHouse 只能采用单切片形式部署,否则会导致数据写入不均衡,这在一定程度上限制了数据的查询速度。

## 准备工作
Expand Down Expand Up @@ -169,6 +169,7 @@ docker compose -f deepflow-docker-compose/docker-compose.yaml up -d
通过 docker compose 部署后,将浏览器指向 `http://<$NODE_IP_FOR_DEEPFLOW>:3000` 即可登录 Grafana 控制台

默认凭据:

- 用户名:admin
- 密码:deepflow

Expand Down
1 change: 1 addition & 0 deletions docs/zh/02-ce-install/08-serverless-pod.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ helm install deepflow-agent -n deepflow deepflow/deepflow-agent --version 6.6.01
```

上述命令将会部署两组 deepflow-agent:

- watcher:一个 deepflow-agent 的 deployment,用于同步 K8s 资源。
- 部署时会自动注入 `K8S_WATCH_POLICY=watch-only` 的环境变量,此时 deepflow-agent 仅会同步 K8s 资源,不会采集可观测性数据。
- daemonset:在每个 serverless pod 中以 sidecar 的方式注入 deepflow-agent,用于采集可观测性数据。
Expand Down
51 changes: 27 additions & 24 deletions docs/zh/02-ce-install/09-ai-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,21 @@ DeepFlow 部署时,默认不会启用 AI 组件,需手动在 `values-custom.
stella-agent-ce:
enabled: true
replicas: 1
hostNetwork: "false"
hostNetwork: 'false'
dnsPolicy: ClusterFirst
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
nameOverride: ''
fullnameOverride: ''
podAnnotations: {}

image:
repository: "{{ .Values.global.image.repository }}/deepflowio-stella-agent-ce"
repository: '{{ .Values.global.image.repository }}/deepflowio-stella-agent-ce'
pullPolicy: Always
# Overrides the image tag whose default is the chart appVersion.
tag: latest

podSecurityContext: {}
podSecurityContext:
{}
# fsGroup: 2000

securityContext:
Expand All @@ -40,19 +41,19 @@ stella-agent-ce:
## Configuration for ClickHouse service
annotations: {}
labels: {}
clusterIP: ""
clusterIP: ''

## Port for ClickHouse Service to listen on
ports:
- name: tcp
port: 20831
targetPort: 20831
nodePort:
protocol: TCP
- name: tcp
port: 20831
targetPort: 20831
nodePort:
protocol: TCP
# Additional ports to open for server service
additionalPorts: []
externalIPs: []
loadBalancerIP: ""
loadBalancerIP: ''
loadBalancerSourceRanges: []

## Denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints
Expand Down Expand Up @@ -81,18 +82,19 @@ stella-agent-ce:
df-llm-agent.yaml:
daemon: true
api_timeout: 500
sql_show: "false"
log_file: "/var/log/df-llm-agent.log"
log_level: "info"
instance_path: "/root/df-llm-agent"
sql_show: 'false'
log_file: '/var/log/df-llm-agent.log'
log_level: 'info'
instance_path: '/root/df-llm-agent'
mysql:
host: "{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.ip}}{{ else }}{{ $.Release.Name }}-mysql{{end}}"
port: "{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.port}}{{ else }}30130{{end}}"
user_name: "{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.username}}{{ else }}root{{end}}"
user_password: "{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.password}}{{ else }}{{ .Values.global.password.mysql }}{{end}}"
database: "deepflow_llm"

resources: {}
host: '{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.ip}}{{ else }}{{ $.Release.Name }}-mysql{{end}}'
port: '{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.port}}{{ else }}30130{{end}}'
user_name: '{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.username}}{{ else }}root{{end}}'
user_password: '{{ if $.Values.global.externalMySQL.enabled }}{{$.Values.global.externalMySQL.password}}{{ else }}{{ .Values.global.password.mysql }}{{end}}'
database: 'deepflow_llm'

resources:
{}
# limits:
# cpu: 100m
# memory: 128Mi
Expand All @@ -108,7 +110,8 @@ stella-agent-ce:
podAntiAffinityTermLabelSelector: []
podAffinityLabelSelector: []
podAffinityTermLabelSelector: []
nodeAffinityLabelSelector: []
nodeAffinityLabelSelector:
[]
# - matchExpressions:
# - key: kubernetes.io/hostname
# operator: In
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ K8s 使用 macvlan CNI 时,在 rootns 下只能看到所有 POD 共用的一
参考[文档](../configuration/agent/#inputs.cbpf.af_packet.inner_interface_capture_enabled),开启 deepflow-agent 的 `inputs.cbpf.af_packet.inner_interface_capture_enabled`,可采集 PodNS 中的网卡流量。

注意需要同时调整如下配置:

- `inputs.cbpf.af_packet.tunning.ring_blocks_enabled`:使得能够让 AF_PACKET 的内存消耗可调整。
- `inputs.cbpf.af_packet.tunning.ring_blocks`:使得能够精简所有 AF_PACKET 的总体内存消耗。
- `inputs.cbpf.af_packet.inner_interface_regex`:使得能够正确匹配 PodNS 内部的网卡名称。
Expand All @@ -138,7 +139,6 @@ K8s 使用 macvlan CNI 时,在 rootns 下只能看到所有 POD 共用的一

唯一需要注意的是,Agent 的 tap_interface_regex 只需配置为 Node NIC 列表。


# K8s 运行 Agent 权限受限

## 无 K8s Daemonset 部署权限
Expand Down
Loading
Loading