diff --git a/modules/administration-guide/examples/snip_che-create-a-role-and-rolebinding-for-prometheus-to-view-metrics.adoc b/modules/administration-guide/examples/snip_che-create-a-role-and-rolebinding-for-prometheus-to-view-metrics.adoc deleted file mode 100644 index 48f6543aa3..0000000000 --- a/modules/administration-guide/examples/snip_che-create-a-role-and-rolebinding-for-prometheus-to-view-metrics.adoc +++ /dev/null @@ -1,46 +0,0 @@ -. Create a Role and RoleBinding to allow Prometheus to view the metrics. - -+ -.Role -==== -[source,yaml,subs="+quotes,+attributes,+macros"] ----- -kind: Role -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: prometheus-k8s - namespace: openshift-operators -rules: - - verbs: - - get - - list - - watch - apiGroups: - - '' - resources: - - services - - endpoints - - pods ----- -==== - -+ -.RoleBinding -==== -[source,yaml,subs="+quotes,+attributes,+macros"] ----- -kind: RoleBinding -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: view-che-openshift-monitoring-prometheus-k8s - namespace: openshift-operators -subjects: - - kind: ServiceAccount - name: prometheus-k8s - namespace: openshift-monitoring -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: prometheus-k8s ----- -==== diff --git a/modules/administration-guide/pages/monitoring-che.adoc b/modules/administration-guide/pages/monitoring-che.adoc index bb2e75effb..1c702e7d88 100644 --- a/modules/administration-guide/pages/monitoring-che.adoc +++ b/modules/administration-guide/pages/monitoring-che.adoc @@ -11,6 +11,6 @@ You can configure {prod-short} to expose JVM metrics such as JVM memory and clas include::partial$proc_enabling-and-exposing-che-metrics.adoc[leveloffset=+1] -include::partial$proc_collecting-che-metrics-with-prometheus.adoc[leveloffset=+1] +include::partial$con_collecting-che-metrics-with-prometheus.adoc[leveloffset=+1] include::partial$proc_viewing-che-metrics-on-grafana-dashboards.adoc[leveloffset=+1] diff --git a/modules/administration-guide/pages/monitoring-the-dev-workspace-operator.adoc b/modules/administration-guide/pages/monitoring-the-dev-workspace-operator.adoc index ca14505223..8a57469f90 100644 --- a/modules/administration-guide/pages/monitoring-the-dev-workspace-operator.adoc +++ b/modules/administration-guide/pages/monitoring-the-dev-workspace-operator.adoc @@ -10,7 +10,7 @@ You can configure the OpenShift in-cluster monitoring stack to scrape metrics exposed by the {devworkspace} Operator. -include::partial$proc_collecting-dev-workspace-operator-metrics-with-prometheus.adoc[leveloffset=+1] +include::partial$con_collecting-dev-workspace-operator-metrics-with-prometheus.adoc[leveloffset=+1] include::partial$ref_devworkspace-specific-metrics.adoc[leveloffset=+1] diff --git a/modules/administration-guide/partials/con_collecting-che-metrics-with-prometheus.adoc b/modules/administration-guide/partials/con_collecting-che-metrics-with-prometheus.adoc new file mode 100644 index 0000000000..2293ac1e95 --- /dev/null +++ b/modules/administration-guide/partials/con_collecting-che-metrics-with-prometheus.adoc @@ -0,0 +1,44 @@ +// monitoring-{prod-id-short} +:page-aliases: .:proc_collecting-che-metrics-with-prometheus.adoc + +[id="collecting-{prod-id-short}-metrics-with-prometheus"] += Verifying {prod-short} Server metrics collection with Prometheus + +[role="_abstract"] +The {prod-operator} automatically creates and reconciles the required Prometheus resources (ServiceMonitor, Role, and RoleBinding) and configures {orch-namespace} labeling for monitoring {prod-short} Server JVM metrics. +No manual configuration is required. + +.Verification + +. In the *Administrator* view of the OpenShift web console, go to *Observe* -> *Metrics*. + +. Run a PromQL query to confirm that the metrics are available. For example, enter `process_uptime_seconds{job="che-host"}` and click *Run queries*. + +[TIP] +==== + +To troubleshoot missing metrics, view the Prometheus container logs for possible RBAC-related errors: + +. Get the name of the Prometheus pod: ++ +[source,terminal,subs="+quotes"] +---- +$ oc get pods -l app.kubernetes.io/name=prometheus -n openshift-monitoring -o=jsonpath='{.items[*].metadata.name}' +---- + +. Print the last 20 lines of the Prometheus container logs from the Prometheus pod from the previous step: ++ +[source,terminal,subs="+quotes"] +---- +$ oc logs --tail=20 ____ -c prometheus -n openshift-monitoring +---- + +==== + +[role="_additional-resources"] +.Additional resources + + +* link:https://prometheus.io/docs/prometheus/latest/querying/basics/[Querying Prometheus] + +* link:https://prometheus.io/docs/concepts/metric_types/[Prometheus metric types] diff --git a/modules/administration-guide/partials/con_collecting-dev-workspace-operator-metrics-with-prometheus.adoc b/modules/administration-guide/partials/con_collecting-dev-workspace-operator-metrics-with-prometheus.adoc new file mode 100644 index 0000000000..d7d4e3080f --- /dev/null +++ b/modules/administration-guide/partials/con_collecting-dev-workspace-operator-metrics-with-prometheus.adoc @@ -0,0 +1,43 @@ +:page-aliases: .:proc_collecting-dev-workspace-operator-metrics-with-prometheus.adoc + +[id="collecting-dev-workspace-operator-metrics-with-prometheus"] += Verifying {devworkspace} Operator metrics collection with Prometheus + +[role="_abstract"] +The {prod-operator} automatically creates and reconciles the required Prometheus resources (ServiceMonitor, Role, and RoleBinding) and configures {orch-namespace} labeling for monitoring the {devworkspace} Operator. +No manual configuration is required. + +.Verification + +. For a fresh installation of {prod-short}, generate metrics by creating a {prod-short} workspace from the Dashboard. + +. In the *Administrator* view of the OpenShift web console, go to *Observe* -> *Metrics*. + +. Run a PromQL query to confirm that the metrics are available. For example, enter `devworkspace_started_total` and click *Run queries*. ++ +For more metrics, see xref:ref_devworkspace-specific-metrics[]. + +[TIP] +==== + +To troubleshoot missing metrics, view the Prometheus container logs for possible RBAC-related errors: + +. Get the name of the Prometheus pod: ++ +[source,terminal,subs="+quotes"] +---- +$ oc get pods -l app.kubernetes.io/name=prometheus -n openshift-monitoring -o=jsonpath='{.items[*].metadata.name}' +---- + +. Print the last 20 lines of the Prometheus container logs from the Prometheus pod from the previous step: ++ +[source,terminal,subs="+quotes"] +---- +$ oc logs --tail=20 ____ -c prometheus -n openshift-monitoring +---- + +==== + +.Additional resources +* link:https://prometheus.io/docs/prometheus/latest/querying/basics/[Querying Prometheus] +* link:https://prometheus.io/docs/concepts/metric_types/[Prometheus metric types] diff --git a/modules/administration-guide/partials/proc_collecting-che-metrics-with-prometheus.adoc b/modules/administration-guide/partials/proc_collecting-che-metrics-with-prometheus.adoc deleted file mode 100644 index b86014256a..0000000000 --- a/modules/administration-guide/partials/proc_collecting-che-metrics-with-prometheus.adoc +++ /dev/null @@ -1,134 +0,0 @@ -// monitoring-{prod-id-short} - -[id="collecting-{prod-id-short}-metrics-with-prometheus"] -= Collecting {prod-short} Server metrics with Prometheus - -To use the in-cluster Prometheus instance to collect, store, and query JVM metrics for {prod-short} Server: - -.Prerequisites - -* Your organization's instance of {prod-short} is installed and running in Red Hat OpenShift. - -* An active `oc` session with administrative permissions to the destination OpenShift cluster. See link:https://docs.openshift.com/container-platform/{ocp4-ver}/cli_reference/openshift_cli/getting-started-cli.html[Getting started with the CLI]. - -* {prod-short} is exposing metrics on port `8087`. See xref:enabling-and-exposing-{prod-id-short}-metrics[Enabling and exposing {prod-short} server JVM metrics]. - -.Procedure - -. Create the ServiceMonitor for detecting the {prod-short} JVM metrics Service. -+ -.ServiceMonitor -==== -[source,yaml,subs="+quotes,+attributes,+macros"] ----- -apiVersion: monitoring.coreos.com/v1 -kind: ServiceMonitor -metadata: - name: che-host - namespace: {prod-namespace} <1> -spec: - endpoints: - - interval: 10s <2> - port: metrics - scheme: http - namespaceSelector: - matchNames: - - {prod-namespace} <1> - selector: - matchLabels: - app.kubernetes.io/name: {prod-deployment} ----- -<1> The {prod-short} namespace. The default is `{prod-namespace}`. -<2> The rate at which a target is scraped. -==== - -. Create a Role and RoleBinding to allow Prometheus to view the metrics. - -+ -.Role -==== -[source,yaml,subs="+quotes,+attributes,+macros"] ----- -kind: Role -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: prometheus-k8s - namespace: {prod-namespace} <1> -rules: - - verbs: - - get - - list - - watch - apiGroups: - - '' - resources: - - services - - endpoints - - pods ----- -<1> The {prod-short} namespace. The default is `{prod-namespace}`. -==== - -+ -.RoleBinding -==== -[source,yaml,subs="+quotes,+attributes,+macros"] ----- -kind: RoleBinding -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: view-{prod-id-short}-openshift-monitoring-prometheus-k8s - namespace: {prod-namespace} <1> -subjects: - - kind: ServiceAccount - name: prometheus-k8s - namespace: openshift-monitoring -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: prometheus-k8s ----- -<1> The {prod-short} namespace. The default is `{prod-namespace}`. -==== - -. Allow the in-cluster Prometheus instance to detect the ServiceMonitor in the {prod-short} namespace. The default {prod-short} namespace is `{prod-namespace}`. -+ -[source,terminal,subs="+attributes,quotes"] ----- -$ oc label namespace {prod-namespace} openshift.io/cluster-monitoring=true ----- - -.Verification - -. In the *Administrator* view of the OpenShift web console, go to *Observe* -> *Metrics*. - -. Run a PromQL query to confirm that the metrics are available. For example, enter `process_uptime_seconds{job="che-host"}` and click *Run queries*. - -[TIP] -==== - -To troubleshoot missing metrics, view the Prometheus container logs for possible RBAC-related errors: - -. Get the name of the Prometheus pod: -+ -[source,yaml,subs="+quotes"] ----- -$ oc get pods -l app.kubernetes.io/name=prometheus -n openshift-monitoring -o=jsonpath='{.items[*].metadata.name}' ----- - -. Print the last 20 lines of the Prometheus container logs from the Prometheus pod from the previous step: -+ -[source,yaml,subs="+quotes"] ----- -$ oc logs --tail=20 ____ -c prometheus -n openshift-monitoring ----- - -==== - -[role="_additional-resources"] -.Additional resources - - -* link:https://prometheus.io/docs/prometheus/latest/querying/basics/[Querying Prometheus] - -* link:https://prometheus.io/docs/concepts/metric_types/[Prometheus metric types] diff --git a/modules/administration-guide/partials/proc_collecting-dev-workspace-operator-metrics-with-prometheus.adoc b/modules/administration-guide/partials/proc_collecting-dev-workspace-operator-metrics-with-prometheus.adoc deleted file mode 100644 index b198f24bd1..0000000000 --- a/modules/administration-guide/partials/proc_collecting-dev-workspace-operator-metrics-with-prometheus.adoc +++ /dev/null @@ -1,90 +0,0 @@ -[id="proc_collecting-dev-workspace-operator-metrics-with-prometheus"] -= Collecting {devworkspace} Operator metrics - -[role="_abstract"] -To use the in-cluster Prometheus instance to collect, store, and query metrics about the {devworkspace} Operator: - -.Prerequisites - -* Your organization's instance of {prod-short} is installed and running in Red Hat OpenShift. - -* An active `oc` session with administrative permissions to the destination OpenShift cluster. See link:https://docs.openshift.com/container-platform/{ocp4-ver}/cli_reference/openshift_cli/getting-started-cli.html[Getting started with the CLI]. - -* The `devworkspace-controller-metrics` Service is exposing metrics on port `8443`. This is preconfigured by default. - -.Procedure - -. Create the ServiceMonitor for detecting the Dev Workspace Operator metrics Service. -+ -.ServiceMonitor -==== -[source,yaml,subs="+quotes,+attributes,+macros"] ----- -apiVersion: monitoring.coreos.com/v1 -kind: ServiceMonitor -metadata: - name: devworkspace-controller - namespace: {prod-namespace} <1> -spec: - endpoints: - - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token - interval: 10s <2> - port: metrics - scheme: https - tlsConfig: - insecureSkipVerify: true - namespaceSelector: - matchNames: - - openshift-operators - selector: - matchLabels: - app.kubernetes.io/name: devworkspace-controller ----- -<1> The {prod-short} namespace. The default is `{prod-namespace}`. -<2> The rate at which a target is scraped. -==== - -include::example$snip_{project-context}-create-a-role-and-rolebinding-for-prometheus-to-view-metrics.adoc[] - -. Allow the in-cluster Prometheus instance to detect the ServiceMonitor in the {prod-short} namespace. The default {prod-short} namespace is `{prod-namespace}`. -+ -[source,subs="+attributes"] ----- -$ oc label namespace {prod-namespace} openshift.io/cluster-monitoring=true ----- - -.Verification - -. For a fresh installation of {prod-short}, generate metrics by creating a {prod-short} workspace from the Dashboard. - -. In the *Administrator* view of the OpenShift web console, go to *Observe* -> *Metrics*. - -. Run a PromQL query to confirm that the metrics are available. For example, enter `devworkspace_started_total` and click *Run queries*. -+ -For more metrics, see xref:ref_devworkspace-specific-metrics[]. - -[TIP] -==== - -To troubleshoot missing metrics, view the Prometheus container logs for possible RBAC-related errors: - -. Get the name of the Prometheus pod: -+ -[source,yaml,subs="+quotes"] ----- -$ oc get pods -l app.kubernetes.io/name=prometheus -n openshift-monitoring -o=jsonpath='{.items[*].metadata.name}' ----- - -. Print the last 20 lines of the Prometheus container logs from the Prometheus pod from the previous step: -+ -[source,yaml,subs="+quotes"] ----- -$ oc logs --tail=20 ____ -c prometheus -n openshift-monitoring ----- - -==== - -.Additional resources -* link:https://prometheus.io/docs/prometheus/latest/querying/basics/[Querying Prometheus] -* link:https://prometheus.io/docs/concepts/metric_types/[Prometheus metric types] - diff --git a/modules/administration-guide/partials/proc_viewing-dev-workspace-operator-metrics-on-grafana-dashboards.adoc b/modules/administration-guide/partials/proc_viewing-dev-workspace-operator-metrics-on-grafana-dashboards.adoc index d6fa309b81..6486f5fe1e 100644 --- a/modules/administration-guide/partials/proc_viewing-dev-workspace-operator-metrics-on-grafana-dashboards.adoc +++ b/modules/administration-guide/partials/proc_viewing-dev-workspace-operator-metrics-on-grafana-dashboards.adoc @@ -9,7 +9,7 @@ After configuring the in-cluster Prometheus instance to collect {devworkspace} O * An active `oc` session with administrative permissions to the destination OpenShift cluster. See link:https://docs.openshift.com/container-platform/{ocp4-ver}/cli_reference/openshift_cli/getting-started-cli.html[Getting started with the CLI]. -* The in-cluster Prometheus instance is collecting metrics. See xref:proc_collecting-dev-workspace-operator-metrics-with-prometheus[]. +* The in-cluster Prometheus instance is collecting metrics. See xref:collecting-dev-workspace-operator-metrics-with-prometheus[]. .Procedure