Prometheus和Openshift

  1. prometheus使用的配置文件在openshift-monitoring/secret/prometheus-k8s中。
    有添加ServiceMonitor时会动态更新该文件
  2. 为system:serviceaccount:openshift-monitoring:prometheus-k8s添加应用所在project的view权限
1
$ oc policy add-role-to-user view system:serviceaccount:openshift-monitoring:prometheus-k8s -n myproject
  1. 查看prometheus的配置，查看对于servicemonitor的过滤器
1
2
3
4
5
6
$ oc get prometheus k8s -o yaml
...
  serviceMonitorSelector:
    matchExpressions:
    - key: k8s-app
      operator: Exists
  • serviceMonitorSelector表示，只有设置了k8s-app的Label的serviceMonitor的状态才能被该prometheus监听到
  1. 添加servicemonitor 按照3的说明，serviceMonitor必须设置k8s-appLabel
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: my-cluster-kafka
  namespace: openshift-monitoring
  labels:
    k8s-app: prometheus
spec:
  selector:
    matchLabels:
      strimzi.io/name: my-cluster-kafka-bootstrap
  namespaceSelector:
    any: true
  endpoints:
  - port: metrics
  • selector：匹配的exporter的service的Label
  • namespaceSelector：匹配exporter所在的namespace。any为匹配所有的namespace
  • endpoints：对应为exporter服务port名

注意：经过实验serviceMonitor必须创建在prometheus相同的project下，默认为openshift-monitoring
另外如果OpenShift网络开启的是多租房/NetworkPolicy模式，需要确保prometheus访问exporter节点的网络通畅。

原理部分可参考：
官方手册 Prometheus Cluster Monitoring，包含对Openshift集群etcd的监控的详细操作
使用Prometheus Operator優雅的監控Kubernetes
基于 Prometheus 的集群监控