运维监控报表太麻烦？用 Grafana-Reporter 一键搞定！-51CTO.COM

引言

为什么要部署这样的一个组件呢？前几天的早上，我们这边提出了一个需求，能否把 Grafana 的 Dashboard 生成为 PDF 文件呢，然后再发送到相应的邮件或者聊天软件，这样做的好处是，比较好玩吧，新鲜，好看，哈哈，开玩笑的。

既然提出了这样的需求，那肯定是咱有这个实力的，哈哈，自卖自夸。但总之，这个需求还是挺有意思的。我喜欢挑战，喜欢新技术。

这边折腾了会，就发现了 Grafan-reporter 这样一款组件。

介绍

Github地址：https://github.com/IzakMarais/reporter，对中文不支持，dashboard 中含有中文字符会报错。

首先介绍一下Grafana Image Renderer，它是一个用于将Grafana仪表盘转换为静态图像的插件。它提供了在Grafana中生成PNG、JPEG和SVG格式图像的功能，这些图像可以用于导出仪表盘的快照、生成报告、在文档中嵌入等用途。通过Grafana Image Renderer插件，用户可以通过简单的HTTP请求将特定的Grafana仪表盘转换为静态图像。用户可以指定要转换的仪表盘、图像的宽度和高度等参数，并通过HTTP响应获取图像的二进制数据。该插件在处理图像时还支持渲染时过滤器的应用，例如应用时间范围、自定义变量、主题等，以确保生成的图像准确地显示所需的数据和样式。

Grafana Image Renderer插件可以与其他工具和系统集成，例如将生成的图像嵌入到报告生成工具、文档管理系统或自定义的应用程序中。这样，用户就能够方便地分享、保存和展示Grafana仪表盘的快照，而无需访问Grafana本身。

Grafana Reporter 是一个基于Grafana的插件，用于生成和导出报告。它允许用户将Grafana中的监控数据和仪表盘内容导出为报告，以便与团队成员、管理层或其他人共享。Grafana Reporter支持导出为PDF、PNG、SVG和CSV等格式，可以定制报告的样式、布局和内容。用户可以设定报告的时间范围、图表和指标，并在报告中包含注释、摘要和其他信息。Grafana Reporter使用户能够以可视化和易读的方式展示和分享他们的监控数据。　

部署

本环境采用 Kubernetes 安装，因为我喜欢用 Kubernetes，我看网上的文章几乎没有使用 Kubernetes 安装的。你也可以使用 Docker 或者 Docker-compose，又或者源码编译安装。

这边的监控采用的是 kube-prometheus-stack 的 Helm chart 安装，这边的重点不在安装 Prometheus，所以就直接跳过了。

因为有些信息比较敏感，这边的域名都做了修改。

部署 Grafana-reporter

这边使用 StatefulSet 部署，他这边需要指定下我们的 Grafana 的地址，还有我们的 Localtime 也需要修改为本地的。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: grafana-reporter
  namespace: monitoring
  labels:
    app: grafana-reporter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana-reporter
  serviceName: grafana-reporter
  template:
    metadata:
      labels:
        app: grafana-reporter
    spec:
      hostNetwork: true 
      containers:
        - name: grafana-reporter
          image: izakmarais/grafana-reporter
          imagePullPolicy: Always
          ports:
            - containerPort: 8686
          args:
            - "-proto=https://"
            - "-ip=grafana.example.com"
            - "-ssl-check=true" # 如果 Grafana 使用了自签名证书,我这边因为有证书，所以就true了
          volumeMounts:
            - name: localtime
              mountPath: /etc/localtime
              readOnly: true
      volumes:
        - name: localtime
          hostPath:
            path: /etc/localtime
            type: File

我们还需要创建一个 Service：

apiVersion: v1
kind: Service
metadata:
  name: grafana-reporter
  namespace: monitoring
  labels:
    app: grafana-reporter
spec:
  selector:
    app: grafana-reporter
  ports:
    - protocol: TCP
      port: 8686        
      targetPort: 8686  
  type: ClusterIP

再来一个 Ingress：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: id-test-reporter
  namespace: monitoring
spec:
  ingressClassName: nginx
  rules:
  - host: reporter.example.com
    http:
      paths:
      - backend:
          service:
            name: grafana-reporter
            port:
              number: 8686
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - reporter.example.com
    secretName: hiwin-test-ph-com-tls

Apply 所有；

kubectl apply -f .

我们来看下它的页面：

图片

这就算成功了。

部署 Grafana-image-render

这边使用 StatefulSet 控制器部署：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: grafana-image-renderer
  namespace: monitoring
  labels:
    app: grafana-image-renderer
spec:
  serviceName: grafana-image-renderer-headless
  replicas: 1
  selector:
    matchLabels:
      app: grafana-image-renderer
  template:
    metadata:
      labels:
        app: grafana-image-renderer
    spec:
      containers:
        - name: renderer
          image: grafana/grafana-image-renderer:latest
          ports:
            - containerPort: 8081
          env:
            - name: RENDERER_CALLBACK_URL
              value: "https://grafana.example.com" # Grafana 服务地址

service：

apiVersion: v1
kind: Service
metadata:
  name: grafana-image-renderer
  namespace: monitoring
spec:
  selector:
    app: grafana-image-renderer
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: grafana-image-renderer-headless
  namespace: monitoring
spec:
  selector:
    app: grafana-image-renderer
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
  clusterIP: None

Ingress：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-image-renderer-ingress
  namespace: monitoring
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "50m" # 调整请求体大小限制
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
    nginx.ingress.kubernetes.io/limit-connections: "50"

spec:
  ingressClassName: nginx
  rules:
    - host: render.example.com # 替换为您自己的域名
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: grafana-image-renderer-headless   # 这边使用的是 Headless 服务
                port:
                  number: 8081
  tls:
    - hosts:
        - render.example.com
      secretName: example-com-tls

Apply 部署：

kubectl apply -f .

测试一些是否有问题：

图片

可以看到，没有问题。

在一开始的部署中，我是不知道还需要部署 Grafana-image-render，所以当时因为这个，一直不能生成 PDF 文件。行了，往前看。

配置 Grafana

接下来我们配置下 Grafana，我们需要为 Grafana 添加下变量：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app.kubernetes.io/instance: kube-prometheus-stack
    app.kubernetes.io/name: grafana
    app.kubernetes.io/version: 11.3.1
    helm.sh/chart: grafana-8.6.3
  name: kube-prometheus-stack-grafana
  namespace: monitoring
spec:
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Retain
    whenScaled: Retain
  podManagementPolicy: OrderedReady
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: kube-prometheus-stack
      app.kubernetes.io/name: grafana
  serviceName: kube-prometheus-stack-grafana-headless
  template:
    metadata:
      annotations:
        checksum/config: e1e314c7fdc64d2ecfb41c8c9c1d43e5dbf8d4fc87a11b2373bfbba95ff86a24
        checksum/dashboards-json-config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
        checksum/sc-dashboard-provider-config: ce3cfbda1358535f70c551752a34e1d92271c95272f1891dfef8d65a403063e4
        checksum/secret: f89c841c5ea6371a733417993185fefcf0760914325c94b37519ea2e919222a5
        kubectl.kubernetes.io/default-container: grafana
      labels:
        app.kubernetes.io/instance: kube-prometheus-stack
        app.kubernetes.io/name: grafana
        app.kubernetes.io/version: 11.3.1
        helm.sh/chart: grafana-8.6.3
    spec:
      automountServiceAccountToken: true
      containers:
      - env:
        - name: METHOD
          value: WATCH
        - name: LABEL
          value: grafana_dashboard
        - name: LABEL_VALUE
          value: "1"
        - name: FOLDER
          value: /tmp/dashboards
        - name: RESOURCE
          value: both
        - name: NAMESPACE
          value: ALL
        - name: REQ_USERNAME
          valueFrom:
            secretKeyRef:
              key: admin-user
              name: kube-prometheus-stack-grafana
        - name: REQ_PASSWORD
          valueFrom:
            secretKeyRef:
              key: admin-password
              name: kube-prometheus-stack-grafana
        - name: REQ_URL
          value: http://localhost:3000/api/admin/provisioning/dashboards/reload
        - name: REQ_METHOD
          value: POST
        image: quay.io/kiwigrid/k8s-sidecar:1.28.0
        imagePullPolicy: IfNotPresent
        name: grafana-sc-dashboard
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /tmp/dashboards
          name: sc-dashboard-volume
          
      - env:
        - name: METHOD
          value: WATCH
        - name: LABEL
          value: grafana_datasource
        - name: LABEL_VALUE
          value: "1"
        - name: FOLDER
          value: /etc/grafana/provisioning/datasources
        - name: RESOURCE
          value: both
        - name: REQ_USERNAME
          valueFrom:
            secretKeyRef:
              key: admin-user
              name: kube-prometheus-stack-grafana
        - name: REQ_PASSWORD
          valueFrom:
            secretKeyRef:
              key: admin-password
              name: kube-prometheus-stack-grafana
        - name: REQ_URL
          value: http://localhost:3000/api/admin/provisioning/datasources/reload
        - name: REQ_METHOD
          value: POST
        image: quay.io/kiwigrid/k8s-sidecar:1.28.0
        imagePullPolicy: IfNotPresent
        name: grafana-sc-datasources
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /etc/grafana/provisioning/datasources
          name: sc-datasources-volume
          
      - env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: GF_SECURITY_ADMIN_USER
          valueFrom:
            secretKeyRef:
              key: admin-user
              name: kube-prometheus-stack-grafana
        - name: GF_SECURITY_ADMIN_PASSWORD
          valueFrom:
            secretKeyRef:
              key: admin-password
              name: kube-prometheus-stack-grafana
        - name: GF_PATHS_DATA
          value: /var/lib/grafana/
        - name: GF_PATHS_LOGS
          value: /var/log/grafana
        - name: GF_PATHS_PLUGINS
          value: /var/lib/grafana/plugins
        - name: GF_PATHS_PROVISIONING
          value: /etc/grafana/provisioning
        - name: GF_RENDERING_SERVER_URL      # Grafana 将向该地址发送渲染请求，并由 Image Renderer 服务生成图片（例如 PNG、JPEG）
          value: https://render.example.com
        - name: GF_RENDERING_CALLBACK_URL     # 当 Image Renderer 完成渲染任务后，它会将生成的图片或渲染结果回传给 Grafana
          value: https://grafana.example.com
        - name: GF_LOG_FILTERS
          value: rendering:debug
        image: docker.io/grafana/grafana:11.3.1
        imagePullPolicy: IfNotPresent
        livenessProbe:
          httpGet:
            path: /api/health
            port: 3000
            scheme: HTTP
          initialDelaySeconds: 60
          periodSeconds: 10
          timeoutSeconds: 30
        name: grafana
        ports:
        - containerPort: 3000
          name: grafana
        - containerPort: 9094
          name: gossip-tcp
        - containerPort: 9094
          name: gossip-udp
        readinessProbe:
          httpGet:
            path: /api/health
            port: 3000
            scheme: HTTP
          periodSeconds: 10
          timeoutSeconds: 1
        volumeMounts:
        - mountPath: /etc/grafana/grafana.ini
          name: config
          subPath: grafana.ini
        - mountPath: /var/lib/grafana
          name: storage
        - mountPath: /tmp/dashboards
          name: sc-dashboard-volume
        - mountPath: /etc/grafana/provisioning/dashboards/sc-dashboardproviders.yaml
          name: sc-dashboard-provider
          subPath: provider.yaml
        - mountPath: /etc/grafana/provisioning/datasources
          name: sc-datasources-volume
      initContainers:
      - command:
        - chown
        - -R
        - 472:472
        - /var/lib/grafana
        image: docker.io/library/busybox:1.31.1
        imagePullPolicy: IfNotPresent
        name: init-chown-data
        securityContext:
          capabilities:
            add:
            - CHOWN
          runAsUser: 0
        volumeMounts:
        - mountPath: /var/lib/grafana
          name: storage
      restartPolicy: Always
      securityContext:
        fsGroup: 472
        runAsUser: 472
      serviceAccountName: kube-prometheus-stack-grafana
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          name: kube-prometheus-stack-grafana
        name: config
      - emptyDir: {}
        name: sc-dashboard-volume
      - configMap:
          name: kube-prometheus-stack-grafana-config-dashboards
        name: sc-dashboard-provider
      - emptyDir: {}
        name: sc-datasources-volume
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: storage
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
      storageClassName: alicloud-nas-subpath

等待更新完成后，可以测试一些是否有问题。

测试

如何测试呢，不要着急，跟紧我的步骤。

首先我们需要先在 Grafana 中创建一个 SA (serviceaccount)，用于授权和访问。

图片

这边起个名字，再选择一个 Admin 的 Role ：

图片

创建完成之后，再创建一个 Token，保存好：

图片

接下来我们还需要获取到 Grafana Dashboard 的 ID，这边选择一个你想要生成 PDF 文件的 Dashboard：

然后进入，注意如何选择：

https://grafana.example.com/d/efa86fd1d0c121a26444b636a3f509a8/kubernetes-compute-resources-cluster?orgId=1&from=now-1h&to=now&timeznotallow=utc&var-datasource=default&var-cluster=&refresh=10s

域名后面的 d/ 到下一个 / 之间的就是我们需要的 ID

图片

接下来我们开始真正的测试：

完成格式为：

https://reporter.example.com/api/v5/report/alertmanager-overview?apitoken=glsa_M4a256mzUizdmdGsSNK8SZW7h9QfvhGr_6616e37c

我们来解析下：

https://reporter.example.com/

• 这个是你的 Grafana-reporter 的域名

api/v5/report/

• 这个是固定的格式，不能改

alertmanager-overview

• 这个也就是你的 Grafana Dashboard 的 ID

?apitoken=glsa_M4a256mzUizdmdGsSNK8SZW7h9QfvhGr_6616e37c

• 这个就是你的 Grafana SA 生成的 Token

接下来我们访问下，看下是否有问题。

可以看到失败了，为什么呢？我们来排查下：

提前说明下，我这边 kubectl 的命令都配置了 Alias。如果你也需要，请查看这篇文章。

进入容器

$ kgpo -nmonitoring
NAME                                                        READY   STATUS    RESTARTS   AGE
alertmanager-kube-prometheus-stack-alertmanager-0           2/2     Running   0          19d
grafana-image-renderer-0                                    1/1     Running   0          156m
grafana-image-renderer-1                                    1/1     Running   0          156m
grafana-image-renderer-2                                    1/1     Running   0          156m
grafana-reporter-0                                          1/1     Running   0          92m
kube-prometheus-stack-grafana-0                             3/3     Running   0          34m
kube-prometheus-stack-kube-state-metrics-5b58cf95c8-blv79   1/1     Running   0          19d
kube-prometheus-stack-operator-b48974f48-m59d2              1/1     Running   0          19d
prometheus-kube-prometheus-stack-prometheus-0               2/2     Running   0          19d

$ kex grafana-reporter-0 -nmonitoring -- bash
root@grafana-reporter-0:/# cd /tmp/   # 这个目录是它默认存放生成的 PDF 文件的位置
root@grafana-reporter-0:/tmp# ls
e4b21f3b-b031-4fa2-9443-a37f1095261e
root@grafana-reporter-0:/tmp# cd e4b21f3b-b031-4fa2-9443-a37f1095261e/
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e# ls
images  report.aux  report.log  report.tex
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e# cd images/
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# ls
image2.png  image3.png  image5.png  image6.png

我们可以使用 file 这个命令查看一些这个文件的类型

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# file image2.png 
bash: file: command not found

那就安装下这个命令；

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt update
Ign:1 http://deb.debian.org/debian stretch InRelease
Ign:2 http://security.debian.org/debian-security stretch/updates InRelease
Ign:3 http://deb.debian.org/debian stretch-updates InRelease
Err:4 http://security.debian.org/debian-security stretch/updates Release
  404  Not Found [IP: 151.101.2.132 80]
Err:5 http://deb.debian.org/debian stretch Release
  404  Not Found [IP: 199.232.46.132 80]
Err:6 http://deb.debian.org/debian stretch-updates Release
  404  Not Found [IP: 199.232.46.132 80]
Reading package lists... Done
E: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

报错了，那就修改下。

问题是由于容器中的 apt 使用了一个过期的 Debian 软件源 (stretch)，这些源已经不再维护或者被替换，因此无法正确获取包列表

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian stretch main contrib non-free" > /etc/apt/sources.list
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian-security stretch/updates main contrib non-free" >> /etc/apt/sources.list

# 安装
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt update
Ign:1 http://deb.debian.org/debian stretch InRelease
Ign:2 http://security.debian.org/debian-security stretch/updates InRelease
Ign:3 http://deb.debian.org/debian stretch-updates InRelease
Err:4 http://security.debian.org/debian-security stretch/updates Release
  404  Not Found [IP: 151.101.2.132 80]
Err:5 http://deb.debian.org/debian stretch Release
  404  Not Found [IP: 199.232.46.132 80]
Err:6 http://deb.debian.org/debian stretch-updates Release
  404  Not Found [IP: 199.232.46.132 80]
Reading package lists... Done
E: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian stretch main contrib non-free" > /etc/apt/sources.list
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian-security stretch/updates main contrib non-free" >> /etc/apt/sources.list
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt update
Ign:1 http://archive.debian.org/debian stretch InRelease
Get:2 http://archive.debian.org/debian-security stretch/updates InRelease [59.1 kB]
Get:3 http://archive.debian.org/debian stretch Release [118 kB]
Get:4 http://archive.debian.org/debian stretch Release.gpg [3177 B]
Get:5 http://archive.debian.org/debian-security stretch/updates/main amd64 Packages [782 kB]
Get:6 http://archive.debian.org/debian-security stretch/updates/non-free amd64 Packages [14.1 kB]
Get:7 http://archive.debian.org/debian-security stretch/updates/contrib amd64 Packages [1760 B]
Get:8 http://archive.debian.org/debian stretch/main amd64 Packages [7080 kB]
Get:9 http://archive.debian.org/debian stretch/contrib amd64 Packages [50.7 kB]
Get:10 http://archive.debian.org/debian stretch/non-free amd64 Packages [78.3 kB]
Fetched 8186 kB in 1s (6008 kB/s)                         
Reading package lists... Done
Building dependency tree       
Reading state information... Done
31 packages can be upgraded. Run 'apt list --upgradable' to see them.

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt install -y file
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libmagic-mgc libmagic1
The following NEW packages will be installed:
  file libmagic-mgc libmagic1
0 upgraded, 3 newly installed, 0 to remove and 31 not upgraded.
Need to get 397 kB of archives.
After this operation, 5265 kB of additional disk space will be used.
Get:1 http://archive.debian.org/debian stretch/main amd64 libmagic-mgc amd64 1:5.30-1+deb9u3 [222 kB]
Get:2 http://archive.debian.org/debian stretch/main amd64 libmagic1 amd64 1:5.30-1+deb9u3 [111 kB]
Get:3 http://archive.debian.org/debian stretch/main amd64 file amd64 1:5.30-1+deb9u3 [64.2 kB]
Fetched 397 kB in 0s (4579 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libmagic-mgc.
(Reading database ... 6831 files and directories currently installed.)
Preparing to unpack .../libmagic-mgc_1%3a5.30-1+deb9u3_amd64.deb ...
Unpacking libmagic-mgc (1:5.30-1+deb9u3) ...
Selecting previously unselected package libmagic1:amd64.
Preparing to unpack .../libmagic1_1%3a5.30-1+deb9u3_amd64.deb ...
Unpacking libmagic1:amd64 (1:5.30-1+deb9u3) ...
Selecting previously unselected package file.
Preparing to unpack .../file_1%3a5.30-1+deb9u3_amd64.deb ...
Unpacking file (1:5.30-1+deb9u3) ...
Setting up libmagic-mgc (1:5.30-1+deb9u3) ...
Setting up libmagic1:amd64 (1:5.30-1+deb9u3) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Setting up file (1:5.30-1+deb9u3) ...

# 看一下这个文件
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# ls
image2.png  image3.png  image5.png  image6.png
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# file image2.png 
image2.png: ASCII text, with no line terminators

可以看到，这是文本，不是 png 图片，所以问题是 grafana 没有成功调用到 Grafana-image-render，去生成相应的 PNG 图片。

我们这边再去看下我们的 Grafana 的 logs

# 截取一部分 logs，看起来没有什么特别大的问题：
logger=rendering renderer=http t=2024-12-26T10:41:49.307925807Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=lFiW6KBBhrA8vY2BQ9OOXdPSZUU2FL5b&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D2%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=context userId=1 orgId=1 uname=admin t=2024-12-26T10:41:53.196481964Z level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=127.0.0.1 time_ms=5 duratinotallow=5.670954ms size=0 referer= handler=/api/live/ws status_source=server
logger=live t=2024-12-26T10:41:53.389851883Z level=info msg="Initialized channel handler" channel=grafana/dashboard/uid/efa86fd1d0c121a26444b636a3f509a8 address=grafana/dashboard/uid/efa86fd1d0c121a26444b636a3f509a8
logger=rendering renderer=http t=2024-12-26T10:42:18.992113007Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=6&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:18.995074149Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=2&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:19.010142812Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=ElH0jx7AYodbbkv2lEzvV7Qf1OkeiRac&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D6%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=rendering renderer=http t=2024-12-26T10:42:19.011440398Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=5&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:19.024147345Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=FGMPcxnruw1ZbPXEeH4v0BBrEeUSKvjR&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D5%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=rendering renderer=http t=2024-12-26T10:42:19.031615719Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=3&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:19.044274424Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=a47WWfW9PiO75DoNSHKwfdMwPSeTIiNo&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D3%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=rendering renderer=http t=2024-12-26T10:42:19.105601469Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=KnO2tIXJg9EvhcbhfDjbadYxAqChU0eN&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D2%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"

我们来排查下，看一下我们的 Grafana 的 YAML 文件里面的 image-render 的地址，其它多余的就不看了，问题重要在这里，去 Google 了一会，发现我可能少一个路径 /render，这个是默认的，那我们就加上去，看下它是否 OK：

k edit sts kube-prometheus-stack-grafana -nmonitoring
····
        - name: GF_RENDERING_SERVER_URL
          value: https://render.example.com/render
        - name: GF_RENDERING_CALLBACK_URL 
          value: https://grafana.example.com
····

访问下：

图片

可以看到，我们成功了，如果你这边还有问题，可以查看下 Grafana 和 Grafana-reporter 的 logs。