一篇学会 Metadata 巧妙使用动态元数据-51CTO.COM

1什么是metadata

metadata就是元数据信息，他提供了基于匹配的 listeners, filter chains, routes and endpoints的额外的输入参数到过滤器，他是一种map的格式，通常是filter的名字(反向dns格式)。过滤器元数据的键值对在请求处理和连接发生时会别合并，后面的值会覆盖前面的值。元数据有一个名称空间的概念，然后是键值对。比如提供额外数据给httpConnectionManager的元数据名称空间 envoy.http_connection_manager.access_log 。另一个例子是每个service使用的cluster的元数据信息，他可能被多个过滤器使用。对于负载均衡来说，元数据提供了一种方法，来子集端点信息。关联元数据的endpoint，路由一个特定的元数据来选择端点。元数据有四种类型，分别是request类型，route类型，cluster类型，host类型。

2metadata有什么作用

tracing customtag值得来源，路由元数据匹配，负载均衡子集决策，ratelimit 动作配置，基于元数据的权限控制，本地响应映射元数据过滤，等。

3metadata数据来源

3.1 envoy.filters.http.ext_authz

当使用grpc授权服务器时，当CheckResponse包含dynamic_metadata字段时，会产生动态元数据信息。

当使用http授权服务器时，当来自授权服务器的响应头匹配 dynamic_metadata_from_headers 的配置值会产生元数据信息。动态元数据的key是匹配的头，动态元数据的值是匹配头的值。

3.2 envoy.filters.network.ext_authz

当使用grpc授权服务器时，当CheckResponse包含dynamic_metadata字段时，会产生动态元数据信息。

3.3 envoy.filters.http.header_to_metadata

配置一些规则，每条规则有header或cookie，当配置的值存在或不存在时就会触发规则，用来设置动态元数据。

比如：

http_filters: 
  - name: envoy.filters.http.header_to_metadata 
    typed_config: 
      "@type": type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config 
      request_rules: 
        - header: x-version 
          on_header_present: 
            metadata_namespace: envoy.lb 
            key: version 
            type: STRING 
          on_header_missing: 
            metadata_namespace: envoy.lb 
            key: default 
            value: 'true' 
            type: STRING 
          remove: false

上面规则显示，当x-version头存在时设置envoy.lb名称空间的key为version元数据为x-version的值，当x-version不存在时，设置envoy.lb的名称空间的key为default的元数据的值为true。

3.4 envoy.filters.http.jwt_authn

可以配置 **payload_in_**metadata ，成功验证jwt payload会写到metadata中，名称空间是envoy.filters.http.jwt_authn，例子：

envoy.filters.http.jwt_authn: 
  my_payload: 
    iss: https://example.com 
    sub: test@example.com 
    aud: https://example.com 
    exp: 1501281058

可以配置 **header_in_**metadata ，成功验证的头会写到metadata中，名称空间是envoy.filters.http.jwt_authn，例子：

envoy.filters.http.jwt_authn: 
  my_header: 
    alg: JWT 
    kid: EF71iSaosbC5C4tC6Syq1Gm647M 
    alg: PS256

3.5 envoy.filters.network.mongo_proxy

当 emit_dynamic_metadata 为true时会产生metadata，格式如下

Name	Type	Description
key	string	The resource name in db.collection format.
value	array	A list of strings representing the operations executed on the resource (insert/update/query/delete).

3.6envoy.filters.network.mysql_proxy

当发送到服务端的sql被解析后，会产生动态元数据信息，格式如下：

Name	Type	Description
<table.db>	string	The resource name in table.db format. The resource name defaults to the table being accessed if the database cannot be inferred.
[]	list	A list of strings representing the operations executed on the resource. Operations can be one of insert/update/select/drop/delete/create/alter/show.

3.7envoy.filters.network.postgres_proxy

语句被解析后，会产生动态元数据，格式如下：

Name	Type	Description
<table.db>	string	The resource name in table.db format.
[]	list	A list of strings representing the operations executed on the resource. Operations can be one of insert/update/select/drop/delete/create/alter/show.

3.8 envoy.filters.http.rbac

会产生如下元数据：

shadow_effective_policy_id	string	The effective shadow policy ID matching the action (if any).
shadow_engine_result	string	The engine result for the shadow rules (i.e. either `allowed` or `denied`).
access_log_hint	boolean	Whether the request should be logged. This metadata is shared and set under the key namespace ‘envoy.common’ (See Shared Dynamic Metadata).

3.9 envoy.filters.network.rbac

会产生如下元数据：

Name	Type	Description
shadow_effective_policy_id	string	The effective shadow policy ID matching the action (if any).
shadow_engine_result	string	The engine result for the shadow rules (i.e. either `allowed` or `denied`).
access_log_hint	boolean	Whether the request should be logged. This metadata is shared and set under the key namespace ‘envoy.common’ (See Shared Dynamic Metadata).

3.10envoy.filters.network.zookeeper_proxy

当每个消息被解析后，会产生如下元数据：

Name	Type	Description
	string	The path associated with the request, response or event
	string	The opname for the request, response or event
<create_type>	string	The string representation of the flags applied to the znode
	string	The size of the request message in bytes
	string	True if a watch is being set, false otherwise
	string	The version parameter, if any, given with the request
	string	The timeout parameter in a connect response
<protocol_version>	string	The protocol version in a connect response
	string	The readonly flag in a connect response
	string	The zxid field in a response header
	string	The error field in a response header
<client_state>	string	The state field in a watch event
<event_type>	string	The event type in a a watch event

3.11 envoy.filters.http.ratelimit

当ratelimit服务返回 RateLimitResponse 带有dynamic_metadata时，会产生元数据信息。

4metadata怎么使用

4.1type.tracing.v3.CustomTag

{ 
  "tag": "...", 
  "literal": "{...}", 
  "environment": "{...}", 
  "request_header": "{...}", 
  "metadata": "{...}" 
}

metadata：自定义tag，值从metadata中获取

案例：

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: httpconnectionmanager 
spec: 
  workloadSelector: 
    labels: 
      istio: ingressgateway 
  configPatches: 
  - applyTo: NETWORK_FILTER 
    match: 
      context: GATEWAY 
      listener: 
        portNumber: 8080 
        filterChain: 
          filter: 
            name: "envoy.filters.network.http_connection_manager" 
    patch: 
      operation: MERGE 
      value: 
              name: envoy.filters.network.http_connection_manager 
              typedConfig: 
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager 
                route_config: 
                  name: test 
                  virtual_hosts: 
                  - name: test 
                    domains: 
                    - "*" 
                    routes: 
                    - name: testroute 
                      match:  
                        prefix: / 
                      direct_response: 
                        status: 200 
                        body:  
                          inline_string: "prefix" 
                      tracing: 
                          customTags: 
                          - metadata: 
                              kind: 
                                request: {} 
                              metadataKey: 
                                key: envoy.filters.http.rbac 
                                path: 
                                - key: istio_dry_run_allow_shadow_effective_policy_id 
                            tag: istio.authorization.dry_run.allow_policy.name 
                          - metadata: 
                              kind: 
                                request: {} 
                              metadataKey: 
                                key: envoy.filters.http.rbac 
                                path: 
                                - key: istio_dry_run_allow_shadow_engine_result 
                            tag: istio.authorization.dry_run.allow_policy.result 
                          - metadata: 
                              kind: 
                                request: {} 
                              metadataKey: 
                                key: envoy.filters.http.rbac 
                                path: 
                                - key: istio_dry_run_deny_shadow_effective_policy_id 
                            tag: istio.authorization.dry_run.deny_policy.name 
                          - metadata: 
                              kind: 
                                request: {} 
                              metadataKey: 
                                key: envoy.filters.http.rbac 
                                path: 
                                - key: istio_dry_run_deny_shadow_engine_result 
                            tag: istio.authorization.dry_run.deny_policy.result 
                          - literal: 
                              value: latest 
                            tag: istio.canonical_revision 
                          - literal: 
                              value: istio-ingressgateway 
                            tag: istio.canonical_service 
                          - literal: 
                              value: mesh1 
                            tag: istio.mesh_id 
                          - literal: 
                              value: istio-system 
                            tag: istio.namespace 
                          overallSampling: 
                            numerator: 100 
                            denominator: HUNDRED 
                          randomSampling: 
                            numerator: 1 
                            denominator: HUNDRED 
                          clientSampling: 
                            numerator: 100 
                            denominator: HUNDRED

4.2路由设置metadata

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: httpconnectionmanager 
spec: 
  workloadSelector: 
    labels: 
      istio: ingressgateway 
  configPatches: 
  - applyTo: NETWORK_FILTER 
    match: 
      context: GATEWAY 
      listener: 
        portNumber: 8080 
        filterChain: 
          filter: 
            name: "envoy.filters.network.http_connection_manager" 
    patch: 
      operation: MERGE 
      value: 
              name: envoy.filters.network.http_connection_manager 
              typedConfig: 
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager 
                route_config: 
                  name: test 
                  virtual_hosts: 
                  - name: test 
                    domains: 
                    - "*" 
                    routes: 
                    - name: testroute 
                      match:  
                        prefix: / 
                      metadata: 
                        filter_metadata: 
                          "envoy.lb":  
                            canary: true 
                      direct_response: 
                        status: 200 
                        body:  
                          inline_string: "prefix"

4.3local_reply_config

{ 
  "mappers": [], 
  "body_format": "{...}" 
}

mappers:

{ 
  "filter": "{...}", 
  "status_code": "{...}", 
  "body": "{...}", 
  "body_format_override": "{...}", 
  "headers_to_add": [] 
}

filter:

{ 
  "status_code_filter": "{...}", 
  "duration_filter": "{...}", 
  "not_health_check_filter": "{...}", 
  "traceable_filter": "{...}", 
  "runtime_filter": "{...}", 
  "and_filter": "{...}", 
  "or_filter": "{...}", 
  "header_filter": "{...}", 
  "response_flag_filter": "{...}", 
  "grpc_status_filter": "{...}", 
  "extension_filter": "{...}", 
  "metadata_filter": "{...}" 
}

metadata_filter：

{ 
  "matcher": "{...}",匹配条件 
  "match_if_key_not_found": "{...}"key不存在时是否匹配 
}

matcher:

{ 
  "filter": "...",过滤名称 
  "path": [],metadata路径 
  "value": "{...}",匹配值 
  "invert": "..."反向匹配 
}

value:

{ 
  "null_match": "{...}",null匹配 
  "double_match": "{...}",double匹配 
  "string_match": "{...}",string匹配 
  "bool_match": "...",bool匹配 
  "present_match": "...",存在性匹配 
  "list_match": "{...}"列表匹配 
}

string_match:

{ 
  "exact": "...", 
  "prefix": "...", 
  "suffix": "...", 
  "safe_regex": "{...}", 
  "contains": "...", 
  "ignore_case": "..." 
}

body_format:

{ 
  "text_format": "...", 
  "json_format": "{...}", 
  "text_format_source": "{...}", 
  "omit_empty_values": "...", 
  "content_type": "...", 
  "formatters": [] 
}

ef-local_reply_config.yaml

kubectl apply -f ef-local_reply_config.yaml -n istio-system

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: httpconnectionmanager 
spec: 
  workloadSelector: 
    labels: 
      istio: ingressgateway 
  configPatches: 
  - applyTo: NETWORK_FILTER 
    match: 
      context: GATEWAY 
      listener: 
        portNumber: 8080 
        filterChain: 
          filter: 
            name: "envoy.filters.network.http_connection_manager" 
    patch: 
      operation: MERGE 
      value: 
              name: envoy.filters.network.http_connection_manager 
              typedConfig: 
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager 
                route_config: 
                  name: test 
                  virtual_hosts: 
                  - name: test 
                    domains: 
                    - "*" 
                    routes: 
                    - name: testroute 
                      match:  
                        prefix: /product 
                      route: 
                        weighted_clusters: 
                          clusters: 
                          - name: outbound|9080||productpage.istio.svc.cluster.local 
                            weight: 100 
                          total_weight: 100 
                          runtime_key_prefix: test 
                local_reply_config: 
                  mappers: 
                  - status_code: 200 
                    filter: 
                      metadata_filter: 
                        matcher: 
                          filter: envoy.lb 
                          path:  
                          - key: canary 
                          value: 
                            string_match: 
                              exact: "true" 
                          invert: false 
                        match_if_key_not_found: true 
                    body:  
                      inline_string: "test" 
                    body_format_override: 
                      text_format: "%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n"  
                    headers_to_add: 
                    - header: 
                        key: test 
                        value: test 
                      append: true 
                  body_format: 
                    text_format: "%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n"

4.4 envoy.filters.http.set_metadata

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: set 
spec: 
  workloadSelector: 
    labels: 
      app: productpage 
  configPatches: 
  - applyTo: HTTP_FILTER 
    match: 
      context: SIDECAR_INBOUND 
      listener: 
        portNumber: 9080 
        filterChain: 
          destinationPort: 9080 
          filter: 
            name: "envoy.filters.network.http_connection_manager" 
            subFilter: 
              name: "envoy.filters.http.router" 
    patch: 
      operation: INSERT_BEFORE 
      value:          
          name: envoy.filters.http.set_metadata 
          typedConfig: 
            '@type': type.googleapis.com/envoy.extensions.filters.http.set_metadata.v3.Config 
            metadata_namespace: envoy.lb 
            value: 
              canary: "true"

4.5route match

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: httpconnectionmanager 
spec: 
  workloadSelector: 
    labels: 
      istio: ingressgateway 
  configPatches: 
  - applyTo: NETWORK_FILTER 
    match: 
      context: GATEWAY 
      listener: 
        portNumber: 8080 
        filterChain: 
          filter: 
            name: "envoy.filters.network.http_connection_manager" 
    patch: 
      operation: MERGE 
      value: 
              name: envoy.filters.network.http_connection_manager 
              typedConfig: 
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager 
                route_config: 
                  name: test 
                  virtual_hosts: 
                  - name: test 
                    domains: 
                    - "*" 
                    routes: 
                    - name: testroute 
                      match:  
                        path: /tEst 
                        case_sensitive: false 
                        dynamic_metadata: 
                          filter: envoy.lb 
                          path: 
                          - key: canary 
                          value: "true" 
                          invert: false 
                      direct_response: 
                        status: 200 
                        body:  
                          inline_string: "runtime_fraction"

4.6基于元数据的权限控制

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: apply-to 
spec: 
  workloadSelector: 
    labels: 
      app: mysqldb 
  configPatches: 
  - applyTo: NETWORK_FILTER 
    match: 
      listener: 
        portNumber: 3306 
        filterChain: 
          filter: 
            name: "envoy.filters.network.tcp_proxy" 
    patch: 
      operation: INSERT_BEFORE 
      value: 
        name: envoy.filters.network.mysql_proxy 
        typed_config: 
          "@type": type.googleapis.com/envoy.extensions.filters.network.mysql_proxy.v3.MySQLProxy 
          stat_prefix: mysql 
  - applyTo: NETWORK_FILTER 
    match: 
      listener: 
        portNumber: 3306 
        filterChain: 
          filter: 
            name: "envoy.filters.network.tcp_proxy" 
    patch: 
      operation: INSERT_BEFORE 
      value: 
        name: envoy.filters.network.rbac 
        typed_config: 
          "@type": type.googleapis.com/envoy.extensions.filters.network.rbac.v3.RBAC 
          stat_prefix: rbac 
          rules: 
             action: DENY 
             policies: 
               "product-viewer": 
                 permissions: 
                 - metadata: 
                     filter: envoy.filters.network.mysql_proxy 
                     path: 
                     - key: t1.test 
                     value: 
                       list_match: 
                         one_of: 
                           string_match: 
                             prefix: update 
                 principals: 
                 - any: true 
          enforcement_type: CONTINUOUS

4.7负载均衡决策

略，等讲cluster再看

4.8header-To-Metadata Filter

{ 
  "request_rules": [], 
  "response_rules": [] 
}

request_rules:

{ 
  "header": "...", 
  "cookie": "...", 
  "on_header_present": "{...}", 
  "on_header_missing": "{...}", 
  "remove": "..." 
}

response_rules:

{ 
  "header": "...", 
  "cookie": "...", 
  "on_header_present": "{...}", 
  "on_header_missing": "{...}", 
  "remove": "..." 
}

on_header_present,on_header_missing:

{ 
  "metadata_namespace": "...", 
  "key": "...", 
  "value": "...", 
  "regex_value_rewrite": "{...}", 
  "type": "...", 
  "encode": "..." 
}

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: set 
spec: 
  workloadSelector: 
    labels: 
      app: productpage 
  configPatches: 
  - applyTo: HTTP_FILTER 
    match: 
      context: SIDECAR_INBOUND 
      listener: 
        portNumber: 9080 
        filterChain: 
          destinationPort: 9080 
          filter: 
            name: "envoy.filters.network.http_connection_manager" 
            subFilter: 
              name: "envoy.filters.http.router" 
    patch: 
      operation: INSERT_BEFORE 
      value:          
          name: envoy.filters.http.header_to_metadata 
          typedConfig: 
            '@type': type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config 
            request_rules: 
            - header: x-version 
              on_header_present: 
                metadata_namespace: envoy.lb 
                key: version 
                type: STRING 
              on_header_missing: 
                metadata_namespace: envoy.lb 
                key: default 
                value: 'true' 
                type: STRING 
              remove: false 
            response_rules: 
            - header: x-version 
              on_header_present: 
                metadata_namespace: envoy.lb 
                key: version 
                type: STRING 
              on_header_missing: 
                metadata_namespace: envoy.lb 
                key: default 
                value: 'true' 
                type: STRING 
              remove: false

4.9ratelimit actions

apiVersion: networking.istio.io/v1alpha3 
kind: EnvoyFilter 
metadata: 
  name: httpconnectionmanager 
spec: 
  workloadSelector: 
    labels: 
      istio: ingressgateway 
  configPatches: 
  - applyTo: NETWORK_FILTER 
    match: 
      context: GATEWAY 
      listener: 
        portNumber: 8080 
        filterChain: 
          filter: 
            name: "envoy.filters.network.http_connection_manager" 
    patch: 
      operation: MERGE 
      value: 
              name: envoy.filters.network.http_connection_manager 
              typedConfig: 
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager 
                route_config: 
                  name: test 
                  virtual_hosts: 
                  - name: test 
                    domains: 
                    - "*" 
                    routes: 
                    - name: testroute 
                      match:  
                        prefix: /product 
                      route: 
                        rate_limits: 
                        - stage: 0 
                          disable_key: test 
                          actions: 
                          - metadata:  
                              descriptor_key: test 
                              default_value: test 
                              metadata_key: 
                                key: envoy.lb 
                                path: 
                                - key: canary 
                              source: DYNAMIC 
                          limit: 
                            dynamic_metadata: 
                              metadata_key: 
                                key: envoy.lb 
                                path: 
                                - key: canary 
                        weighted_clusters: 
                          clusters: 
                          - name: outbound|9080||productpage.istio.svc.cluster.local 
                            weight: 100 
                          total_weight: 100 
                          runtime_key_prefix: test

本文转载自微信公众号「k8s实战」，可以通过以下二维码关注。转载本文请联系k8s实战公众号。