Metadata

Every new connection to the APM Server starts with a metadata stanza. This provides general metadata concerning the other objects in the stream.

Rather than send this metadata information from the agent multiple times, the APM Server hangs on to this information and applies it to other objects in the stream as necessary.

Metadata is stored under context when viewing documents in Elasticsearch.

Kubernetes data

edit

APM agents automatically read kubernetes data and send it to the APM Server. In most instances, agents are able to read this data from inside the container. If this is not the case, or if you wish to override this data, you can set environment variables for the agents to read. These environment variable are set via the kubernetes Downward API. Here’s how you would add the environment variables to your kubernetes pod spec:

         - name: KUBERNETES_NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: KUBERNETES_POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: KUBERNETES_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: KUBERNETES_POD_UID
            valueFrom:
              fieldRef:
                fieldPath: metadata.uid

The table below maps these environment variables to the APM metadata event field:

Environment variable	Metadata field name
KUBERNETES_NODE_NAME	system.kubernetes.node.name
KUBERNETES_POD_NAME	system.kubernetes.pod.name
KUBERNETES_NAMESPACE	system.kubernetes.namespace
KUBERNETES_POD_UID	system.kubernetes.pod.uid

Metadata Schema

edit

The APM Server uses JSON Schema for validating requests. The specification for metadata is defined below:

{
    "$id": "doc/spec/metadata.json",
    "title": "Metadata",
    "description": "Metadata concerning the other objects in the stream.",
    "type": ["object"],
    "properties": {
        "service": {
            "$ref": "service.json",
            "type": "object",
            "required": ["name", "agent"],
            "properties.name.type": "string",
            "properties.agent.type": "string",
            "properties.agent.required": ["name", "version"],
            "properties.agent.properties.name.type": "string",
            "properties.agent.properties.version.type": "string",
            "properties.runtime.required": ["name", "version"],
            "properties.runtime.properties.name.type": "string",
            "properties.runtime.properties.version.type": "string",
            "properties.language.required": ["name"],
            "properties.language.properties.name.type": "string"
        },
        "process": {
            "$ref": "process.json"
        },
        "system": {
            "$ref": "system.json"
        },
        "user": {
            "description": "Describes the authenticated User for a request.",
            "$ref": "user.json"
        },
        "labels": {
            "$ref": "tags.json"
        }
    },
    "required": ["service"]
}

Service Schema

edit

{
    "$id": "doc/spec/service.json",
    "title": "Service",
    "type": ["object", "null"],
    "properties": {
        "agent": {
            "description": "Name and version of the Elastic APM agent",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "description": "Name of the Elastic APM agent, e.g. \"Python\"",
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "description": "Version of the Elastic APM agent, e.g.\"1.0.0\"",
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "ephemeral_id": {
                    "description": "Free format ID used for metrics correlation by some agents",
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "framework": {
            "description": "Name and version of the web framework used",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "language": {
            "description": "Name and version of the programming language used",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "name": {
            "description": "Immutable name of the service emitting this event",
            "type": ["string", "null"],
            "pattern": "^[a-zA-Z0-9 _-]+$",
            "maxLength": 1024
        },
        "environment": {
            "description": "Environment name of the service, e.g. \"production\" or \"staging\"",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "runtime": {
            "description": "Name and version of the language runtime running this service",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "version": {
            "description": "Version of the service emitting this event",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "node": {
            "description": "Unique meaningful name of the service node.",
            "type": ["object", "null"],
            "properties": {
                "configured_name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        }
    }
}

Process Schema

edit

{
  "$id": "doc/spec/process.json",
  "title": "Process",
  "type": ["object", "null"],
  "properties": {
      "pid": {
          "description": "Process ID of the service",
          "type": ["integer"]
      },
      "ppid": {
          "description": "Parent process ID of the service",
          "type": ["integer", "null"]
      },
      "title": {
          "type": ["string", "null"],
          "maxLength": 1024
      },
      "argv": {
        "description": "Command line arguments used to start this process",
        "type": ["array", "null"],
        "minItems": 0,
        "items": {
           "type": "string"
        }
    }
  },
  "required": ["pid"]
}

System Schema

edit

{
    "$id": "doc/spec/system.json",
    "title": "System",
    "type": ["object", "null"],
    "properties": {
        "architecture": {
            "description": "Architecture of the system the agent is running on.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "hostname": {
            "description": "Deprecated. Hostname of the system the agent is running on. Will be ignored if kubernetes information is set.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "detected_hostname": {
            "description": "Hostname of the host the monitored service is running on. It normally contains what the hostname command returns on the host machine. Will be ignored if kubernetes information is set, otherwise should always be set.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "configured_hostname": {
            "description": "Name of the host the monitored service is running on. It should only be set when configured by the user. If empty, will be set to detected_hostname or derived from kubernetes information if provided.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "platform": {
            "description": "Name of the system platform the agent is running on.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "container": {
            "properties": {
                "id" : {
                    "description": "Container ID",
                    "type": ["string"],
                    "maxLength": 1024
                }
            },
            "required": ["id"]
        },
        "kubernetes": {
            "properties": {
                "namespace": {
                    "description": "Kubernetes namespace",
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "pod":{
                    "properties": {
                        "name": {
                            "description": "Kubernetes pod name",
                            "type": ["string", "null"],
                            "maxLength": 1024
                        },
                        "uid": {
                            "description": "Kubernetes pod uid",
                            "type": ["string", "null"],
                            "maxLength": 1024
                        }
                    }
                },
                "node":{
                    "properties": {
                        "name": {
                            "description": "Kubernetes node name",
                            "type": ["string", "null"],
                            "maxLength": 1024
                        }
                    }
                }
            }
        }
    }
}

User Schema

edit

{
    "$id": "docs/spec/user.json",
    "title": "User",
    "type": ["object", "null"],
    "properties": {
        "id": {
            "description": "Identifier of the logged in user, e.g. the primary key of the user",
            "type": ["string", "integer", "null"],
            "maxLength": 1024
        },
        "email": {
            "description": "Email of the logged in user",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "username": {
            "description": "The username of the logged in user",
            "type": ["string", "null"],
            "maxLength": 1024
        }
    }
}

« Events Intake API Transactions »