Metadataedit

Every new connection to the APM Server starts with a metadata stanza. This provides general metadata concerning the other objects in the stream.

Rather than send this metadata information from the agent multiple times, the APM Server hangs on to this information and applies it to other objects in the stream as necessary.

Metadata is stored under context when viewing documents in Elasticsearch.

Kubernetes dataedit

APM agents automatically read kubernetes data and send it to the APM Server. In most instances, agents are able to read this data from inside the container. If this is not the case, or if you wish to override this data, you can set environment variables for the agents to read. These environment variable are set via the kubernetes Downward API. Here’s how you would add the environment variables to your kubernetes pod spec:

         - name: KUBERNETES_NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: KUBERNETES_POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: KUBERNETES_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: KUBERNETES_POD_UID
            valueFrom:
              fieldRef:
                fieldPath: metadata.uid

The table below maps these environment variables to the APM metadata event field:

Environment variable Metadata field name

KUBERNETES_NODE_NAME

system.kubernetes.node.name

KUBERNETES_POD_NAME

system.kubernetes.pod.name

KUBERNETES_NAMESPACE

system.kubernetes.namespace

KUBERNETES_POD_UID

system.kubernetes.pod.uid

Metadata Schemaedit

APM Server uses JSON Schema to validate requests. The specification for metadata is defined on GitHub and included below:

{
    "$id": "docs/spec/metadata.json",
    "title": "Metadata",
    "description": "Metadata concerning the other objects in the stream.",
    "type": "object",
    "properties": {
        "service": {
            "type": [
                "object"
            ],
            "properties": {
                "agent": {
                    "description": "Name and version of the Elastic APM agent",
                    "type": [
                        "object"
                    ],
                    "properties": {
                        "name": {
                            "description": "Name of the Elastic APM agent, e.g. \"Python\"",
                            "type": [
                                "string"
                            ],
                            "maxLength": 1024,
                            "minLength": 1
                        },
                        "version": {
                            "description": "Version of the Elastic APM agent, e.g.\"1.0.0\"",
                            "type": [
                                "string"
                            ],
                            "maxLength": 1024
                        },
                        "ephemeral_id": {
                            "description": "Free format ID used for metrics correlation by some agents",
                            "type": [
                                "string",
                                "null"
                            ],
                            "maxLength": 1024
                        }
                    },
                    "required": [
                        "name",
                        "version"
                    ]
                },
                "framework": {
                    "description": "Name and version of the web framework used",
                    "type": [
                        "object",
                        "null"
                    ],
                    "properties": {
                        "name": {
                            "type": [
                                "string",
                                "null"
                            ],
                            "maxLength": 1024
                        },
                        "version": {
                            "type": [
                                "string",
                                "null"
                            ],
                            "maxLength": 1024
                        }
                    }
                },
                "language": {
                    "description": "Name and version of the programming language used",
                    "type": [
                        "object",
                        "null"
                    ],
                    "properties": {
                        "name": {
                            "type": [
                                "string"
                            ],
                            "maxLength": 1024
                        },
                        "version": {
                            "type": [
                                "string",
                                "null"
                            ],
                            "maxLength": 1024
                        }
                    },
                    "required": [
                        "name"
                    ]
                },
                "name": {
                    "description": "Immutable name of the service emitting this event",
                    "type": [
                        "string"
                    ],
                    "pattern": "^[a-zA-Z0-9 _-]+$",
                    "maxLength": 1024,
                    "minLength": 1
                },
                "environment": {
                    "description": "Environment name of the service, e.g. \"production\" or \"staging\"",
                    "type": [
                        "string",
                        "null"
                    ],
                    "maxLength": 1024
                },
                "runtime": {
                    "description": "Name and version of the language runtime running this service",
                    "type": [
                        "object",
                        "null"
                    ],
                    "properties": {
                        "name": {
                            "type": [
                                "string"
                            ],
                            "maxLength": 1024
                        },
                        "version": {
                            "type": [
                                "string"
                            ],
                            "maxLength": 1024
                        }
                    },
                    "required": [
                        "name",
                        "version"
                    ]
                },
                "version": {
                    "description": "Version of the service emitting this event",
                    "type": [
                        "string",
                        "null"
                    ],
                    "maxLength": 1024
                },
                "node": {
                    "description": "Unique meaningful name of the service node.",
                    "type": [
                        "object",
                        "null"
                    ],
                    "properties": {
                        "configured_name": {
                            "type": [
                                "string",
                                "null"
                            ],
                            "maxLength": 1024
                        }
                    }
                }
            },
            "required": [
                "name",
                "agent"
            ]
        },
        "process": {
            "$ref": "process.json"
        },
        "system": {
            "$ref": "system.json"
        },
        "user": {
            "description": "Describes the authenticated User for a request.",
            "$ref": "user.json"
        },
        "cloud": {
            "$ref": "cloud.json"
        },
        "labels": {
            "$ref": "tags.json"
        }
    },
    "required": [
        "service"
    ]
}
Service Schemaedit
{
    "$id": "docs/spec/service.json",
    "title": "Service",
    "type": ["object", "null"],
    "properties": {
        "agent": {
            "description": "Name and version of the Elastic APM agent",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "description": "Name of the Elastic APM agent, e.g. \"Python\"",
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "description": "Version of the Elastic APM agent, e.g.\"1.0.0\"",
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "ephemeral_id": {
                    "description": "Free format ID used for metrics correlation by some agents",
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "framework": {
            "description": "Name and version of the web framework used",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "language": {
            "description": "Name and version of the programming language used",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "name": {
            "description": "Immutable name of the service emitting this event",
            "type": ["string", "null"],
            "pattern": "^[a-zA-Z0-9 _-]+$",
            "maxLength": 1024
        },
        "environment": {
            "description": "Environment name of the service, e.g. \"production\" or \"staging\"",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "runtime": {
            "description": "Name and version of the language runtime running this service",
            "type": ["object", "null"],
            "properties": {
                "name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "version": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        },
        "version": {
            "description": "Version of the service emitting this event",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "node": {
            "description": "Unique meaningful name of the service node.",
            "type": ["object", "null"],
            "properties": {
                "configured_name": {
                    "type": ["string", "null"],
                    "maxLength": 1024
                }
            }
        }
    }
}
Process Schemaedit
{
  "$id": "docs/spec/process.json",
  "title": "Process",
  "type": ["object", "null"],
  "properties": {
      "pid": {
          "description": "Process ID of the service",
          "type": ["integer"]
      },
      "ppid": {
          "description": "Parent process ID of the service",
          "type": ["integer", "null"]
      },
      "title": {
          "type": ["string", "null"],
          "maxLength": 1024
      },
      "argv": {
        "description": "Command line arguments used to start this process",
        "type": ["array", "null"],
        "minItems": 0,
        "items": {
           "type": "string"
        }
    }
  },
  "required": ["pid"]
}
System Schemaedit
{
    "$id": "docs/spec/system.json",
    "title": "System",
    "type": ["object", "null"],
    "properties": {
        "architecture": {
            "description": "Architecture of the system the agent is running on.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "hostname": {
            "description": "Deprecated. Hostname of the system the agent is running on. Will be ignored if kubernetes information is set.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "detected_hostname": {
            "description": "Hostname of the host the monitored service is running on. It normally contains what the hostname command returns on the host machine. Will be ignored if kubernetes information is set, otherwise should always be set.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "configured_hostname": {
            "description": "Name of the host the monitored service is running on. It should only be set when configured by the user. If empty, will be set to detected_hostname or derived from kubernetes information if provided.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "platform": {
            "description": "Name of the system platform the agent is running on.",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "container": {
            "properties": {
                "id" : {
                    "description": "Container ID",
                    "type": ["string"],
                    "maxLength": 1024
                }
            },
            "required": ["id"]
        },
        "kubernetes": {
            "properties": {
                "namespace": {
                    "description": "Kubernetes namespace",
                    "type": ["string", "null"],
                    "maxLength": 1024
                },
                "pod":{
                    "properties": {
                        "name": {
                            "description": "Kubernetes pod name",
                            "type": ["string", "null"],
                            "maxLength": 1024
                        },
                        "uid": {
                            "description": "Kubernetes pod uid",
                            "type": ["string", "null"],
                            "maxLength": 1024
                        }
                    }
                },
                "node":{
                    "properties": {
                        "name": {
                            "description": "Kubernetes node name",
                            "type": ["string", "null"],
                            "maxLength": 1024
                        }
                    }
                }
            }
        }
    }
}
User Schemaedit
{
    "$id": "docs/spec/user.json",
    "title": "User",
    "type": ["object", "null"],
    "properties": {
        "id": {
            "description": "Identifier of the logged in user, e.g. the primary key of the user",
            "type": ["string", "integer", "null"],
            "maxLength": 1024
        },
        "email": {
            "description": "Email of the logged in user",
            "type": ["string", "null"],
            "maxLength": 1024
        },
        "username": {
            "description": "The username of the logged in user",
            "type": ["string", "null"],
            "maxLength": 1024
        }
    }
}