Working with plugins that require additional files

Running Logstash may require additional files, such as JAR files needed to load JDBC drivers when using a JDBC or JMS plugin. To add these files, there are two options available - using an initContainer to add files before the main container start, or creating a custom Docker image that includes the required files. Refer to Custom configuration files and plugins for a run down of which option might be most suitable for you.

Adding files using an initContainer

edit

This example creates an initContainer to download a PostgreSQL JDBC driver JAR file, and place it in a volume mount accessible to the main container, and then use it in a JDBC input in the pipeline configuration.

spec:
  podTemplate:
    spec:
      initContainers:
      - name: download-postgres
        command: ["/bin/sh"]
        args: ["-c", "curl -o /data/postgresql.jar -L https://jdbc.postgresql.org/download/postgresql-42.6.0.jar"]
        volumeMounts:
          - name: workdir
            mountPath: /data
      containers:
        - name: logstash
          volumeMounts:
            - name: workdir
              mountPath: /usr/share/logstash/jars 
  pipelines:
    - pipeline.id: main
      config.string: |
        input {
          jdbc {
             jdbc_driver_library => "/usr/share/logstash/jars/postgresql.jar"
             jdbc_driver_class => "org.postgresql.Driver"
             
          }
        }

	Referring to the external file should match the `mountPath` of the `container`
	Remainder of plugin configuration goes here

Adding files using a custom image

edit

This example downloads the same postgres JDBC driver, and adds it to the Logstash classpath in the Docker image.

First, create a Dockerfile based on the Logstash Docker image. Download the JDBC driver, and save it alongside the other JAR files in the Logstash classpath:

From docker.elastic.co/logstash/logstash:8.16.1, run:

curl -o /usr/share/logstash/logstash-core/lib/jars/postgresql.jar -L https://jdbc.postgresql.org/download/postgresql-42.6.0.jar

Placing the JAR file in the /usr/share/logstash/logstash-core/lib/jars folder adds it to the Logstash classpath.

After you build and deploy the custom image, include it in the Logstash manifest. (Check out Create custom images for more details.)

  count: 1
  version: {version} 
  image: <CUSTOM_IMAGE>
  pipelines:
    - pipeline.id: main
      config.string: |
        input {
          jdbc {
              
             jdbc_driver_class => "org.postgresql.Driver"
              
          }
        }

	Providing the correct version is always required as ECK reasons about APIs and capabilities available to it based on the version field.
	Note that when you place the JAR file on the Logstash classpath, you do not need to specify the `jdbc_driver_library` location in the plugin configuration.
	Remainder of plugin configuration goes here

« Working with custom plugins Scaling Logstash »