Filebeat Autodiscover: Dynamic Input Source Configuration

The 'filebeat.autodiscover' configuration option in Filebeat enables automatic discovery and configuration of input sources. It allows Filebeat to dynamically discover and start new input sources based on predefined conditions or patterns.

When 'filebeat.autodiscover' is enabled, Filebeat continuously scans for new files, containers, or other input sources that match the specified conditions. It automatically starts and configures new input sources without requiring manual intervention.

The 'filebeat.autodiscover' option accepts a list of configurations, each defining a separate input source. The configuration can include conditions, such as file path patterns or container labels, to filter and match the input sources. It also allows specifying the input type, processing options, and output destinations for each discovered input source.

Here's an example configuration for 'filebeat.autodiscover':

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - "/var/lib/docker/containers/*/*.log"

In this example, Filebeat is configured to automatically discover input sources from Docker containers. It enables hints for Docker autodiscovery and specifies the default configuration for discovered containers. The default configuration sets the input type to 'container' and defines the log file paths to be collected.

By using 'filebeat.autodiscover', you can simplify the configuration of Filebeat and ensure that new input sources are automatically detected and processed without manual intervention.