docker_logdna—deep dive into Docker plugins
Christopher Besch (IBM) • 9th February 2024


Deep Dive into Docker Plugins

What we'll talk about

  1. What is the docker_logdna Plugin?
  2. Performance
  3. Testing
  4. Logspout Alternative

What is a Log Aggregator?

  • Example:
    • Authentication request arrives at proxy -> proxy logs original IP
    • LDAP only sees proxy's IP -> LDAP logs rejection of request
  • Proxy and LDAP logs need to be combined
  • Solution: ELK, Graylog, Fluentd, Logdna, ...

Why Logdna?

  • IBM Log Analysis uses it
  • We don't want to host own aggregator

What is a Docker Logging Plugin?

docker run --log-driver fluentd debian


What do I have to do

  1. Check installed docker_logdna version with
  2. $ docker plugin ls
  3. Add logging section to your docker-compose.yaml
  5. Documentation on


©NASA 1969

  • Never block the application!
  • Read logs from fifo file ASAP!
  • Low resource consumption
  • No resource consumption when blocked waiting

Simple Architecture


  1. Multiple Container can't be handled
  2. Busy waiting wastes CPU time

Solution: Async

  • Run tasks concurrently
  • Not parallel with multithreading
  • Great for IO-bound tasks

Introducing the Future

  • Code to be executed when the time is right
  • Can await external condition
  • Async runtime schedules polling of futures

Simple Architecture

Async Architecture


Let's get that Green Color!

What do we Want?

  • Handle edge cases
  • Handle rare conditions (stress, external environment slow, ...)
  • Prevent regressions

  • Consistent results
  • Errors easy to debug

How do we get there?

  • Unit tests (white box)
  • End-to-End tests (black box)

  • Automated at every change
  • Fast (short turnaround time)
  • Full control over environment (i.e. tests not dependant on external API)

Is docker_logdna easy to test?

The Good
  • Little state
  • Almost entirely linear pipeline
  • No UI/UX
The Bad
  • IO-heavy code
  • External input/output difficult to mock

What to test for

Unit Tests
  • Config parsing
  • Docker HTTP socket interaction
  • Application Container protobuf fifo file interaction
  • Logdna interaction
End-to-End Tests
  • Many log lines per second
  • Long log lines
  • Many containers
  • Unresponsive Logdna server


  • Minimize differences between mock and original

Unit Tests


Unit Tests

  • Test all edge cases
  • At least one unit test per outcome

End-to-End Tests

  • Use Docker-in-Docker
    • full control over environment
    • don't clutter host with 100+ containers

End-to-End Tests



  • ~900 lines of code
  • ~1600 lines of tests

Comparison to Logspout

docker_logdna Plugin
Logdna Logspout

Coming Soon: IBM Open-Source Approval Talk

Coming Soon: IBM Open-Source Approval Talk