Setting up Traefik with Let's Encrypt certificates is safe and easy, but does it fully protect you against malicious users? The answer is no. You're still vulnerable to a variety of attacks. A common one would be a DDoS attack. This where a malicious user(s) overloads your server with a massive amount of data, sent all at once. How do we protect our services from these kinds of attacks? This is where Crowdsec can help us.

DISCLAIMER: Crowdsec is NOT an all-in-one solution for protecting your publically exposed services. Crowdsec will help mitigate attacks and is an overall great project! There are many practices and standards to help enforce cyber security, so do NOT relie on a single service for projection. Remember to regularly maintain your server and perform security updates.

What is Crowdsec?

Crowdsec is a free, open-sourced project to help analyize the logs of a proxy and determine malicious behavior. Crowdsec was built on the idea of collaberation to have a community-powered security. Crowdsec keeps a public collection of known malicious IPs to help prevent attacks ahead of time.

Acquisition

The acquisition provides a list of logs for Crowdsec to parse and analyze. It requires the source of the logs, as well as the type of logs. To configure an acquisition for traefik, create an acquis.yaml file and put the following text inside:

filenames:
  - /var/log/traefik/*
labels:
  type:	traefik 

Place this file in a secure location inside your server, as we will be mounting this file to your Crowdsec container.

Crowdsec Setup

Now that we have our aquis.yaml file ready, we can start building out our Docker Compose stack.

version: '3.9'
 
networks:
  public-network:
    name: public
    driver: bridge
    
volumes:
  network-logs:
    name: traefik-logs
  certs-volume:
    name: traefik-certs
  crowdsec-db:
    name: crowdsec-database
  crowdsec-config:
    name: crowdsec-config

services:

  traefik:
    image: traefik:v2.4
    container_name: traefik
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - certs-volume:/letsencrypt
      - network-logs:/var/log/
    networks:
      - public-network
    ports:
      - 80:80
      - 443:443
    command:
      - --api.insecure=true
      - --certificatesresolvers.letsencrypt.acme.httpchallenge=true
      - --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.letsencrypt.acme.email=support@example.com # CHANGE ME
      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entrypoints.web.http.redirections.entrypoint.scheme=https
      - --entrypoints.websecure.address=:443
      - --entrypoints.websecure.http.tls=true
      - --entrypoints.websecure.http.tls.certResolver=letsencrypt
      - --log=true
      - --log.level=DEBUG
      - --log.filepath=/var/log/traefik.log
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --providers.docker.network=public

  crowdsec:
    image: crowdsecurity/crowdsec
    restart: unless-stopped
    container_name: crowdsec
    expose:
      - 8080
    environment:
      COLLECTIONS: "crowdsecurity/linux crowdsecurity/traefik"
      GID: "${GID-1000}"
    depends_on:
      - traefik
    volumes:
      - /path/to/crowdsec/acquis.yaml:/etc/crowdsec/acquis.yaml # Change Me!
      - network-logs:/var/log/traefik/:ro
      - crowdsec-db:/var/lib/crowdsec/data/
      - crowdsec-config:/etc/crowdsec/
    networks:
      - public-network

In this stack we have two services: Traefik and Crowdsec. Traefik is configured to use Let's Encrypt certs, redirect HTTP request to HTTPS, and perform HTTP challenges.

This is a solid setup, but something to notice is the logs volume. We have our logs defined with the /var/log/traefik.log path. If we look at our acquis.yaml file, we notice the paths match the same location. These paths MUST MATCH, otherwise Crowdsec will not find your logs.

Looking into our Crowdsec configurations, we are setting Crowdsec to depend on traefik, Crowdsec is on the same network as traefik, we expose port 8080, and we mount the aquis.yaml file. For our environment, we have two important variables: COLLECTIONS and GID. COLLECTIONS help define a set of tools our Crowdsec will need to perform checks on our Traefik logs. These collections can come with many different tools. Such as a parser to help parse the logs of your web server. The GID needs to point a group that has permissions to read the logs of Traefik. We also mount the same volume that carries the Traefik logs and give it read-only permissions. Crowdsec now has Traefik logs and can read them.

Now that you have Traefik and Crowdsec running, you probably don't notice anything different. Thats because Crowdsec isn't protecting you. You're probably upset with me for wasting your time, but I can assure you that we are not done yet.

Bouncers

A bouncer is a component to Crowdsec that reads the events Crowdsec broadcasts. Its essentially a middleware to your Traefik and will make the decisions to reject a user or not.

Crowdsec Traefik Bouncer

Now that you have Traefik running and Crowdsec is reading/parsing your logs, lets set up the Traefik Bouncer. Before we do that, we need to generate an API key from your Crowdsec instance.

To do that, we'll need to run a command to your Docker container. You can run this command in the terminal (don't forget sudo if you need it):

docker exec crowdsec cscli bouncers add bouncer-traefik

If you are using portainer to manager your containers, you run a console session for your crowdsec container. Then run this command instead:

cscli bouncers add bouncer-traefik

Keep that generated key somewhere safe so you can use for your Crowdsec bouncer.

Now that you have an API key to your Crowdsec container you can configure the Traefik Bouncer.

  crowdsec-bouncer:
    image: fbonalair/traefik-crowdsec-bouncer
    container_name: crowdsec-bouncer
    environment:
      GIN_MODE: release # default is debug (more logs)
      CROWDSEC_BOUNCER_API_KEY: ChangeME # Change to bouncer api key
      CROWDSEC_AGENT_HOST: crowdsec:8080
    expose:
      - 8080
    networks:
      - public-network

Don't forget to put your API key inside of the CROWDSEC_BOUNCER_API_KEY environment variable.

Next, you'll need to add the crowdsec-bouncer service as middleware to your Traefik container. You're entire docker-compose.yaml should look something like this:

version: '3.9'
 
networks:
  public-network:
    name: public
    driver: bridge
    
volumes:
  network-logs:
    name: traefik-logs
  certs-volume:
    name: traefik-certs
  crowdsec-db:
    name: crowdsec-database
  crowdsec-config:
    name: crowdsec-config

services:

  traefik:
    image: traefik:v2.4
    container_name: traefik
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - certs-volume:/letsencrypt
      - network-logs:/var/log/
    networks:
      - public-network
    ports:
      - 80:80
      - 443:443
    labels:
      - traefik.enable=true
      - traefik.http.middlewares.crowdsec-bouncer.forwardauth.address=http://crowdsec-bouncer:8080/api/v1/forwardAuth
      - traefik.http.middlewares.crowdsec-bouncer.forwardauth.trustForwardHeader=true
    command:
      - --api.insecure=true
      - --certificatesresolvers.letsencrypt.acme.httpchallenge=true
      - --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.letsencrypt.acme.email=support@example.com # CHANGE ME
      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.middlewares=crowdsec-bouncer@docker
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entrypoints.web.http.redirections.entrypoint.scheme=https
      - --entrypoints.websecure.address=:443
      - --entrypoints.websecure.http.middlewares=crowdsec-bouncer@docker
      - --entrypoints.websecure.http.tls=true
      - --entrypoints.websecure.http.tls.certResolver=letsencrypt
      - --log=true
      - --log.level=DEBUG
      - --log.filepath=/var/log/traefik.log
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --providers.docker.network=public

  crowdsec:
    image: crowdsecurity/crowdsec
    restart: unless-stopped
    container_name: crowdsec
    expose:
      - 8080
    environment:
      COLLECTIONS: "crowdsecurity/linux crowdsecurity/traefik"
      GID: "${GID-1000}"
    depends_on:
      - traefik
    volumes:
      - /path/to/crowdsec/acquis.yaml:/etc/crowdsec/acquis.yaml # Change Me!
      - network-logs:/var/log/traefik/:ro
      - crowdsec-db:/var/lib/crowdsec/data/
      - crowdsec-config:/etc/crowdsec/
    networks:
      - public-network

  crowdsec-bouncer:
    image: fbonalair/traefik-crowdsec-bouncer
    container_name: crowdsec-bouncer
    environment:
      GIN_MODE: release # default is debug (more logs)
      CROWDSEC_BOUNCER_API_KEY: ChangeME # Change to bouncer api key
      CROWDSEC_AGENT_HOST: crowdsec:8080
    expose:
      - 8080
    networks:
      - public-network

Be sure to go through this and make sure you update all settings commented with a "change..." statement. After everything is running you should see some logs inside of your crowdsec-bouncer container.

I hope this helps you get started with securing your services and have a better understand of Crowdsec. Thank you for reading!

Docker compose and acquisition configurations can be found in my GitHub.