Exterminate them all, kill spam using GitHub Actions

In previous articles, I explained how to fight against spam if you've a cpanel or PlanetHoster's N0C infrastructure.

What if we automated as many things as possible?

This article uses GitHub Actions to deploy a generated file over FTP — exactly the mechanism described in GitHub - Use Actions to deploy this blog.

In the PlanetHoster's N0C infrastructure article, I showed that you need to write rules for RoundCube to identify sources of spam and remove them. These rules have to be created from the RoundCube interface and, well, ok, that's not optimized at all.

Let's look at how to do as little as possible and still be a hardcore spam fighter.

What you get​

At the end of this article, a script turns a plain list of domains into a complete RoundCube filter file:

# rule:[Identify as spam: *.buzz]
if allof (header :matches "from" "*.buzz")
{
  fileinto "spam";
}
# rule:[Identify as spam: *.cf]
if allof (header :matches "from" "*.cf")
{
  fileinto "spam";
}
# rule:[Identify as spam: *.cn]
if allof (header :matches "from" "*.cn")
{
  fileinto "spam";
}

That file lands on your hosting account and RoundCube reads it as its filter list:

To put it simply: you add one line to a JSON file on your computer, you push it, and RoundCube receives the new rules automatically a few seconds later. No more clicking through the filter interface, one rule at a time.

The idea​

As already explained, RoundCube is using a sieve file to store your rules. Such file can look like this:

# rule:[Identify as spam: *.su]
if allof (header :matches "from" "*.su")
{
    fileinto "spam";
}

And, therefore, the objective is simple and obvious: we're going to create a little script that only needs two things.

1. a list of values, e.g. domain names to be blocked
2. a sieve template

Our action plan​

Building the generator​

Create​

Please run mkdir ~/sieve-generator && cd $_ to create the sieve-generator in your home directory and jump in it.

Storing the list of values in a JSON file​

Please create a file called patterns.json with this content:

{
  "spam": [
    "*.buzz",
    "*.cf",
    "*.cn",
    "*.ga",
    "*.gq",
    "*.host",
    "*.icu",
    "*.india",
    "*.info",
    "*.live",
    "*.ml",
    "*.net",
    "*.online",
    "*.ru",
    "*.su",
    "*.tk",
    "*.top",
    "*.ua",
    "*.us",
    "*.wang",
    "*.xyz"
  ]
}

Create our template​

Please create a file called spam.template with this content:

# rule:[Identify as spam: {{ pattern }}]
if allof (header :matches "from" "{{ pattern }}")
{
 fileinto "spam";
}

Create the generation script​

Please create a file called generate.sh with this content:

#!/usr/bin/env bash

clear

if [ ! -f patterns.json ]; then
    echo "Please create a patterns.json file."
    echo "Run 'cp patterns.json.dist patterns.json' to use an example."
    exit 1
fi

if [ ! -f spam.template ]; then
    echo "Please create a spam.template file. Below an example of what you can put in it:"
    echo ""
    echo "# rule:[Identify as spam: {{ pattern }}]"
    echo "if allof (header :matches \"from\" \"{{ pattern }}\")"
    echo "{"
    echo "  fileinto \"spam\";"
    echo "}"
    echo ""
    exit 1
fi

# File to create
outFile="${PWD}/build/roundcube.sieve"

[ ! -d "$(dirname "${outFile}")" ] && mkdir "$(dirname "${outFile}")"

rm -f "${outFile}" && touch "${outFile}"

cat patterns.json \
    | jq '.spam[]' \
    | sort \
    | while read -r pattern; do \
        # Trim quotes
        pattern=$(echo "$pattern" | tr -d '"')
        # Read the spam.template file, make the replace and append in file roundcube.sieve
        sed "s/{{ pattern }}/${pattern}/g" spam.template >>"${outFile}"; \
    done

echo "File ${outFile} has been created."

Also run chmod +x generate.sh to make it executable.

At this stage, you've thus three files in your folder:

• generate.sh,
• patterns.json and
• spam.template

In your console, run ./generate.sh and if everything is running fine, you'll get a message like File /home/xxx/sieve-generator/build/roundcube.sieve has been created..

So now, you should have a new folder called build where the file roundcube.sieve has been created. Open it: its content is the rule list shown at the beginning of this article, one block per pattern, sorted alphabetically.

Congratulations: you've successfully created your anti-spam generator.

Deploying the file by FTP​

To validate the roundcube.sieve file, you can start your FTP client, go to your N0C server (hosted by PlanetHoster), go to the folder /mail/DOMAIN.TLD/ACCOUNT/sieve (refers to the article PlanetHoster's N0C infrastructure if needed).

Push the file there, on your host.

Now, go to your web email account (direct link: https://mg.n0c.com/email/accounts) and start RoundCube. Open your account, click on Settings then Filters and tadaaa — the filter list shown at the beginning of this article is now yours, without a single click in the filter editor.

Automating it with GitHub Actions (optional)​

Create your own GitHub repository​

In order to be able to create automation using GitHub Actions, first, we need to create a GitHub repository.

I'll suppose you already have a GitHub account and you know how to use GitHub.

Go to https://github.com/new and create a new private repository (it's free).

Then just copy/paste the example given by GitHub; something like:

$ echo "# Sieve generator" >> README.md
$ git init

$ git add README.md

$ git commit -m "first commit"

$ git branch -M main

$ git remote add origin XXXXXXXXXXXX     '\<-- put here your own path

$ git push -u origin main
▋

Now push the other files:

$ git add .

$ git commit -m "Initial upload"

$ git push
▋

Back to GitHub, you'll obtain a repo like this one:

So, now, each time you'll update the list of domains (in the patterns.json file), our objective is to ask GitHub to run the ./generate.sh script itself, generate the build/roundcube.sieve file, and publish it to your FTP account.

Adding GitHub Actions​

Go back to your computer, jump back in the ~/sieve-generator folder and create a new file called .github/workflows/build_and_deploy.yaml.

Here is the content for that new file:

name: Create roundcube.sieve and deploy

on:
  push:
    branches:
      - main

  workflow_dispatch:
    inputs:
      version:
        type: string
        required: false
        description: 'Version of jq to install'
        default: '1.7'
      force:
        type: boolean
        required: false
        description: 'Do not check for existing jq installation before continuing.'
        default: false

permissions:
  contents: write

jobs:
  deploy:
    name: Create roundcube.sieve and deploy
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6

      - name: 'Setup jq'
        uses: dcarbone/install-jq-action@v2
        with:
          version: '${{ inputs.version }}'
          force: '${{ inputs.force }}'

      - name: 'Build'
        run: |
          ./generate.sh

      - name: Push files
        uses: SamKirkland/FTP-Deploy-Action@v4.3.5
        with:
          server: ${{ secrets.ftp_server }}
          username: ${{ secrets.ftp_login }}
          password: ${{ secrets.ftp_password }}
          local-dir: ./build/

I won't go into detail about this file, but it asks GitHub to:

1. Install JQ on its server,
2. Run the generate.sh script and thus build the build/roundcube.sieve file
3. Use FTP-Deploy-Action to push the file on your server.

As you can see, there are three secrets.

Back to your browser, go to your GitHub repository you've created in the previous chapter.

Click on Settings then Secrets and variables and finally, click on Actions

Click on the green button New repository secret and create the first one called FTP_SERVER (seems to be in uppercase). Put there the name of your FTP server like node99-xx.N0c.com (get the name from your My planethoster page).

You'll need to add two new secrets FTP_LOGIN and FTP_PASSWORD but, before, create a new FTP user.

Go to https://mg.n0c.com/files/ftp-accounts and create a new user. Use a very explicit name like deploy_spam_roundcube_xxx and define a very strong password (30 chars or more).

Restrict the path for that user to, exactly, where the sieve file is located on your FTP. As said in PlanetHoster's N0C infrastructure, for me, it's /mail/avonture.be/christophe/mail/avonture.be/christophe/sieve. Make sure to find the path for you.

Ok, so, now, you've a new FTP login, its very secure password, and access restricted to only your sieve folder.

Copy the login and the password in the GitHub secrets page.

You'll then have this:

Everything is now in place for automation​

If you've followed the GitHub chapters, now, everything is set up.

Back to your computer and edit the patterns.json file and add a new pattern, f.i. newsletters@*.

Push the change to GitHub (git add . ; git commit -m "chore: new pattern" ; git push).

Just for the curiosity, go to GitHub - Actions tabs:

If you've configured everything correctly, you should get this:

Conclusion​

From now on, all you have to do is copy/paste the email addresses you receive spam from, see if you can standardize them as much as possible (by using the asterisk like in newsletter@*) and then copy/paste them into the JSON file that you just have to upload to GitHub.

Fighting spam has become a one-line commit, and the blocklist is versioned: you can see when a domain was added, and remove it just as easily.

Et voilà!