<!--
  canonical-url: https://www.avonture.be/blog/planethoster-n0c-spam-roundcube-action
  published:     2024-01-28
  This is a static plain-Markdown mirror generated at build time.
  Visit the canonical URL above for the fully rendered page, with images and interactive components.
-->

# Exterminate them all, kill spam using GitHub Actions

> Kill spam automatically! This guide shows how to leverage GitHub Actions, JSON, and a Bash script to generate and deploy Roundcube Sieve filters on your PlanetHoster N0C hosting environment, automating your blocklist management.

---
![Exterminate them all, kill spam using GitHub Actions](/img/v2/fighting_against_spam.webp)

**TL;DR**

This article automates the RoundCube Sieve spam-filter workflow from the previous PlanetHoster N0C post: a Bash script generates a `roundcube.sieve` file from a `patterns.json` domain list and a template, and a GitHub Actions workflow rebuilds and FTP-deploys that file automatically whenever `patterns.json` is pushed, removing the need to manually click through RoundCube's filter UI.

In previous articles, I explained how to fight against spam if you've a [cpanel](/blog/cpanel-spam) or [PlanetHoster's N0C infrastructure](/blog/planethoster-n0c-spam).

What if we automated as many things as possible?

*This article uses GitHub Actions to deploy a generated file over FTP — exactly the mechanism described in [GitHub - Use Actions to deploy this blog](/blog/github-action).*

In the [PlanetHoster's N0C infrastructure](/blog/planethoster-n0c-spam) article, I showed that you need to write rules for RoundCube to identify sources of spam and remove them. These rules have to be created from the RoundCube interface and, well, ok, that's not optimized at all.

Let's look at how to do as little as possible and still be a hardcore spam fighter.

## What you get

At the end of this article, a script turns a plain list of domains into a complete RoundCube filter file:

```none
# rule:[Identify as spam: *.buzz]
if allof (header :matches "from" "*.buzz")
{
  fileinto "spam";
}
# rule:[Identify as spam: *.cf]
if allof (header :matches "from" "*.cf")
{
  fileinto "spam";
}
# rule:[Identify as spam: *.cn]
if allof (header :matches "from" "*.cn")
{
  fileinto "spam";
}
```

That file lands on your hosting account and RoundCube reads it as its filter list:

![RoundCube filters](./images/filters.webp)

> **I've already added more spams domain for my own account:**
>
> If you don't have the same list of filters; it's absolutely normal.

To put it simply: **you add one line to a JSON file on your computer, you push it, and RoundCube receives the new rules automatically a few seconds later.** No more clicking through the filter interface, one rule at a time.

## The idea

As already explained, RoundCube is using a sieve file to store your rules. Such file can look like this:

```none
# rule:[Identify as spam: *.su]
if allof (header :matches "from" "*.su")
{
    fileinto "spam";
}
```

And, therefore, the objective is simple and obvious: we're going to create a little script that only needs two things.

1. a list of values, e.g. domain names to be blocked
2. a sieve template

## Our action plan

1. We'll create a directory and jump in it: `mkdir ~/sieve-generator && cd $_`,

2. We'll create our JSON file,

3. We'll create our sieve template,

4. We'll create our script, in Bash (but you can do this in any language)

5. We'll create a repository on GitHub and push our code there

6. And we'll finally set up GitHub Actions to automate the build and deploy process.

## Building the generator

### Create

Please run `mkdir ~/sieve-generator && cd $_` to create the `sieve-generator` in your home directory and jump in it.

### Storing the list of values in a JSON file

Please create a file called `patterns.json` with this content:

```json
{
  "spam": [
    "*.buzz",
    "*.cf",
    "*.cn",
    "*.ga",
    "*.gq",
    "*.host",
    "*.icu",
    "*.india",
    "*.info",
    "*.live",
    "*.ml",
    "*.net",
    "*.online",
    "*.ru",
    "*.su",
    "*.tk",
    "*.top",
    "*.ua",
    "*.us",
    "*.wang",
    "*.xyz"
  ]
}
```

### Create our template

Please create a file called `spam.template` with this content:

```none
# rule:[Identify as spam: {{ pattern }}]
if allof (header :matches "from" "{{ pattern }}")
{
 fileinto "spam";
}

```

> **Please keep the last empty line:**
>
> The last line of the `spam.template` file has to be an empty line so the file should have 6 lines, not 5.

### Create the generation script

Please create a file called `generate.sh` with this content:

```bash
#!/usr/bin/env bash

clear

if [ ! -f patterns.json ]; then
    echo "Please create a patterns.json file."
    echo "Run 'cp patterns.json.dist patterns.json' to use an example."
    exit 1
fi

if [ ! -f spam.template ]; then
    echo "Please create a spam.template file. Below an example of what you can put in it:"
    echo ""
    echo "# rule:[Identify as spam: {{ pattern }}]"
    echo "if allof (header :matches \"from\" \"{{ pattern }}\")"
    echo "{"
    echo "  fileinto \"spam\";"
    echo "}"
    echo ""
    exit 1
fi

# File to create
outFile="${PWD}/build/roundcube.sieve"

[ ! -d "$(dirname "${outFile}")" ] && mkdir "$(dirname "${outFile}")"

rm -f "${outFile}" && touch "${outFile}"

cat patterns.json \
    | jq '.spam[]' \
    | sort \
    | while read -r pattern; do \
        # Trim quotes
        pattern=$(echo "$pattern" | tr -d '"')
        # Read the spam.template file, make the replace and append in file roundcube.sieve
        sed "s/{{ pattern }}/${pattern}/g" spam.template >>"${outFile}"; \
    done

echo "File ${outFile} has been created."
```

Also run `chmod +x generate.sh` to make it executable.

At this stage, you've thus three files in your folder:

* `generate.sh`,
* `patterns.json` and
* `spam.template`

In your console, run `./generate.sh` and if everything is running fine, you'll get a message like `File /home/xxx/sieve-generator/build/roundcube.sieve has been created.`.

> **`jq` is required:**
>
> The script reads the JSON list with `jq`. If you get an error about it, install it by running `sudo apt-get update && sudo apt-get install jq` then start `./generate.sh` again.

So now, you should have a new folder called `build` where the file `roundcube.sieve` has been created. Open it: its content is the rule list shown at the beginning of this article, one block per pattern, sorted alphabetically.

Congratulations: you've successfully created your anti-spam generator.

## Deploying the file by FTP

To validate the `roundcube.sieve` file, you can start your FTP client, go to your N0C server (hosted by PlanetHoster), go to the folder `/mail/DOMAIN.TLD/ACCOUNT/sieve` (refers to the article [PlanetHoster's N0C infrastructure](/blog/planethoster-n0c-spam) if needed).

Push the file there, on your host.

Now, go to your web email account (direct link: <https://mg.n0c.com/email/accounts>) and start RoundCube. Open your account, click on `Settings` then `Filters` and tadaaa — the filter list shown at the beginning of this article is now yours, without a single click in the filter editor.

## Automating it with GitHub Actions (optional)

### Create your own GitHub repository

> **Optional chapter:**
>
> Skip this chapter if you don't want full automation.

In order to be able to create automation using GitHub Actions, first, we need to create a GitHub repository.

I'll suppose you already have a GitHub account and you know how to use GitHub.

Go to <https://github.com/new> and create a new **private** repository (it's free).

Then just copy/paste the example given by GitHub; something like:

```bash
$ echo "# Sieve generator" >> README.md
$ git init

$ git add README.md

$ git commit -m "first commit"

$ git branch -M main

$ git remote add origin XXXXXXXXXXXX     '\<-- put here your own path

$ git push -u origin main
```

Now push the other files:

```bash
$ git add .

$ git commit -m "Initial upload"

$ git push
```

Back to github, you'll obtain a repo like this one:

![GitHub repo](./images/github_repo.webp)

So, now, each time you'll update the list of domains (in the `patterns.json` file), our objective is to ask GitHub to run the `./generate.sh` script itself, generate the `build/roundcube.sieve` file, and publish it to your FTP account.

### Adding GitHub Actions

> **Optional chapter:**
>
> Skip this chapter if you don't want full automation.

Go back to your computer, jump back in the `~/sieve-generator` folder and create a new file called `.github/workflows/build_and_deploy.yaml`.

Here is the content for that new file:

```yaml
name: Create roundcube.sieve and deploy

on:
  push:
    branches:
      - main

  workflow_dispatch:
    inputs:
      version:
        type: string
        required: false
        description: 'Version of jq to install'
        default: '1.7'
      force:
        type: boolean
        required: false
        description: 'Do not check for existing jq installation before continuing.'
        default: false

permissions:
  contents: write

jobs:
  deploy:
    name: Create roundcube.sieve and deploy
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6

      - name: 'Setup jq'
        uses: dcarbone/install-jq-action@v2
        with:
          version: '${{ inputs.version }}'
          force: '${{ inputs.force }}'

      - name: 'Build'
        run: |
          ./generate.sh

      - name: Push files
        uses: SamKirkland/FTP-Deploy-Action@v4.3.5
        with:
          server: ${{ secrets.ftp_server }}
          username: ${{ secrets.ftp_login }}
          password: ${{ secrets.ftp_password }}
          local-dir: ./build/
```

I won't go into detail about this file, but it asks GitHub to:

1. Install JQ on its server,
2. Run the `generate.sh` script and thus build the `build/roundcube.sieve` file
3. Use FTP-Deploy-Action to push the file on your server.

As you can see, there are three `secrets`.

Back to your browser, go to your GitHub repository you've created in the previous chapter.

Click on `Settings` then `Secrets and variables` and finally, click on `Actions`

![GitHub secrets](./images/settings_actions_secrets.webp)

Click on the green button `New repository secret` and create the first one called `FTP_SERVER` (seems to be in uppercase). Put there the name of your FTP server like `node99-xx.N0c.com` (get the name from your *My planethoster* page).

You'll need to add two new secrets `FTP_LOGIN` and `FTP_PASSWORD` but, before, create a new FTP user.

Go to <https://mg.n0c.com/files/ftp-accounts> and create a new user. Use a very explicit name like `deploy_spam_roundcube_xxx` and define a very strong password (30 chars or more).

Restrict the path for that user to, exactly, where the sieve file is located on your FTP. As said in [PlanetHoster's N0C infrastructure](/blog/planethoster-n0c-spam), for me, it's `/mail/avonture.be/christophe/mail/avonture.be/christophe/sieve`. Make sure to find the path for you.

> **Note:**
>
> In case the `sieve` folder didn't exists yet, the best way to create it, I think, is to manually create a filter in Roundcube. Here is the [official PlanetHoster documentation](https://kb.n0c.com/en/knowledge-base/redirecting-emails-with-a-filter-in-roundcube-2/).

Ok, so, now, you've a new FTP login, its very secure password, and access restricted to only your sieve folder.

Copy the login and the password in the GitHub secrets page.

You'll then have this:

![Secrets](./images/secrets.webp)

## Everything is now in place for automation

If you've followed the GitHub chapters, now, everything is set up.

Back to your computer and edit the `patterns.json` file and add a new pattern, f.i. `newsletters@*`.

Push the change to GitHub (`git add . ; git commit -m "chore: new pattern" ; git push`).

Just for the curiosity, go to GitHub - Actions tabs:

![Build and deploy](./images/build_and_deploy.webp)

If you've configured everything correctly, you should get this:

![FTP Deploy Action](./images/ftp_deploy_action.webp)

## Conclusion

From now on, all you have to do is copy/paste the email addresses you receive spam from, see if you can standardize them as much as possible (by using the asterisk like in `newsletter@*`) and then copy/paste them into the JSON file that you just have to upload to GitHub.

Fighting spam has become a one-line commit, and the blocklist is versioned: you can see when a domain was added, and remove it just as easily.

Et voilà!
