Node-RED Pills 1 - How to use Node-RED with Docker Container

Hello folks! Today we’ll learn how to containerize Node-RED with Docker. I’ll breafily explain what Node-RED is, what can we use it for and how to structure a simple Dockerfile and docker-compose.yml and some strategies to persist Node-RED data.

GitHub Repository

If you want to give a quick look at the project solution or maybe you feel lost, click on the button below to be redirect on the GitHub Repository. Remember to switch on the node-red-pills-1 branch.

Github Repository

What is Node-RED

Node-RED is a flow-based programming language tool which describes an application’s behavior as a network of nodes. Each node has a well-defined purpose. Given some input data, a node does something with this data and then the result data is passed to the next node. Breaking down a problem into discrete steps (Node-RED nodes) creates a flow of nodes. It’s possible to look at those nodes and get a sense of what they are doing without reading the line of codes within each node.

Runtime/Editor

Node-RED consists of a Node.js based runtime. The way to code in Node-RED is using a flow editor accessible through web browser. You create your application dragging nodes into the workspace and wiring them together. With a click is possible to deploy back to the runtime where it is running without restarting the application. It’s possible to create and define some logic inside nodes using Javascript language.

Node-RED base project

A basic Node-RED project, usually consists of three files:

flows.json

Inside the flows.json file you’ll find all your flows that you’ve created with the editor.

settings.js

All the settings of your application will be defined inside the settings.js file. Here you can export some configurations to modify the appearance or the behaviour of your Node-RED application. The complete settings which can be configurated can be found here.

flows_cred.json

Because of security reasons when you export Node-RED flows, the properties identified as credentials inside nodes are skipped. The flows_cred.json file will be normally filled with credentials inserted inside communication nodes like the one used to connect to Database or MQTT broker. I will be using a Node-RED node called node-red-contrib-credentials to demonstrate the usage. Whenever you deploy your Node-RED application the flows_cred.json file will be overwrite and encrypted by Node-RED with some random string if you don’t export yours inside the settings.js file. This string is used also to decrypt the credentials to be read by Node-RED which are used to fill the nodes wherever they were used. If, for some reason, the credential secret string is lost or it changed, the flows_cred.json could not be decrypted so the nodes will not be filled with the credentials. What you have to do is to delete the flows_cred.json file, fill the credentials inside nodes and then deploy again your Node-RED application. A new flows_cred.json will be created.

I usually save my credential secret inside the docker-compose.yml or maybe inside the Azure Key Vault.

If you have missed my pills on how load Azure Key Vault secrets within your local environment, you can recover it clicking here.

Then what I do is something like this:

module.exports = {
    credentialSecret: process.env.CREDENTIAL_SECRET || false
}

If you export credentialSecret:false and then deploy you Node-RED application, every credentials saved inside flows_cred.json will not be encrypted.

Dockerfile

A tipical Dockerfile for a Node-RED project will look something like this:

FROM nodered/node-red:2.2.3-12

RUN npm install --unsafe-perm --no-update-notifier --no-fund --only=production

WORKDIR /usr/src/node-red

# install your favourites nodes here
RUN npm install node-red-node-email
RUN npm install node-red-contrib-credentials

# insert the copy command here so Docker will use the previous cached layers
COPY settings.js /data/settings.js
COPY flows_cred.json /data/flows_cred.json
COPY flows.json /data/flows.json

In order for the container to start with everything it needs, the necessary files are copied. It’s important to mantain the structure of these steps so, on a new Docker build, Docker will use the previous cached layers. The flows.json file will change more often compared to the settings.js or compare to a new version of some nodes.

Persistent Data Strategies

It’s possible to use two strategies to avoid to lose all the data inside flows.json and flows_cred.json and persist data whenever the Docker container is destroyed. Otherwise you need to export all the Node-RED flows from the editor everytime and then save them inside the flows.json file.

Bind mounting

To avoid the permission error for creating folders and files from the Docker Container to the Docker host, remember to create the ./node-red-data folder with flows.json, settings.js and flows_cred.json files inside of it by yourself.

This is the method which gives you the ability to version your files. You can create a bind mount specifing two folders named ./node-red-data:/data in the Docker Compose, where the first folder is inside the Docker host and last folder is inside the Docker Container. Inside of the node-red-data folder you will add the flows.json, flows_cred.json and settings.js files. At runtime Docker will create the binding from the Docker host and the Docker container and will overwrite all the files inside /data with everything it finds inside ./node-red-data. Doing so it’s possible to delete the COPY steps inside the Dockerfile because Docker will skip them during build time. To avoid versioning configurations, backup and lib folders and files coming from Node-RED, just add inside the .gitignore file those few lines:

# node-red
.config*
lib
*backup

Docker Volume

First of all you need to create a volume with this command:

docker volume create node_red_data

Using this strategy you will need the COPY steps inside the Dockerfile because Docke will prepopulate the /data folder with the data coming from the Docker image which was built with the flows.json, flows_cred.json and settings.js files.

Docker Compose

Below you can see the two Docker Compose files for every Persistent Data Strategy.

Bind Mounting

version: '1'

services:
  node-red:
    volumes:
      - './node-red-data:/data' 
    build:
      context: .
      dockerfile: Dockerfile-bind
    ports:
      - 1880:1880
    environment:
      - CREDENTIAL_SECRET=SUPER_SECRET_PASSWORD

Docker Volume

version: '1'

services:
  node-red:
    volumes:
      - 'node_red_data:/data'     
    build:
      context: .
      dockerfile: Dockerfile-volume
    ports:
      - 1880:1880
    environment:
      - CREDENTIAL_SECRET=SUPER_SECRET_PASSWORD

volumes:
  node_red_data:
    external: true

Try the Docker Compose

You can try the Docker Compose files with those commands:

# use the bind mounting strategy
docker compose -f docker-compose-bind.yml up --build

# use the docker volume strategy
docker compose -f docker-compose-volume.yml up --build

Happy Docker Composing!