Overview

This is a extendable chatbot written in python. Currently it only supports XMPP/Jabber, but Matrix can surely be made working again and other protocols added.

Its source code should be available here (soon).

1. Features

The bot knows its users, who can configure their language (currently german and bad english) and some details.
Fine grained permission system for every user controls which extensions are available, which data can be seen (with or without history) and which data can be written to.
To secure against unintended or unauthorized changes, you can enable a kind of write protection which comes into play when executing command scripts or instantiating rules. A chatting user is asked for a passcode in this case (help).
Users can create and use macros.
Depending on the configuration, users may just use the bot, maybe with enforced mutual presence subscription, or need to be invited by the bot’s admin.
A MUC (multi user chat) can be a user, too.
It includes kind of a time series database. Data is fed into this database from mqtt or from the bot itself. For every value included you can configure how many datapoints should be kept in a circular buffer, and how to compress older datapoints. Data can be queried/downloaded/plotted. There also is a http server which can provide access to the data (used with some html generator and javascript code to provide a web interface to home automation, for example).
Core functionality can easily be extended by the admin, even by using the chat interface. A single file provides a script (question or command from a users point of view) including its help and meta information. The same holds for rules, which can execute code because of a time condition or when (certain) values are changed.
Integrated help. All those help texts can be read here, too.
Some precautions have been taken to prevent users from overloading the bot’s server (per user config by using the permission system).
A dialog system can be used to provide functionality without having to know syntax (and is used in some extensions)

2. Use case examples

read data from the internet, distribute alerts
implement an enhanced wetterbot or jabrss
Monitor a system’s log files and usage data (CPU, RAM, disks)
Connect to a home automation (or other) system (or even be its central controller) by reading from and writing to a mqtt broker. Useful to test new sensors, notify about important events, configure the system. No need to access/open up your network from the outside (by using a VPN or something).
Make your data available to others: the initial use case for my public bot was to help someone living nearby by notifying him when outside dew point was low/high so he knew when to open/close the windows to his moist basement.
Monitor changes in the family’s CalDav calendar and notify about changes (in use but unpublished)
Without any extension or connections to other services, it can be used to test XMPP things:
- echo test
- periodically send messages including timestamp and message counter to test notifications or general receive behaviour
- notify about presence changes

3. Try it out, chat about it

A publicly available instance is running at bex@jotwewe.de (you need to make sure mutual presence subscription exists, otherwise it won’t talk to you. You can turn it off again once it has accepted you as a user).

There are two MUCs where you can discuss it and also try it out (because it is a member there): english support and test chatroom and german support and test chatroom.

This bot does only use code which is available in the base package. In addition to base features it receives some data via mqtt which you can view, monitor (using included rules), download, and plot. The data currently available to everyone is enviromental data from Karlsruhe, Germany.

From outside of my home:

K3Lueft_T_0     Temperature, °C
K3Lueft_T_TP_0  Dew point, °C. Value is too high usually
K3Lueft_RH_0    Relative humidity, percent. Value is too high usually
K3Lueft_u12_3   Rain sensor. <1950 means it is wet
Logger_P_PV     Current power of solar electric cells

From https://web1.karlsruhe.de/wetter:

Mome1_T_LUBW           Temperature
Mome1_T_LUBW_TP        Dew point
Mome1_RH_LUBW          Relative humidity
Mome1_RegenSum_LUBW    Rain

4. Installation

As of July 2023, I am using this bot with python 3.7.3 and 3.11, so using something older or newer than that may work…or not.

Installing prerequisites on a system based on debian linux:

sudo apt-get install python3-aiohttp python3-dateutil cython3 python3-levenshtein wget python3-slixmpp python3-slixmpp-lib python3-paho-mqtt python3-setuptools python3-systemd

In order to produce plots you need one of those (depends on whether you have/want X11 on the system or not):

sudo apt-get install gnuplot
sudo apt-get install gnuplot-nox

Instead of directly modifying a bunch of config files, edit the file 10_config (no need to edit it in case of a manual test on localhost) and run 20_etccfg.sh afterwards. This will take of settings which have to be consistent across some files and provide some useful information.

Edit the configuration file:

nano mome/cfg/mome_SYS_NAME.ini  # or whatever your favorite editor and SYS_NAME is...

You need to enter xmpp addresses of bot and admin in the first two sections for a basic test.

If all those files reside on your desktop, but you want to run on a server (raspi?), transfer them to the target. 30_upload_rsync.sh might need a little editing (ssh port) but should otherwise do the job for you.

Go ahead on the target system: some code needs to be compiled (which takes some minutes on a small single board computer):

cd mome ; python3 setup.py build_ext --inplace ; cd -

4.1. Logging to a RAM disk

You can skip this section in case of a test with default settings and manual start.

On a single board computer using a SD card for mass storage, you like to prevent writing log files to this card in order to extend its lifetime. So /var/log resides in a RAM disk (https://github.com/azlux/log2ram).

When running this bot as a user who does not have privileges to write there, you could use etc/systemd/pymsgbot_logdir.service which has been created before.

sudo cp etc/systemd/pymsgbot_logdir.service /etc/systemd/system/

sudo systemctl daemon-reload
sudo systemctl enable pymsgbot_logdir
sudo service pymsgbot_logdir start

I don’t know whether this is a truly dumb idea, but it works.

4.2. Manual start

Try to start the bot:

cd ./mome
./mome.py cfg/mome_SYS_NAME.ini

The admin should receive a message from the bot now and be able to chat with it.

4.3. …and automatically on system startup

After the previous steps, a usable etc/systemd/pymsgbot.service should exist.

sudo cp etc/systemd/pymsgbot.service /etc/systemd/system/

sudo systemctl daemon-reload
sudo systemctl enable pymsgbot
sudo service pymsgbot start

4.4. In case of errors

4.4.1. Plots not working

In case the bot won’t send plots or other files and you see something like this in the log:

File "/usr/lib/python3/dist-packages/slixmpp/plugins/xep_0030/disco.py", line 326, in get_info_from_domain
  info_futures, _ = await asyncio.wait(
                    ^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/tasks.py", line 415, in wait
  raise TypeError("Passing coroutines is forbidden, use tasks explicitly.")
TypeError: Passing coroutines is forbidden, use tasks explicitly.

…you possibly have slixmpp < 1.8.4 and python >= 3.11 on your system and need to update. Installing from pip is not easy on a current debian system due to https://www.debian.org/releases/stable/amd64/release-notes/ch-information.en.html#python3-pep-668 but this is a possible way out, although not the one that’s recommended:

sudo apt-get remove python3-slixmpp python3-slixmpp-lib
sudo apt-get install python3-aiodns python3-pyasn1 python3-pyasn1-modules python3-emoji python3-pip
sudo pip3 install 'slixmpp==1.8.4' --break-system-packages

5. Let it receive data via mqtt

Install a mqtt broker:

sudo apt-get install mosquitto mosquitto-clients

Maybe edit its mosquitto.conf:

change persistence_location to somewhere on the RAM disk

It should already be running afterwards (sudo service mosquitto status).

Edit the config:

nano mome/cfg/mome_SYS_NAME.ini  # or whatever your favorite editor and SYS_NAME is...

…and edit the mqtt section to connect it to mosquitto:

[mqtt]
host = localhost
port = 1883
# username and password: empty in case of anonymous access
username =
password =

Add mosquitto.service to the Wants= line in your /etc/systemd/system/pymsgbot.service

Restart the bot:

sudo service pymsgbot restart

Send some data via command line:

mosquitto_pub -t "mome/s/devicename/u16/descr" -m 42
mosquitto_pub -t "mome/s/devicename/T/descr" -m 21.3

…or using python:

import paho.mqtt.publish as publish
publish.single("mome/s/devicename/T/descr", "21.7")

Send this to the bot via xmpp and expect it to answer with the values you recently sent:

vlist dev.+

5.1. mqttbridge

It does only handle Tasmota switches at the moment. Their device name has to begin with "tasmota_". Apparently this string is configurable? See topic at https://tasmota.github.io/docs/MQTT/#configure-mqtt-using-webui

Start manually

./mqttbridge/src/mqttbridge.py --tasmota

or permanently via systemd:

sudo cp etc/systemd/mqttbridge.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable mqttbridge
sudo service mqttbridge start

6. web interface

Edit mosquitto.conf (sudo nano /etc/mosquitto/mosquitto.conf):

listener 1883
protocol mqtt
allow_anonymous true

listener 9001
protocol websockets
allow_anonymous true
http_dir /directory/containing/static/files

…but 20_etccfg.sh has told you about the directory to use for http_dir before.

Download libraries:

cd ./webui ; ./download_libs.sh ; cd -

Create static website data:

cd ./webui/src ; ./build.sh ; cd -

Restart mosquitto:

sudo service mosquitto restart

If that fails: see its logfile, maybe this is /var/log/mosquitto/mosquitto.log. Is it able to read from http_dir?

7. Todo list

The list of ideas is long and growing. Some of the next or important steps:

screenshots?
document cfg/values.ini
document how to use the website generator
add shelly switches to mqttbridge
add rules/scripts to read/write shelly switches via http
add solar inverter example
add wildcard status display (raw) to webui
iron out presence subscription stuff
internal clean up
publish android app for alarm notifications (works on top of conversations and surely other apps)
values should have a unit of measure

Letzte Änderung: 16.07.2023 10:57

Jens W. Wulf