ejabberd 18.12
This new ejabberd 18.12 release includes new major features, but also several improvements and bug fixes.
New features
XML Compression in message archive storage
Message Archive Management is a very nice features of XMPP. It allows your users to store their message history directly on the XMPP server. It is pretty handy, but as the feature set used by clients grows, the overhead of the message in the data store grow as well. This is due to the various XML flags that are added to the message, serving both as metadata for the server or the message recipient.
We have added a custom compression to ejabberd to reduce the message size in the database. This mechanism uses custom compression algorithm operating on xml trees, which knows about most common tags used by xmpp, and allows better compression ratios that generic compression algorithms like zlib.
For example, we will compress this:
<message to="bob@server.com/test" from="anne@server.com/web" type="chat">
<body>Hello there!</body>
<request xmlns='urn:xmpp:receipts'/>
<thread>thread1</thread>
<message>
to: (everything represented as […] is actually encoded as single byte)
[jabber:client message]["to"=recipient with suffix]test["from"=sender]["type"="chat"][children list]
[jabber:client body]Hello there![end of a tag]
[urn:xmpp:receipts request][end of a tag]
[jabber:client thread]thread1[end of tag]
[end of tag]
This feature can be enabled by adding compress_xml: true option inside mod_mam module options. With this feature enabled, new messages added to archives are compressed. Depending on message pattern, the gain can vary. Your storage space can take between 35% of the original database size in the best case for typical standard message. In the worst case, the storage was still 80% of the original size, which is already an interesting saving. The worst case is typically triggered on large messages body, for example when the client use end-to-end encryption storage like OMemo (Encryption adds a significant data chunk that cannot be compressed).
Proxy protocol support
Ejabberd now supports version 1 and 2 of the proxy protocol.
This feature allows ejabberd to get clients real ip addresses and ports when running behind a proxy server which supports this protocol. It means that real IP addresses can be used for access control directly inside ejabberd, even when running behind a proxy.
That feature can be enabled by adding option use_proxy_protocol: true in listeners sections which use ejabberd_c2s or ejabberd_http module. It’s important to make sure that this option is enabled on listeners that are used ONLY by proxy server which supports this protocol, otherwise connections will be closed due to protocol mismatch.
MUC Self-Ping server optimisation (XEP-0410)
When using Multi-User Chat across XMPP federated servers, the client view about its MUC rooms status is not always properly in sync with the server view (for various reason like server-to-server connection loss, crashes, restart, etc.).
It is then recommended that the client sends periodically a ping probe to know if he is actually a participant in the remote server chat room or if it should rejoin the room. To make that feature totally reliable, proper collaboration of the server is required to skip a roundtrip in the self-ping flow and reply directly on behalf of the client. That way, you can expect to get a proper error reply or a ping response, making the features more predictable and easier to support on the client. It is also more efficient for the server.
The server optimisation is automatically enabled on ejabberd 18.12.
This self-ping packet is supposed to be send from the client to room that are hosted on remote servers, not on his own server.
This kind of probe is not required in the context of MucSub, as the goal of MucSub is to provide a way to interact with MUC rooms that is not based on presence.
We hope you will update your client to improve the reliability of your chat session in remote chat room.
Bookmarks Conversion (XEP-0411)
Ejabberd now supports a new extension which allows migrating to PEP-based bookmarks while keeping compatibility with clients that still use private XML storage by syncing both mechanisms. The synchronisation is performed whenever a client updates one of the two bookmark stores. However, until this happens for the first time, the bookmark stores of a given user won’t be synced automatically. A new command can be used to export the contents of the private XML storage to PEP manually.
In order to do this for all users of all XMPP domains, a shell loop such as the following could be executed:
$ for host in $(ejabberdctl registered-vhosts); do
ejabberdctl registered-users "$host" | while read user; do
ejabberdctl bookmarks-to-pep "$user" "$host"
done
done
Note that this might take a while if the number of users is large. Also note that this will overwrite any preexisting PEP bookmarks. However, if this is not performed, users might end up with empty bookmark lists after upgrading to the new ejabberd version, as clients might rely on PEP bookmarks once they detect that ejabberd’s XEP-0411 support.
Changes overview
Important changes to web access
ejabberd now supports a TLS-secured access to its web and admin panels on a new port 5443 (previously, 5280).
This means the new URL to log into the admin dashboard is https://localhost:5443/admin, if installed on localhost, of course.
This is only relevant for new installation. The config file is not modified and it will not change anything for existing users.
Configuration changes
Since ejabberd 18.12, the option ca_path is deprecated and has no effect anymore. If needed, you MUST use the option ca_file instead.
Database changes
The MySQL database schema has changed for the pubsub_item table: the ‘payload’ field has changed from TEXT to MEDIUMTEXT. Upgrade process is described in the documentation.
Download and install ejabberd 18.12
If you intend to upgrade your server, it’s recommended to wait for ejabberd 18.12.1 bugfix release, which will be available shortly. For brand new installations, go ahead with ejabberd 18.12.
The source package and binary installers are available at ejabberd XMPP & MQTT server download page.
As usual, the release is tagged in the Git source code repository on Github. If you suspect that you’ve found a bug, please search or fill a bug report in Issues.
Admin
– Add new command unban_ip
– Enable mod_proxy65 in default config
– Avoid using * in ‘certfiles’ option in default config
– Remove references in default config to http_poll; it was removed long ago
– Report available options in lexical order
– Fix missing file priv/lua/redis_sm.lua
– Fix push_roster description and provide UTF8 example
Documentation
– New file CONTRIBUTING.md
– New file CONTRIBUTORS.md
– Rename README to README.md as the markdown version is the preferred one now
Multi-User Chat
– Get rid of ‘catch-all’ statements
– Replace dict with maps to improve performance and memory consumptions of large MUCs
– Recover logging of user joining room, lost in 32de9a56
– Affiliations other than admin and owner cannot invite to members_only rooms
– mod_mam: Archive messages with type=normal and pubsub payload (like MucSub messages)
– Advertise disco#info and disco#items by MUC room
PubSub and PEP
– Only advertise conversion feature when mod_pubsub is loaded
– Don’t suppress notifications on PEP node removal
– Fix explanation in configuration file for enabling OMEMO support
– MySQL: Use MEDIUMTEXT for PubSub payload, so it supports up to 16 MiB
Tests
– Improve tests for mod_private
– Fix test case for invalid language
– Fix mod_http_apt_test
Miscellanea
– Add ejabberd_stopping hook, which allows modules to detect when ejabberd is stopping
– Use newer fast_tls that fixes some issues with tls1.3
– ejabberd_captcha: Add specific Var names to CAPTCHA fallback form fields
– ejabberd_http: Handle some malformed URL requests
– ejabberd_http: Do no add 127.0.0.1 address to trusted_proxies list by default
– ejabberd_logger: Disable debug logging of xmpp properly
– ejabberd_pkix: Move certificates processing code to pkix application
– ejabberd_regexp: Support Unicode
– ejabberd_stun: Add forgotten TURN options to validator
– ejabberd_sql: Relax result matching to fix pgsql keepalive
– mod_avatar: Reduce log level for too large avatars
– mod_http_api: Don’t preprocess arguments, format_args verifies and prepares them
– mod_http_api: Recognize not_exists error in http_api
– mod_multicast: Local stanzas are routed one by one, not by multicast
– mod_multicast: Track presences sent via a multicast service
– mod_private: Add support for XEP-0411: Bookmarks Conversion
– mod_offline_sql: Fix PostgreSQL compatibility in remove_old_messages
– web access with TLS security on new port 5443