ejabberd 16.12

After ejabberd 16.09 which brings a lot of improvements, ejabberd 16.12 includes a big refactor we have been preparing for long time, and a couple of new features that pursue the central effort on API, commands and web integration started early this year.

ejabberd 16.12 includes: a new API permissions framework for commands, a new more reliable BOSH module, major code refactor, more integration and unit tests, improved support for Erlang/OTP R19, compatibility with rebar3 build system, many fixes and optimizations, and Docker containers.

Thank you to all users who have send back their feedback on ejabberd 16.12 beta version. This is helping a lot make ejabberd better.

We hope you will enjoy this new release and we wish you a merry Christmas ! Stay tuned for even more new features in 2017 !

New BOSH module

This new BOSH implementation improves BOSH users experience, as it’s more robust and efficient and overall over lower latency.

New API permissions framework

This new framework is one of the big changes in this release. It makes our API very flexible, allowing administrators to fine grain access to some users to a group of API, depending on the path used to call a given command.

While it should make it possible to implement any kind of access control on top of ejabberd management API, it also makes it much easier to configure ejabberd API for simple needs.

ejabberd API operations are organised around the concept of commands. ejabberd modules provide many commands, but the mechanism is generic and any module can provide its own set of commands.

All commands can be exposed through interfaces. Available interfaces are: ejabberdctl command-line tool, ejabberd ReST API and ejabberd XML-RPC API. The XML-RPC API still works but is deprecated in favor of the ReST API. You should migrate to ReST if you are using it.

Finally, ReST API can be accessed through two authentication mechanisms: HTTP Basic Authentication and OAuth 2.

Each command interface can have different restrictions based on how exposed and sensitive the commands are.

Note that the following configuration snippets assume ejabberd API listeners are properly configured, as defined in API listener configuration

By default, when no api_permission option is provided, ejabberd would use the following default permissions:

api_permissions:
 "console commands":
   - from:
     - ejabberd_ctl
   - who: all
   - what: "*"
 "admin access":
   - who:
     - admin
     - oauth:
       - scope: "ejabberd:admin"
       - admin
   - what:
     - "*"
     - "!stop"
     - "!start"

This will grant access to all commands when ejabberdctl is used (this is what “console commands” group does), and additionally each command that is authenticated by user that match acl auth from any source will be able to call all commands except start and stop.

It’s possible to extend this by adding new section like this:

api_permissions:
 ...
 "allow statbot to get server stats"
   - who:
     - user: "statbot@server.example.com"
   - what:
     - connected_users_number
     - num_active_users

This will allow user statbot to execute commands connected_users_number and num_active_users.

You can even relax the need to authenticate on ReST API, for example, if you only want to restrict the API to localhost usage, for admin tasks:

api_permissions:
  "local admin can use all commands"
    - who:
      - ip: "127.0.0.1/8"
    - what:
      - "*"
      - "!stop"
      - "!start"

As you can see, for simple cases, the API permission management is now very simple. Yet, the system is powerful and can accommodate your most advanced needs.

For more details, see full api permission documentation.

You can now rely on ejabberd hundreds of commands to integrate in a precise and secure way with your back ends.

Major code refactor

Finally, the most important change (in terms of line of code and impact on ejabberd code base) is refactor of XMPP packet handling in the entire code base. This improvement makes code simpler, safer, and smaller. ejabberd now uses a dedicated XMPP library, which helps developers packing/unpacking XMPP packets.

Example is always better than words:

Don’t build entire XML element, but rely on common “templates”

Before:

#xmlel{name = <<"identity">>,
       attrs = [{<<"category">>, <<"pubsub">>},
                {<<"type">>, <<"pep">>}]}
IQ#iq{type = error, sub_el = [Error, SubEl]}

After:

#identity{category = <<"pubsub">>, type = <<"pep">>}
xmpp:make_iq_result(IQ, Error);

Match packets efficiently

Before:

normal_state({route, From, <<"">>,
       #xmlel{name = <<"message">>,
              attrs = Attrs,
              children = Els} = Packet},
       StateData) ->
   case is_user_online(From, StateData) of
     true ->
       case fxml:get_attr_s(<<"type">>, Attrs) of
         <<"groupchat">> ->
              ...

After:

normal_state({route, From, <<"">>,
     #message{type = Type,
              lang = Lang} = Packet},
     StateData) ->
   case is_user_online(From, StateData) of
      true when Type == groupchat ->
          ...

Don’t use macros to build error response

Before:

jlib:make_error_reply(Packet, ?ERRT_BAD_REQUEST(Lang, Txt));

After:

xmpp:make_error(Packet, xmpp:err_bad_request(Txt, Lang));

The code is safer, as all XMPP processing in the core of ejabberd can be typed-checked through static analyzer like Dialyzer.

Overall, this change paves the way to improvements of ejabberd API and will make it much more pleasant for contributors to write new modules and extensions. Improving the contribution documentation is next on our list and you are welcome to join the effort.

This big refactor impacts the entire code base. Now that all tests are successful, we provide an early beta before the final 16.12 release. 16.12-beta1 is not yet intended for production use. We are waiting for your feedback to check that special cases did not sleep through our careful refactor. Please, test it and send us feedback in your use case, so that we can catch the tricky bugs before the final version is released.

Docker containers

Ejabberd docker container contributed by Rafael Römhild is now included in ejabberd sources.
We also provide simple containers to build and run ejabberd from sources. Thanks to this image, you can build ejabberd with dependencies provided in our Docker image, without the need to install any specific tools (beside Docker) directly on your own machine. This works on Linux, FreeBSD, MacOSX, Windows.
See more in this blogpost.

External modules

Ejabberd’s ext_mod has been updated to build external modules with the new xmpp library. You should port your contribution to make it compile and work with ejabberd 16.12. While this will improve your code, it will also offer possibility to implement post_install and pre_uninstall directly in your module. By exporting these functions, ejabberd will call them after module has been successfully installed and right before uninstalling it to allow any custom tasks at these stages.

Windows installer

The installer now moves ejabberd data files (configuration, database and logs) in %ProgramData% instead of %APPDATA%. This move fixes running ejabberd as a service. As a result, installer now registers ejabberd service by default. If you’re upgrading ejabberd from an older version, the installer will automatically move your configuration from %APPDATA% to %ProgramData% to keep your existing data without any manual operation.

Changes

This is just a summary of the most relevant ones:

API / integration

• New API permissions framework

Commands

• Add configurable weight for ejabberd commands
• add_rosteritem: Support several groups separated by ;
• create_rooms_file: Fix reading room jids from file
• delete_old_messages: Fix command for SQL backends
• send_message: Don’t duplicate the message
• Remove obsolete remove_node command (use leave_cluster)
• Fix reload_config
• Cleanup mod_admin_extra, add few functions
• Expose unregister API command

Core XMPP

• New BOSH module
• Use fxml_gen XML generator
• Use our new stand-alone XMPP library instead of jlib.erl
• Don’t let MAM messages go into offline storage
• Add xdata generator for XMPP data form
• Get rid of excessive (io)list_to_binary/1 calls

HTTP

• Add authentication support to mod_http_fileserver
• ejabberd_http: Handle missing POST data gracefully
• Use inets instead of lhttpc in http_p1
• Add http_p1.erl, rest.erl, and oauth2 ReST backend for OAuth2 tokens

MUC

• Create room on configuration request as per XEP-0045, 10.1.3
• Ensure that presence_broadcast room option is stored
• Fix conference disco#items when running multiple virtual hosts
• Fix Result Set Management (RSM) for conference disco#items
• Introduce muc_invite hook
• Make the constant MAX_ROOMS_DISCOITEMS configurable
• mod_carboncopy: Don’t copy MUC private messages

MUC/Sub

• Store the flag “Allow Subscription” room option in database
• When getting list of subscribed rooms, also check temporary ones
• Add password support in muc_subscribe
• When unsubscribes, check if room should get closed

Pubsub

• Enforce pubsub node removal

Relational databases support

• Append ; to privacy_list_data exporting lines
• Improve relational database import

Build

• Make build system compatible with rebar3
• Produce ejabberd.service and fix for systemd usage
• Cleanup ext_mod and fix compilation path
• Fix compilation of external module with new xmpp lib
• Add optional post_install and pre_uninstall hooks for external module

Miscelanea

• Add docker container from Rafael Römhild
• fast_tls is now compatible with LibreSSL
• Delete obsolete module mod_configure2
• Rename #error{} record to #stanza_error{}
• Bugfix: Ignore offline sessions in ejabberd command statistics
• Bugfix: Don’t let MAM messages go into offline storage

Feedback

As usual, the release is tagged in the Git source code repository on Github.

The source package and binary installers are available at ProcessOne.

If you suspect that you’ve found a bug, please search or fill a bug report on Github.