Angie: plugin-based publish and subscribe is a huge leap toward rich presence
The XMPP Standards Foundation has defined a standard protocol to add publish and subscribe mechanisms to XMPP (XEP-0060). It is currently not largely used because of one missing implementation element: it should be implemented as the base of an application server to spread adoption among service developers. The next version of ejabberd will introduce a plugin-based publish and subscribe implementation
Introduction
The XMPP Standards Foundation has defined a standard protocol to add publish and subscribe mechanisms to XMPP (XEP-0060). It is currently not largely used because of one missing implementation element: it should be implemented as the base of an application server to spread adoption among service developers. The next version of ejabberd will introduce a plugin-based publish and subscribe implementation that will open up many opportunities for rich presence applications. The PEP (Personal Eventing Protocol) is currently being adapted to rely on the pluggable ejabberd pubsub API.
Pubsub node types: Why a pluggable API ?
Pubsub has the notion of node types built-in. The ability to write plugin-based pubsub modules is here from the start. However, we think that the node types should not be limited as in the current specification, but be free to add by applications or services developers. For the client, it does not matter as the API will be the same, but for the application developer, the power at hand is much bigger.
For now XEP-0060 says that you can have two types of node: Leaf or collection. If we remove this limitation, you open up a world of opportunities for application developers. The API is strictly the same for the client point of view, but on the back-end, different type of nodes can react differently.
For example, the collections could be implemented on the server side as a pubsub plugin. When you publish to a node, the API can define that you warn your parent (in a tree like hierarchy) about the event . If the parent is a collection, it can implement the corresponding method as a publish and in turn pass the publish event to its father. You can also implement a new node type plugin that does the reverse very easily: a node of type dispatch can republish its event to its children, maybe with some filtering rules or conditions.
Let’s take another example: The pubsub approach let you build complex event based application by mixing backends. Some plugin nodes can be stored in one or several relational databases. Some other node types can be simple gateways to backend applications whose goal is to publish events for themselves or on the behalf of users (Time tracking applications, reporting tool, accounting or invoicing system, etc). Users can publish events (and update backends) or subscribe to those event transparently. This simple approach is a huge leap toward rich presence as many applications can be plugged as backend to plug event based engine as ejabberd backend.
We are currently working with partners to provide actual plugins for ejabberd pubsub plugin architecture. Do not hesitate to contact us if you want to plug your application with ejabberd. When ejabberd 2.0 will be released, the integration potential will be big :)
The beauty of that approach is that the API does not change on the client side. No matter what the pubsub node type does in the background, the pubsub event API is still compliant. The clients send subscriptions, publish messages, and receive events the same way, no matter the node type you are using.
Plugin for hierarchy
The node type is not the only plugin you can write in our pubsub architecture.
You can also write a plugin to store / manage the pubsub node hierarchy. This make it possible to store the hierarchy in a relational database of your choice. It makes it possible also to create a purely virtual pubsub nodes tree.
The virtual pubsub tree approach can lead to large pubsub performance improvements when the node hierarchy can be defined by rules (For example, each user as a node geoloc and tunes). There is no need to perform database operation in this case as the tree is purely read-only and pre-defined. This has proved to be an important addition for our large scale pubsub deployments.
Performance rewrite
At the same time, we have rewriten the pubsub module to increase raw performance. The new implementation should be much faster, sustaining a much higher messages throughtput. Some of our mesurements show a 4 folds performance increase for site doing an heavy use of pubsub.
Compliance update
The new version of pubsub is also getting a compliance update to the latest version of the specification (XEP-0060 version 1.9). Clients will have to update their API to this new version.
Where are we now ?
We have developed this pubsub plugin-based version with one of our high-profile customer and partner (more to come soon). We have been granted an agreement to include this new code into the main ejabberd code base. This module is currently ready for production, even if without a doubt the plugin API will still be improved over time. Before being able to push this pubsub module to ejabberd trunk we need to adapt the PEP implementation to make use of the new feature.
We are only a few weeks away from a final release. What we describe here is not science fiction, but is here now.
We are sure that you will not be disappointed by the upcoming ejabberd 2.0. It is a very ambitious release, providing a mix of performance, stability and scalability improvements along with innovative XMPP features.
References
See our previous posts on Angie, the code name for the upcoming ejabberd 2.0: