Migrating from Jabberd2 to Prosody

After trying (and failing!) to find a guide, how-to, or anything that could help me in the migration from Jabberd2 to Prosody on my personal server, I decided to write my own version here. I hope it can help other people who want to do this somewhat painful procedure!

Struggling with Jabberd2

When I installed my personal server, I chose Jabberd2 as my Jabber server. At that time, this choice seemed the most logical to me because of a few reasons:

1. It is written in C, which is my favorite language and, therefore, would make it easier for me if I ever wanted to fix something in the software (as it really happened afterwards).
2. Looking at the page comparing different Jabber servers at Wikipedia, I found that Jabberd2 was mature enough to be used seriously.
3. I knew some people who uses it in their own servers, and they said good things about it.

So, the decision seemed pretty simple for me: Jabberd2 would be my choice! And then the problems started…

The first issue I had to solve was not Jabberd2’s fault: I am using Debian Wheezy (stable) in my server, and Jabberd2 is only available for Debian Jessie (testing) or Sid (unstable). Therefore, I had to create my own version of the Jabberd2 Debian package (and all its dependencies that were not packaged) for Wheezy, which took me about 1 day. But after that, I managed to install the software in my server. Then, the configuration hell began…

Jabberd2 uses configuration files written in XML. They are well documented, with helpful comments inside. But they are confuse, as confuse as XML can be. Of course you have to take into account that it was my first time configuring a Jabber server, which added a lot to the complexity of the task. However, I feel compelled to say that the way Jabberd2 organizes its configuration files makes it a much more complex work than it should be. Nevertheless, and after lots of fails, I managed to set the server up properly. Yay!

Now, before I continue complaining, one good thing about Jabberd2: it has never crashed with me. I consider this to be something good because I am a software developer myself and I know that, despite our best efforts, bad things can happen. But Jabberd2 takes the gold medal on this one…

However… My confidence on Jabberd2’s security was severily damaged when I found that the SQLite backend could not encrypt the users’s passwords!!! I stumbled on this issue by myself, while naively dumping my SQLite database to check something there… You can imagine how (badly) impressed I was when I saw my password there, in plaintext. I decided to fix this issue ASAP. Hopefully next users will benefit from this fix.

After that, the bell rang in my head and I started to look for alternatives for Jabberd2. Though I still want to contribute to the project eventually (I am even working on a patch to merge all the database backends), I wanted to have a little bit more confidence in the software that I use as my Jabber server.

Meeting Prosody

Prosody came to my attention when I was setting up the server for our local Free Software group in Brazil. You can reach our wiki here (in pt_br, portugues) if you are interested. We wanted to offer a few services to our members/friends, and Jabber was obviously one of them. This happened after I discovered the bug in Jabberd2’s SQLite backend, so using Jabberd2 was not a choice anymore. We had heard ejabberd, which was being used by Jabber-BR (they recently migrated to Prosody as well), but the fact that it is written in Erlang, a language that I am not familiar with, has contributed to our decision of dropping the idea. So, the only choice left was Prosody itself.

Since I am brazilian, I also feel a little bit proud of Prosody because it is writte in Lua, a programming language designed by brazilians.

We installed Prosody on our server, and it was amazingly easy to configure it! The configuration file is writte in Lua as well, which makes it a lot easier to read than XML. It is also well documented, and I felt that they were more organized too: you have small configuration files splitted by categories, instead of one big XML to edit.

The modular structure of Prosody also impressed me. You can load and unload many modules very easily, generally just by (un)commenting lines on the configuration file. Neat.

Prosody also offers a command-line program to manage the server, which is really helpful if you want to automatize some tasks and write scripts. There is a little thing that still annoys me, which is the fact that this command-line program does not have a very useful “–help” command, but I plan to propose a patch to fix that.

And at last, but definitely not least, Prosody is also very robust, and have not crashed one single time with us. It runs smoothly in the server, and although I haven’t really compared the memory footprint of Jabberd2 and Prosody, I have nothing to complain about it too.

The Migration Process

Well, so after all this story, I think it is clear why I decided to migrate to Prosody. However, it was not an easy task.

Before we begin to understand the procedure needed to do the migration, I would like to say a few things. First, I would like to thank the guys at the Prosody chatroom, who were very helpful and provided several resources to make this migration possible. And I would also like to say that these instructions apply if you are running jabberd2_2.2.17-1 and prosody-0.8.2-4+deb7u2!! I have not tested with other versions of those softwares, so do it at your own risk.

The first thing you have to do is to convert Jabberd2’s database to XEP-0227. This XEP is very nice: it defines a standard format to import/export user data to and from XMPP servers. Unfortunately, not every server supports this XEP, and Jabberd2 is one of those… So I started looking for ways to extract the information which was inside Jabberd2’s SQLite database in a XEP-0227 compatible way. Thanks to the guys at the Prosody chatroom, I found a tool called sleekmigrate. It allowed me to generate a XEP-0227 file that could be imported into Prosody. Nice! But… I needed to extract this information from Jabberd2, and sleekmigrate could not do it. Back to the beginning…

It took me quite a while to figure out how to extract this info from Jabberd2. I was initially looking for ways (other than using sleekmigrate) that would allow me to extract this info directly from Jabberd2’s SQLite database, but could not find it. Only when I read that sleekmigrate could actually work with jabberd14 data directories directly, I had the idead to find a way to convert my SQLite database into a jabberd14 data directory, and then I found this link: it teaches how to migrate from Jabberd2 to ejabberd, and has separate instructions on how to do the Jabberd2 -> Jabberd14 conversion! Sweet!

The first thing you have to do is to download the j2to1 Perl script. I had to patch the script to make it work with SQLite, and also to fix a little bug in a SQL query; you can grab my patched version here. Save the file as j2to1.pl, and run the script (don’t forget to edit the source code in order to provide the database name/file):

$> perl j2to1.pl jabberd14-dir/
Converting user@host...
$>

This will convert the database from Jabberd2 to Jabberd14, and put the XML file of each Jabber user in the server into jabberd14-dir/host/. Now, you have a Jabberd14 version of your user data. Let’s proceed with the migration.

After following the instructions on the sleekmigrate page on how to set it up, you can run it on your Jabberd14 data directory in order to finally generate a XEP-0227 XML file that will be imported into Prosody.

$> ./sleekmigrate.py -j /path/to/jabberd14-dir/

This should create a file called 227.xml on your current directory, which is the exported version of the Jabberd14 data directory. As a side note, it is always recommended to check those generated files in order to see if everything is OK.

Right, so now you have 227.xml, which means you can finally import it into Prosody. Fortunately, Prosody has a tool to help you with that: it is a Lua script called xep227toprosody.lua. However, if you are doing this using Debian and the same versions of the softwares that I was using, you may find it harder than it seems to run this script without errors. Here is what I had to do.

First, grab a copy of version 0.8.2 of Prosody. I had to do that because using the latest version of the script was not working. I also had to build some POSIX module of Prosody in order to make everything work. To do that, unpack the tar.gz file, go to the Prosody source code directory, and do:

$> apt-get build-dep prosody && ./configure --ostype=debian && make

Only after I did that I could finally run the conversion script successfully. The script is locate inside the tools/ directory. To run it:

$> cd tools && lua ./xep227toprosody.lua /path/to/227.xml

And yay! I finally had everything imported into Prosody!!!! Then it was just a matter of finishing the server configuration, initializing it, and everything was there: my contacts, my user, etc.

Conclusion

The migration was not very easy, especially because Jabberd2 does not support XEP-0227. I found a bug against Jabberd2 that requested this feature to be implemented, but it was not receiving any attention. Of course, if Jabberd2 implemented XEP-0227 it would make it easier for people to migrate from it, but it would also make it easier to migrate to it, so it is definitely not a bad thing to have.

Despite some difficulties, Prosody made it really easy to import my data, so kudos to it. The Prosody community is also very responsive and helpful, which made me feel very good about it. I hope I can contribute some patches to the project :-).

So, that’s it. I hope this guide will be helpful to anyone who is planning to do this migration. Feel free to contact me about mistakes/comments/suggestions.

Happy migration!