My eighth semester project at Aalborg University was about mobile development. I’ve been looking for excuses for a long time to dedicate some time to peer-to-peer Matrix, which extends Matrix in such a way that users may participate without having a separate server running for their messages.

You can find my code and report on GitHub.

Approach

I decided to look at current implementations, and as it turns out, there was already a branch of the Dendrite implementation which implemented P2P in pure Go! Great. Two pull requests later, and this is ready to go for Mobile.

Design

When you register with a Matrix server, you need to know where it is. For web clients, this typically isn’t a problem, since the web client automatically chooses a server for you. But on mobile, the default client is the main Matrix homeserver, and you’ll have to know the address of your custom instance if you want to register with that one. You want to connect to a server running on your phone, things get a little more complicated, since we can’t guarantee that the address will be the same every time the server starts. This is simply because there is a limited amount of available ports on any network device, including your mobile phone. Other apps may be greedy and hog some ports, which means that we must be flexible when we launch.

In other words, we need to make sure that whenever the server starts, the server gives the client its current address. For that purpose, I came up with the following design.

Riot for Android spawns a Dendrite process on the mobile phone.

This design shows two processes. The blue process is the client. When the user launches the client on their mobile phone, the client starts the server and awaits for synchronization. The synchronization is the mechanism by which the server notifies the client of its latest address.

go bind using Go Mobile

With that out of the way, we need to figure out how to get the Go binary running on Android. The Go team has developed a couple of ways for us to use Go in mobile development. The first approach assumes you’ll be writing your app entirely in Go, whereas the other provides Java bindings to your Go binary. For us, the second approach makes more sense, since we don’t want to reimplement the client side.

The Library

First of all, go bind requires a Go library, as opposed to a Go binary. In Go, the difference is the existence of a main function, which in binaries serves as the entry point of the application. Instead, we want to provide an interface which another binary (such as Riot for Android or RiotX) can use to initiate the server.

The p2p pull request on Dendrite did not include such a library per se, but rather a binary called dendrite-demo-libp2p.

However, if we extract the logic from the main function there, and replace it with a call to an init function in a separate file, then we’ve created a way for us to eat our cake and have it, too.

I’ve done exactly that and you can find the code here.

// main.go
func main() {
    // ...
    server.Init(
        *instancePath,
        *instanceName,
        *instancePort,
        simpleCallback{},
    )
}

// server.go
// Init starts the Dendrite server in p2p mode
func Init(path string, instanceName string, instancePort int, callback Callback) { ... }

You may notice that the function takes instancePort. This allows the caller to specifically set the port if necessary. If the value of the parameter is 0, then the server automatically chooses an available port.

In any case, once the server is listening on the port, it passes the address through Callback to the caller. We have to do it this way due to the limitations of gomobile, which can’t handle complex types across the boundary of Go and Java.

Instead, the caller has to pass an object which implements the Callback interface which is defined on the Go side, allowing us to receive the port on the Java side.

// Callback provides the the caller a way to respond to the port being set.
type Callback interface {
    SetPort(int)
}

Android Development and Android Studio

Personally, I’ve never been fond of a workflow that requires its very own IDE. I prefer to write all my code in vim, or the more heavyweight VSCode if I’m being lazy. However, Android Studio is probably the way to go if you want to keep your sanity when working with Android development. I encountered the following problems:

  1. Gradle wouldn’t recognize my imported .aar file.
  2. Android Studio (adb) wouldn’t recognize my OnePlus.

Both problems were solved rather trivially, but were time consuming and annoying nevertheless.

1. Importing .aar files

The common consensus seems to be that there are two methods of importing .aar files into Android. I followed this stackoverflow guide and it worked seamlessly, until I changed the name of my .aar file to something less generic, and Android Studio wouldn’t recognize my package. That’s because the .aar file contains its own package information, which is what you use to reference it from code. For example, I imported the Dendrite p2p lib as dendrite, even though the actual .aar file was server.aar.

Turns out it wouldn’t recognize dendrite, and I was left scratching my head for an embarrassing 6 hours trying to figure it out, until just writing import server.Server did the trick.

2. adb and Android on Linux

Sometimes Android Studio would recognize my phone, other times it wouldn’t. I’m still not sure what made the difference before I applied this solution, but now it works consistently.

TL;DR? Make sure your user is in the plugdev group.

$ usermod -aG plugdev $USER

Riot for Android

For my project, I decided to use Riot for Android. Although it is not as new and fancy as RiotX, it supports Dendrite out of the box, whereas RiotX does not.

Most of the code I wrote directly modified the Application class which handles the basic lifecycle events for Riot. This class is in VectorApp, and my code was within the onCreate method.

I began by implementing the Go interface in Java, which you can see below.

// The implementation of the Callback interface
private static class PortActivatedCallback implements Callback {
    private long mDendritePort = 0;

    // Not the most elegant solution,
    // this spin-locks until Dendrite is done initializing
    // and then returns the port.
    long getPort() {
        this.blockOnPort();
        return mDendritePort;
    }

    // The spin-lock implementation.
    private void blockOnPort() {
        while (mDendritePort == 0) {

        }

        Log.d(LOG_TAG, "Port accessed :" + mDendritePort);
    }

    // This is called by Dendrite once it's done initializing.
    @Override
    public void setPort(long l) {
        mDendritePort = l;
    }
}

Unfortunately, the Go/Java language bindings provided by Gomobile don’t support complex types, but they do support implementing interfaces, which we abuse here to allow the Go server to call setPort once the server is initialized.

The client can then call getPort, which returns instantly if the Dendrite server has already assigned a port, otherwise it blocks until the port has been set.

// VectorApp.java
/* ... */
import server.Server;       // Bindings for the Init method.
import server.Callback;     // Interface which we must implement.

public class VectorApp extends MultiDexApplication { // The Application class which handles the app lifecycle methods.
    /* ... */

    // I never actually used this.
    // But storing the thread handle allows you to control the Dendrite process in a more granular manner.
    // For example, by suspending it explicitly when the app is suspended.
    private Thread mDendriteProcess;
    private PortActivatedCallback mDendriteCallback = new PortActivatedCallback();
    private Uri mDendriteUrl;

    // Convenience getter for fetching the url to override the one stored with credentials
    public Uri getDendriteUrl() {
        return mDendriteUrl;
    }

    // onCreate is a lifecycle method which is called when the app is initialized
    @Override
    public void onCreate() {
        Log.d(LOG_TAG, "onCreate");
        super.onCreate();

        Runnable dendriteTask = () -> Server.init(
                getFilesDir().getPath(),
                "dendrite-server",
                0,
                mDendriteCallback
        );
        mDendriteProcess = new Thread(dendriteTask);
        mDendriteProcess.start();
        long port = mDendriteCallback.getPort();
        mDendriteUrl = Uri.parse("http://localhost:" + port);
        /* ... */
    }
    /* ... */
}

The rest of the code shows how I integrated this into the main application. I modified the onCreate method to spawn Dendrite in a separate thread and to block until the port was set. After that, the client could continue as before.

Dendrite Port Problems

With this code, I was able to start the app and register an account! And running the peer-to-peer server on my laptop and phone simultaneously allowed them to discover each other.

However, once the Dendrite server finds a new port, you’re out of luck, since the server address is stored with your credentials. In order to work around that, we need to modify all instances of using the address stored in the credentials with the address reported by Dendrite once it initializes. You can see how I did that on GitHub.

Conclusion

I’ve shown how you can port a Dendrite server to Android using Go bindings, and thanks to the hard work of Go and Matrix developers, it is relatively straightforward. I expect my code can be re-used with different Dendrite implementations, and eventually repeated for RiotX when it supports Dendrite.

Matrix is a very exciting project, with a dedicated and capable team of developers implementing features at an astounding pace. Keeping up this semester was difficult!

P.S. dendrite-demo-yggdrasil:

Not too long ago, the Dendrite team created a new peer-to-peer demo based on Yggdrasil. This is exciting, because it supports messaging over the internet, whereas the libp2p demo is limited to mDNS.

In theory, my process should be applicable to the Yggdrasil demo as well.