Firefox Sync Server on an Uberspace

Update (11.01.2014): Currently, there is an error when trying to build the sync server using the latest version of virtualenv (1.11). It is possible to build it using an earlier version of virtualenv, e.g., 1.10.1, which can be installed by running:

easy_install virtualenv==1.10.1

Update (01.10.2012): In case you have an account on an older server on Uberspace you might not be able to create subdomains by yourself. Contact Uberspace Support or try the solution mentioned in this comment.

Update (22.08.2012): As t.animal pointed out, it’s necessary to change the path of a log file to run multiple sync servers on a single machine. This additional modification is now described in the Configuration section after the port settings.

Update (19.08.2012): Added information for users who do not have an own domain but instead use <username>.<server>.uberspace.de.

Update (15.08.2012): As suggested by typo3guy, I changed most commands to use paths relative to the home directory. E.g. instead of [server-full]$ vim etc/sync.conf it’s now $ vim ~/ffsync/server-full/etc/sync.conf. However, commands for building (like make) weren’t changed but in general they are used in a block of commands where the first command ensures that they’re run under the right directory.
Also the style for commands was changed to better distinguish them from text in pre-blocks which are mostly used to display options for configuration files.

This tutorial is based on Mozilla’s installation instruction for Firefox Sync (link) and the Wiki of Uberspace (link).

In this tutorial, shell commands have either the form [dir]$ foo if the command foo has to be run under the directory dir or $ foo if the directory can’t be confused (e.g. if a cd was made directly before) or if the directory shouldn’t matter. Generally dir is not the complete path but only the current directory’s name.

About Firefox Sync

Firefox Sync provides an easy synchronization service for bookmarks, browsing history, settings etc. across multiple Firefox installations on different devices. However, a synchronization server (sync server) is needed. Users can either create an account on a third-party server (e.g. from Mozilla) or run the synchronization software on their own server. The setup is very easy, at least if most required tools are already preinstalled or can be added quickly.

Creating the Necessary Directories

We’ll need three directories:

  1. to download and install the sync server. It’s not trivial to move the installation to another directory so this directory should be carefully chosen.
    mkdir ~/ffsync
  2. for the URL under which the sync server can be contacted via the Internet. By default, the server accepts new registrations by anyone. Either a secret URL can be chosen or, when all users are registered, registration of new accounts can be disabled as described at the end of this tutorial.
    If you don’t have an own domain, use „<username>.<server>.uberspace.de“ instead of „<domain>“.

    mkdir /var/www/virtual/<username>/ubersecretsubdomain.<domain>
  3. (optional) to store the sync server’s sqlite database file (it’s also possible to store this file somewhere else, but keeping such databases in one place might be considered comfortable. Alternatively, e.g. a MySQL database could also be used, consider Mozilla’s instructions to do so.)
    mkdir ~/db

Installing Additional Requirements

Please note the update from 11.01.2014!
The only requirement we need to install is virtualenv:

easy_install virtualenv

Installing Firefox Sync Server

Checkout the repository and build the sync server:


[ffsync]$ 
[ffsync]$ 
[server-full]$ 
cd ~/ffsync
hg clone https://hg.mozilla.org/services/server-full
cd server-full
make build

Optionally, we can test the build:

[server-full]$ 
make test

Configuration

Two configuration files need to be modified:

vim ~/ffsync/server-full/etc/sync.conf

In the [storage] section: Specify the path to the sqlite database file (the server will create it on the first run but the directory must already exist):

sqluri = sqlite:////home/<username>/db/ffsync.db

and, if you want, raise the quota (size in bytes):

quota_size = 51200

The path to the database file must be set under [auth], too:

sqluri = sqlite:////home/<username>/db/ffsync.db

Under [nodes], enter the client-visible URL (and uncomment the line):

fallback_node = https://ubersecretsubdomain.<domain>/

The last necessary configuration is the port under which the sync server runs locally. It’s not necessary to open this port for outbound access. Choose a free port number and put it into development.ini:

vim ~/ffsync/server-full/development.ini

In [server:main] set:

port = <your port>

By default, the sync server uses /tmp/sync-error.log for error logging. With this setting, only one sync server could be run per machine. Thus we change the error log file, e.g. to /home/<username>/tmp/sync-error.log (make sure that the directory /home/<username>/tmp exists).
Under [handler_syncserver_errors] set:

args = ('/home/<username>/tmp/sync-error.log',)

At this point we could already run the sync server locally but we couldn’t access it via the Internet. Thus we add a .htaccess file in our sync server’s web directory which tells Apache to proxy HTTP requests from the Internet to the local port under which the sync server runs:

vim /var/www/virtual/<username>/ubersecretsubdomain.<domain>/.htaccess

Put the following there:

RewriteEngine On
RewriteRule (.*) http://localhost:<your port>/$1 [P]

Start the Server as a Daemon (aka Service)

As the server should be running all the time, it’s useful to run it as a service which gets started automatically when the host is rebooted.
If not already done, create your own instance of daemontools:

uberspace-setup-svscan

Because the sync server accesses some files via relative paths and thus must be run under the right path, the service must first change into the server-full directory. We first setup up the service with echo as a dummy command and then modify the service’s run file to change to server-full and start the sync server:

uberspace-setup-service ffsync_daemon echo

Now edit the run file:

vim ~/service/ffsync_daemon/run

Remove the last line and insert the following two lines at the end:

cd ~/ffsync/server-full
exec bin/paster serve development.ini 2>&1

Restart the daemon via:

svc -du ~/service/ffsync_daemon

Now you can create new accounts via the menu in Firefox‘ settings and use the synchronization service.

Disabling Registration for New Accounts

To disable new registrations, edit the sync.conf file as follows:

vim ~/ffsync/server-full/etc/sync.conf

Under [auth], insert:

allow_new_users = false

and restart the sync server:

svc -du ~/service/ffsync_daemon

Updating the Sync Server

Before updating, the sync server has to be stopped:

svc -d ~/service/ffsync_daemon

To update the sync server, first go into its directory and pull the update. (When updating your working copy you’ll probably need to resolve conflicts between your modified configuration files and the ones from the repository). Then build it again:


[server-full]$ 
[server-full]$ 
[server-full]$ 
cd ~/ffsync/server-full
hg pull
hg update
make build

Finally start the sync server again:

svc -u ~/service/ffsync_daemon

Any comments are highly appreciated. As this is my first blog post/tutorial, I’d be very glad if you’d give me any feedback on how well I did. Feel free to post comments or contact me via email or the contact form. Thanks.

This entry was posted in General and tagged , , , . Bookmark the permalink.

18 thoughts on “Firefox Sync Server on an Uberspace

  1. I just tried your tutorial and I think it’s really great. It all worked like a charm. I got here from the Uberspace wiki.

    One small thing: since I switched directories between steps, it would help if all paths were relative to the home directory, so you could paste the commands anywhere.

    Thanks for sharing!

  2. Hi,

    nice Tutorial.

    I try to install it on my onw Uberspace, but „make“ failed with error code 1 while installing MoPyTools.

    Does someone run into the same error? Is there an quick solution?

  3. Hi, i just tried your tutorial and got stuck at the creation of new accounts through the firefox menu…

    I don’t have no domain connected with my uberspace-account, therefore I’m not sure if i set up the subdomain correct:

    I just created a new directory named „ffsync“ under /var/www/virtual// therefore i tried to use the following domain as my server: https://ffsync.username.server.uberspace.de

    Unfortuantely firefox tells me to use a valid server-url and doesn’t allow me to create the new account.

    I would really appreciate any help with this!

    Thanks, mik

    • Hi, create /var/www/virtual/<username>/ffsync.<username>.<server>.uberspace.de and try using this directory instead. Please comment if it works or not.
      Edit: fixed the < and >

  4. Ok, that’s just what i did.
    The <username> is only missing in my previous post but actually i already created the directory „/var/www/virtual/<username>/ffsync“.

    What url do i have to set up as the fallback_node and as the server in the firefox-settings?
    https://<username>.<server>.uberspace.de or
    https://ffsync.<username>.<servername>.uberspace.de

    Thanks

    Edited by admin: Fixed < and > and the lost „username“ and „server“. If I edited something wrong, please contact me. When writing <username> in a comment, this is interpreted as a HTML tag. Thus (until I find a better solution) you need to write &lt;username&gt;.
    Edited by admin 2: Swapped the &gt; and &lt; in the previous edit.

      • I used the second URL and i created the following directory:

        /var/www/virtual/>username</ffsync

        So the subdomain & the settings should be correct? Do you have any idea how i can check what the problem is?

        Thanks

        • Sorry, in my first reply there is an awkward linebreak. You should create „/var/www/virtual/<username>/ffsync.<username>.<server>.uberspace.de“ (mind the quotation marks).

  5. On my server someone is obviously already using a sync server so /tmp/sync-error.log had already been created. This means that my server crashed when it tried to touch this file. I had to change the path in the development.ini under [handler_syncserver_errors].

    Just wanted you to know, just in case someone else runs into this problem.

  6. Nice tutorial!
    On orion.uberspace.de I had to use

    easy_install-2.7 virtualenv

    (See https://uberspace.de/dokuwiki/development:python , seems that newer hosts use Python 2.7 as default)

  7. Thank you for the great manual. I have found a issue, on old uberspace servers (e.g pyxis) you could not create a uberspace subdomain!

    You could create a folder with htaccess in your html folder and you could use e.g <username>.<server>.uberspace.de/ffsync

    Or you could write a message to uberspace support. In my case the support give me the option to create subdomains.

    Edited by admin: fixed the lost <username>.<server>

  8. How do I tell Firefox to use my own syncserver instead of Firefox's syncserver?

  9. ffsync_daemon doesn't work. The error log:

    2015-06-21 21:30:12.100329500 ./run: line 12: /home/parson/etc/run-ffsync_daemon/bin/paster: No such file or directory
    2015-06-21 21:30:12.100421500 ./run: line 12: exec: /home/parson/etc/run-ffsync_daemon/bin/paster: cannot execute: No such file or directory