Replicating Over SSH Port-Forwarding

Introduction

In some situations, it's not possible to initiate a direct TCP connection to the source repository you want to replicate from.  Your destination repository could be behind a firewall with strict rules about outgoing connections, or the source repository could be behind a firewall with strict rules about incoming connections.

Replication only requires connections to a single TCP port.  (Specifically the one in the config file setting [Repository]VestaSourceSRPC_port, usually 21776.)  Because of this, it's possible to securely tunnel a replication with the port forwarding feature provided by OpenSSH and other SSH implementations.

Hosts

Replication over SSH port-forwarding involves at least two different hosts (an SSH client and an SSH server), but possibly as many as five:

  1. The host where the replicator is run
  2. The destination repository
  3. A host which the destination repository can connect to (which can be either the SSH client or the SSH server)
  4. A host which can connect to the source repository (the other end of the SSH connection)
  5. The source repository

In the simplest case, 1-3 are all the same host and 4 and 5 are the same host, but they could all be different.

Port forwarding can go in either direction (from the SSH client to the SSH server, or from the server to the client).  Because of this flexibility, host 3 can act as an SSH client to host 4, or host 4 can act as an SSH client to host 3.  However, the port-forward must always go from the destination side to the source side (from host 3 to host 4).

Access Control

One important prerequisite is ensuring that the source repository will allow read access over the port-forwarded connection.  This usually requires using one of the following methods:

In addition to having access to the source repository over the SSH port forward, the destination repository will also have to be told to allow replication from the destination end of the SSH connection.  This requires adding an appropriate "#replicate-from" attribute in the destination repository.  (See the vrepl(1) man page for a description of this attribute.)

Example #1

A user with Vesta installed behind a corporate firewall that wishes to replicate from pub.vestasys.org, but can't because of outgoing restrictions on the firewall.  However, the corporate firewall does allow outgoing SSH connections, and they want to use this to tunnel out to the public repository.  They have a Linux machine at home on a broadband connection which is running an SSH server.

In this example the hosts will be:

  1. workstation.example.com: The user will run the replicator on their workstation, which is behind the corporate firewall and is a client of the destination repository
  2. vesta.example.com: The destination repository, also behind the corporate firewall
  3. workstation.example.com: The user will also run the SSH client on their workstation
  4. user1234.dsl.exampleisp.net: The user's Linux machine at home on a broadband connection, which will be used as an SSH server.
  5. pub.vestasys.org: The source repository

The user first arranges to allow replications using their workstation as the source repository:

jsmith@workstation.example.com% su vadmin
Password:
vadmin@workstation.example.com% vattrib -a "#replicate-from" workstation.example.com:21776 /vesta

Of course their workstation isn't really the source repository, but it will look like it is to the replication and the destination repository, since that's where the incoming end of the port-forward will be.

Next the user starts the SSH client, connecting out to their home Linux machine with a port forward through to pub.vestasys.org:

jsmith@workstation.example.com% ssh -L 21776:pub.vestasys.org:21776 -g jsmith@user1234.dsl.exampleisp.net

Leaving the ssh client running, they start the replication on their workstation:

jsmith@workstation.example.com%  vrepl -s workstation.example.com:21776 -e@ /vesta/vestasys.org/vesta/release/LAST/linux_i386.main.ves

It's worth noting that this requires no changes to the access control configuration on the source repository, because pub.vestasys.org allows anyone to have read-only access.

Example #2

A user with Vesta installed both at work behind a corporate firewall and at home behind a personal firewall wishes to replicate from their repository at home to their repository at work.  Both machines are behind firewalls that would prevent this.  However, the corporate firewall allows incoming SSH connections, which the user will use to tunnel between the two repositories.

In this example the hosts will be:

  1. workstation.example.com: The user will run the replicator on their workstation, which is behind the corporate firewall and is a client of the destination repository
  2. vesta.example.com: The destination repository, also behind the corporate firewall
  3. user1234.dsl.exampleisp.net: The user will run the SSH client on their home Linux machine to connect into the corporate network
  4. ssh.example.com: The corporate SSH server.
  5. user1234.dsl.exampleisp.net: The source repository is also running on the user's home Linux machine

First, the user adds a line to the source repository's export file to allow the realm of the destination repository access:

jsmith@user1234.dsl.exampleisp.net% su vadmin
Password:
vadmin@user1234.dsl.exampleisp.net% echo "127.0.0.1: ro example.com" >> /etc/vesta/repos.export

(Note that unless you've used the RPM or Dedian packages to install Vesta, this may not be where your export file is.)

Next the user starts the SSH client, connecting into the company network with a port forward pointing back to the repository running on their home computer:

jsmith@user1234.dsl.exampleisp.net% ssh -R 21776:localhost:21776 -g jsmith@ssh.example.com

Now that they've logged into the company network, the user arranges to allow replications using the company SSH server (which is be the destination end of the port forward) as the source repository:

jsmith@workstation.example.com% su vadmin
Password:
vadmin@workstation.example.com% vattrib -a "#replicate-from" ssh.example.com:21776 /vesta

Before trying a replication, just to make sure everything's working, the user runs a simple client to test the port forwarded connection to their home repository:

jsmith@workstation.example.com%  vid -R ssh.example.com
User names and aliases:
  jsmith@example.com
Groups:
Unix (NFS) user ID:          203
Unix (NFS) primary group ID: 202

Satisfied that everything's working, they start the replication on their workstation:

jsmith@workstation.example.com%  vrepl -s ssh.example.com:21776 -e+ /vesta/jsmith_exampleisp.net/*/LAST

Limitations

While replication only requires a uni-directional connection to the source repository, mastership transfer requires bi-directional connections.  That means that SSH port-forwarding alone is not sufficient for checking out an object mastered at a remote repository.  For that, you need a more significant VPN solution.

Troubleshooting

Connection refused

Port forwarding can go either from the SSH client to the SSH server, or from the server to the client.  Make sure you have it going in the right direction (from the destination side to the source side).

Other possible causes include:

SRPC::failure unauthorized user

As described above, the source repository must allow read access from the source end of the SSH connection.  Check the export file at the source repository, and make sure it allows access from that host with the realm used at the destination.  You can check whether you've gotten it right by running simple commands that will access the source repository over the SSH port forward:

vid -R ssh-host:forwarded-port
vattrib -R ssh-host:forwarded-port /vesta
vrepl: No permission, copying ...

Make sure you've added a "#replicate-from" attribute at the destination repository to allow using the destination end of the SSH port forward as a replication source.


Kenneth C. Schalk
Last modified: Tue Nov 16 13:26:40 EST 2004