DevOps NY

Category: Subversion

  • svnsync problems

    I have a little cron job that syncs two svn repos. It runs every ten minutes. For some reason it stopped working so I killed the job. Now when I try to manually run the job, I get:

    Failed to get lock on destination repos, currently held by...

    Great, the sync is broken!
    Here is how to clear it up. Use the following command to remove the lock:

    svn propdel --revprop -r0 svn:sync-lock file:///path/to/the/repository

    This is the sync script:

    #!/bin/sh
svnsync synchronize file:///path/to/the/repository --username svnuser --password svnpassword

  • Rsync Between Servers Automatically

    Part 1 – Setting up SSH key authentication

    First, we need to make sure the DESTSERVER has the ability to use key authentication enabled. Find your sshd configuration file (usually ‘/etc/ssh/sshd_config’) and enable the following options if they are not already set.

    RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

    If you edit the file be sure to restart sshd afterwards.

    # /etc/init.d/sshd restart

    Next, on the SOURCESERVER we will create the public / private key pair to be used for authentication with the following command.

    # ssh-keygen -t rsa
    • Note: Do not enter a passphrase for this, just hit enter when prompted.

    This should create 2 files, a public key file and a private key file. The public key file (usually [homedir]/.ssh/id_rsa.pub) we will upload to the DESTSERVER. The private key file (usually [homedir]/.ssh/id_rsa) we will keep on the SOURCESERVER.

    • Be sure to keep this private key safe. With it anyone will be able to connect to the DESTSERVER that contains the public key.

    Now we will plant the public key we created on to the DESTSERVER. Choose the user account which you will use to connect to on DESTSERVER, we’ll call this user ‘destuser’ for now.

    In that account’s home directory, create a ‘.ssh’ subdirectory, and in that directory create a new text file called ‘authorized_keys’. If it already exists, great, use the existing file. Open the ‘authorized_keys’ file and paste in the contents of the public key you created in the previous step (id_rsa.pub). It should look something like the following

    ssh-rsa <lots and lots of characters…> sourceuser@SOURCESERVER

    Save the file and change the permissions to 600 for the file and 700 for the ‘.ssh’ directory.

    Now to test that the keys are working. From the SOURCESERVER try logging in as normal using ssh to the DESTSERVER.

    # ssh destuser@DESTSERVER

    If all is working you should not be prompted for a password but instead connected directly to a shell on the DESTSERVER.

    Part 2 – Creating the rsync script

     

    Now for the rsync script.

    #!/bin/sh

SOURCEPATH=’/var/svn/workingcopies/beta.app.jp/’
DESTPATH=’/var/www/lamp_root/wwwapps/app.jp’
DESTUSER=’updater’
LOGFILE=’/var/log/rsync_beta.log’

echo $’\n\n’ >> $LOGFILE
for SERVER in web01 web02
do
        rsync -av –rsh=ssh $SOURCEPATH $DESTUSER@$SERVER:$DESTPATH 2>&1 >> $LOGFILE
        echo “Completed at: `/bin/date`” >> $LOGFILE
done
    Part 3 – Setting up the cron job

    This is optional. I assume web updates will be done manually.

    Assuming everything has worked so far all that’s left is to setup a cron job to run the script automatically at a predefined interval.

    Use the ‘crontab’ command to create a new cron job.

    >sudo crontab -e -u updater

    This will open an editor where you can schedule the job.

    Enter the following to have the script run once every hour

    0 * * * * /var/scripts/rsync_betajp.sh

  • Setting up a subversion repository

    This is a bit old. From 2008 to be exact. But, I am putting it out there for reference anyway.

    1. Install packages:
    sudo apt-get install subversion libapache2-svn libapache-mod-dav apache2

     

    Note: libapache-mod-dav is included in libapache2.2-common, so it may be unavailable or you do not need it.

     

    2. Enable SSL:
    sudo a2enmod ssl

     

    check the ports.conf file, if ssl is already enabled we do not need to do the following:
    sudo sh -c “echo ‘Listen 443’ >> /etc/apache2/ports.conf”

     

    3. Generate Certificate:
    Ubuntu < Feisty:
    sudo apache2-ssl-certificate
    Use the server name to be used for access the web server.

    Ubuntu >= Feisty:
    sudo apt-get install ssl-cert
    sudo mkdir /etc/apache2/ssl
    sudo /usr/sbin/make-ssl-cert /usr/share/ssl-cert/ssleay.cnf /etc/apache2/ssl/apache.pem

     

    4. Create Virtual Host:
    sudo cp /etc/apache2/sites-available/default /etc/apache2/sites-available/$SITENAME
    sudo vim /etc/apache2/sites-available/$SITENAME
    change:
    NameVirtualHost *:443
    <VirtualHost *:443>
    add:
    SSLEngine on
    SSLCertificateFile /etc/apache2/ssl/apache.pem
    SSLProtocol all
    SSLCipherSuite HIGH:MEDIUM

     

    5. Enable the site:
    sudo a2ensite $SITENAME
    sudo /etc/init.d/apache2 restart

    A warning that complaints about failure of server name determination can be fixed by adding ServerName $SERVERNAME to the main Apache config /etc/apache2/apache2.conf

     


    6. Adding repository(ies):
    The following setup assumes we want to host multiple repositories.

    $REPOS is “nbty” currently
    sudo mkdir /var/svn/repositories
    sudo svnadmin create /var/svn/repositories/$REPOS
    sudo chown -R www-data:www-data /var/svn/repositories/$REPOS
    sudo chmod -R g+ws /var/svn/$REPOS

     

    7. Adding Basic Authentication:

    $AUTH_USER is svn with password mypassword
    sudo htpasswd -c -m /etc/apache2/dav_svn.passwd $AUTH_USER

     

    8. Enable and configure WebDAV and SVN:
    Add to /etc/apache2/mods-available/dav_svn.conf
    DAV svn
    SVNParentPath /var/svn/repositories
    AuthType Basic
    AuthName “Subversion Repository”
    AuthUserFile /etc/apache2/dav_svn.passwd
    Require valid-user
    SSLRequireSSL

    and for non-anonymous access comment out:
    #<LimitExcept GET PROPFIND OPTIONS REPORT>
    #</LimitExcept>

    (optionally the same configuration can be set for particular virtual host only, i.e. /etc/apache2/sites-available/$SITENAME)

     

    $SITENAME is svn (ssl on svn01 and svn02)

     

    Add to $SITENAME in /etc/apache2/sites-available:

    # Specially log any Subversion operations.

    CustomLog /var/log/apache2/svn.log “%t %u %{SVN-ACTION}e” env=SVN-ACTION

     

    <Location /svn/company>

    DAV svn

    SVNPath /var/svn/repositories/company

    SVNReposName “Companies Main Subversion Repository”

    </Location>

     


    9. Finalization:
    sudo /etc/init.d/apache2 restart

     

    Testing:

    $REPOS = company
    Web access:
    lynx https://svn01/svn/$REPOS exposes the repository.
    lynx http://localhost/svn/$REPOS says: eat my shorts , i.e. 403-forbidden.

     

    Now the sync:

     

    1. cd /var
    2. sudo mkdir svn
    3. sudo mkdir svn/repositories
    4. sudo svnadmin create /var/svn/repositories/company
    5. sudo adduser svn
      1. yourpassword is pwd
    6. sudo vipw
      1. change svn line from /bin/bash to /bin/false….we don’t want this user to log in to shell

     

     

    Need to

     

     

    An initial import:
    svn import –username $AUTH_USER $A_FILE https://localhost/svn/$REPOS/testdir -m “Testing”

    … and check-out:
    svn co –username $AUTH_USER https://localhost/svn/$REPOS

    To add a new repository just repeat the step 6 (without making the root directory of course).
    If you wish to configure a single repository only, instead of point 6:
    sudo svnadmin create /var/svn
    sudo chown -R www-data:www-data /var/svn
    sudo chmod -R g+ws /var/svn

    and in /etc/apache2/mods-available/dav_svn.conf (step 8) use this instead of SVNParentPath:
    SVNPath /var/svn

     

    sudo svnsync initialize file:///var/svn/repositories/nbty https://svn01/svn/company –username svn –password yourpassword

     

    sudo svnsync synchronize file:///var/svn/repositories/company –username svn –password yourpassword

  • Fix a broken svn repo

    First.
    svn propdelete svn:sync-lock --revprop -r 0 file:///path/to/repo
    then do an svn sync