Posted by Jim Morris
Sun, 18 Feb 2007 23:32:00 GMT
I have updated the Capistrano local subversion module and added a perforce one.
The original article is here.
I have added rsync as an option to speed up
the deployment to remote servers, To use that option install this
version,
in the example below I have installed the file into my rails projects
lib/tasks directory.
When the rsync option is set the deployment method will use rsync over
ssh to synchronize between a local cache of the subversion project and
a remote cache, minimizing the amount of data uploaded to the server.
To use rsync use this in the deploy.rb...
require 'lib/tasks/local_subversion_rsync.rb'
set :scm, Capistrano::SCM::LocalSubversionRsync
set :repository_is_not_reachable_from_remote, true
set :use_rsync, true
set :local_rsync_cache, "/home/user/projects/aproject/cache"
set :remote_rsync_cache, "/var/www/webapp/cache"
set :rsync_username, "ausername"
set :rsync_excludes, ["*.bak", "*.log"]
The rsync_username and rsync_excludes are optional.
The :rsync_username option sets the username that ssh uses to login
to use rsync. Leave it blank if it is the same as your current login
name.
The :rsync_excludes option allows you to pass --exclude options to
rsync, using the rsync syntax for exclusions, this is an array of
patterns to exclude.
Note you must do a local SVN checkout to the directory specified by
local_rsync_cache at least once before using this method.
Also the directory specified by remote_rsync_cache must exist on the server.
The perforce version of the scm module that allows the perforce server
to be accessed locally only is
here,
note that this does not currently support the rsync option
or write to the revisions.log.
Read more...
Posted in Rails | Tags capistrano | 1 comment | no trackbacks
Posted by Jim Morris
Sun, 04 Feb 2007 06:32:35 GMT
The version of beagle available as a standard package with Ubuntu
Edgy 6.10, is horribly buggy and uses a lot of memory (2Gbytes on my
machine).
I wanted to build the latest version of Beagle (0.2.15.1) on my system which is
actually KUbuntu, and this required a lot of effort! I had to install
a bunch of added support libraries for gtk which do not appear to be
installed by default on KUbuntu. I downloaded the latest source from
here.
Here is a list of what I installed using sudo aptitude install many are suggested
here the rest I
discovered by trial and error.
libmono-sharpzip2.84-cil
mono
mono-devel
libmono-dev
libgdiplus
libxml-parser-perl
libsqlite0
libsqlite0-dev
libexif12
libexif-dev
shared-mime-info
libgmime2
libgmime2-dev
gtk-sharp2
libgmime1
libgmime-2.0-2-dev
libgmime2.2-cil
libbeagle0
libmono-sqlite1.0-cil
mono-gmcs
mono-classlib-2.0
gnome-vfs-extfs
libgnome-vfs-dev
libxml2-dev
libgconf2-dev
libbonobo2-dev
libbz2-dev
fam
libfam-dev
libgnomevfs2-dev
libgnome2-dev
libgtk2.0-dev
librsvg2-dev
python-gtk2-dev
gnome-sharp2
gtk-sharp
It turned out that the installed version of mono is not high enough to build the
latest version of beagle, so I installed the latest version of mono
manually into /opt/mono. You can get it
here, I got the Generic Mono 1.2.2.1_1 Linux
installer from
here.
To use this the following exports need to be done, usually in ~/.bashrc
export GTK2_RC_FILES=$HOME/.gtkrc-2.0
export PATH="/opt/mono/mono-1.2.2.1/bin:$PATH"
export PKG_CONFIG_PATH="/opt/mono/mono-1.2.2.1/lib/pkgconfig:$PKG_CONFIG_PATH"
export MANPATH="/opt/mono/mono-1.2.2.1/share/man:$MANPATH"
export LD_LIBRARY_PATH="/opt/mono/mono-1.2.2.1/lib:$LD_LIBRARY_PATH"
Additionally I needed to do this...
export MONO_PATH=/opt/mono/mono-1.2.2.1/lib/:/opt/mono/mono-1.2.2.1/lib/mono/gtk-sharp-2.0:/usr/lib/cli/gmime-sharp-2.2:/usr/lib/cli/gsf-sharp-0.0
when I got a bunch of build errors, and run errors, presumably from mono.
I am not a c sharp programmer, and know nothing about mono, so I am
not sure why these search paths are needed, but they are.
After all the above you can use the standard...
./configure
make
sudo make install
to build and install beagle.
I also turned on extended attributes on my ext3 partition as explained
here.
Then run beagle (making sure the above export is done first otherwise
it uses the wrong version of mono and can't find some of the mono
libraries.)
As far as I can see it is now running and indexing my disk, and the
memory usage is pretty low, and the cpu usage is tolerable.
If anyone can add good explanations please do so in the comments
section.
UPDATE This recipe also works with beagle 0.2.16
Posted in Linux | Tags beagle, edgy, kubuntu | 1 comment | no trackbacks
Posted by Jim Morris
Sun, 17 Dec 2006 10:57:15 GMT
My current development system is Ubuntu 6.06 using an Athlon 2500+ Barton
processor and an Asus A7V8X-X motherboard and 1GB of DRAM. It has been
pretty stable, and relatively fast, but it is about 3 years old, and I
like to upgrade when I can at least double my perceived performance. (ie I
have to notice the difference, not just going on specs).
So I decided the latest hot system seems to be based on the Intel Core
2 Duo 6600, and I like a quiet system so I decided to build one spec'd
out by Silent PC Review. The one in
particular was SPCR Model One: Modern General Purpose
PC.
This is a build based on an Asus P5B-E board, in an Antec Solo case,
with the silencer foam etc. I wanted 2 GB memory this time around as I
like to use VMware workstation to run winxp.
So I ordered the CPU, Mobo and Memory from NewEgg, and I ordered the
Antec Solo case with pre-installed silencer foam from EndPCNoise.com, they
have a pre-built
version
but by buying the components myself I could save about $500, and I enjoy
building systems.
The specs are:
- Intel Core Duo E6600
- Corsair TWIN2X2048-6400C4 memory
- Asus P5B-E Motherboard
- Antec Solo case
- Scythe Ninja Plus Rev. B CPU Cooler
I already have Seagate Barracuda drives (but had to get a SATA drive see
later).
I had an Antec 380 Watt PSU, which is not really quiet but I will
switch that with a 430 Watt Seasonic S12 later.
I had to get a new Video card, as the newer Motherboards no longer
have AGP slots, so I wanted to get as close to what I already have and
setup for dual monitor, which meant sticking with ATI Radeon, not the
fastest but I don't play games so don't need the fastest out there,
but it does need to support dual monitor, Fry's had a special on
Diamond Stealth x300e ATI based PCI express cards, so I got one of
those. (And as it turns out I may as well have gotten anything, as
even though it is a Radeon like my current card, it couldn't use the
same driver).
I had already read the horror stories of people trying to get Ubuntu
to work on these new C2D Mobos, but it seemed most of the problems
related to trying to run off of ATA/IDE disks, and the newer mobos all
have SATA. So I got an SIIG UltraATA 133 PCI card, just in case. This
was because the on-board JMicron based IDE controller does not work
with the version of the Kernel Ubuntu 6.06 comes with and the plan was
to try to continue using Dapper Drake and not have to upgrade to Edgy
Edge or worse Fiesty. I thought I could just stick my existing drive in
the new system and be off and running. (Boy was I wrong!)
I built the system pretty quickly, everything fitted in nicely
although the Ninja heatsink is huge, and made it hard to plug in some
connectors after I installed it, probably should have plugged them in
first. Also replacing the power supply later on is going to be tough
without removing the heatsink first.
The first thing I tried was to boot off of my existing IDE drive using
the SIIG IDE PCI board, and I was surprised that it booted OK. The
first thing I always do is run memtest over night, and as that comes
with Ubuntu it was easy to do. I needed to tweak the memory settings
in the BIOS first though. I don't overclock as I want stability, but
the defaults the BIOS picked up were wrong, so I just changed the
latencies to the Corsair specified ones (4,4,4,12). Memtest ran all
night with no problems, and the CPU temp was around 50 deg C, which
isn't too bad. I really like Corsair RAM it always seems to work for
me. So the first step seemed to go ok.
I then discovered I was not able to get the system to boot from an
IDE CDROM drive connected to the SIIG controller, whatever I tried.
I also noticed that the HD performance of the ata100 IDE disk drive
was not what it should be on the SIIG controller, I'm not sure why,
but it was about half the speed I expected, so I popped down to Fry's
again (I practically live there ;) as they had a Seagate 300GB SATA
drive for just under $100, and hooked that up, and it was nice and
fast, at least 3 times as fast as the IDE drive, I suspect the SIIG
PCI/IDE adapter was using a slower mode than necessary, but there was
no way of fixing it, and given I can't boot from CDROMs either I
returned the card (luckily Fry's has a great return policy).
I found I could boot off of a USB CDROM drive though, which I had
lying around so I ran Spinrite 6 on the SATA drive,
as I like to test all my drives as I've had a few bad ones in the past,
this comes on a bootable CD and I left that running for about 14
hours! The disk checked out OK.
Given all these problems, plus the fact that the sound card wasn't
recognized and the Ethernet card wasn't either I figured I'd see what
happened if I tried to boot off of the Edgy (Ubuntu 6.10) Live CD
using the built-in JMicron based IDE, I thought after all the bug
reports I read they have fixed it by now.
I stuck it into the IDE CDROM drive plugged into the JMicron slot, and
watched it boot, it came up and saw all the drives on the IDE, and the
sound worked too. Only the Ethernet was still invisible. So it seems
Edgy has fixed most of the compatibility problems, so this was enough
incentive for me to upgrade from Dapper to Edgy, which turned out to
not be too much of a hassle, and most of the problems I ran into other
people have already run into and can be easily fixed with some
Googling. (For instance the acpid didn't want to update, it was just a
matter of killing off hald).
I plugged my existing Dapper IDE drive into the JMicron slot, booted
into the Edgy Live CD, and copied the Dapper system onto the new SATA
drive, basically just cp -ax /mnt/old /mnt/new. This meant I could
keep that Dapper disk as a backup in case something went terribly
wrong down the road. I did some Grub incantations to get the SATA
drive to boot, this is best done in the grub boot prompt rather than
using grub on a running system...
grub> root (hd0, 0)
grub> setup (hd0)
Figuring out it's hd0 or hd1 takes some effort, but if its the only
drive it will be hd0, if you do this from a running system (ie the
Live cd)...
> grub
grub> device (hd0) /dev/sda
grub> root (hd0, 0)
grub> setup (hd0)
I then booted off the SATA drive into single user mode of Dapper.
(Dapper boots fine from a SATA drive on this MoBo).
I followed the KUbuntu instructions to upgrade from Dapper to Edgy,
basically edit the /etc/apt/sources.list and replace all occurrences
of dapper to edgy, then...
> apt-get update
> apt-get dist-upgrade
> apt-get install kubuntu-desktop python-qt3 python-kde3 ubuntu-minimal
Then keep doing it until everything downloads ok, that's when I found
the hald problem mentioned above which was blocking the upgrade
finishing.
I had to do a few apt-get upgrades afterwards, and manually upgrade
the list of things that got held back for some reason.
At the end of that I had my existing workstation image upgraded to
Edgy on the SATA drive, able to see the IDE drives on the JMicron
controller, and the sound worked fine. I had to plug in an old PCI
Ethernet card I had lying around as the Gig Ethernet controller is not
recognized even in Edgy.
In the process of all this I discovered that even though the JMicron
IDE works under Edgy, it thinks all the HDs are UDMA33, ie SLOW, I
haven't played around much with that, but I'm not sure why as it says
it handles UDMA133. This is ok for my CDROM, but I want my second IDE
drive to be a bit faster, even though my primary will now be a genuine
SATA. Fry's didn't have anymore SATA to IDE dongles to try (the one
from Syba I tried didn't work at all), so I ordered the newer SYBA one
from NewEgg, along with the 430 Seasonic S12 PSU. Hopefully this will
work and make my second IDE drive fast again. In the meantime I'll
live with the slow second drive.
Now I needed to find all the things that broke from my upgrade from
Dapper to Edgy (Something I was hoping I would have to avoid).
The first thing was to get dual monitors working again, and as fglrx
didn't seem to work anymore (not sure if it was Edgy or the different
video card), I had to do some research again and found that the Radeon
Xorg driver should work the same way withthe same setup I had before.
I took my existing xorg.conf that I had finally got to work with dual
screen a while back using fglrx and changed the driver to radeon, and
everything worked OK, except for some problem with the second screen
having a huge rectangle instead of a cursor, again Googling discovered
others had solved that problem, and I think adding "composite" "false"
to my xorg.conf fixed it. (Or rebooting, who knows).
Most everything else seemed to work except...
Had to uninstall and reinstall Mysql to get the server to run
I had to remove Beagle as it was using 2GB of memory! and no
backport in sight of the latest version, which apparently fixes that.
lm-sensors no longer worked, and it appears the device drivers
needed for my motherboard have not been ported over yet, and probably
won't make it into Edgys kernel anyway, so no more CPU temp sitting in
GKrellM.
Minor mods needed to smarttools (smartd) setup (add -d ata for the SATA drive)
Oh yea, it is really fast, well worth the effort and expense of the
upgrade, a very noticeable speed improvement over the Athlon XP 2500+
Barton system. I won't have to upgrade again for years, until there
are native quad core chips at 5GHz, at least ;)
This version is quieter than my old system, but not silent yet, I'm
hoping replacing the case fan with my Nexus fan, and the PSU with the
Seasonic S12, and removing the fan from the heat sink, will make it
almost silent.
update I got the new PSU it is a lot quieter. It is still not 100%
silent as I can just hear the nexus case fan, but the pc is right next
to my right ear, so you could say it is a good as it gets.
The syba silicon image based sata to ide dongle works well, the ide
drive now runs about 2-3 times faster, make sure you get the silicon image
based one and not the jmicron one. The one I have is in a clear
plastic case, available from egghead for about $12.
Posted in Linux | Tags coreduo, dapper, edgy, ubuntu | no comments | no trackbacks
Posted by Jim Morris
Thu, 07 Dec 2006 05:42:00 GMT
UPDATE 2007-06-09 This method has been deprecated in Cap 2.0.
UPDATE 2007-02-21 I have updated the files to correctly update revisions.log
UPDATE I added rsync as suggested see end of article for more info.
I previously published a custom deploy recipe that allowed subversion
to checkout from a local repository and send to the remote server.
This was a suboptimal solution, as it wouldn't work with things like
deprec.
So here is the way I should have done it in the first place. It is a
modified version of the standard subversion scm module, it should be
backward compatible with the built in version, but adds a couple of
features.
- Handles the subversion repository only being accessible from the local
machine
- If the subversion repository is accessible from the remote server
allows for different URLs for access from the local and remote
machines.
If you use a standard deploy.rb and your subversion is accessible
from the server, then it should work exactly as the built-in version
(no need to use it then, but it should work)
However if you have your subversion server behind your local firewall,
you just add these three lines to your deploy.rb
require 'lib/tasks/local_subversion_rsync.rb'
set :scm, Capistrano::SCM::LocalSubversionRsync
set :repository_is_not_reachable_from_remote, true
And everything will work as before, even though the server has no idea
what subversion is.
Of course for this to work you need to download
this de-tar
it and put it in your lib/tasks directory. (That's where the require
line gets this extension from). There is also a unit test for the
new_subversion module which is extended from the one shipped with
Capistrano, which passes.
Everything else in deploy.rb should be the same as before, except
that you set the repository to the URL your local machine uses to
access your locally accessible subversion server or repository.
(Should even work with the URL file://...), for instance...
set :repository, "svn://your.svnserver.host/#{application}/trunk"
The assumption is that both the local and remote machines can create
files and directories in /tmp, if this is not true then one or both
of these should be set...
set :tmpdir_local, "/usr/tmp"
set :tmpdir_remote, "/home/user/tmp"
The way this module works is to export the relevant version of the
project from Subversion into a temporary directory on the local
machine (default is /tmp/unique_name). Then I create a gzip'd tar
archive into the temp directory. The tar file is then transferred to
the target servers, using the Capistrano put command into a
temporary directory (/tmp by default) on the server. I then create the
target directory on the server and un-tar the file into that
directory. Then clean up the various temporary files. From the
perspective of the server the end result should be identical to doing
an svn export into the target directory on the server.
The other facility this SCM Module provides is when your subversion
server is accessible from the server as is the "normal" use case (per
the Capistrano developers, but bad practice IMHO), but you need a
different URL to access the SVN repository from the local machine vs
the remote server, eg
server accesses with this URL svn://localhost/myapp/trunk
local machine accesses with this URL svn+ssh://myserver.com/myapp/trunk
in this case you add these lines...
require 'lib/tasks/local_subversion_rsync.rb'
set :scm, Capistrano::SCM::LocalSubversionRsync
set :local_repository_path, "svn+ssh://myserver.com/myapp/trunk"
set :repository, "svn://localhost/myapp/trunk"
In addition if the actual SVN binary is different on server and local
you can add these for instance...
set :remote_svn, "/usr/local/bin/svn"
set :local_svn, "/usr/bin/svn"
If the relevant configuration variable is not set then the standard
svn configuration variable is used, and then just "svn" if that is not set.
Please beware I have not fully tested the use cases other than the
first one, where I use my local subversion server to deploy to my
remote servers that know nothing about subversion, however the unit
tests do pass for the other use cases.
UPDATE 2007-02-15 check this article for using the rsync option.
Posted in Ruby, Rails | Tags capistrano, subversion | 26 comments | no trackbacks
