Wolfmans Howlings

A programmers Blog about Ruby, Rails and a few other issues

Using Ruby SVN bindings to get file status

Posted by Jim Morris Mon, 04 Sep 2006 18:51:35 GMT

If you Google around for information or even some documentation on the ruby SVN bindings you will find plenty of comments that it simply is not documented, so when I wanted to add an SVN status check to a UI I was working on (a project browser window for Rails), I had to "Use the source Luke". However given the bindings are actually mostly automatically generated by SWIG, and the actual details are hidden in a goo of swig generated c code, even that was a challenge.

Eventually I realized that the API was almost identical to the c level subversion API, not surprisingly, and for the most part it works the way you would expect. However I could not find any examples of the client status call, so here it is for anyone else struggling with this issue.

require "svn/core"
require "svn/client"
require "svn/wc"
require "svn/repos"

# define Consts for all the numeric status values
NONE = 1
UNVERSIONED = 2
NORMAL = 3
ADDED = 4
MISSING = 5
DELETED = 6
REPLACED = 7
MODIFIED = 8
MERGED = 9
CONFLICTED = 10
IGNORED = 11
OBSTRUCTED = 12
EXTERNAL = 13
INCOMPLETE = 14
# I added these so I can amalgamate repo and local status
UPDATED = 15
MODIFIED_NEWER = 16
NEWFILE = 17

$modes= { # Map status to English
    NONE => "None",
    UNVERSIONED => "unversioned",
    NORMAL => "normal",
    ADDED => "file added",
    MISSING => "missing",
    DELETED => "file removed",
    REPLACED => "deleted and then re-added",
    MODIFIED => "modified",
    MERGED => "received repos mods",
    CONFLICTED => "file modified and in conflict",
    IGNORED => "ignored",
    OBSTRUCTED =>  "unversioned resource is in the way of the versioned resource",
    EXTERNAL => "unversioned path populated by an svn:external property",
    INCOMPLETE => "directory doesn't contain a complete entries list",
    UPDATED => "newer version in repository",
    MODIFIED_NEWER => "modified and newer version in repository",
    NEWFILE => "new file in repository",
    nil => "Unknown status"
}

# this amalgamates the most common status between local and repository
def compute_status(status)
  # if modified in repo and localy
  return MODIFIED_NEWER if status.repos_text_status == MODIFIED && status.text_status != NORMAL
  # if added in repo and does not exist locally
  return NEWFILE if status.repos_text_status == ADDED && status.text_status == NONE
  # if repo is none then use local status
  return status.text_status if status.text_status != NORMAL
  # if normal locally but changed in repo
  return UPDATED if status.repos_text_status == MODIFIED
  # otherwise it must be normal
  NORMAL
end

ctx = Svn::Client::Context.new()

rev = ctx.status("/home/user/project/myproject", "HEAD", true, true) do |path, status|
  astat= compute_status(status)
  puts "#{path}: #{status.text_status},#{status.repos_text_status} = #{$modes[astat]}"

  unless status.entry.nil?
    puts(".....name: #{status.entry.name}")
    puts(".....url: #{status.entry.url}")
    puts(".....repos: #{status.entry.repos }")
    puts(".....revision: #{status.entry.revision}")
    puts(".....kind: #{status.entry.kind}")
    puts(".....schedule: #{status.entry.schedule}")
    puts(".....deleted: #{status.entry.deleted}")
    puts(".....absent: #{status.entry.absent}")
    puts(".....incomplete: #{status.entry.incomplete}")
    puts(".....cmt_date: #{status.entry.cmt_date}")
    puts(".....cmt_rev: #{status.entry.cmt_rev}")
    puts(".....cmt_author: #{status.entry.cmt_author}")
    puts(".....prop_time: #{status.entry.prop_time}")
    puts(".....text_time: #{status.entry.text_time}")
  end
end

This is a sample, it shows how to setup the client context, call the status call, and parse the results. I also added some sugar by defining the numeric status codes and adding a hash to translate them to english. I also compute an amalgamated status code from the local status and status in the repository, to get the extra status codes I added above.

The status parameter returned by the status() call turns out to be a standard svn status structure, containing various fields, the most interesting one as far as status is concerned are text_status and repos_text_status fields which are numeric fields which specifies the SVN status of the file locally and in the repository, I have created in the example above a bunch of constants to define each of the states, and a hash to turn them into english.

Note the compute_status() method which looks at both the local status and repository status to see what the relative state of any file is wrt the repository.

The other interesting field is the entry field, which contains bunch of data as shown above, and in the structure definition below.

There are more parameters that can be passed to the status call, but the last two I show should be set to true, as that allows recursion into sub directories and shows all files, by default recursion is true but all_files is false which only show files not under svn control.

The developers have made some attempt to make this relatively easy to use, you can forget about the apr pools and stuff, and the second parameter is usually a structure that tells it the version or revision to search for, this has been simplified so you can pass in a string describing the revision ("HEAD", "TAIL", etc) or a number denoting the revision number.

The status and entry structures have also had the types converted into convenient ruby types, but for reference I show the c struct definitions from the various svn header files...

Here is the full status structure from svn_wc.h

typedef struct svn_wc_status2_t
{
  /** Can be @c NULL if not under version control. */
  svn_wc_entry_t *entry;

  /** The status of the entries text. */
  enum svn_wc_status_kind text_status;

  /** The status of the entries properties. */
  enum svn_wc_status_kind prop_status;

  /** a directory can be 'locked' if a working copy update was interrupted. */
  svn_boolean_t locked;

  /** a file or directory can be 'copied' if it's scheduled for
   * addition-with-history (or part of a subtree that is scheduled as such.).
   */
  svn_boolean_t copied;

  /** a file or directory can be 'switched' if the switch command has been
   * used.
   */
  svn_boolean_t switched;

  /** The entry's text status in the repository. */
  enum svn_wc_status_kind repos_text_status;

  /** The entry's property status in the repository. */
  enum svn_wc_status_kind repos_prop_status;

  /** The entry's lock in the repository, if any. */
  svn_lock_t *repos_lock;

}

and the entry structure

typedef struct svn_wc_entry_t
{
  /* IMPORTANT: If you extend this structure, check svn_wc_entry_dup() to see
     if you need to extend that as well. */

  /* General Attributes */

  /** entry's name */
  const char *name;

  /** base revision */
  svn_revnum_t revision;

  /** url in repository */
  const char *url;

  /** canonical repository URL or NULL if not known */
  const char *repos;

  /** repository uuid */
  const char *uuid;

  /** node kind (file, dir, ...) */
  svn_node_kind_t kind;

  /* State information */

  /** scheduling (add, delete, replace ...) */
  svn_wc_schedule_t schedule;

  /** in a copied state */
  svn_boolean_t copied;
  /** deleted, but parent rev lags behind */
  svn_boolean_t deleted;

  /** absent -- we know an entry of this name exists, but that's all
      (usually this happens because of authz restrictions)  */
  svn_boolean_t absent;

  /** for THIS_DIR entry, implies whole entries file is incomplete */
  svn_boolean_t incomplete;

  /** copyfrom location */
  const char *copyfrom_url;

  /** copyfrom revision */
  svn_revnum_t copyfrom_rev;

  /** old version of conflicted file */
  const char *conflict_old;

  /** new version of conflicted file */
  const char *conflict_new;

  /** working version of conflicted file */
  const char *conflict_wrk;

  /** property reject file */
  const char *prejfile;

  /** last up-to-date time for text contents (0 means no information available)
   */
  apr_time_t text_time;

  /** last up-to-date time for properties (0 means no information available) */
  apr_time_t prop_time;

  /** base64-encoded checksum for the untranslated text base file,
   * can be @c NULL for backwards compatibility.
   */
  const char *checksum;

  /* "Entry props" */

  /** last revision this was changed */
  svn_revnum_t cmt_rev;

  /** last date this was changed */
  apr_time_t cmt_date;

  /** last commit author of this item */
  const char *cmt_author;

  /** lock token or NULL if path not locked in this WC
   * @since New in 1.2.
   */
  const char *lock_token;
  /** lock owner, or NULL if not locked in this WC
   * @since New in 1.2.
   */
  const char *lock_owner;
  /** lock comment or NULL if not locked in this WC or no comment
   * @since New in 1.2.
   */
  const char *lock_comment;
  /** Lock creation date or 0 if not locked in this WC
   * @since New in 1.2.
   */
  apr_time_t lock_creation_date;

  /* IMPORTANT: If you extend this structure, check svn_wc_entry_dup() to see
     if you need to extend that as well. */
} svn_wc_entry_t;

For the most part you can simply reference the fields in the structure and get a relatively understandable result.

The status call is defined in ruby as this

def status(path, rev=nil, recurse=true, get_all=false,
           update=true, no_ignore=false,
           ignore_externals=false, &status_func)

The first parameter is the path on the local file system to get the status of, the second is the revision to get, something like "HEAD", or 2345 can be passed in, the next parameter is whether to recurse into subdirectories, the next tells it to return the status of all files if set to true, default of false and only returns "interesting files" ie local mods and/or out-of-date or not versioned. Update is set to true it will contact the repository to get more information wrt to the version specified in the first parameter (which is ignored otherwise). I couldn't find any documentation in the c stuff about the no_ignore parameter, but the ignore_externals tells it to return status on externals or not. See the header file svn_client.h and the call svn_client_status2 for more documentation.

Posted in ,  | Tags , , ,  | 3 comments | no trackbacks

Experiences with Komodo Pro and Ruby

Posted by Jim Morris Thu, 10 Aug 2006 05:39:00 GMT

I got a license for Komodo Pro, (Komodo-Professional-3.5.3-262321-linux-libcpp5-x86) and started trying to use it, I have the latest KUbuntu, with GTK installed so I had no problems installing Komodo as far as libraries and requirements were concerned, however running it had me stumped, I ran into immediate problems (some of my own making I'll admit).

First I saw a stream of errors on the screen, they didn't seem fatal but are annoying, for instance...

(Gecko:24458): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_width: assertion `pixbuf != NULL' failed
(Gecko:24458): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_height: assertion `pixbuf != NULL' failed
(Gecko:24458): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_rowstride: assertion `pixbuf != NULL' failed
(Gecko:24458): GLib-GObject-CRITICAL **: g_object_unref: assertion `G_IS_OBJECT (object)' failed
(Gecko:24458): Gdk-CRITICAL **: gdk_pixbuf_get_from_drawable: assertion `gdk_window_is_viewable (src)' failed

For a commercial product costing around $299 that seemed a little unnecessary.

The first problem was my making I launched Komodo from the command line in the installation directory, then deleted the installation directory, which caused it to have hysterics about chdir etc, although to be honest an xwindows IDE probably shouldn't care where it is launched from, as it would normally be launched from a button or desktop icon.

Tech support was very fast in responding to my email though, and they didn't even know I had a pro license. They asked a bunch of questions, which I answered, but seeing as the remaining problems were non-fatal I wasn't too concerned about the response, which I haven't got yet, but I would like to know if all those GTK errors are normal.

The more disturbing problem was when I tried to write a simple Ruby script, I expected completion as you would see with Java and Eclipse, but instead typing:

str= ""
str.

gave me an error:

 error determining members: don't know how to make a suitable dumb_get_members for Ruby

This error was what I got at first, after going back and typing str. again I got the expected list of possible completions, seems this happens the first time every time, either it is slow figuring out what to complete or it is very buggy. (Update see later, it is the former, it is slow figuring out what type something is).

Time.

Did give me a list of completions most times. When it does decide to offer a list of completions, it also shows the parameters for the calls, like Eclipse/Java does, that's nice, I hope it is reliable though.

OK got word back from tech support, apparently it can take "a couple of seconds" for Komodo to catch up with typing, so my example of...

str= ""
str.

Would probably not work unless I waited a few seconds between the two lines, I think this may be acceptable as it is unlikely to be an issue in normal use. In practice it seems to be more than a couple of seconds BTW.

The main reason for me trying Komodo was for the code completion, without intelligent code completions I really don't see the use for an IDE, I can use my trusty Epsilon editor from Lugaru, with the ruby syntax highlighting and formatting extensions and my own contribution to the Epsilon library snippets, much like the great Textmate snippets (although not nearly as complete), there are also various user contributed extensions for various source code code control systems etc here.

Yes Epsilon is quite expensive for an editor, but I have been using it for nearly 20 years, upgrading every few years or so, it is basically Emacs with a c-like extension language (instead of lisp), it is very solid, and runs on all the platforms I use (Mac OS/X, Windows and of course Linux). It does not have code completion of course (although I think I can write an extension ala Vim7 to do that, watch this space).

Maybe I should mention what I am looking for in an IDE for Ruby and Rails...

I want:

  • syntax highlighting
  • auto indentation
  • auto-correction (or at least error highlighting, or suggestions)
  • snippets (or templates)
  • code completion (showing the possible methods for a variable called intellisense or intelligent code completion by some)
  • word completion based on something in the source already, or from a tags list or from a known-word list
  • showing the parameters for a given method I am currently typing
  • full debugging (with breakpoints, watches, stepping and variable examination)
  • SCCM integration (subversion, CVS, perforce)

Basically everything I get when I use Eclipse and Java.

I get some of this with Epsilon, and Textmate but they are not IDEs so do not have debugging or code completion, or error detection.

Back to Komodo...

It has Perforce, CVS and Subversion bindings built-in, and does show the files status in the project view.

The auto-indent works nicely, unlike RadRails which I tried but just didn't get along with as I kind of expect it to be as good as Java and Eclipse, but my main complaint is it doesn't even auto-indent. (I think intentionally).

The syntax highlighting in Komodo is OK but not great, I think that can be tweaked, but I haven't looked that closely.

The debugging does seem to work well, you can set breakpoints, and step over, into and watch variables, everything you would expect. This may be worth the price of admission (YMMV).

It does show you highlighting when it detects errors as you type, within the limitation of Ruby syntax.

The Snippets implementation is pathetic compared to Textmate or Eclipse, you can't specify tab stops, you can't type the trigger word for a snippet and have it expanded on TAB, you have to either bind it to a key sequence, or double click the snippet name in the toolbox, that is simply clunky, and pretty much unusable. There are *NO* built-in snippets, you have to create them all yourself, so if you like typing "for" then tab and have a for loop created forget it, unless you create it yourself then you have to find the "for" snippet in the toolbox and double click it. I also couldn't find any on the web, I think for the price of this product they should supply most of the expected snippets. The other thing I don't like is the snippets will pop up a dialog box asking for input, instead of the usual way of just highlighting the text and typing over it to replace it.

My initial impressions is the editor part is clunky, given the excellent examples of Eclipse and Textmate, it simply doesn't compare as far as modern editing conveniences such as snippets are concerned.

The debugging seems capable, and probably the best I have seen for Ruby.

The code completion is slow and doesn't always work.

The source code control seems a little buggy in some places, although only from a UI standpoint. (Pops up strange dialog boxes on updates saying files need to be added, when they don't).

A serious bug is if the file changes on disk, (like if you do a revert), you get a dialog popping up saying the file has changed on disk, but you can't get it to go away, clicking OK just pops it up again, ad infinitum, you need to kill the program to move on.

All in all I think this would be a nice IDE if it were free, and the personal version may well be useful for some if you want decent debugging, but I simply can't see a justification for $299 for the pro-version. (At least not for this version). There is an alpha for the next version that claims to support Rails development, if they also fix the snippets and the code completion (which they have not in the current alpha), then we may be getting close to value for the money, but right now I think they have a long long way to go before this product justifies the price tag, and comes close to competing with the few other editors out there let alone IDEs.

Update

Tech support have been very responsive, and they came up with a macro that inserts snippets almost the way I expect,

komodo.view.setFocus();
try {
    komodo.editor.vCHomeExtend();
    var snippet = komodo.interpolate('%s');
    var tmpl = komodo.findPart("snippet", snippet, "container");
    if(tmpl) {
        Snippet_insert(tmpl);
    } else {
        var msg = "no snippet found named " + snippet;
        StatusBar_AddMessage(msg,"debugger",5000,true);
    }
} catch(e) {
    alert(e);
}

Add this as a macro, bind it to a key (say ALT-=), and then add your snippets to the same folder as the macro is in, name them appropriately, and the type im the name hit ALT-= and the snippiet is found and inserted, pretty neat.

Disclaimer: These are my personal opinions, I am not affiliated with any editor or IDE manufacturers.

Posted in ,  | Tags , , , , ,  | 1 comment | no trackbacks

Capistrano deploy from local repository

Posted by Jim Morris Sat, 27 May 2006 00:08:00 GMT

UPDATE 2006-12-06 I have replaced this with a full blown SCM module that works much better, see this posting

It seems that many people are in the same position as I was, they want to deploy to a remote server farm that does not have access to the SCCM (subversion,CVS,perforce etc), which is usually behind a corporate firewall.

The following recipe overcomes this by checking out the source code from the local repository (using svn in this case) and goes into your deploy.rb.

What it does is checkout the latest copy of the application from your local subversion repository, tars it up, and copies the tar file to the remote server, then detars it and does the normal deploy tasks from then on.

desc <<DESC
Update all servers with the latest release of the source code.
This is a modified version that copies a local copy to the remote site
DESC

task :update_code, :roles => [:app, :db, :web] do
    on_rollback { delete release_path, :recursive => true }

    #puts "doing my update_code"
    temp_dest= "tmp_code"

    #puts "...get a local copy of the code into #{temp_dest} from local svn"
    # but this could also just be your local development folder
    system("svn export -q #{configuration.repository} #{temp_dest}")

    #puts "...tar the folder"
    # you could exclude files here that you don't want on your production server
    system("tar -C #{temp_dest} -c -z -f code_update.tar.gz .")

    #puts "...Sending tar file to remote server"
    put(File.read("code_update.tar.gz"), "code_update.tar.gz")

    #puts "...detar code on server"
    run <<-CMD
        mkdir -p #{release_path} &&
        tar -C #{release_path} -x -z -f code_update.tar.gz &&
        rm -rf code_update.tar.gz &&
        rm -rf #{release_path}/log #{release_path}/public/system &&
        ln -nfs #{shared_path}/log #{release_path}/log &&
        ln -nfs #{shared_path}/system #{release_path}/public/system
    CMD

    #puts "...cleanup"
    system("rm -rf #{temp_dest} code_update.tar.gz")
end

This recipe does what many people need todo which is replace the database.yml with the production version, and also repalces the .htaccess with the production version (YMMV)

desc "fix up database and .htaccess"
task :after_update_code do
  run "cp #{release_path}/config/database.yml.templ #{release_path}/config/database.yml"
  run "cp #{release_path}/public/dot.htaccess.deploy #{release_path}/public/.htaccess"
end

Posted in ,  | Tags , , ,  | no comments | no trackbacks

Howto format ruby code for blogs

Posted by Jim Morris Fri, 26 May 2006 07:14:00 GMT

How do I get that nice formatted ruby code inline?

Well if you are on typo trunk use this…

<div class="typocode"><pre><code class="typocode_ruby ">       <span class="punct">...</span><span class="ident">ruby</span> <span class="ident">code</span><span class="punct">...</span>
</code></pre></div>

If you are not on typo trunk (which I am not yet) you can do the following…

I had to search around for this so I thought I’d put the recipe here.

I got most of my information from this site, you can also use this site to convert the code for you, but I found that more cumbersome than what I do below.

Basically all the hard work is done by the syntax gem install it as…

> gem install syntax

Then I use this little script called code2html.rb which converts the code in the clipboard and puts it back in the clipbboard…

require 'rio'
require 'rubygems'
require 'syntax/convertors/html'

if ARGV.size > 0
    code= File.read(ARGV[0])
else
    code= `dcop klipper klipper getClipboardContents`
end

convertor = Syntax::Convertors::HTML.for_syntax "ruby"
@code_html = convertor.convert( code )

puts @code_html

if ARGV.size > 0
    fn= "#{File.basename(ARGV[0], File.extname(ARGV[0]))}.html"
    rio(fn) << @code_html
else
    # put the results back on the clipboard, NB this may fail if there are shell specific characters
    system("dcop klipper klipper setClipboardContents \"#{@code_html}\"")
end

The clipboard stuff is kind of kde specific.

Alternatively you can specify a filename on the command line, and it will convert that file and put the results in a file with .html as the extension. (Note this requires the rio gem).

You will need this CSS available to your web page to render it nicely.

pre {
    background-color: #f1f1f3;
    color: #112;
    padding: 10px;
    font-size: 1.1em;
    overflow: auto;
    margin: 4px 0px;
          width: 95%;
}



/* Syntax highlighting */
pre .normal {}
pre .comment { color: #005; font-style: italic; }
pre .keyword { color: #A00; font-weight: bold; }
pre .method { color: #077; }
pre .class { color: #074; }
pre .module { color: #050; }
pre .punct { color: #447; font-weight: bold; }
pre .symbol { color: #099; }
pre .string { color: #944; background: #FFE; }
pre .char { color: #F07; }
pre .ident { color: #004; }
pre .constant { color: #07F; }
pre .regex { color: #B66; background: #FEF; }
pre .number { color: #F99; }
pre .attribute { color: #5bb; }
pre .global { color: #7FB; }
pre .expr { color: #227; }
pre .escape { color: #277; }

Posted in  | Tags , ,  | 8 comments | no trackbacks

Older posts: 1 2 3