Friday, January 5. 2007svn:externalsI was recently working with someone who was using Zend Framework in their project. To keep things stable and releasable, he was doing an export of framework into his repository and checking it in. Since files change so much in the ZF project currently, instead of doing an rsync from a checkout into his own repository, he decided instead to delete the directory from the repository and re-add it everytime he was updating framework. This seemed really inefficient to me, especially considering that it made it incredibly difficult to merge changes from his development branch into his production branch (deleting and re-adding directories breaks the merge process considerably). I knew there had to be a better way. I'd heard of the svn:externals property before, but never really played with it. As it turns out, it exists for just this very type of situation. The problem is that the documentation of svn:externals in the SVN book doesn't indicate at all how the property should be set, and most howto's I've read omit one or more very important details. I finally figured things out through some trial and error of my own, so I'm going to share the process so others hopefully can learn from the experience as well. It's actually pretty easy. This assumes that your project layout looks something like this:
project/
branch/
production/
tag/
trunk/
One thing to note: any directory you specify for an svn:externals checkout should not already exist in your repository. If it does, you will get an error like the following: svn: Working copy 'sharedproject' locked svn: run 'svn cleanup' to remove locks I show using revisions above; you could also pin to tags by simply checkout the external repository from a given tag. Either way works well. Then, when moving from one branch to another, or from the trunk to a branch, you simply set a different svn:externals for each branch. For instance, your current production might check from one particular revision, but your trunk might simply track head; you then simply determine what the current revision being used is on your trunk, and update svn:externals in your production branch when you're ready to push changes in. Hope this helps some of you out there! Comments
Display comments as
(Linear | Threaded)
svn:externals is nice, but including third party development versions might be a bad thing, since they might break some APIs and such things, so you need to retest everything quite often. Usually it's way better to check in a version of the foreign software and use vendor branches for maintaining it. So you alway know which version of the external stuff you're using and can test your software after every update of the base framework (and might even apply unofficial patches...) See http://svnbook.red-bean.com/en/1.0/ch07s04.html
I completely agree about the 'third party development versions' -- which is why I mentioned the bit about setting a specific revision or using a tag when using the technique. The idea is that you update the third party software, test against it, and make changes accordingly. Then, when you push to your production branch, you change the revision used in your production branch.
The problem my friend had is that, by removing and re-adding a new checkout everytime, he made the merge process brittle. Using svn:externals will fix that in his particular use case. Hi Matthew,
even using tags (in the svn-way aka copies where no other commit /should/ be made - that'S really bad, SVN should really support CVS like tags, else it's a great tool, but tags aren't just copies of a specific revision...) don't assure you're having a stable version. In most cases I don't care too much about bugs in a used Framework/Library and more about bugs in my project's code. And from time to time uplink won't accept a (good) patch and it's better to use a custom patch. Tracking these is hard, too when just linking too an remote repository. I use svn:externals, too, but only for linking to my base libraries where I assure that I won't introduce anything which might break my application - or I at least know about it, but for third party libraries I really tend to an approach as described in the link I gave above. You might be interested in Piston: http://piston.rubyforge.org/
I've moved wholesale from svn:externals to Piston and couldn't be happier with the results. When I was using svn:externals to link to the Zend Framework subversion repository I was using one of the tags to ensure I was using a stable point-in-time snapshot of ZF.
I was dealing with exactly this problem just yesterday; thanks for the timely and informative post!
Problem 2 is that using a link to a tag actually still allows the target to change. You the user will not get a warning that the tag was changed. I use the -r syntax to lock down my svn:externals declarations as soon as I release from a branch.
Yes, it's possible for a tag to change -- but it's incredibly rare. But for absolute certainty, yes, use a revision number when defining the externals definition.
On a slightly different note; has anyone done a tool to scan a repo for svn:externals declarations recursively?
I suffer from creative developers, who use the features themselves. Gives me a bit of a headacke when branching larger projects. svn:externals is really handy if you have NUMBER of projects which are using the same external library. If you have couple projects only, you may import external library into project trunk, no problems with that. If number of projects will extremely grow, supporting of all imported libraries will be nightmare. svn:externals is a way to go.
If you need to have control over external library source code (apply patches, fix bugs, etc) it Great article - this is something I was just looking to implement, and didn't find the documentation in the Subversion book good enough.
Thanks Matthew! The fact that you can't svn:externals into a directory that already exists was precisely the problem I had when trying to do this with the Zend Framework. I had to have all my other code in, eg, lib/ and all the zend code in zflib/ because it's not possible to checkout Zend.php on its own.
I found this quite messy as I had 2 library folders, 2 include paths, etc etc, and I have sadly just reverted back to committing the ZF files directly. On the plus side, I only need to commit the files for the components that I need. If there is any way of checking the library out into lib/ then I'd definitely be interested in hearing it though! (Alternatively I suggested doing Zend.php, as I can't see any real reason why the ZF *needs* a base class... needless to say, that suggestion didn't go down very well!! Richard,
SVN supports symlinks, this means you might put the framework in zflib/ and then create a symlink to your lib/ folder and check that in. On platforms not supporting symlinks (Windows) you get a copy checked out which is kept in sync to the original file by svn. See http://svnbook.red-bean.com/nightly/en/svn.tour.cycle.html#svn.tour.cycle.edit Thankfully this also works on subdirectories. In my case, I set my externals to:
http://framework.zend.com/svn/framework/tag/release-0.6.0/library/ Thus I was able to grab just the library bits, without worrying about all of the other things. Good to find this: So an interesting case, java source trees. Eclipse like all source code in one area for it's projects so you can browse them all together but often the ifnal builds have different *NODES* of the tree isolated to make .jar files.
I might want // an "actual tree" actual/dir1/dir2 actual/dir1/dir4 // a "mirroed tree" mirror/dir1 -> actual/dir1 mirror/dir1/dir4 -> actual/dir1/dir4 (note no dir2 exists) *** Or actual/proj2/dir1/dir2 actual/proj1/dir1 // and the mirror mirror/dir1 -> actual/proj1/dir1 mirror/dir1/dir2 -> actual/proj2/dir1/dir2 I know I'm being bonkers here But is non-recursive each node indirected possible? for a master-project compilation do
for i in */.svn/entries;do echo $(dirname $( dirname $i)) $(head $i|grep url|cut -f 2 -d= );done whatever you do, AVOID irc://freenode.net/#svn these guys are a bit resistant to the idea of documentation that is unclear being imperfect in any way. cheers i don't understand how to use svn:externals even i read "official svn manual", but it's very easy after i read your article, very useful , thanks
I just wanted to note that for Windows users, you'll need to define the environment variable SVN_EDITOR and set it to notepad (or your editor of choice). Here is a nice how-to for that:
http://blogs.vertigosoftware.com/teamsystem/archive/2006/01/16/Setting_up_a_Subversion_Server_under_Windows.aspx Hope that helps. It helped me. Hmm, I'm having trouble with that "svn: Working copy 'sharedproject' locked" problem.
It's possible that in some past revision I had a directory where I now want to link in the external. But I don't anymore! I decided to use an external instead. Is there any way to recover from this situation? I really want an external at that location, but I'm getting the lock problem. Make sure you removed your directory using 'svn rm' and comitted your changes.
Wow, I love the Internet, I really do. I too noticed how most tutorials just not tell you how to really set it.
I am not sure why the SVN handbook is so poor when it comes to real world examples. They could save a couple pages here and there if they would only show you how to do it. Anyway, great post! I just googled this up and it solved my problem perfectly. Thanks 2 years later for this post!
If you are using a secure connection (svn+ssh:// ) then this command appears to be less usefull than tits on a boar.
Or maybe I don't understand it, but you have to use a FULLY qualified path on the right... (DUH? WTF??) Ok, so I have a directory of RTF files that I want to also put into bin/debug/RTF and bin/release/RTF... Should be no problem with an externals of: bin/debug/RTF RTF bin/release/RTF RTF Oh, wait, no, HAS to be fully qualified, ok, well, for me that URL is: bin/debug/RTF svn+ssh://Me@traderhut.com/SVN/Program/RTF And I just have to post my password to the entire group so everyone can pull down the source code... Entering my password to get this source code... Sweet huh? Security >> toilet I can't imagine why realitive paths weren't implemented the day after the full URL versions were... Inconcievalbe... Or am I missing something... -Chert ---- http://www.traderhut.com Chert,
The way I would approach this (and I'm going to try it in the next few days) is to set up ssh keys for each user. This has the effect of changing your fully qualified URL from: svn+ssh://Me@traderhut.com/SVN/Program/RTF to: svn+ssh://traderhut.com/SVN/Program/RTF Under Windows this can be done easily with PuTTY, or on Unix (or in my case FreeBSD and OSX) just be setting up the ssh keys correctly in .ssh Should solve your problem? Hi Matthew,
thanks for sharing I used TortoiseSVN > Right Click to a folder and then TortoiseSVN > Settings > Properties > New choose from the "property name" dropdown "svn:externals". NOTE: There is a "Apply property recursively" message which really works Slavi Thanks Matthew - 2 years on and your effort keeps on helping the rest of us out. Brilliant.
Ha, just fell over your instructions when I was looking to do something similar, just what I needed. Thanks
Good instructions - my only addition would be that Subclipse doesn't recognise the folder that the external dependency is checked out into and hence, it thinks there are uncommitted changes in the project. I solved this by adding the folder to svn:ignore which works well.
Add Comment
|
Calendar
QuicksearchLinks
CategoriesSyndicate This BlogShow tagged entries |
|||||||||||||||||||||||||||||||||||||||||||||||||





Continuing on with yesterday’s theme of revision control, I decided to post a little bit about how I’m using VCS these days. I’ve moved to using git as a front-end for SVN, using a workflow that follows along the paths laid out here and
Tracked: Oct 10, 16:13
*More Links* * [Maven Book - A Multi-Module Project|http://www.sonatype.com/books/maven-book/reference/multimodule.html] * [The Maven 2 POM demystified|http://www.javaworld.com/javaworld/jw-05-2006/jw-0529-maven.html?page=1] * [svn:externals|http:/...
Tracked: Apr 23, 17:38