What's annoying me with the build system is that it usually writes a lot to the disk. Not only is that quite unnecessary (as the files will be overridden in no time) but the laptop hard-disk is also quite slow and all this writing is trashing it.
The solution: write to a ramdrive ! As you all know, a ramdrive is a "virtual" hard-disk that sits in your RAM and goes away when you shutdown the machine (not that I care, the build files are temporary).
First step: create the ramdisk
There are some utilities that do this, but it's quite doable from the terminal (eventually with a shellscript).
- First, get your disksize in MB and multiply it by 2048. So 256MB means 524288.
- Next, create the device: hdik -nomount ram://524288 The command will also display the name of the new device file.
- Create a HFS filesystem in there: newfs_hfs -v "disk name"/dev/diskXXX , where diskXXX is whatever the previous command printed
- Mount the filesystem: mkdir /Volumes/diskXXX &&
diskutil mount /dev/diskXXX
At this point you should have a new 256MB drive mounted.
Second step: link maven folders
Now, I have to set the "target" folders on the ramdisk. Normally the orthodox way is to change the pom but I just didn't get it working. So my old-school solution is to use symbolic links.
This could be smarter as a "mvn clean" will remove the links we just create (but just keep a script in place that recreates this).
My script is:
for i in *; do
echo $i;
mkdir -p "$1/$i/target";
ln -s "$1/$i/target" "$i/target";
done
and I run it in the folder that keeps all my Maven projects (a flat hierarchy). Note the $1 which is the argument to the script. I use it like this:
$./linkramdisk.sh /Volumes/diskXXX
Building
All the links are in place so you can try a mvn install and see how fast it works. I my case, I reduced the build time (with no unit-tests) from 27 seconds to 17 seconds.
That doesn't seem much, but it does add up for larger projects and most importantly, it keeps the hard-drive out of the loop.
Oh, did I mention I also use FileVault on my account ? That's another reason one would like to avoid writing to disk: no need to encrypt something useless like a build artifact...
6 comments:
Perhaps the symlink creation script for the target dirs could be triggered as part of the build? Maybe with a snippet of Ant script added to the maven build? Not sure that Ant, or Java for that matter, handles symlink creation without NIO.2? In which case the Ant script could just exec a shell script process to handle it, just not in a portable way.
Nice idea, I'm going to try this out because build times are getting annoying.
From 48 seconds down to 23, so like you I've reduced the build time by around 50%.
I have a couple of custom code generation scripts that run on every build, without checking timestamps to see if they really need to be run. If I add that in, I could cut another ten seconds or so off of the time.
Yes, the disk is the slowest part of the system and it shows especially for big projects.
I have also been using a ramdisk lately as a temporary place for movies. It seems that if you have other IO processes running, the movie player might compete with those over disk access and lose. The result is skipped frames and seconds of sound-only 'movies'. Moving the movie to a 1G ramdisk (having over 2GB total helps) makes all this go away.
I wonder what the improvement will be like for the "mvn site:site" command. I run that under continuous build with Hudson, several times a day. The site generation creates *lots* of files, far more than a regular build. I think the improvement there could be even greater.
Nice, continuous build time down from 10 minutes to 3.
I now have this set up so that it automatically runs as part of the maven build. Its not portable between unix/windows, and there is currently no flag to disable it. These are features I may add later. Here's how its done:
Created script called 'create_target_dirs.sh'
#!/bin/bash
RAMDISK=/ramdisk
mkdir -p "$RAMDISK$1/target";
ln -s "$RAMDISK$1/target" "$1/target";
Created script called
'clean_target_dirs.sh'
#!/bin/bash
RAMDISK=/ramdisk
rm -f "$1/target";
rm -fR "$RAMDISK$1/target";
Added an antrun script to the root POM, that gets run with every clean.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>preclean</id>
<phase>pre-clean</phase>
<configuration>
<tasks>
<echo>Pre clean...</echo>
<echo>Delete symlink and target directory on the ramdrive.</echo>
<exec executable="${topdir}/clean_target_dir.sh">
<arg value="${basedir}"/>
</exec>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
<execution>
<id>postclean</id>
<phase>clean</phase>
<configuration>
<tasks>
<echo>Post clean...</echo>
<echo>Create target directory on the ramdrive and create symlink.</echo>
<exec executable="${topdir}/create_target_dir.sh">
<arg value="${basedir}"/>
</exec>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
That goes in the build/plugins section of the root POM, and by being there is automatically called by every sub-project.
Sorry for not putting this code in 'pre' blocks, but the comment thing wouldn't let me.
Post a Comment