Friday, March 12, 2010

The OSGi Bundle Buddy

The Problem

As probably every developer, I want to keep my compile-test-deploy cycle as short and fast as possible. It shouldn't take longer than a few seconds to test a new feature or a bug fix. As long as I have unit tests I can directly run the tests in my IDE or use a tool like sbt (when writing Scala code) to have continuous test execution. However, often I want to evaluate the final product as well, and not only an isolated piece covered by a unit test. Do all my bundles work together? How does the UI look like? How responsive is the AJAX search box?

In order to do this, I need to combine my build artifacts and my target runtime. Since I am a full time OSGi developer, 99% of my build artifacts are bundles. The target runtime varies and depends on the project and task. Sometimes I deploy on Eclipse Equinox, sometimes (and more often recently) I deploy on Apache Felix Karaf and every now and then I embed a OSGi runtime within a WAR and deploy everything in Apache Tomcat. Obviously, executing this cycle takes much longer than running a few unit tests:

  1. Compile the source code
  2. Package all bundles
  3. Assemble/collect all bundles
  4. Copy bundles to target runtime
  5. Make sure the target runtime applies the changes
Especially 3 and 4 are most annoying. What if I only updated one bundle? I can easily do a "mvn package" in the corresponding module directory but I am still forced to execute the top level assembly process. This will collect all bundles, regardless of which I changed, and put them in a common location. From there, I have to manually copy the bundles to the target runtime. Given the presence of a tool like "mvn jetty:run", which automatically watches my class files and restart the web application on changes, I always thought that this process is way to complex. One of OSGi strengths is the dynamic, fine-grained update mechanism so with a bit of plumbing we should be able to improve our build process with it. Tools like fileinstall only help with step 5, since they rely on someone who collects all bundles and copies them to the watched directory.

The Solution

What I wanted is a tool that takes a running target runtime and my build environment and synchronizes them whenever the build changes. This tool should be independent from a specific target runtime (Equinox, Felix, ...) and independent from the build tool (Maven, Ant, Bnd in Eclipse, ...) and it only took approx. 150 lines of code (probably my best LOC/usefulness ratio so far) to implement this: The Bundle Buddy (TBB)

TBB is one simple bundle that you add to your target runtime during the development. Before you start the target runtime, you have set the environment variable "tbb" to the top-level directory of the project you are working on, e.g.

export tbb=/home/roman/workspace/myfancyproject

When you now start the runtime, TBB will recursively collect all jar files in that directory and align the result with the bundles already installed. Whenever a jar file in the build directory gets changed, TBB will update the corresponding bundle in the running OSGi framework automatically. It doesn't matter that the bundle location is different from the build directory path. TBB will synchronize by using the Bundle-SymbolicName and Bundle-Version.


You can get the TBB bundle file at github:


Currently I have these additional features in mind. Feel free to contact me if you have a fancy idea!
  • Specify more than one directory
  • Watch the class files as well and re-create the bundle jar file on the fly (only works if the manifest does not need to change)


Chris Aniszczyk (zx) said...

You could simply use PDE and self-host which takes care of this workflow. We have been doing it for years. We don't need to actually package the bundles and can run them from out of the workspace.

Roman Roelofsen said...

1. I don't want to use PDE

2. Your approach does not work with e.g. Maven

3. If a workflow doesn't integrate BND, I consider it to be broken

4. No everybody is using Eclipse

Hendy Irawan (Spring vs Java EE Dev) said...

I agree this problem is where OSGi is backwards compared to "plain Java". Not OSGi's fault though but IDE tooling support.

But since Java IDE tooling support is excellent, OSGi development is pressured to be on the same level.

Thanks Roman for your excellent work. Hope it continues.

I myself prefer both Eclipse and Maven. I like PDE for its convenience, but I want the option to not depend on PDE. i.e. on the same (Maven-ized) project, it should be possible to work on the project with or without PDE.

And I guess The Bundle Buddy is one way towards that utopia. :-) Thanks again Roman.

Iris said...
This comment has been removed by the author.
André said...

I currently work in a project that chose the pom first approach (pom manually written, manifests get generated by the maven bundle plugin). The maven projects are just plain java projects, PDE does not see any bundle. I might not be experienced enough building osgi apps with maven (used plain pde and pde build/athena so far) but I didn't find any way to have the best of both worlds so far. We used the sling maven plugin to install bundles to the felix console and the dev cycle's just a pain to me. I'm very pleased to get a better option now, your Bundle Buddy. Thanks!
I wonder though if a switch to a manifest first approach (hand-written or bnd-created manifests and generated/augmented poms) with tycho wouldn't bring that ultra-fast dev cycle I knew from PDE back again.

André said...
This comment has been removed by the author.
Riduidel said...

Hi, I find your idea very interesting, however there's one thing that keep buggin' me.
Suppose I have a bundle project which has maven dependencies (well, they're also OSGi bundles, but they're only available in one maven repository).
During my dev, those dependencies will appear in my Eclipse as "maven dependencies". But how will they be installed in my OSGi server ? Can TBB install them as well as my bundle ? I think in your case it works because your dependencies are in your proejct's library ... no ?

Roman Roelofsen said...

@Riduidel: The idea is that you use TBB *after* you deployed everything. TBB will then sync your project directory with the bundles currently installed. To install everything (including your dependencies) in the first place, you need the normal e.g. Maven assembly-plugin setup as usual.

Riduidel said...

Hi again, Roman.
I wonder if TBB can't be replaced by a good combination of felix fileinstall and copy on package of both generated artifact and its dependencies. Is it equivalent, to your mind ?

Ch@rbel said...

Hi Roman,

I am trying to compile Java files on the fly and package it in Jar then deploy it on an OSGi Framework at runtime.
I am using Janino to compile (at least I am trying not so easy when trying to add imported Jars).

Do you think I can use BBT to compile and install bundles on the fly?

Thanks and Regards,

Tor Hershman said...

It nice to know someone is takin' care of the nuts & bolts of this internet thingy.