Skip to main content

Posts

Showing posts from 2016

Migrating the extra large NetBeans Mercurial repository to Git

Note: This article is a living document and will be updated as I learn new useful information (last update 31st December 2016). I will move the helper scripts to a dedicated repository and copy part of this article into the Apache NetBeans wiki.

Introduction

The NetBeans source code has been stored in a Mercurial repository for almost a decade now.

But starting October 2016 NetBeans is preparing to become an Apache project.

And all incubating projects must store their source code on Apache Software Foundation infrastructure which only provides Subversion or Git hosting.

So, NetBeans must migrate its Mercurial repository to Git.

Size concerns

The NetBeans Mercurial repository covers 17 years of history and has grown to over 3GB.

Apache projects are mirrored on GitHub and they have a limit of 2GB or so. As such, any talk of migration started with ways of reducing the size by potentially splitting up the repository or removing some of the history.

Luckily, it turns out the NetBeans Mercuri…

Minimal NetBSD for Raspberry Pi 2

For no reason other than fiddling with the NetBSD codebase and build system plus the desire to reduce the world bandwidth waste, I've published a minimal NetBSD image for the Raspberry Pi 2 containing only the base set.

You only have to download this 76MB compressed image which becomes under 500MB uncompressed so it fits any recent microSD card you have in your drawer.

I have successfully used such a NetBSD system as a customer proxy of sorts this whole year.

Because lots of times you just need a machine with SSH and a shell.

Of course, my patches are minor, it's not like I managed to fit NetBSD onto a floppy disk but I liked doing it none the less!

NBnotify - Native NetBeans Notifications

NBnotify is back with a vengeance: Windows, macOS and Linux notifications from NetBeans plus the very useful build notifications!
NBnotify started 6 years ago as a way to integrate NetBeans with OSX and to provide me with build notifications.

I have almost always used a MacBook Pro as a work laptop for my customers so it was important that NetBeans is hooked up into the Mac ecosystem.

Back then Growl was the rage, providing customizable notifications for a myriad of Mac applications.

NBnotify entered into a short limbo while I was busy doing NetBeans customizations, Javascript type inference, JDBC drivers, ANTLR parsers and all in all Java code for paying customers...

But as soon as I had some free time this winter I had to bring it up to date!

And not only that -- but I've added first Linux and then Windows support too!

Actually, since NBnotify natively binds to libnotify it should work on BSD systems too.

So, if you are using NetBeans, you have to use NBnotify!

NBnotify is close…

NetBSD cross compilation is a marvel

While trying to get a patched variant of NetBSD 7.0.2 to run on my Raspberry Pi 3 I had to build NetBSD many times.

And it was pretty amazing how this all worked.

You have a single entry point: ./build.sh And it does everything! It builds the kernel, the userland apps up to the install image you write on the microSD card.

Of course, having a top-level script that does everything is nothing new, it's basically the second question in the Joel Test.

What is amazing is the cross-compilation.

You have an x64 machine and build.sh will happily create the install image for any supported platform.

The Raspberry Pi image is just:

./build.sh -m evbarm -a earmv7hf -u -U release

then you look into the release dir, find arm7.img.gz and you are ready to go.

I ran the builds on a NetBSD virtual machine, but apparently build.sh will happily run with some small preparation even on Linux or macOS.

NetBSD on Raspberry Pi 3

I have used quite successfully a Raspberry Pi 2 running NetBSD 7 as a customer proxy and I assumed 7.0.2 would run on a Raspberry Pi 3.

As it turns out, the Raspberry Pi 3 ARM Cortex-A53 processor is different enough from the previous Cortex A7 processor we have on the Pi 2 that it needs some kernel changes.

And while the bleeding edge NetBSD current- does work on the Raspberry Pi 3, the stable NetBSD 7.0.2 does not.

I even tried to compile my own NetBSD 7.0.2 and backport the patch that added Pi 3 support, plus up-to-date firmware, but it seems it's not sufficient. There are probably more related commits that need backporting.

So, if you have a Raspberry Pi 3 you can only use current- builds. Go on the nightly build server, grab an .img.gz and test it out!

But if you want a stable NetBSD release, you'll have to wait some more or get a Raspberry Pi 2 instead.

Getting the Raspberry Pi 2 is not such a bad thing because the Raspberry Pi 3 wireless will probably not work for a wh…

Time Machine eating its own tail on a FreeNAS ZFS share

I work on a Mac and the OSX Time Machine is a very nice and simple feature to have backups.

While I have some other Apple gear I didn't get an Airport Time Capsule because I already had a tiny Airport Express, the Time Capsule is pricy and I'm not certain it does RAID.

So, what I use instead is the FreeBSD-based FreeNAS on an N54L MicroServer. FreeNAS allows you to easily create a Time Machine Apple Share and it works seamlessly.

The nice thing here is that the FreeNAS share is stored on a ZFS disk! And I have periodic snapshots for those shares.

So, on one side you have Time Machine storing periodic snapshots and only deleting when it needs space.

And on the other side you have the ZFS snapshots that guarantee that nothing is truly deleted!

This proves to be an interesting combination because Time Machine does not expect such a situation when you are low on disk space.

When you are low on disk space OSX's Time Machine will try to remove some data to free some space. But …

Signing NetBeans modules with a Time Stamping Authority (TSA)

Signing JAR files is a very good practice. And while a proper certificate is not worth the price and effort, self-signing is still a step in the right direction.

Ever since Java 5 jarsigner supported a Time Stamping Authority (TSA) with the --tsa and --tsacert parameters. A Time Stamping Authority is basically an online digital notary that certifies the point in time the jar was signed -- it is designed to prevent signing files after the certificate expired.

It turns out that while you can sign NetBeans modules using the FAQ steps, there is no support in the build harness for a TSA.

I found bug #243213 which also mentions NBM problems and I submitted a patch there.

So, if you want to also add a timestamp to your NBMs, apply this small patch on top of your NetBeans source repository and rebuild NetBeans.

Then, you just have to define in nbproject/project.properties another key with your TSA (I'm using StartSSL's here):

tsaurl=http://tsa.startssl.com/rfc3161


Forcing export of internal API in Java 9 with -XaddExports

I've long been a fan of NetBeans' module system and of OSGi so Java 9's modules are a big improvement to me.

Except modules are really good at enforcing API boundaries and stop allowing one to freely use any public class.

An error such as this is no fun:

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.IllegalAccessError: superclass access check failed: class A$1 (in unnamed module @0x3fb6a447) cannot access class jdk.nashorn.internal.ir.visitor.NodeVisitor (in module jdk.scripting.nashorn) because module jdk.scripting.nashorn does not export jdk.nashorn.internal.ir.visitor to unnamed module @0x3fb6a447
I've assumed that this has to be tweaked at SecurityManager level and played with -Djava.security.manager and -Djava.security.policy and the very handy -Djava.security.debug.

Alas, that doesn't help. (Although I'm still convinced it should, unless there is a bug somewhere).

What one needs…

BSD on Raspberry Pi 2

Raspberry Pi 2 is the version that finally seems a good purchase to me with its quad core processor and 1GB of memory. If you need WiFi, Pi 3 is even better since an external WiFi dongle consumes more power.

But I would prefer to use an official release of a known operating system instead of using custom forks like Raspbian.

FreeBSD provides an official image, but to them ARM is a tier 2 platform "not supported by the security officer and release engineering teams". So, no freebsd-update. The existing image is for CURRENT ie. bleeding edge. RELEASE would be nice.

OpenBSD has no plans to support it.

The winner is NetBSD: they have an official image since NetBSD 7!

So now NetBSD and this little Pi 2 will replace a much bigger and nosier machine for a dedicated task. I'm curious how it holds up!