Friday, August 16, 2019

Wayback Machine Downloader

Internet Archive's Wayback Machine is a gift to the world. For quick checks you just enter the URL and you get the archived version going years back.

A whole little cottage industry seems to have been formed around the Wayback Machine. They offer you whole-site download and conversions for the low price of $5 or $15 or $45 or however much they can convince you their service is worth.

Among these busy bees, the free Ruby based Wayback Machine Downloader is a little gem.

You just install it then run

wayback_machine_downloader -c 10 -s http://www.example.com

and you get everything! Total cost: $0.

Installing the actual gem on macOS as a non-admin user seems to have contradicting info online. There's a `gem install --local` command but it doesn't seem to do what one expects -- installing in the home folder of the current user.

What did the trick for me was:

gem install -i ~/.gem/ruby/2.3.0/ wayback_machine_downloader

and this after I manually downloaded the proper .gem file from rubygems.org

Some were even recommending to add a http (versus the default https) source to gem but that seemed foolish and even gem itself complained about using http in 2019.

Whatever road you pick with downloading from Wayback Machine, remember all the work the Internet Archive is doing for all this to be available to you and donate to them.

Saturday, June 08, 2019

Fair Source and the Fair Source Initiative

There's been some uproar about the MongoDB Server Side Public License which tries to prevent cloud vendors like Amazon take all the money in the MongoDB market.

Many are pointing out that this new license does not respect the Open Source Definition published by the Open Source Initiative.

In truth many users and companies would find the license acceptable. A legal advisor will clear the license, the software will be used and nobody except a vendor in a similar position like Amazon will care.

What this move towards a financially sustainable open source ecosystem needs is branding.

I suggest calling this new type of open source "fair source". Most people and companies understand that some money is necessary to keep a project alive and would find it palatable that the once you are big enough to disrupt the market for the author you should pay.

In order to help smaller companies that do not have a legal advisor at hand, a Fair Source Initiative foundation should be created. This foundation would review such fair source licenses and define them as acceptable or not.

In many ways in the same way as "open source" was introduced to make free software more acceptable to businesses, "fair source" will be about making an open source business model more sustainable.

Open Source was about dethroning the Free Software Foundation. Fair Source must dethrone the Open Source Initiative.

Perhaps the Open Source Initiative board will realize this and redefine the way they classify licenses. Otherwise they will find themselves irrelevant for a buzzing section of the software world.

Saturday, April 27, 2019

Apache NetBeans interview

As Apache NetBeans became a top level Apache project and finished the incubation process I was asked for an interview and my photo.

Only a single quote was taken from the interview and used on a not too positive article about NetBeans. The quote was presented as coming from me as a member of the 'Project Management Committee' to give it even more weight.

Bellow is my full interview for historical reference:

> Do you think Apache is the best place for NetBeans?

Churchill said that 'democracy is the worst form of Government except for all those other forms that have been tried from time to time'.

In the current context, there is no better place.

Maybe in some alternate universe Sun Microsystems didn't spend a full $1 billion on MySQL but took a chunk of that to create a NetBeans Foundation that rivals the Eclipse Foundation... but I'm not entirely certain it would have been better for the project.
 

> What kind of future do you anticipate for NetBeans under Apache?

This depends on how Apache and the other Apache projects value NetBeans. There is a lot of integration that would help both the projects and the end users.

The ASF is a large Java house now so having a programming language (Groovy), an IDE (NetBeans), build systems (Ant, Maven) and application runtimes (Tomcat, TomEE) means you can do some interesting things in sync.

If you think about it, the IDE is the last major piece of the puzzle missing from Apache. So now, you can push a new feature all the way to developers using the IDE really easy.

Imagine you want to introduce, say, reproducible builds to the Java developer world. Well, you change the build systems, have the runtimes also reproducible then push this to the default project types in the IDE and suddenly all new Java projects created by developers using the IDE are reproducible. You can really change and educate the world really fast.

And education is not to be understated. An IDE suggestion that the developer sees *while editing code* is educational. Want people to use libraries better? A blog post might help, but people have to find it and read it. But if a suggestion about how to use the library better is part of NetBeans, all developers will see it!

It's not clear to me yet if the ASF takes such a holistic approach, but there's a big opportunity here. Everything fits.
 

> What will your project management committee do to advance NetBeans?

This is a hard question for me since I'm not the 'manager' of NetBeans. Nobody is. The whole point under Apache is to participate as individuals, regardless of the employer (that might sometimes be sponsoring said involvement).

So, I don't necessarily see hard targets like under a strictly hierarchical corporation.

Changes happens somewhat chaotically but towards betterment. Many people submit specific bug fixes for their particular problems while other work on more big picture changes (like a new Java version being supported, etc).

On the Java front we have a lot of community members that work on it, a few of them full time as part of their job at Oracle. On PHP we also have some folks, particularly Jun-ichi Yamamoto from Japan. JavaEE is also quite popular. We had people add support for JUnit 5, etc.

Basically the community will self-organize to overcome obstacles. We had somebody fix a really hard bug in the Java profiler. I would have thought that only a handful of core Java developers from Oracle knew how to fix that. But the individual looked into it, worked hard and fixed it! There's a lot of hidden talent like that.
 

> Also, what is the role of the committee?

See https://www.apache.org/foundation/governance/pmcs.html

The PMC decides which new committers get added (which in turn decide how the code is changed) and then votes when a release is to be made as an act of the Apache foundation. We also oversee how the NetBeans trademark is used and basically take care of the NetBeans project and brand as a whole.

> Thank you very much!

No problem. BTW, the individual doing the hard Profiler bugfix I mentioned is called Peter Hull.

Thursday, April 04, 2019

The Apache Software Foundation is a record label not a rock band

What shocked me most during my involvement with NetBeans, now an Apache Software Foundation project, is that The Apache Software Foundation is a record label not a rock band.

Imagine you like a given band. You go to their concerts regardless of the location, enjoy their music, buy their records, maybe proudly wear a T-shirt. You deeply care about that band and the band cares about the music they make and their fans.

Once your band joins The Apache Record Label things might seem unchanged. The band still makes good music, released obviously exclusively through their new record label.

But something did change: while the band and the fans care about their future, the record label has a lot of bands to look after and only tangentially cares about a particular band. The band is also not doing much better since all their sales go towards the maintenance of the main music venue, lawyers, trademark protection, distribution fees, etc.

The misunderstanding about The Apache Software Foundation must have been caused by the fact that initially the Foundation was about a big and important project: the Apache HTTP Server. At that time I believe the fate of the project was quite important. Nowadays I believe the Foundation could retire the Apache HTTP Server and survive unscathed.

The other misunderstanding is caused by the fact that the technology landscape has some other software foundations like the FreeBSD Foundation, the OpenBSD Foundation, Mozilla Foundation which are all about a single project. These foundations basically live and die by that project.

It's an odd situation. The Apache Software Foundation provides competent support for its projects but has no skin in the game and if a project fails they will eventually acknowledge it in a board meeting and move on.

There's also no way to directly support a project via the Apache Software Foundation. The Foundation does not sponsor any kind of project software development. All the donations go to infrastructure and administrative costs. But projects rarely hurt for infrastructure while targeted development could help them and their users a whole lot.

Sunday, August 26, 2018

NetBeans Web Toolkit

I'm exploring NetBeans Web Toolkit with the articles here

NetBeans Web Toolkit is the new name I'm trying to give to Jaroslav Tulach's HTML/Java API, a rather impressive library that deserves more use.


Friday, March 02, 2018

Guards in Java

Haskell functions have this nice concept called 'guards' which allow you to define a condition and return a value when that condition is true.

For example:

abs n
  | n < 0     = -n
  | otherwise =  n

This makes the code rather readable, especially when you have more guards.

Guards build one upon another since you know that if your guard condition is checked, all the other failed:

something n
  | n < -2 = 10
  -- bellow we know that n > =-2
  | n < 0 = 8
  | otherwise = n

Back in Java land, where I get paid, I sometimes wondered if I should write a method as:

X method(Y param) {
  if (!param.isSomething()) {
    return null;
  } else {
    return param.getX();
  }
}

or if I should write it as

X method(Y param) {
  if (!param.isSomething()) {
    return null;
  }

  return param.getX();
}

I generally prefer the 2nd variant and now I realised these are a form of function guards!



Wednesday, October 04, 2017

The case of the different jsch 0.1.54 binaries

As part of the Apache NetBeans IP clearance we are combing through all the code and dependencies.

One interesting thing we bumped into was that the jsch 0.1.54 binary JAR we are using has a different hash (and size) than the binary JAR from Maven Central.

The old hash is 0D7D8ABA0D11E8CD2F775F47CD3A6CFBF2837DA4, the new one is DA3584329A263616E277E15462B387ADDD1B208D.

The binaries are 278,612 bytes vs 280,515 bytes in Maven Central.

Our version is actually the same as the one found on http://www.jcraft.com/jsch/

Also, the Maven JAR is properly signed with the author's CA7FA1F0 key.

This is where it becomes clear that reproducible builds are important. You do not want to have to wonder why a binary differs, especially years later when you are doing a review. And this one is a library doing SSH!

So, why the different binaries?

It seems the original JAR was compiled on Aug 30, 2016 with Java 1.4 (major version 48) while the Maven Central JAR was compiled Sep 3, 2016 with Java 5 (major version 49).

The original JAR also concatenates strings using StringBuffer while the Maven Central JAR uses the newly introduced in 1.5 StringBuilder. Which should also be a bit faster since it's not synchronized.

Next, most of the cypher classes use some reflection via a static static java.lang.Class class$(java.lang.String) method.

What is this? It's just the way class literals worked in Java 1.4. As explained here, in Java 5 the ldc_w instruction was introduced to load a Class object.

In 1.4 the class literal was helped by the compiler by actually introducing the helper Class class$(java.lang.String className) method and replacing the Person.class with a class$("Person") call.

It conclusion, it seems that excluding the Java 1.4 to Java 5 compiler changes, the two JARs are identical. With the Maven Central JAR even a bit better due to StringBuilder being used.

There is no check so far that the sources do produce the specific JAR. This is an exercise left for the reader.

Note: I have also cross-posted this blog post to the Apache NetBeans blog.

Wayback Machine Downloader

Internet Archive's Wayback Machine is a gift to the world. For quick checks you just enter the URL and you get the archived version go...