Andy Hedges’ Tech Blog

This Beta release of Java has had the typical Sun treatment of not being promoted hard enough. Maybe I’m being unfair and maybe they are saving the hype for JavaOne - we’ll see.

Anyway one of the main feature of the snappyly names Java 6 update 10 (nee Java 6 update n) is that it has a modular basis. The kernel - all you need to run most things is 200K (flash player is 1.6M, Silverlight 1M, .NET Runtime 23M). It will then dynamically fetch what it needs to run each application or applet and cache that in case anything else needs it.

This isn’t the end of the story however there is now good support for 3D acceleration of 2D and 3D graphics and some great configuration parameters for Applets (like turning the system tray icon off).

As always a Java Applet can be run as a Java application and a webstart application.

As a demo of this here is Quake 2 running as an applet (you’ll need Java 6 update 10+ for the applet version):

• applet versions/embeded
• this will run the no embeded version

Yes, multiple player works you can run a Quake server within the applets and then anyone else running the applet on your network can join. This is a great demo of low latency high response network requirements being delivered in a browser and standalone environment.

Other examples showing JavaScript interacting with a Java applet is the NASA World Wind demos

• World Wind JavaScript Demo
• Other World Wind Demos

Most of this stuff I think will run with good old Java 5 but who would know nobody from Sun seems to be promoting it hard enough.

I have now moved the jmimeinfo project to a more suitable home at java.net. The project now has all the trimmings of a well run project: defect tracking, source control, forums, mailing lists and file distribution. Thanks to Jeff Dinkins for setting that up.

There have been no changes to the license it is still GPL and I will hopefully be doing a bug fix release in the near future.

jmimeinfo has moved please update your links!

I’ve been looking around for a java library that will do a similar task to the unix file command. The reason I need such a library is because, shortly, I’ll be allowing the upload of files from the public to a website of mine. When uploading a file via http it is encapsulated as a multipart message which includes the mime type. However what mime type gets put in the multipart is entirely dependent upon the uploaders browser and/or operating system. So if the user uploads a flash movie file but their system is unaware of the flash movie format the mime-type will likely be set to default such as application/octet-stream which basically tells me that it is a bunch of binary stuff - not much use.

Windows also provides a number of headaches (just for a change) in this areas, as every program installed on the machine is allowed to set mime-types at the OS level there is often some confusion. So depending on what programs a user has installed the same file maybe encapsulated with a different mime-type. For example on a Windows system I recently looked at there was a file called video.mpg, the *.mpg extension was associated with the movie/quicktime mime-type and upon looking at the file in a hex editor it was actually an RIFF based avi file - as a colleague of mine would say ‘genius’.

So this is how Windows deals with things, how do other OSes do it? Well I’m glad you asked. Linux or more correctly freedesktop.org which produces standard and software for numerous OSes (Linux, BSD, Solaris, HP-UX, AIX etc) have a standard for this, it is called ‘Shared Mime Info‘. Shared Mime Info has a specification and a reference implementation (in ANSI C). Shared Mime Info uses what are colloquially know as magic numbers and globbing to determine a file’s type.

Magic numbers simplistically work by looking for sequences of byte patterns in a file. So for example a java class file starts with the hex string ‘CA FE BA BE’ or an access database file starts with a normal string ‘\x00\x01\x00\x00Standard Jet DB’ or to pick a more complex example ELF executables start with ‘\177ELF’ and then 5 bytes in have the byte 1 then a further 16 bytes in have a little-endian byte 16.

Globbing is just a simple form of regular expressions matched against a file path. So for example the glob for a word document would be *.doc and the glob of a GNU make file would be [Mm]akefile.

Shared Mime Info also goes one step further providing plain English (or anyone of 30 other languages) description of each file format as well as suggesting applications that can deal with the file.

Anyway, so Java doesn’t have an implementation of Shared Mime Info… until now. I’ve timeboxed myself 2 hours over 2 days to put this together, it is a basic implementation of Shared Mime Info for Java, currently the code is a messy and it isn’t a full implementation yet, but it does provide a full implementation of magic and glob based mime identification and it’s licensed under the GPL. It requires Java SE 5 or higher.

The source and compiled jars can be downloaded here:

jmimeinfo-0.0.1.tar.gz (08-Nov-2006)

jmimeinfo-0.0.2.tar.gz (12-Nov-2006)
Fixed concurrency issue, I (stupidly) assumed that Matchers in ORO are threadsafe.

jmimeinfo-0.0.3.tar.gz (20-Apr-2007)
Fixed problem with RandomAccessFiles not being closed under certain conditions in net.hedges.mimeinfo.magic.Rule - Thanks to Ryan and KewlCat for point that out (see comments below)

To determine the mime type of file code the following:


File file = new File("/path/to/my/file.ext");
MimeInfo mimeInfo = new MimeInfo();
System.out.println("mime-type: " + mimeInfo.getMimeType(file));


To test the library run the following from the expanded archive:


java -cp jmimeinfo-0.0.1.jar:lib/jakarta-oro-2.0.8.jar net.hedges.mimeinfo.ui.Test /path/to/file.ext


All comments and feedback welcome as well as patches and bug reports

Technorati Tags: mime, shared mime info, freedesktop

The two lead developers on the Open Source project JRuby have been hired by Sun. Charles Nutter and Thomas Enebo will be working on JRuby fulltime now at Sun.

Sun seem to be putting a lot of effort into supporting other languages on the VM and specifically compiling to Java bytecode. It’s a truely interesting development although Visual Basic on the VM still makes me shudder.

Charles has some more on his blog.

(via Tor Norbye)

Technorati Tags: java, ruby, jruby

There has been some talk on the Java Posse and other places too of a presentation at JavaOne where they took a boring looking Swing based email client and ‘pimped’ it up to look pretty ace. As I couldn’t make J1 this year I missed the presentation but have been searching desperately for some screen shots, a copy of the binaries or even better the source. To that end does anyone know where I can find said application?

Technorati Tags: swing, ui, look & feel

This is the message I get when I naively try to install Sun’s J2SE on Gentoo. Now I thought that Sun had gone to lengths to reach out and make distributing Java easier for Linux Distros or was that just Ubuntu, either way it makes me pissed off every time I have to download Java separately.

* Please download jdk-1_5_0_07-linux-i586.bin from:
* http://javashoplm.sun.com/ECom/docs/Welcome.jsp[truncated]
* Select the Linux self-extracting file
* and move it to /usr/portage/distfiles


Update: Looks like Gentoo and Sun have been talking and this restriction will be removed soon! Fantastic news. Now if they just open source Java… (via Kit Peters)

Technorati Tags: java, j2se, freedbacking, sun, open source

Will Sun have the grace to reverse their bad decision?

Technorati Tags: java, derby, freedbacking

Much of the excitement the technology today is what can be done with media, by combining micro formats like RSS with audio formats like MP3 and devices like iPods with the increasing high speed Internet people are empowered to broadcast and listen to anything they want. With major media distributers such as the BBC putting their weight behind podcasts and vodcasts (video podcasts) the time to start using the video support in Java is now (actually it was yesterday).

So what support is there for Video on Java? Well there is JMF (Java Media Framework), however this was last updated in May 2003 (yep it’s 3 years old). It is also a pain in the arse to install and you can forget about a good user story for using it in applets. I was considering providing the Java applet video player on my website so that I could guarantee that if Java was installed on the user machine then they could watch video (sort of like YouTube’s flash player). However these are the install notes for JMF, I can’t see Joe Six Pack or Grandma managing that and I can see why both Google Video and YouTube choose Flash now for their players.

What other video support is their for Java? There is Java QuickTime but this only supports the platforms that normal QuickTime supports because it is all backed with native calls to the QuickTime libraries. So Java QuickTime only works on Mac and Windows, not much use for me on Linux.

Also I have my reservations on what the licensing terms are on both these APIs I think it is fairly unlikely I could distribute them as freely as I would need to if I was using them in a applet.

So far these are the only two free APIs I have found for Video with Java, if anyone knows of any others then please let me know. As it is I can see this issue losing Java mind share quickly in the new world of pub/sub media on consumer devices.

Technorati Tags: JMF, freedbacking, Java

So it seems everyone is saying it now, christ, people are even meta saying it pick one feed format and stick with it. All the aggregators worth their salt support all of them so it really doesn’t matter. I know when I add a feed to Bloglines I hate it when I get presented with 4 versions of the same thing just delivered to a different specification.

As the old ironic adage goes ‘the great thing about standards is that there are so many to choose from’. Having just reimplemented The Humor Archives Feed using Rome I fully sympathise with Sam, Nick and Dion’s position. My first implementation of a feed simply cranked out RSS 2.0 directly using JDOM, however with Rome one is encouraged to use an abstraction to populate the feed and then specify the feed type. Once this is done the specific feed specification compliant XML is cranked out to a Writer. This is all very good, however, it takes the minimal subset approach, so for example on an RSS 2.0 Feed type it is impossible without going to the specific implementation to set the TTL field as it isn’t a minimal subset feature of feeds.

I looked at using just the RSS 2.0 beans within the Rome framework but couldn’t see any obvious way to write it to an output stream or writer - I only look briefly however, if anyone knows please let me know. As I only want RSS 2.0 perhaps Rome isn’t for me.

Mark Shuttleworth founder of Twarte, Canonical Ltd and The Ubuntu Foundation (which distributed the popular Ubuntu Linux) comments on his blog regarding the open sourcing of Java following from Sun license changes that make it easier to distribute the JRE with Linux. Mark was presumably involved in the license change as he was wheeled out on stage during the announcement at JavaOne. Also at JavaOne it was stated that it wasn’t a case of if Java would be open sourced but when.

Sadly, for personal reasons, I couldn’t make JavaOne this year so I wasn’t there to hear the keynotes but having read reports from the keynotes it isn’t immediately clear what they intending to open. The licensing term changes only effect the JRE (the software that let’s you run Java applications but not create them) however there are numerous pieces to the Java Platform, there is of course the JDK (which allows you to create Java apps), the Language Specification, Java EE, Java ME, the Byte Code Specification and on and on.

Anyway Mark has some interesting thoughts on this and also the damage that has been caused by not Open Sourcing Java 5 years ago.

No I’m not joking, sun have actually gone and done this! Some screenshots, but very little in the way of justification can be found here. I’m hoping it’s to try and get people away from VB6 by providing a migration path to a sensible platform, but who knows.

I can already predict what will happen. I’ll be working away happily on a client application and suddenly the Java/Groovy/JRuby/Whatever code will drop out into some VB6 code that someone thought would be a good idea to copy/paste across from their spreadsheet macro. I’ll then have to spend hours fighting my way though a never refactored, organically fudged wad of VB6 spoodge. All we need is more excuses for people to hang on to their sacred code.

On the Mustang JSR there has been a lot of lively discussion on the benefits and drawbacks of providing reflective access to method parameter names. Currently in Java using the standard APIs there is no way to access the parameter names of a compiled java classes methods. In fact unless a Java class is complied using the -g flag with javac then the parameter names are completely discarded from the resultant bytecode. If the class is compiled with -g then they are preserved in bytecode and can be access by bytecode reading libraries (this is what IDE sometimes do in order to present more useful method information when source and javadoc are absent).

There have been calls prompted by this RFE for the addition of reflective methods to the java.lang.reflect.Method class to access the names of parameters as specified by the developer at coding time. These calls have been made by some members of JSR 270. To try and summaries a very long thread here are the arguments for and against:

• code that used the parameter names would become dependent on that parameter name remaining the same across revisions

Actually that’s it as far as I can see, if anyone can think of others please let me know.

Arguments for:

• Use for dynamic language support in java; many dynamic languages such as Python and Ruby benefit from having access to parameter names for certainly language features such as method calls like this (pseudo code):

  calculateArea(width := 10, height: = 20):

  This is cool because you don’t need to remember which way around the parameters go and it is also more readable.

• Logging: if you have reflective access to the parameter names then you can record them for the purposes of logging.
• IDE support: at the moment if you only have access to the compiled binary that hasn’t been compiled with the -g option then the ide will display generated variable names such as string1, int0 and so on which don’t aid the developer at all
• External add-ons to a Java application such as a rules engine could certainly make use of parameter names, enabling the user to define rules and validation using the same names as the methods parameters, this without having access to the source or specially compiled binaries
• AOP frameworks could certainly use these features too
• Binding of Java classes to webservice by the generation of WSDL from classes files would create much more meaningful/readable WSDL with the parameter names included
• It will be easier to add features such as closures to Java in the future using a well understood method of reflective parameter name access with the appropriate syntatic sugar

I would been keen to have this features in Java SE but as far as I’m aware it won’t be in Mustang (Java SE 6) but will be put forward as a recommendation to the Dolphin (Java SE 7) JSR

It seems at the moment that there is an arms race on with language specifications. I’m particularly referring to Java and C# but there are languages pushing them (the spec leads) on to include more and more language features. C# is introducing a raft of new features in C# 3.0, this is old news and the fact that Java introduced a set new features in J2SE 5.0 is even older news. I don’t think anyone would disagree that Java is behind on this race and it was really only the emergence of features such as autoboxing and generics in C# that push the guys at Sun to introduce these features in Java.

Now there is even more momentum to introduce even more language features to Java, things such as closures. Now from a selfish point of view I would love closures and many more features in the language when I’m coding. This is because I have been writing code now for almost 10 years and coding Java for about 8 years, plus working with other languages that have these features (Ruby, Python and even Perl) and so more features doesn’t pose much of a learning curve to me. However what it will do is make Java more inaccessible to people new to coding. It will also introduce more ways to do one things which for a commercial power house and main stay language is almost certainly a bad thing.

I have dealt with many code bases produced by inexperience or misguided developers throughout my career and one thing that has saved these code bases from unmitigated disaster is the fact that there is only a few ways for achieving things in Java. It keeps the developer under control, stops them straying too far from the path. Also for those who are experienced developers the simplicity and lack of features is also a good thing. A concept (program) based upon a few simple rules (features) is often more elegant than one based on many complex ones. I think studying mathematics has given me an appreciation of the beauty in keeping things simple and this principle apply just as well to coding and language specifications.

I’m, after intial hesitation, now keen to add scripting support to Java, this will hopefully stem the tide of additions to the Java language features and keep it clean. Obviously the use of these languages will need to be governed correctly to ensure projects don’t turn into the tower of babel but in the end I think the Java platform if not the language will be a richer technology.

Oh and C# can add what they want because I’ll be able to code in JRuby which is nice

Came across Ivy recently, which seems to have complete slipped under my radar until now. Ivy is an incredibly configurable dependency management tool. Now hold on it’s not one of these stupid head in the clouds tools it is an actual practical tool for managing extremely complex sets of dependencies.

It hooks in with ant or runs standalone and allows the declarative statement of your Java (or anything else actually, Ruby, .NET, C++ whatever) dependencies. It resolves these dependencies from repositories that could be on your local machine, in a subversion repository or even in one of the main Maven repositories.

So why is this tool better than Maven? Well it comes from the old Unix school of thought, do one thing well. It does dependency management extremely well and it doesn’t worry about anything else. The other stuff is left to applications better suited to that job (mostly ant actually). It can be used straight out of the box for modest requirements or configured to totally custom and/or complex requirements. [TBH I really don’t like Maven2 it’s: poorly documented, lacks flexibility and it’s just too much work to get going properly.]

Ivy also does a nice graphs for your dependencies along with HTML reports which help plan new releases of your dependent projects.

It’s documentation could be a little better but hey you can’t have everything.

Why does every Java IDE that supports Web development force the structure of your coding artifacts on disk to be the same as a war file? To my mind it is an extremely un-user friendly structure. I’ve just been looking at the latest copy of Webtools for the Eclipse IDE and it is no different.

Take a look at the attached screen shot to see what I mean.

Now not only does it force me to keep everything so it is diplayed in the horrible war file format, but because of this is doesn’t leave me anyway to organise my extraneous artifacts. I mean what the hell does WEB-INF mean to anyone intuatively? Do I have to dump all my hibernate configuration with my log4j configuration with my application configuration and my urlfilter configuration, quite quickly we are going to end up with a mess of crap in the classes directory. Where as I would like to have a configuration directory with subdirectory for each component that needs configuration files.

Where do I put my unit tests? Where do I put classes and jars that I depend upon at compile time but not at runtime, for example JAXB compiler jar files. I could have them as “external jars” but that stops my web project being self-contained, I like my projects be self contained. I want to checkout from source control and be able to compile without spending two weeks tracking down dependancies.

The solutions to my mind is fairly simple, some form of mapping needs to be allowed that enables you to declaritively instruct the IDE what should be where for the war but still allow you to structure things in the way you wish. This is what I usually do with ant but Eclipse isn’t yet clever enough to understand my ant file.

So although I’ll be evaluating Webtools some more I’m immediately turned off by it.

Now that I have installed Ubuntu 5.10 (Breezy Badger) I thought I would share my first impressions. The installation was fairly painless, very similar to Fedora’s really. Once installed sound didn’t work, quickly discovered that it was using my onboard sound card rather than my PCI card and that was altered fairly quickly though system preference.

I then decided to install Eclipse from the package manager. This went fairly smoothly but when Eclipse was run it was s-l-o-w. As Ubuntu uses gcj as the Java VM I thought I would give the Sun VM a try instead. Obviously because of Sun’s restrictive licensing I couldn’t just grab this from the package manager. I had to sign a EULA twice, once to download and once to install. I then had to offer up the the binary to the package manager and wait for it to pull it apart and put it back together in the correct manner. When will Sun get the message that this isn’t doing Java any good? For god’s sake Sun relax the distribution restrictions on your VM! Anyway the Java problems I faced can be laid squarely at Sun’s door and Eclipse ran much more smoothly with the Sun J2SE 5.0 VM.

Next issue I have hit is that Flash sound doesn’t work. I tried various methods of fixing this, creating Sym linking for the socket in /tmp and reconfiguring AOSS and offering it up to Firefox without success. Eventually after running the flash shared object through strings I noticed the linking was wrong and the following solved the problem (partially):

sudo ln -s /usr/lib/libesd.so.0 /usr/lib/libesd.so.1

So not really stuff for the faint hearted. Even once this is done there is still a problem with the sound lagging slightly behind the video - I haven’t resolved this yet and it is bloody annoying.

The next problem that I’m facing is that video (WMV, AVI) plays at twice (I’m guessing twice) the speed is should do. I have no idea why but I do suspect that the binary codecs I’m using are to fault.

To summaries I’m surprised that Ubuntu has thrown up so many issues after just a few hours of using it. I was lead to believe that it was a leader in terms of desktop linux usability, quite franky both Fedora and Gentoo are easier to use than Ubuntu so far.

People say if you just want to check emails and surf the web then Ubuntu is for you. Well that, in my experience, is not true. If you want to read emails and surf the web without Flash sound and without streaming video and that is it then maybe Ubuntu will do for you. I will keep on try to get along with Ubuntu for a week or so and see if I can figure out what all the hype around it is.

As multicore processors become more prevalent it seems to me that developers are going to have to get smarter toward writing the applications to exploit this fact. Most programming languages do work well for parallel processing simple tasks (other than the ones used on Supercomputers).

However it has occured to me that it wouldn’t be too hard to add some basic feature to languages to denote that things can be done at the sametime.

For example in Java (5.0) I could write a for loop as follows:


for (Customers : customer){
   customer.giveDiscount();
}


However wouldn’t it be good if I could run this in parallel (where possible, the VM can work out when) by writing this.


toevery (Customer : customer){
   customer.giveDiscount();
}


In a number of cases the data structures being iterated through in classic for loops don’t guarentee order anyway and so this new ‘loop’ would be better. This construct should be a signal to the VM that the coder doesn’t mind if the task in the code block executes in parallel, it makes no odds to the use case he is coding. The VM should monitor where there is resouce available which would be best spent performing this task.

Just a thought.

Now I like Ruby as much as the next guy, I really do, but there are some advocates out there that give it a bad name. This blog entry for example seems to imply that the fewer lines of code a particular method can be written in the better the language. Whilst I’m all for brevity in code I also like my code to be readable. The first piece of code whilst terrible code could be figured out by a junior developer whilst the second piece written in Ruby is a little more cryptic IMO. Whilst it’s hard for me to judge as I have been programming both languages for sometime I reckon anyone who has programmed in any language would be able to figure out the Java code whilst I’m not so sure about the Ruby. I certainly wouldn’t implement this method in this way in Ruby (and for that matter I wouldn’t write the Java the way the Java is written).

Apparently someone called Fred wrote that lines of code is proportional to time to develop a project and to overall maintenance costs. Now I can’t find a copy of the paper but I’m guess he is using SLOC (statement lines of code) as an indicator of complexity within the same language and with some assumptions about levels of clarity. So if a project has 2000 line of code and another has 4000 lines of code built in the same technology with the same level of readability then yes it take twice as long to develop and maintain. This is somewhat of a no brainer.

Anyway for fun I thought I would see how many lines of code in Java I could write the function in… I got two (excluding method sig and braces and unnecessary one before the System.out.print call so that it would fit on the page):


static void fmtString(String format, String number){
  int k = 0;
  for(int i = 0; i < format.length(); i++)
    System.out.print((format.charAt(i)+"").equals("#") ? number.charAt(k++) : format.charAt(i));
}


Now this is horrible and completely unmaintainable but according to the logic of the above mentioned blog post this will reduce my delivery time and maintenance costs - BOLLOCKS.

Whilst I believe in certain situations Ruby can bring down delivery and maintenance cost as well as delivery time it is no silver bullet and SLOC has nothing to do with anything. Ruby is not always the right solution and Java is not always the right solution.

Here is an interesting and funny satire on the state of Java.

Do I agree with it? A bit, quite a lot actually, although I don’t think a whole new programming language is the solution. Maybe I’m too attached to Java, maybe I’m falling foul of A Dozen Ways To Sustain Irrational Technology Selections.

I must make a note to checkout Ruby but somehow I can’t see it being used on the commercial projects I’m involved with in the near term.

So after all my tweaking did my Hibernate effort pay off? YES! Here is some log output of my production server (sounds a lot grander than it is) when I turned on DEBUG for Hibernate management code:


getMostRecentlyPublished took 54 millis
getAllCategories took 3 millis
getPublishedBefore took 19 millis
getPublishedBefore took 4 millis
getMostRecentlyPublished took 85 millis
getAllCategories took 4 millis
getMostRecentlyPublished took 55 millis
getPublishedBefore took 2 millis
getPublishedBefore took 5 millis
getAllCategories took 4 millis
getMostRecentlyPublished took 54 millis
getPublishedBefore took 69 millis
getPublishedBefore took 26 millis
getAllCategories took 3 millis
getMostRecentlyPublished took 86 millis
getPublishedBefore took 4 millis
getPublishedBefore took 4 millis
getAllCategories took 3 millis
getMostRecentlyPublished took 56 millis
getPageOfPublishedArticlesByCategory(Law, 0) took 86 millis
getAllCategories took 3 millis
getMostRecentlyPublished took 57 millis
getPublishedBefore took 59 millis
getPublishedBefore took 37 millis
getAllCategories took 4 millis
getMostRecentlyPublished took 58 millis


I’m amazed quite how fast some of these queries run considering the machine is just a single processor pentium 3 with 384M of RAM. Most of these queries are running 20x faster some are running 100x faster. Load average on the machine has dropped from about 4 to around 0.6 [load average is the number of processes competing for the CPU averaged over a window of time - usually 1 minute, 5 minutes and 15 minutes].

Now if I could get my hands on some proper hardware I could try out some funky JBoss Cache cacheing.