Archive for the ‘Open Source’ Category

Microsoft buying Yahoo

Tuesday, February 5th, 2008

There is uproar in many Open Source circles that Microsoft might be buying Yahoo. Yahoo reportedly runs on a predominantly FreeBSD platform and Microsoft would likely change this as they did when they bought Hotmail to run on their own proprietary operating system.

Whilst this deal is almost certainly a blow for Open Source I don’t think it is that much of a big deal. There are plenty of large successful companies running their core systems on Open Source - I know, I’ve worked on countless numbers of them. Yahoo gradually moving to Windows will take an extremely long time. Firstly because Windows and Microsoft technology isn’t built for use at this kind of scale and secondly there will be a huge amount of legacy at Yahoo that is tied to their platform. If it weren’t a PR nightmare for Microsoft running their platforms on Open Source then it would make much more economic and technical sense to leave the systems at Yahoo the way they are.

Anyway I’m happy this is happening it is about time Google had some competition in the search space. Search hasn’t moved on since 2001 when Google reached tipping point as the dominant search engine.

My Prediction Came True

Thursday, December 20th, 2007

Some may remember I predicted in a post back in April 2006 that Firefox would overtake Internet Explorer in terms of user base by December 2007 (based on the statistics from my website The Humor Archives). It looks as though I was right. The statistics for December 2007 stands as follows:

Firefox: 50.9%
MSIE: 45.1%
Safari: 2%
Opera: 0.5%

based on 261,386 unique visitors to date in December.

Figures for last month were still in MSIE’s favour

MSIE: 53.1 %
Firefox: 42.5 %
Safari: 2.1 %
Opera: 0.6 %

based on 351,316 unique visitors in November 2007

Raytracing with Povray

Monday, September 17th, 2007

I have been messing around with Povray again. Below is my effort

table-still-life-huge-thumb.png

Hurray Open Source, boooo Apple

Monday, September 17th, 2007

Looks like the new checksum in the iPod database to restrict non-Apple products has been reverse engineered in the open source libgpod (a library that many open source music library managers use to talk with iPods) and so iPods can now work with Linux again. Given all the problems I have had with my iPods and their interoperability with any standards I will not be buying any Apple products for a long time.

jmimeinfo

Wednesday, July 11th, 2007

I have now moved the jmimeinfo project to a more suitable home at java.net. The project now has all the trimmings of a well run project: defect tracking, source control, forums, mailing lists and file distribution. Thanks to Jeff Dinkins for setting that up.

There have been no changes to the license it is still GPL and I will hopefully be doing a bug fix release in the near future.

PostgreSQL pounds MySQL

Sunday, December 3rd, 2006

Another reason, if you needed any more, to use PostgreSQL over MySQL is independent benchmarks have shown Postgres to be far faster than MySQL, 50% in fact. The gap gets even bigger when multicores are taken into account; MySQL adds 37% performance going from 1 to 4 cores; postgresql adds 226%, that really is an absolute pounding.

(via spyced)

Technorati Tags: , , , ,

Java Shared Mime Info

Wednesday, November 8th, 2006

jmimeinfo has moved please update your links!

I’ve been looking around for a java library that will do a similar task to the unix file command. The reason I need such a library is because, shortly, I’ll be allowing the upload of files from the public to a website of mine. When uploading a file via http it is encapsulated as a multipart message which includes the mime type. However what mime type gets put in the multipart is entirely dependent upon the uploaders browser and/or operating system. So if the user uploads a flash movie file but their system is unaware of the flash movie format the mime-type will likely be set to default such as application/octet-stream which basically tells me that it is a bunch of binary stuff - not much use.

Windows also provides a number of headaches (just for a change) in this areas, as every program installed on the machine is allowed to set mime-types at the OS level there is often some confusion. So depending on what programs a user has installed the same file maybe encapsulated with a different mime-type. For example on a Windows system I recently looked at there was a file called video.mpg, the *.mpg extension was associated with the movie/quicktime mime-type and upon looking at the file in a hex editor it was actually an RIFF based avi file - as a colleague of mine would say ‘genius’.

So this is how Windows deals with things, how do other OSes do it? Well I’m glad you asked. Linux or more correctly freedesktop.org which produces standard and software for numerous OSes (Linux, BSD, Solaris, HP-UX, AIX etc) have a standard for this, it is called ‘Shared Mime Info‘. Shared Mime Info has a specification and a reference implementation (in ANSI C). Shared Mime Info uses what are colloquially know as magic numbers and globbing to determine a file’s type.

Magic numbers simplistically work by looking for sequences of byte patterns in a file. So for example a java class file starts with the hex string ‘CA FE BA BE’ or an access database file starts with a normal string ‘\x00\x01\x00\x00Standard Jet DB’ or to pick a more complex example ELF executables start with ‘\177ELF’ and then 5 bytes in have the byte 1 then a further 16 bytes in have a little-endian byte 16.

Globbing is just a simple form of regular expressions matched against a file path. So for example the glob for a word document would be *.doc and the glob of a GNU make file would be [Mm]akefile.

Shared Mime Info also goes one step further providing plain English (or anyone of 30 other languages) description of each file format as well as suggesting applications that can deal with the file.

Anyway, so Java doesn’t have an implementation of Shared Mime Info… until now. I’ve timeboxed myself 2 hours over 2 days to put this together, it is a basic implementation of Shared Mime Info for Java, currently the code is a messy and it isn’t a full implementation yet, but it does provide a full implementation of magic and glob based mime identification and it’s licensed under the GPL. It requires Java SE 5 or higher.

The source and compiled jars can be downloaded here:

jmimeinfo-0.0.1.tar.gz (08-Nov-2006)

jmimeinfo-0.0.2.tar.gz (12-Nov-2006)
Fixed concurrency issue, I (stupidly) assumed that Matchers in ORO are threadsafe.

jmimeinfo-0.0.3.tar.gz (20-Apr-2007)
Fixed problem with RandomAccessFiles not being closed under certain conditions in net.hedges.mimeinfo.magic.Rule - Thanks to Ryan and KewlCat for point that out (see comments below)

To determine the mime type of file code the following:


File file = new File("/path/to/my/file.ext");
MimeInfo mimeInfo = new MimeInfo();
System.out.println("mime-type: " + mimeInfo.getMimeType(file));

To test the library run the following from the expanded archive:


java -cp jmimeinfo-0.0.1.jar:lib/jakarta-oro-2.0.8.jar net.hedges.mimeinfo.ui.Test /path/to/file.ext

All comments and feedback welcome as well as patches and bug reports :)

Technorati Tags: , ,

JRuby developers join Sun

Monday, September 11th, 2006

The two lead developers on the Open Source project JRuby have been hired by Sun. Charles Nutter and Thomas Enebo will be working on JRuby fulltime now at Sun.

Sun seem to be putting a lot of effort into supporting other languages on the VM and specifically compiling to Java bytecode. It’s a truely interesting development although Visual Basic on the VM still makes me shudder.

Charles has some more on his blog.

(via Tor Norbye)

Technorati Tags: , ,

Mencoder magic incantation

Tuesday, September 5th, 2006

I have been working on having flash based videos on The Humor Archives as Flash seems to be the most widely supported video format on the web, mpeg, real, quicktime and windows video can not compete for the it-just-works factor.

However not all the video clips (read none) that I get given or find for The Humor Archives are in Flash video format (flv). Therefore I needed to find a way to transcode every format under the sun into flv. The formats I currently have are Quicktime (mov), Windows Media (wmv, asf, avi), mpeg (mpg) and Real (rm) in a veritable cornucopia of codex including but not limited to: wmv7, wmv8, wmv9, divx, xvid ,H.264.

I needed to get these all into the FLV format using the Sorenson codec so that a flash player could play them. Now I thought that ffmpeg would do the job but it turns out that it doesn’t support a number of common closed codecs like the horrible Microsoft ones and so I looked to mencoder (from the guys who make mplayer). Mencoder uses binary codecs from vendors such as Microsoft as well as a large collection of open codex such as xvid to decode the Video, it then uses the Sorenson3 codec in ffmpeg to encode the final flv file.

However things are rarely that simple, there are numerous command line options for mencoder. I tried many variations, including the ones in the mencoder documentation. However the results were a little poor - blocky, pixelated. I was beginning to lose hope - I even started to look at commercial software *gasp* spend money! However one of the variations paid off and gives almost perfect reproduction on all format to flv and here it is:


mencoder myfileinshitformat.wmv -o myfileinflvformat.flv \
-of lavf -oac mp3lame \
-lavcopts vcodec=flv:vbitrate=200:cbp:mv0:mbd=2:trell:v4mv:predia=2:dia=2:last_pred=3 \
-vf scale=320:240,harddup \
-lameopts br=64 -ovc lavc -ofps 25 -srate 22050 \
-lavfopts i_certify_that_my_video_stream_does_not_use_b_frames \

Obvious really…

Did I mention that mencoder is completely free?

Technorati Tags: , , , ,

IE7 website

Monday, August 21st, 2006

Well this made me laugh. How long before the cease and desist threats start from MS?

tabs vs spaces

Tuesday, July 4th, 2006

I for one am sick of the debate over whether tabs or spaces should be used to indent code (there are more important things to worry about). We all want the same thing and that is for code to format right no matter what editor we are using. It looks like someone has come up with a simple yet brilliant solution for this, and whats more they have code (which is open source).

Link: Elastic tabstops (via Joel)

I’m not alone, others don’t get Macs either

Tuesday, July 4th, 2006

It seems that a number of A-List geeks agree with me that Macs are a rip-off and quite frankly not that special; both Mark Pilgrim (author a Dive into Python and general Python guy see wikipedia) and Cory Doctorow (journalist, blogger [boingboing] and SF writer: see wikipedia) are both dumping Mac OS in favour of Linux on cheap commodity hardware.

You can spec a Lenovo (who make ThinkPads after buying IBMs PC division) to the absolute maximum of it’s capacity and it is still cheaper than a MacBook Pro and furthermore it is much much higher spec. Combine that with the fact that Ubuntu is easy to use and looks good and you have a very compelling reason not to by a Mac at all.

Mark also sights ideological reasons for not wanting to continue as a Mac user anymore:

but it is particularly galling to realize that if I bought a new Mac, I would be subsidizing the development of an operating system that contains code whose sole purpose is to lock me into a specific hardware platform.

This is an important point and general distinction, Linux is Open and encourages Openness of software design and coding, whilst Mac forces proprietary software and forces you to use your machine in the way they want you to.

Whilst I can’t agree with the choice of Ubuntu, preferring myself to use Gentoo, I do agree that Macs are becoming very overrated, overpriced and DRMed to death. I can feel a ground swell of opinion away from Macs and Mac fanboyism - not before time.

Update: This from another A-Lister Cedric Otaku also echos my thoughts. It seems that more and more people are realising that Mac OS X and Mac hardware are not a silver bullet for OS design. In fact many people think that OS X is actually quite rubbish.

Installing Java on Gentoo

Monday, July 3rd, 2006

This is the message I get when I naively try to install Sun’s J2SE on Gentoo. Now I thought that Sun had gone to lengths to reach out and make distributing Java easier for Linux Distros or was that just Ubuntu, either way it makes me pissed off every time I have to download Java separately.



>>> Emerging (4 of 4) dev-java/sun-jdk-1.5.0.07 to /
!!! jdk-1_5_0_07-linux-i586.bin not found in /usr/portage/distfiles

!!! dev-java/sun-jdk-1.5.0.07 has fetch restriction turned on.
!!! This probably means that this ebuild's files must be downloaded
!!! manually. See the comments in the ebuild for more information.

* Please download jdk-1_5_0_07-linux-i586.bin from:
* http://javashoplm.sun.com/ECom/docs/Welcome.jsp[truncated]
* Select the Linux self-extracting file
* and move it to /usr/portage/distfiles

Update: Looks like Gentoo and Sun have been talking and this restriction will be removed soon! Fantastic news. Now if they just open source Java… (via Kit Peters)

Technorati Tags: , , , ,

Does Sun have any grace?

Thursday, June 22nd, 2006

Will Sun have the grace to reverse their bad decision?

Technorati Tags: , ,

FLOSS Java

Wednesday, May 24th, 2006

Mark Shuttleworth founder of Twarte, Canonical Ltd and The Ubuntu Foundation (which distributed the popular Ubuntu Linux) comments on his blog regarding the open sourcing of Java following from Sun license changes that make it easier to distribute the JRE with Linux. Mark was presumably involved in the license change as he was wheeled out on stage during the announcement at JavaOne. Also at JavaOne it was stated that it wasn’t a case of if Java would be open sourced but when.

Sadly, for personal reasons, I couldn’t make JavaOne this year so I wasn’t there to hear the keynotes but having read reports from the keynotes it isn’t immediately clear what they intending to open. The licensing term changes only effect the JRE (the software that let’s you run Java applications but not create them) however there are numerous pieces to the Java Platform, there is of course the JDK (which allows you to create Java apps), the Language Specification, Java EE, Java ME, the Byte Code Specification and on and on.

Anyway Mark has some interesting thoughts on this and also the damage that has been caused by not Open Sourcing Java 5 years ago.

Ivy Dependency Management

Thursday, March 2nd, 2006

Came across Ivy recently, which seems to have complete slipped under my radar until now. Ivy is an incredibly configurable dependency management tool. Now hold on it’s not one of these stupid head in the clouds tools it is an actual practical tool for managing extremely complex sets of dependencies.

It hooks in with ant or runs standalone and allows the declarative statement of your Java (or anything else actually, Ruby, .NET, C++ whatever) dependencies. It resolves these dependencies from repositories that could be on your local machine, in a subversion repository or even in one of the main Maven repositories.

So why is this tool better than Maven? Well it comes from the old Unix school of thought, do one thing well. It does dependency management extremely well and it doesn’t worry about anything else. The other stuff is left to applications better suited to that job (mostly ant actually). It can be used straight out of the box for modest requirements or configured to totally custom and/or complex requirements. [TBH I really don’t like Maven2 it’s: poorly documented, lacks flexibility and it’s just too much work to get going properly.]

Ivy also does a nice graphs for your dependencies along with HTML reports which help plan new releases of your dependent projects.

It’s documentation could be a little better but hey you can’t have everything.

Java IDEs and Web Development

Saturday, February 11th, 2006

Why does every Java IDE that supports Web development force the structure of your coding artifacts on disk to be the same as a war file? To my mind it is an extremely un-user friendly structure. I’ve just been looking at the latest copy of Webtools for the Eclipse IDE and it is no different.

Take a look at the attached screen shot to see what I mean.

web-inf

Now not only does it force me to keep everything so it is diplayed in the horrible war file format, but because of this is doesn’t leave me anyway to organise my extraneous artifacts. I mean what the hell does WEB-INF mean to anyone intuatively? Do I have to dump all my hibernate configuration with my log4j configuration with my application configuration and my urlfilter configuration, quite quickly we are going to end up with a mess of crap in the classes directory. Where as I would like to have a configuration directory with subdirectory for each component that needs configuration files.

Where do I put my unit tests? Where do I put classes and jars that I depend upon at compile time but not at runtime, for example JAXB compiler jar files. I could have them as “external jars” but that stops my web project being self-contained, I like my projects be self contained. I want to checkout from source control and be able to compile without spending two weeks tracking down dependancies.

The solutions to my mind is fairly simple, some form of mapping needs to be allowed that enables you to declaritively instruct the IDE what should be where for the war but still allow you to structure things in the way you wish. This is what I usually do with ant but Eclipse isn’t yet clever enough to understand my ant file.

So although I’ll be evaluating Webtools some more I’m immediately turned off by it.

Xgl and Compiz

Wednesday, February 8th, 2006

Today Novell announced Xgl and Compiz the new window manager and composition manager to go with it. Whilst this doesn’t sound particularly exciting to the uninitiated this things can do some pretty impressive stuff. Such as:

Miguel de Icaza the lead on the Gnome project and Novell employee blogs about it some more. There is also an official press release for the release of Xgl and Compiz to the open source X.org project. That’s right this stuff is all open source and all free! Woohoo.

Commentary from CNET

Kopete

Sunday, December 18th, 2005

Having used Gaim for years on my Linux machine I have recently discovered a superior IM client for Linux named Kopete. It is a KDE based application (but don’t let that put you off :p it works under GNOME too).

It is a lot more slick than GAIM which to be honest I have always found a little clunky. The interface is small which is exactly what I want from IM and further more it is one of the most intuitive applications I have ever used.

British Firefox

Thursday, December 8th, 2005

It’s really annoying me that there is no offical British version of Firefox yet.

Edit: Since writing this entry Mozilla have released a British version of Firefox - not before time ;)