Skip to content

matthewneale.net

I’ve resisted buying a domain for many years. Around ten years ago all my peers were hurriedly snapping up their names, and I couldn’t really see the point. The whole business of choosing someone to buy the domain from, and choosing who to host it, and administering it, all seemed a bit much when all I really needed was an email address. In the end I signed up for HoTMaiL (as it was then) for free and never looked back. Perhaps it wasn’t cool, but it was simple.

After starting this blog I thought about it again, but the same thoughts came to mind, and anyway, this is not intended to be a pro-blog, it’s just for me to record my progress on.

In the end though I finally succumbed, and this blog is now matthewneale.net. You may have noticed, you may have not, in theory it shouldn’t make any difference and matthewneale.wordpress.com is still active and will remain so.

The reason for the change of heart is that recently I’ve put the URL for the blog all over the place – it’s on Twitter, LinkedIn, Seesmic and Plaxo, it’s on FriendFeed and Facebook, it’s even on my CV. I only realised just how many places when I tried to change them all to the new domain. My feeling is that if I am going to have my main internet presence somewhere, it really should be somewhere as permanent as possible, and somewhere I can control independently of where I host it.

It’s dead easy now anyway, I just paid a tenner via PayPal and WordPress did the rest. [This is not an advert for WordPress.]

The funny part is still to come though, the day before I made my final decision, I saw this little advert on my WordPress Dashboard:

WordPress tip: Be the master of your own domain – make this blog easymattnealeonline.com for just $15 per year.

For those who read my post 2 weeks ago, you’ll know why this made me gawp and then laugh. Strewth! This is one of those moments when you’re sure someone in power must be watching.

Times Fly*

My six months are up! I’m done! Pencils are down! The project has been put to bed!

Time does seem to have flown, and even though I promised to write a lot on this blog, I haven’t. You’ll have to believe that this is because I’ve been far too busy to have had time.

Goodnight!

Goodnight!

From the sofa of a flat on Upper Richmond Road, Putney, through the fantastic Le Bureau in Battersea, to the dining table of an apartment on Rue Montorgueil, Paris, it has been a grand adventure. Though there were a few gaps during the move times, I’ve worked at least 15 hours a day and have written 24,700 lines of code. I’ve learned a lot – one category of things I’m glad I now know, and another category of things I really wish I hadn’t had to go through!

The most important thing is that I do feel I’ve executed on what I said I’d do. My plan was to spend six months prototyping my idea, to achieve three goals:

  • Have a working prototype of the thornier technical areas to demonstrate they can work.
  • Gain some clarity into the major technical hurdles and clarify the purpose.
  • Get a better idea what the rough time and cost investment would be to produce a first release version of the product.

The prototype is quite nice, it does an awful lot with a very simple user interface, which is exactly what I intended. With not much more than a few clicks you can select a number of filesystems, and databases, suck them from their current locations, record them, version them, and spew them (technical jargon for sure!) onto a whole host of other servers that you choose. Everything is transactional and filesystems and databases can be rolled back, and rolled forward as you please.

The ability to do this is one corner of the functionality that is required to be able to build the software I want, though actually in itself, the above could be quite useful.

So what now? I’ve taken a week off, I’ve brushed up my CV, and I’m going to look around for something interesting to do. A contract would be best, as I’m still dedicated to finding a way forward for my idea.

On the project front there are a few things that I want to clean-up in the prototype that didn’t quite get finished by the deadline, but most importantly I’m going to start putting the business plan on paper – starting right after this post.

Finally, there are a few parts of the prototype that may be useful in their own right. The XA Filesystem post was the first example of this and two other areas that are immediately obvious are:

  • To facilitate the ability to read a database and re-construct it elsewhere, I needed an RDBMS-neutral schema format. I looked around but couldn’t find exactly what I wanted, so I wrote one from scratch. I can analyse a database and produce an XML-based, vendor-neutral schema which then can be loaded up anywhere else you like. This could potentially be useful to others so I’ll put something together about it on this blog sometime in the next month.
  • The other is the WS-Coordination and WS-AtomicTransaction support I wrote for Apache CXF. This is slightly more rough, and probably serves more as a proof-of-concept rather than anything that could be used straight away. Also, I imagine we’ll see this in CXF at some point, but I’ll post something on it soon if Apache doesn’t get to it first.

* Title from an Orbital EP Times Fly, 1995.

Product-Naming Mistakes

A Friday morning post in the general amusement category!

After an entire morning working through a mind-numbing list of software packages thrown up by a Google search, I feel I have to make a few observations.

The names have been changed to protect the ‘innocent’.

1. If your product name shortens to an acronym that’s a little dubious, please don’t use the acronym on your website

For example, if your software is a system to help a circus owner seat important VIPs, and say you decided to call this product Seating Of Dignitaries, then that’s fine with me, there is only a limited namespace available. However, don’t use the acronym! Don’t put on your website:

Click here to download Seating Of Dignitaries (SOD)

Saw it this morning!

2. Be careful about what you really mean when you call your product * Manager

Sticking with the circus, if your software is to help lion-tamers manage their lions then calling your product Lion Tamer Manager is not at all correct. That would be a piece of software for the circus owner to manage lion-tamers. See?

3. And finally, putting the word ‘online’ in your url isn’t going to fool me (for long)

There is nothing wrong with PC-based software. The world may be moving towards hosted software like a, erm, leaping circus lion, but this doesn’t make your product obsolete overnight. It’s probably still the best in it’s class. Don’t worry about it. Putting the word ‘online’ in your url to cover-up for the fact isn’t going to fool me.

To go back to Lion Tamer Manager, calling your site http://www.liontamermanageronline.com and then giving me a link to order the cd is just desperate.

Pinch of salt 😉

Positive Reinforcement

I’m keen for my application to keep the employee interested and involved in their job, perhaps even in the gaming sense I’m keen for my application to reward the employee for their positive behaviour. Now while this could sound rather sinister, as does the creepy behavioural title of this post, that’s not my intention, so I’ll try to explain.

In my previous job we used an ageing piece of software called ClearDDTS to track defects and change requests. This had a very simple feature whereby it emailed you when the status of the defect or change-request you were working-on changed. So if the defect went through test successfully, you’d get a mail, or if the defect was finally closed once the customer had approved it, you’d get a mail. The email was useful for example if your defect had been re-opened so you knew to look at it again, but other than that I feel the potential was under-utilised.

I’d like to use similar techniques in my application, but not only to keep people informed of the status of their previous (or upcoming) work, but also to keep people interested. Ideas from modern social media crop-up here too – these days, staff are often spread around the globe and I know only too well how annoying it can be receiving an email or a call from someone you’ve never even heard of, asking you why something you’ve done works the way it does.

Before I go any further, I’m not building a social networking tool, nor am I building a defect tracking system. The software I’m focused-on is designed in part (it has many facets) to aid communication and collaboration within large enterprises, and part of the purpose is to track work done on (and changes made to) software.

As some examples of the kinds of things I’m thinking of: I’d like to see the user informed when work they’ve done goes into test, I’d like them to know who is doing that testing, what that person’s bio is and what they look like. I’d like to see the user informed when the work they’ve done is delivered to the customer, I’d like them to see any customer feedback, a log of what occurred during the installation and who performed it. I’d like to see the user informed when someone else in their team takes their changes to incorporate into their own work. I’d like the user to be able to see the work of those around them, so they can understand at a glance how it impacts them. Most of all I want this visualised in a gui. Part of this is a news feed (how very Facebook), but I’d also like a visual representation of it.

I’ve been thinking about this on and off for a while and it led to a 4:30 or so wake-up one morning with my brain buzzing randomly about the standard Organisation Chart. That led to these thoughts. The video may be slightly jokey, but my point is serious. The Org Chart tells management how many people they manage, but it doesn’t describe to the individual their daily communication, or their responsibilities, or their worth. Let’s face it, if you’re at the bottom of an Org Chart, where most are, you’d be forgiven sometimes for thinking that the work you’re doing is just a drop in the ocean.

If the Org Chart demonstrates to you how insignificant you are, how much benefit would there be to providing an employee with an ever-changing, constantly up-to-date diagram of where they sit in their organisation? What impact does what they do have? Lots, generally. When you fix a defect in a piece of software, how many other people work with what you do? How many people are impacted before it goes out the door? The fact that you introduced a bug, or forgot to document something clearly, how many people did this negatively impact? What were their names? What do they look like? Where did they go on holiday? (You can mention this when you contact them to apologise.) How much better would I feel if I could see visually the good that came of what I did today, and when it’s negative, perhaps it would improve me as an employee to know.

Some of this wasn’t a problem in the ‘old days’ when people generally only worked with others in the same room, but large companies now are run across many countries, and many of us work daily with two or three other countries. I think this is genuinely great for the individual, it’s one of the most wonderful things about my previous job, all the people from so many different countries that I worked with, it’s just that companies haven’t yet evolved to take any other advantage of this other than the reduced cost.

The Org Chart has no place in my application, but I feel a visual representation is important. I’m imagining a diagram that shows you, and your work, and your links to your peers (whether local or remote). I’d like this to be active and clickable so the user can drill down, or scroll around and find out a little more about their peers – who they are and what they are working on.

To my mind this will give an employee not only an idea of what is upcoming for them and what has happened to the work they’ve done previously, but it will also give them an understanding of what is going on around them, and a sense of worth and position in their organisation. So while the title of the post could be seen as creepy, I’d rather the emphasis was on the word positive rather than the word reinforcement, and the ideas thought of as a benefit to the everyday enjoyment of the employee, as well as, as a benefit to the organisation they work for.

An XA Filesystem, update

In a comment on my original post, Alessandro Apostoli asks for an example of how to use the XAFileResource class, and specifically how to pass the Xid.

Digging this out has proved an interesting exercise (the code didn’t work at all any more, as I had expected), and it has raised two points worth noting. The second one I believe may be the answer to Alessandro’s question.

The EJB test code is included at the bottom of this post. It was written for Glassfish v2ur2, and I’ve tested that it all works. If you would like a copy of my Eclipse workspace containing the EJB, drop me an email – mdneale on hotmail.com matt@matthewneale.net.

1. String representation of a Xid

As I wrote in the original post, I’m not actually using EJBs in my application, and I’m not using an application server, though I do have a standalone J2EE Transaction Manager. The original tests were done against an application server so that I could prove the theory worked before piling-in more unknowns.

As is noted in the comment at the top of the XAFileResource class, for it to work, the Xid class that is passed to it must have a toString() method which returns a String representation of the Xid. Sadly the Glassfish Xid implementation com.sun.jts.jtsxa.XID doesn’t – invoking toString() on it always returns null. This probably makes sense, as com.sun.jts.jtsxa.XID can’t really know which String form would be useful.

Going back to my original very rough implementation of the XAFileResource class, I had a method in there which converted a Xid to a String, to workaround this exact problem. The problem doesn’t occur in my application as my Xid implementation does provide a String representation when toString() is called.

How you design this is up to you, but for simplicity’s sake, you can just add the following xidToString() method to the XAFileResource class, and change all calls to xid.toString() to a call to this new method. You’ll probably also want to look at the other uses of ‘xid’ as the rest are in log4j debug statements, and you probably will want to see the String representation there too, rather than just “null”.

protected String xidToString(Xid xid) {
	String xidStr = null;

	if (xid != null) {
        	String globalTransactionIdStr = "";
        	String branchQualifierStr = "";

        	if (xid.getGlobalTransactionId() != null) {
                	globalTransactionIdStr = new BigInteger(1, xid.getGlobalTransactionId()).toString(16);
        	}

        	if (xid.getBranchQualifier() != null) {
                	branchQualifierStr = new BigInteger(1, xid.getBranchQualifier()).toString(16);
        	}

        	xidStr = globalTransactionIdStr + "." + branchQualifierStr;
	}

	return xidStr;
}

One word of warning – the Commons Transaction FileResourceManager uses the transaction Id passed to it (the string representation of the Xid) to create a directory on the filesystem, and therefore the String representation needs to be within the limits the filesystem imposes on filenames. The above works fine for me (Glassfish on OS X 10.5), but watch out for that on other Application Servers and Operating Systems.

2. Connection Class

In my original post I alluded to the need for a Connection class, but I didn’t want to include it as I didn’t want to confuse the original intent, which was to communicate the XAResource implementation.  However, looking back at the original test code I can see it could be confusing without a note about it.

The issue is that when the XAFileResource is enlisted in the transaction, the Transaction Manager passes it the Xid to use. However, outside the Transaction Manager and the XAFileResource class, the Xid isn’t visible, so how do we get hold of the Xid in order to pass it to further calls to FileResourceManager (for instance for creating files, writing to files etc)?

The way I dealt with this in my original EJB test case was to store it in a static – a quick dirty fix for the test case, but of no further use as it won’t work for more than one transaction.

The way I’ve dealt with this in my application is to write an XAFileConnection class, which wraps an XAFileResource, but also implements XAResource itself. The XAFileConnection is what is enlisted in the transaction, and thefore, when the start() method is called on it we can store the Xid for use later. The idea is that you keep one XAFileResource class per FileResourceManger, and one XAFileConnection per transaction. This means you need some kind of tracking to ensure one connection per transaction. In a J2EE environment this is normally taken care of for you, but here you’ll have to write some kind of DataSource.

I don’t want to reproduce my entire XAFileConnection code here, as it’s large and repetitive. You’ll get the idea from a simple example. The following is what is used now in the EJB test case:

package txtest;

import javax.transaction.xa.XAException;
import javax.transaction.xa.XAResource;
import javax.transaction.xa.Xid;

public class XAFileConnection implements XAResource {

	private XAFileResource xaFileResource = null;
	private Xid storedXid = null;

	public XAFileConnection(XAFileResource xaFileResource) {
		this.xaFileResource = xaFileResource;
	}

	/**
	 * The start method stores the Xid for this 'connection' and then invokes
	 * the XAFileResource.
	 */
	public void start(Xid xid, int flags) throws XAException {
		this.storedXid = xid;
		xaFileResource.start(xid, flags);
	}

	// The rest of the methods just pass through to the XAFileResource

	public void commit(Xid xid, boolean onePhase) throws XAException {
		xaFileResource.commit(xid, onePhase);
	}

	public void end(Xid xid, int flags) throws XAException {
		xaFileResource.end(xid, flags);
	}

	public void forget(Xid xid) throws XAException {
		xaFileResource.forget(xid);
	}

	public int getTransactionTimeout() throws XAException {
		return xaFileResource.getTransactionTimeout();
	}

	public boolean isSameRM(XAResource xares) throws XAException {
		return xaFileResource.isSameRM(xares);
	}

	public int prepare(Xid xid) throws XAException {
		return xaFileResource.prepare(xid);
	}

	public Xid[] recover(int flag) throws XAException {
		return xaFileResource.recover(flag);
	}

	public void rollback(Xid xid) throws XAException {
		xaFileResource.rollback(xid);
	}

	public boolean setTransactionTimeout(int seconds) throws XAException {
		return xaFileResource.setTransactionTimeout(seconds);
	}

	public Xid getXid() {
		return storedXid;
	}

	/*
	 * Better to do this, and hide the FileResourceManager inside XAFileConnection.
	 * This way you can remove the getXid() method.
	public void createResource(Object resourceId) throws ResourceManagerException {
	}

	public OutputStream writeResource(Object resourceId) throws ResourceManagerException {
	}
	*/

}

If you look at the EJB test code below you’ll see how it all hangs together. In my real code, I don’t return the Xid as I do here. What I’ve done is add methods to XAFileConnection to perform all the FileResourceManager’s tasks – i.e. createResource, writeResource etc. These don’t take a txid parameter any more, as it’s known to the XAFileConnection class. This way we’ve now completely abstracted the actual FileResourceManager.

3. The EJB test code

The one EJB java file is reproduced below with its remote interface. As I mentioned at the top of this post, if you would like an archive of my Eclipse workspace, just send me an email.

The EJB creates a file, and inserts a row into a table. The first argument to the method test() is used as the filename and the data inserted into the table. The second test() argument says whether we want to commit or not. If we don’t, then an exception is thrown and the container-managed transaction rolls-back.

You’ll notice the xidToString() method is duplicated here – this will go away if you hide the FileResourceManager inside the XAFileConnection class as mentioned above.

package txtest;

import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.math.BigInteger;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;

import javax.ejb.Stateless;
import javax.ejb.TransactionAttribute;
import javax.ejb.TransactionAttributeType;
import javax.ejb.TransactionManagement;
import javax.ejb.TransactionManagementType;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.sql.DataSource;
import javax.transaction.RollbackException;
import javax.transaction.SystemException;
import javax.transaction.Transaction;
import javax.transaction.TransactionManager;
import javax.transaction.xa.XAException;
import javax.transaction.xa.Xid;

import org.apache.commons.transaction.file.FileResourceManager;
import org.apache.commons.transaction.file.ResourceManagerException;
import org.apache.commons.transaction.file.ResourceManagerSystemException;
import org.apache.commons.transaction.util.Log4jLogger;
import org.apache.commons.transaction.util.LoggerFacade;
import org.apache.log4j.Logger;

@Stateless
@TransactionManagement(TransactionManagementType.CONTAINER)
public class XATestEJB implements XATestEJBRemote {

	private static final Logger logger = Logger.getLogger(XATestEJB.class);

	private static FileResourceManager fileResourceManager = null;

	static {
		LoggerFacade loggerFacade = new Log4jLogger(logger);

		fileResourceManager = new FileResourceManager(
				"/Users/mdneale/xa-test/base",
				"/Users/mdneale/xa-test/work",
				false,
				loggerFacade);

		try {
			fileResourceManager.start();
		} catch (ResourceManagerSystemException e) {
			logger.error(e);
		}
	}

	@TransactionAttribute(TransactionAttributeType.REQUIRED)
    public void test(String filename, boolean commit) throws XATestEJBException {
    	logger.debug("test(filename=" + filename + ", commit=" + commit + ")");

		try {
			insertRow(filename);
			createFile(filename);

			if (!commit) {
				throw new XATestEJBException("Forced Rollback");
			}
		} catch (Exception e) {
			logger.error(e);
			throw new XATestEJBException(e);
		}
    }

	protected void insertRow(String filename) throws SQLException, NamingException {
		Context context = new InitialContext();

		DataSource dataSource = (DataSource)context.lookup("jdbc/xatest");

		Connection connection = null;
		PreparedStatement statement = null;

		try {
			connection = dataSource.getConnection();

			statement = connection.prepareStatement("INSERT INTO TEST ( filename ) VALUES ( ? )");
			statement.setString(1, filename);

			statement.executeUpdate();
		} finally {
			if (statement != null) { statement.close(); }
			if (connection != null) { connection.close(); }
		}
	}

	protected void createFile(String filename) throws NamingException, SystemException, XAException, IllegalStateException, RollbackException, ResourceManagerException, IOException, XATestEJBException {
		Context context = new InitialContext();

		TransactionManager transactionManager = (TransactionManager)context.lookup("java:appserver/TransactionManager");

		Transaction transaction = transactionManager.getTransaction();

		// Create the XAFileResource
		XAFileResource xaFileResource = new XAFileResource(fileResourceManager);

		// Wrap the XAFileResource in an XAFileConnection
		XAFileConnection xaFileConnection = new XAFileConnection(xaFileResource);

		if (!transaction.enlistResource(xaFileConnection)) {
			throw new XATestEJBException("Failed to enlist resource in transaction");
		}

		fileResourceManager.createResource(xidToString(xaFileConnection.getXid()), filename);

		OutputStream outputStream = fileResourceManager.writeResource(xidToString(xaFileConnection.getXid()), filename);

		PrintWriter writer = new PrintWriter(outputStream);

		writer.print("HELLO");
		writer.flush();

		outputStream.close();
	}

	protected String xidToString(Xid xid) {
		String xidStr = null;

		if (xid != null) {
	        String globalTransactionIdStr = "";
	        String branchQualifierStr = "";

	        if (xid.getGlobalTransactionId() != null) {
                globalTransactionIdStr = new BigInteger(1, xid.getGlobalTransactionId()).toString(16);
	        }

	        if (xid.getBranchQualifier() != null) {
                branchQualifierStr = new BigInteger(1, xid.getBranchQualifier()).toString(16);
	        }

	        xidStr = globalTransactionIdStr + "." + branchQualifierStr;
		}

		return xidStr;
	}

}

The simple Remote interface for completeness:

package txtest;

import javax.ejb.Remote;

@Remote
public interface XATestEJBRemote {
	
    public void test(String filename, boolean commit) throws XATestEJBException;
    
}

Notes:
1. This test uses Glassfish v2ur2, MySQL 5.0, MySQL Connector/J 5.1.7 and Commons Transactions 1.2.
2. My database has one table CREATE TABLE TEST ( filename VARCHAR(50) NOT NULL ).
3. Filesystem directories are defined in the static block at the top of XATestEJB.
4. Nothing special exists in XATestEJBException, except that it is defined with the annotation @ApplicationException(rollback = true).

Did Java do us a favour, or no?

Gripe of the day alert!

In the old days, when I was a C programmer (when we were all C programmers), reading a null pointer was just about the worst thing you could do, apart from perhaps writing to one. Then along came Java – in many ways much nicer, it has a NullPointerException, so your program prints a nice message and doesn’t just core dump.

My gripe is that this fact has made people lazy. Whereas I used to be completely anal about checking pointers in C, I’ve become much more relaxed. I’ve noticed this laziness in my code recently and I’m going to change because…

A NullPointerException is not acceptable, EVER! (In my opinion of course). I hate getting them, whether in commercial software, in open-source software, in free software, or in my software. It tells you something is badly wrong, but rarely gives a user any insight into which of the million parameters that affect the running of the software is to blame.

One well-respected piece of open-source software was NPEing all over me recently and it was “right wracking me off”. Yes, I know I was fiddling around with it trying to get something obscure to work, and when I found out how to make it work, it didn’t NPE, but a well written piece of software doesn’t NPE, and if you’re calling an exposed API, then that API should be ultra-defensive.

Gripe off.

Advent Calendar Disgrace!

Kinder Advent Calendar 2008

I had to spend a few minutes the other day explaining to my girlfriend, and consoling her –  it didn’t matter that the first door on the advent calendar (which I was opening) was so much larger than the second door (which she was opening), as over the 24 days they are bound to be organised so they even-out. If not they would be the cause of endless fights between children.

Oops. No: first door – large; second door – tiny; third door – large; fourth door – tiny; fifth door – large; sixth door – large (ooh); seventh door – large; eighth door – tiny…

There is one bonus for the person going second though – door number 24 is huge and most likely has a Kinder Surprise behind it! This got me thinking that actually the doors were cleverly organised and would even-out, and as I’ve obviously too much time on my hands, I decided to add up the door areas. [This is getting sad I know]. The results are:

2 Children

  • Child 1 opens 205.36 sq cm of door (41% more door than Child 2!)
  • Child 2 opens 145.57 sq cm of door

3 Children

  • Child 1 opens 106.36 sq cm of door
  • Child 2 opens 91.74 sq cm of door
  • Child 3 opens 152.83 sq cm of door

This doesn’t seem very fair.  I can hear the screams of a million disappointed children from here! When I was young, the three of us shared one advent calendar. Is this not the case any more? Perhaps we were an anomaly, and normally children had one each?

My final thought is – is it possible to order a reasonable number of different sized doors (say 3 sizes) on a calendar so that it wouldn’t matter whether 2, 3 or 4 children used the calendar, they would all get an equal total door size?  Call it a kind of Christmas puzzle!

%d bloggers like this: