Bill Roper's Journal

Turning on basic authentication in the new Java code has caused a great many things to break. I have spent the day patching them and swearing.

I've now run into a serialization problem that I will end up working on some more tonight before going to bed. But I can deal with serialization problems, because they are problems in code that I wrote, not someone else. :)

I've been trying to get the multithreaded version of the calculations running in Java. Well, I finally got something to run today.

It runs more slowly that the single threaded calculation. I have either done something badly wrong or I am being beaten to death by synchronization. To add insult to injury, I'm trying to figure out how to profile the test app in Eclipse, but that isn't working for reasons that I am not yet able to figure out.

*sigh*

Thinking some more about yesterday's Java problem:

In almost every case, for an "ID"-type class, two different instances of the ID that contain the same identifiers should return true when passed to the equals() method and should return identical hash code values when the hashCode() method is called. The problem is that the standard equals() and hashCode() methods that are inherited from the Object class will treat these as different objects. If an ID is supposed to uniquely identify some Object (for example, a PersonID that uniquely identifies a Person), only one instance of the ID will work to identify the unique Object if the ID is used as a key in a HashMap or the like, because a different instance of the ID, although conceptually equal, will generate a different hash code and will return false in an equals() comparison.

The solution, then, is to override the equals() and hashCode() methods for the ID classes so that IDs that are conceptually equal will return identical hash code values and will return true in an equals() comparison. This is, in fact, the case by default if the ID is simply a String.

So let's override the hashCode() and equals() methods for the ID classes. Life is good.

Except now there's no way to identify different instances of your ID object. And when you're writing a serialized stream that may contain duplicate references to the same object, you want to know if you've written an object to the stream before, so you can write the object once and restore the duplicate references to the object when reading the serialized stream back into memory. Normally, you could use a HashMap with the object as a key to keep track of this, but now two different instances of the ID will return the same hash code and will return true for equals(), so different instances are interpreted as being the same instance.

It looks like the right answer to this is to define an interface that all of the serializable objects can implement that will, by default, call back to the default object implementations of hashCode() and equals() (let's call them objectHashCode() and objectEquals()); then implement an ObjectHashMap that will call objectHashCode() and objectEquals() instead of hashCode() and equals().

*sigh*

This is an annoying amount of work to do...

Meanwhile, back in the wonderful world of Java:

Java allows you to declare classes inside of classes. Those classes have access to the internal protected members of each other, which is a handy way of circumventing the absence of "friend" classes when you've got a group of related classes that actually need access to those protected members. And it allows me to hide away any of those inner classes that I choose so that other classes in and out of the package can't get access to them.

So what's that good for?

Well, enforcing locking of objects, for one thing.

Let's assume an outer object that other classes will want to use. In this example, we'll call it a Thing. The Thing object itself is immutable so that it cannot be changed, but it contains an inner class, Thing.Mutable, that contains all of the methods that can change a Thing. This will be important in a moment.

The Thing object also contains an inner class, Thing.Lock, that can be used to acquire either a read or a write lock on a Thing. Each Thing has a unique ThingID that can be used to look up a Thing when it is passed to the Thing.Lock class, which will provide access to either a Thing or a Thing.Mutable depending on whether the user requested a read lock or a write lock. Internally, a Thing is stored as a Thing.Mutable, but we'll cast the reference to a Thing before handing it to the user if they've gotten a read lock. (Yes, they could cast the Thing back to a Thing.Mutable, but if they do that, then I cannot help them. :) )

The only way to gain access to a Thing is to use the Thing.Lock class to acquire a read or a write lock. Of course, all of those Things need to be stored somewhere. Thus, the Thing class has another inner class, which is the Thing.Catalog and which holds all of the Things. This class is protected so that you can't access it from outside the Thing class.

The Thing.Catalog class is immutable, but -- like the Thing class! -- it contains an inner Thing.Catalog.Mutable class and a Thing.Catalog.Lock class. In this particular implementation, the Thing.Catalog is a singleton, so there's only one instance in the application that holds all of the Things.

So let's add a new Thing to the Thing.Catalog. All we do is create a new Thing and call its catalog() method. Since the catalog() method is internal to Thing, it can access the protected Thing.Catalog and obtain a Thing.Catalog.Lock for writing, giving access to the Thing.Catalog.Mutable. This allows us to add the new entity to the Thing.Catalog without ever exposing the Thing.Catalog to an external user -- because if we did expose the Thing.Catalog, then an external user could look up a Thing without getting a lock on the Thing simultaneously. And that would be bad, even if they did have to use the Thing.Catalog.Lock to do it. :)

The happy news is that these objects can inherit most of the locking code from an abstract base class, so you don't have to write a lot of new code in any given Thing to support this. My Thing class is in a file that's only about 275 lines long (with lots of blank lines for readability, but not yet any Javadocs), despite having five different inner (or inner inner) classes.

And it does what I want it to do, which is also happy news.

I was making good progress at porting my grid code to Java, but then I hit the point where I needed the incomplete access control package.

So now I'm porting more of the access control package.

I could make some ham and beans soup.

If I had some ham.

And some beans.

And maybe a pot to cook it in.

Water would be good too...

Back to the Loop again today for another round of meetings.

I did manage to spend a few more minutes on the locking scheme for the server-side of my Java code. At this point, I'm looking at having all of the lockable objects on the server define three inner classes: the first being an immutable class that holds all of the data for the mutable outer class, the second being a private class that provides a locator / index for the objects so that you can find them with a key, the third being a public class that allows you to obtain either a read lock or a write lock and that will give you access to either the immutable inner class or the mutable outer class, depending on the type of lock that you've obtained.

So in skeleton form, it would look like this:

public class MyClass implements LockableObject {
  public class Const {
    boolean    member;

    Const( boolean member ) {
      this.member = member;
    }

    boolean getMember() {
      return this.member;
    }
  }

  private class MyIndex implements Index< Key, MyClass > {
    MyClass find( Key key ) {
      ... implementation details and other useful operations elided
      return object;
    }
  }

  public class Lock {
    Key       key;            // key to use with the index to locate the class instance
    LockType  readOrWrite;    // enum defined elsewhere (READ/WRITE)
    MyClass   object;         // the object, once we've located and locked it, else null

    public Lock( Key key, LockType readOrWrite ) {
      this.key = key;
      this.readOrWrite = readOrWrite;
    }

    public boolean lock() {
      ... probably uses tryLock() etc on ReentrantReadWriteLock
      return success;
    }

    public MyClass get() {
      if ( isLockedForWrite()) { 
        return object;
      }

      return null;
    }

    public MyClass.Const getConst() {
      if ( isLockedForWrite() || isLockedForRead()) {
        return object.Const;   // is this vaguely the right syntax?
      }

      return null;
    }
  }
}

Ok, that's an awful mess of assumptions left out and I may have blown some of the syntax, as I haven't actually tried to compile this yet, but it looks like the right idea.

I think.

I managed to successfully read the user-defined property set out of our file with the POI library. The serialization still doesn't match up between the Java and the C++ code, but we can straighten that out tomorrow.

The substantially rewritten parser is now running.

It is not yet running properly, but that's what debugging is for. :)

(I've already fixed about half-a-dozen minor bugs, mostly introduced by having to use new String handling functions that are different from the ones I used in C++. More to fix on Monday, to be sure...)

I had just about finished rearranging the parser logic in Java, when I realized that I could substantially improve my life by adding a helper class that keeps track of the current bit of the string that's being parsed, the position of that string relative to the original string (for error messaging), and the operator to apply to the bit of string once the parsing is complete.

So I'm refactoring. The resulting code will be much easier to live with and quite likely much easier to convert to some "standard" parsing system in the future, if that proves to be helpful.

Not counting the smaller (but, in some cases, still significant) side packages that amount to another 9000 lines of code, I have now ported 51,201 lines of C++ code to Java.

Using this code base, I am able to open up two entities, add them up into a third entity, and write the results back to disk in a way that our current product can read.

And there was much rejoicing. :)