QVCS/QVCS-Pro Refactoring II

Whew. Posting here has been light while I've been doing the deep dive on refactoring the checkin code. The first pass is now complete, and the QVCS/QVCS-Pro code (not released yet!!) can now apply a label at checkin time without necessarily creating a new revision. I still have to cross some t's and dot some i's to finish things up, but I'm pretty happy with the result. The code is much more maintainable as it finally has transitioned completely to object oriented C++ from the old old C code that has its origins from way back in the early days of QVCS on the Amiga.

Because of the large number of changes, I'll need to put this next release through a beta cycle. Anyone interested in helping out, please send me e-mail (jimv at qumasoft dot com). I'll probably set up a separate forum area to support the beta.

QVCS/QVCS-Pro Refactoring

There has been a 'feature' in QVCS/QVCS-Pro for a while now: if you apply a label at checkin time, then QVCS will always make a new revision for you, whether the file you are checking in is actually different from the revision that was checked out, or not. This is clearly not the correct behavior. The correct behavior for an unchanged file would be to simply apply the label to the revision that had been locked, unlock that revision, and not create a new revision. (The QVCS/QVCS-Pro workaround is to not apply the label until after the checkin, which can be a nuisance).

Well, in the upcoming 1.1 release of QVCS-Enterprise, I've added the ability to apply a label at checkin time, and in QVCS-Enterprise, it works as it should.

If QVCS-Enterprise can do it, then QVCS/QVCS-Pro should do it to! The reason I delayed implementing the same behavior in QVCS/QVCS-Pro is that.... you know how sometimes when you write code, and then add more and more features to it, it becomes.... complex? Well, that portion of QVCS/QVCS-Pro is pretty complex. I've stayed away from changing it for fear of breaking it. Fear no more. I've decided to refactor that whole section of code so that it is much more closely aligned with the Java implementation in Enterprise. The Enterprise code is actually pretty clean in this area, and the goal is to get the C++ code up to that same level of cleanliness. Along the way, there is a whole lot of refactoring going on. When complete, I'll have a body of code that is much easier to maintain, and will also (finally) do the right thing when applying labels at checkin time.

Is there a limit to intelligence?

Suppose the Einstein was the smartest human to ever have lived? I'm not suggesting that he was, but just asking the question rhetorically. Alternately, if we ask the question: If we can find the smartest person in the world, how come there isn't anyone smarter than that? And when we find the person who is smarter (more intelligent, whatever that means) why can't we find someone who is even smarter? Won't we at some point reach some kind of limit where we can't find anyone smarter than the smartest person we can find. Once we've found the smartest person, let's ask the question: why can't we find anyone smarter than that? Is there some inherent limit to human intelligence? Is there some overall limit to the thing that we call intelligence, irrespective of its means of implementation?

My hypothesis is that there is a limit, both to human intelligence, and to the overall category of intelligence, whatever its means of implementation. By this 'means of implementation', I'm proposing that there is also a limit to non-human intelligence -- that there is a kind of ceiling to intelligence. I don't know what the ceiling is, or how close we are to it in the general population, but my hypothesis is simply that there is a limit of some kind.

I did a bit of searching via google to see if I could turn up any links on this, and only came up with: Limits of Artificial Intelligence

This is something that should be covered more extensively, somewhere. Where?

IDE integration improvements

I've been slogging through updates to QVCS-Pro to improve the features it supports within an IDE. So far, I've added the ability for a user to specify either the revision or the label they want to use when doing a 'get latest version'. This is optional, of course. If the user doesn't want to take the trouble of identifying a label or revision, the 'get latest version' will just get the tip revision, as it does in the currently released product.

Along the way, I've been testing with both Visual Studio 6.0, and VS.Net 2003. These IDE's behave quite a bit differently, so it winds up being a good way to test that I'm implementing the API as I'm supposed to. 0

The Causality Trap

Suppose there is a chain of events where event b follows event a. Does that mean that event a causes event b? There are some (many?) cases where the unwarranted inference is made that because b follows a, that therefore somehow a causes b. Correlation is not causation. What goes on in these kinds of fallacies is that along with a and b, there are a gazillion other variables that are part of the stew. What relation these other variables have with respect to a and/or b is ignored, either from genuine ignorance, convenience, sloth, or intellectual dishonesty. I've come to call this kind of mistake a 'causality trap'. It is a fallacy to invoke 'cause and effect' as an analytical method when there are unobserved and/or uncontrolled variables.

A prominent example of a causality trap is the daily reporting on the 'causes' of movements in the stock market. "The market went up today on news of blah, blah, blah." or, "The market went down today after Company X reported lower than expected earnings." Here's a tip: there's a whole lot more going on in the movement of the stock markets than can be summarized in a 10 second sound bite.

I don't mean to deny the law of causality. The problem is that people invoke the law of causality when it is not appropriate.

Another gem that I put in the 'Causality Trap' bucket is the Butterfly Effect (also here) example that is used so prominently when discussing Chaos Theory. The part of the Butterfly Effect that I find a stretch is the implicit assumption that somehow the system remains unperturbed by other larger effects after the initial conditions have been established. The equations that are used to create the beautiful variations in the Chaos Theory domain rarely describe any other stimuli that may affect the system after time zero. In the real world, this is a completely fanciful situation. A butterfly may flap its wings in Brazil, but there are a whole lot more important things that happen after that that will determine whether there is a tornado in Texas. To claim that the butterfly 'caused' a Texas tornado is to fall victim to a 'Causality Trap'. Who's to say that it wasn't some fishing boat in the Gulf of Mexico, or perhaps the guy growing coffee in Columbia? Or better still, perhaps the radiance of the sun had something to do with it? Causality requires that if you remove the cause, then the effect necessarily also goes away, every single time. If we kill all the butterflys in Brazil, will there never be another tornado in Texas?

I think by nature, we have an innate attraction to cause-effect explanations. (A self-referential assertion!). It's a way of taking a shortcut when thinking about very difficult problems. Rather than confess that we don't understand something, we invent a pat cause-effect explanation that allows us to pull the problem back into our comfort zone.

Defining features for the next QVCS/QVCS-Pro release

I bounce back and forth now between Java and C++ depending on the needs of the moment. For the past several days, I've been working on the next QVCS/QVCS-Pro release. This while I've also got the next QVCS-Enterprise release as a work-in-progress.

So far, the biggest item for the next QVCS/QVCS-Pro release will be recursion support for the command line utilities. This I've already completed. It works pretty well, and is very simple conceptually -- you just put 'qrecurse -worktree' in front of the QVCS command that you want to recurse. Alternately, you can use 'qrecurse -archivetree'. The former uses the existing workfile directory tree to 'drive' the recursion; the latter uses the existing archive directory tree to 'drive' the recursion. This is a feature I should have done years ago. Oh well. Sometimes the clarity of the requirements take a while to gell.

Other items on the release wish list (in no particular order) include: (1) The ability to check out and/or get revisions other than the tip revision when working within an IDE. (2) Allow entry of a check-in comment at check-out time. (3) Group named file filters by project name like in QVCS-Enterprise.

There's a whole lot of other potential features to work on. Those named here are the ones highest on the list.

For evening distraction, I've been slowly making my way through the final book of Neal Stephenson's The Baroque Cycle. The books are not ones to speed through, but I've found them to be well worth the effort, and certainly more worthy of my time than any distractions provided by television.

The Knowledge Problem I

In software development, there have been fads of one sort or other that spring up every several years that promise to solve the software development bottleneck. There then are usually a number of books published associated with the fad, and along with the books come training courses, and finally products that promise to use that fad to streamline the development process. This is all well and good, and over the years has yielded some genuine progress. The object oriented 'fad', and the design patterns 'fad' are two obvious examples of progress. The latest fads are extreme programming and aspect oriented programming. It's still a little early to pass judgement on these.

In spite of this progress, there are some areas of our craft that are more resistant to attempts at improvement. Defining the actual nuts-and-bolts of system behavior is one. Yes, we have made progress in this area -- witness the wide adoption of Use Cases to capture a description of the desired behavior of a system. But there is still no real short cut to the writing of the use case itself. A good deal of trial and error, and iteration must go into the description of system behavior in order to get it right, and while there is tool support to help, there's no getting around the give and take that tunes the definition of system behavior. This is especially true of any kind of system that interacts directly with a user. It's also especially true when describing the behavior of an interface between two separate systems.

One reason that this area of development defies attempts at improvement is because there is an information bottleneck. The person who owns the system requirements must communicate those requirements to other project participants. This is often challenging because the knowledge of what the system must do; what behavior it must supply, has never before been clearly articulated. It must be discovered. For a system that's being built in order to replace an older 'manual' system, there are no shortcuts to the process -- actually understanding the problem domain takes a lot of time; the communication bandwidth between the domain expert and the system architect is necessarily limited to their conversations. Tools can assist in providing a common vocabulary, but there can be no shortcuts in transferring the domain knowledge. System architects cannot perform a Vulcan mind-meld to acquire this knowledge. They must get into the trenches beside the domain expert and live that life for a while in order to really understand the domain.

The shortcut term I use for this dilemma is the 'Knowledge Problem'. This same term is widely used in flavors of economics (c.f. Hayek) and its impact in economics has some parallels to the problem we see in software development.