Geir Hedemarks blog

Trygve Reenskaug and Jim Coplien are at it again. They have produced an excellent article on a possible post-OO way of structuring information and behavior.

This is a problem that several people have sought to solve in the past. Aspects and Qi4j are some of the latest attempts.

I heartily recommend Trygve and Jims article. I think it describes the problem clearly, and it doesn’t hurt that they are thinking along the same lines I do, of course.

If you want to see these two guys live, they will be giving a presentation of this paper at the Roots conference. This is a shameless plug, though. One of my very able colleagues are on the organizing committee.

It is fairly well known to most developers that comments should provide value. The following snippet has almost disappeared totally from the codebases I see.

public class Foo {
//Constructor
public Foo() {
//Add one to a
a++;
}
}


The next wave of problems is probably going to be comments elsewhere. Source code repositories like subversion allow comments to be entered during a checkin.

Do you use this feature? If you use it - do your comments provide value, or do they read “checkin”?

I sin against this myself from time to time. Don’t do as I do. I have found that it is very instructive to include my thoughts on what I am attempting to accomplish when I check in. If there is a JIRA (or Bugzilla, RT, Redmine - choose your own poison. Mine is JIRA) issue, I try to remember to enter the issue number.

Then again, people write documents. Most document templates I have used over the years have had a changelog table at the beginning of the document. An example of such a table can be found in my installation guide template. I have now seen some of you use this template, which is why I am writing this post.

The changelog table is there to help the operations team that install the software get a grip on which parts of the document have been updated between versions. If this table does not provide value, the installation document will not be read after the third release, and the installation will break. There will probably be one or more issues registered (”it doesn’t work”, which is an error report that has the same problem I am writing about here), people will get frustrated, and your project will grind to a halt.

Don’t enter comments along the lines of “updated for version 17″ in the description column of this table. Enter something useful. If you rewrote a section, say why and which section(s) was involved. If you have added a configuration parameter, say so, and give a reference to where in the document the parameter is described.

This allows installations to be performed incrementally, which in turn will save money, which in its turn will leave resources for more useful stuff to happen.

Comments can provide lots of value. Make sure you know how to use them wisely.

A couple of people on my team are heavily into security, and I have also been part of several security reviews in the past.

It is high time I had a look at OWASP, which is why I am going to my first ever OWASP meeting in 15 minutes.

For those of you who have not had a look already, OWASP is a community focused on improving application security. Nice project. These people have a comprehensive reference about all things security related.

There is a very real need for this kind of project. I am still seeing SQL injection problems caused by people assembling SQL by concatenating SQL and user-supplied data instead of using prepared statements. I would have hoped we were rid of this kind of problem years ago, but this is not the case.

So - if you haven’t already, have a look at OWASP. There surely is something to learn there for everyone.

I have been silent for a while. There are reasons for that. My two daytime jobs at Objectware take up a lot of time, and so does the family furniture store.

I had thought I would spend the winter writing code. Haven’t. Instead, I have been caught up in reading a number of books. For a change, they are nothing to do with Java. They are to do with teamwork.

I have decided most books about agile methodologies describe a fairytale world, where everyone agrees which methodology should be used. This is almost never the case. In the real world, people disagree.

An example: A while back, I got into a situation where someone wanted to build a wiki page, and we started discussing how the information should be organized. I have been doing document management systems for many years, so my first thought was to have a look at what librarians have been up to. There is a fairly substantial amount of work underlying the Dewey classification system. Recently (the last ten years, that is), people have come up with stuff like topic maps and ontologies. I have been discussing how Amazon and similar sites have taken ontology creation to a new level with a librarian in the family. Instead of some greybeard sitting in an attic somewhere designing an ontology, classifications are made on the spot by the users. If I read “The pragmatic programmer”, Amazon will offer me other books which have been looked at by other people who have looked at “The pragmatic programmer” on their site. This is my world model.

During the discussion about the organization of this wiki, one of the participants stopped the discussion short. “We will use ITIL”, he said. No discussion. Simple. We will use ITIL.

I find what happened in this discussion fascinating. I think this kind of argument is seen a lot in development teams. I am sure I have been guilty of doing it myself. I think this is part of any technology adoption cycle, especially the earlier parts where the evangelists appear.

Evangelist is such a good word for the position that is often taken by someone who wants to adopt a technology. Thou shalt not doubt. Sometimes, this is for the good - the spring framework has killed off EJBs, at least around here. Good riddance.

Other times, we get testing for the sake of writing tests, we get “SOA is the same as web services”, “agile means not writing documentation and standing up” and all sorts of other easy solutions which look good on paper. Buzzwords.

Next time this happens, try to ask “why”. I have great fun doing this - at least as long as I won’t get hurt by the project going bad. If you get a sensible answer that does not lead to a circular reference, walk away. You are probably more needed somewhere else. You clearly have a well-reasoned person on your hands, maybe lacking a bit in communication skills, but someone who has thought things through. Keep that guy happy. He is at least worth his weight in silver.

If you are strung up as a disbeliever, ridiculed, told about “best practices”, or get pitted against God (meaning a reference is given to someone outside the room), there may still be hope. Whoever is talking to you may want to try something out, or have good reasons he is not able to communicate. I have done this myself, sadly. If you are very good, you will be able to get this wish into the open so you can start mitigating risks. There is absolutely nothing wrong with trying something new. It is the only way to get better. The trick is finding the right time.

Then again, there are the pure, honest evangelists. Haven’t figured out what to do with them yet.

This is why I am reading books such as “communities of practice” right now. Someone has to have done this before. Hopefully, there is no need for me to reinvent the wheel. So - no development right now. Gotta get through a couple more books first.

I did a lightning talk this friday on the problems people are experiencing with agile projects when trying to hand them over to a hosting organization. I have been working with this topic several years, and there is a basic problem in that development organizations are measured on change rate, while hosting organizations are measured on stability. There is a basic contradiction in how management goes about rewarding different parts of their organizations. I still think there is quite a lot that can be done at the process manager level, and I also think this is the harder task to solve.

There was an open space discussion on the topic afterwards. I am summarizing the opinions put forth in that discussion here - and adding quite a bit of myself in the process, I am afraid. This is one of the many problems with rewriting a discussion into a blog post. Caveat lector.

These are the problems the people in the open space wanted to raise as candidates for attention:

• The hosting organization needs to be involved early in an agile project. If we are agile, and want to try to communicate with our stakeholders (and the hosting organization is a major stakeholder in the success of a project - without running our code, our customer gets very little benefit), we actually need to have them present.
• ITIL is a set of best practices, but there does not seem to be a great willingness to change how ITIL is applied in the real world. I have had several people outside the open space come up to me and complain about the paperwork that is ITIL. I have also seen this myself - parts of the ITIL crowd has the same problem some agilists are struggling with: “We are ITIL, therefore all we do is good”. Pointing at the good book (in this case, the ITIL book and processes) and stating how the world should be is closer to religion than engineering. “This is how we do it” is never a valid reasoning when your customers raise legitimate concerns they are having. Instead, the paperwork and/or the tooling in a hosting organization should be changed if that is necessary to improve matters.
• Communication needs to happen. It is not enough to have a hosting resource in the project if parts of the organization places obstacles in the way of communication. There have been examples of project managers trying to apply a military chain of command to the flow of communication - a way of organizing communication that even the military has started to leave behind. What happens if a military chain of command is used is that the project manager becomes a bottleneck, slowing the whole project down.
• The hosting resource should deliver whatever technical infrastructure is needed to get the system into production. He also has a “weak veto” on technologies that will not work in a production environment. This will allow the hosting resource to veto bad moves like building the application in the production environment. If the application cannot be built by the developers and handed over, there is something wrong with the tool chain.
• More importantly, the hosting resource should work as a liaison with specialized technology providers inside his own organization. It is the hosting resource which needs to ensure that the load balancer technology can be used in the way planned (or, in extreme cases, dreamt up) by the developers.
• It should be possible to install the application within the first month of development. This will move all the noise that happens in the chaotic last part of the project forward in the project, where it is still possible to make changes.
• It is almost impossible to give any rules to how the workload should be split between the hosting organization and the development team. This depends on the expertise profile of the teams. Someone needs to be able to build a working tool chain for the developers, and someone needs to ensure that the application can be hosted without excessive costs. One person may be perfectly capable of fulfilling both roles, but such people are not common.

Some people reported that it was hard to get the three parts of an organization - the business owner, the hosting organization and the developers - to act towards a common goal.

One of the hosting people in the open space told stories of how some of their customers had ordered the development team not to bother making monitoring and hosting easy - the customer had paid the hosting people, and he expected them to work for the money. The project failed due to excessive hosting costs - the hosting provider had to pass on their elevated costs to the customer, or go out of business.

Other people in the open space told of hosting organizations where the project manager had told the customer straight out that the time of his technicians was too valuable to be wasted talking to the customer.

Next: What should change in the next project?

Read the rest of this entry »

Tomorrow, I am to give a lightning talk on “Agile vs ITIL”. Since I don’t just want to complain about how evil the world is, I am putting up a suggestion for an installation template for server-side, java-based applications. I have been using this template a number of years with good results, but I am sure it can be improved - feel free to give your suggestions.

The template is itself licensed under a creative commons/attribution license, but feel free to create proprietary installation instructions using the template.

The templates, in pages, word and RTF format, can be found here. Enjoy.

(Edited 2008-09-14 to include information from comments)

I have been working with a hosting company and seen what we developers pass off as applications.

These are some musings on what a developer needs to do in order to keep applications deployable.

The installation guide

All developers really, really should create an installation guide. They should then try to forget everything about their application and try to follow the guide.

Did it work? If not, it needs to be fixed. Modern development environments thrive on being able to have massive numbers of deployments. For a far better reasoning behind this than I can ever give in a short blog article, have a look at Refactoring databases by Ambler and Sadalage. Every project, team and developer should isolate themselves from errors made by someone outside their control. Virtualization technologies (or, if you are like me, I prefer to use my own workstation) have made this possible - but only if it is easy enough to deploy the application that people actually bother.

This is somewhat the same problem as testing. Architects need to make it very easy to write good tests, otherwise people won’t bother. So write that deployment guide, and try to make it as easy as possible to get from a source code repository to a running version of the system for someone who does not know the system.

Configuration should be split

A lot of developers seem to think that it is ok to place the configuration inside your application. Since I am a java developer, this often means inside some kind of java archive.

This is a really bad proposition for keeping an application deployable. Often, people will start to check in configurations for the test environments. There never seem to be provisions for configuring the production environment (or, when it comes to that, developer environments) without doing silly stuff.

It is perfectly ok to have all the static configuration inside the application. Whatever changes between installations - database logins and the like - need to go somewhere else. These are some guidelines I try to follow:

• Production passwords must never be checked into a developer repository. Developers should not have access to production passwords without a human giving it to them. Letting everyone tamper with the production database is a very good way to increase the risk of stability problems.
• The hosting organization must not be required to unpack an application, rewrite the configuration, and repack the application. Repacking leads to several different versions of the application floating around, with who knows what bugs. The release manager builds the application archive, period. Nobody messes with it afterwards.
• There must not be defaults for the installation-critical configuration bits that point to the test rig. If you do this, you are placing yourself in a situation where the only protection against accidentally putting production data into a test rig is a routing table entry. If the entry disappears, the best thing that can happen is a very costly reconciliation between the test and production data sets. Worst case, data is lost when someone tries to test the stuff that is in production - the stuff which by now is not behaving as expected - and clears the test database before the test starts. Please don’t write this off as far-fetched. I have seen it happen.
• If installation-critical configuration does not exist, the application needs to stop. This is one of the oversights of the JEE specifications. There is no way for an application to tell the application server that it wants to stop because continuing is not an option. This functionality usually needs to be written inside the application or be application server specific. If this condition is not met, people trying to install the application are going to spend lots of time reading logs, trying to find the log statement saying that it wasn’t possible to connect to the database. This statement should perferably be the last statement before the “I am stopping” statement in the application log.

I usually make some kind of provision for providing the name of a configuration file using either JNDI or command-line options to JAVA. Both are usually acceptable to hosting organizations.

A nice side effect of not hardcoding the location of the configuration file is that several instances of the application may suddenly run on one computer. This does away with a lot of virtual machines.

Use a production-like environment during development

The closer the system the developer uses is to the production enviroment, the fewer the problems.

How do you know that you are close to the production environment before this enviroment has been built? Specify how you want it to look. Lots of projects get into trouble because different team members think something is self evident - like how the production environment looks. It usually isn’t at this level. Knowing how the production environment needs to look also makes it easier to select a hosting provider.

Specifying how the application enviroment should be, seen from the development side, is the first step of starting a constructive information exchange with the hosting provider. A good, professional hosting provider will wish to make changes to the environment of the application in order to reduce the costs associated with hosting your application, and there is absolutely nothing a developer should attempt to do about this. A competent hosting vendor is the hosting technology expert, and needs to be able to exercise this kind of control.

Whatever changes to the application environment is agreed on also needs to be fed back into the development and test environments. As the first paragraph of this section started - the closer the system the developer uses is to the production environment, the fewer the problems.

Share the pain!

All developers should install a local copy of the application. If this is not possible, something is wrong. If it is too cumbersome to do so, only the developers are in a position to fix the problem - and hosting personnel is not much cheaper than developers, so it is better to fix the issue than to let the problem persist for the lifetime of the project.

Next: Installing deployable applications… When I get the time, at least. Shout if I should prioritize it.

Summer. Yup. Ice cream time. Cholocate ice cream.

• 6 tablespoons cocoa powder
• 4 dl skimmed milk
• 40 g sugar

Whisk the cocoa to a paste with some of the milk while heating, add rest of milk while whisking, let simmer until the “powdery taste” is gone. Add 100g of dark chocolate and a knob of butter.

• 3 egg yolks
• 40g sugar

Whisk the egg yolks with the sugar, then add the cocoa slowly while whisking vigorously. Put the stuff back in the pan and heat until the mixture goes more rigid. Do not boil - the result will be similar to an overcooked vanilla sauce.

• 1 ts granulated coffee
• 1 ts vanilla extract
• 2.5 dl cream
• sugar syrup (50% sugar by weight, boiled and chilled)
• 2 cl cointreau or other orange-flavored spirit

Stir in additional flavorings, then the cream. This will lower the temperature of the thing enough that the cointreau can be added without the alcohol disappearing. Chill further.

Adjust the consistency of the ice cream base with skimmed milk or water then it is cool - 20 to 10 centigrade. Milk will take away some of the chocolate-y punch. Your call. The consistency should be like a slightly thick vanilla sauce.

Freeze in an ice cream maker or put in freezer and stir every 30 minutes.

Use a small scoop. This thing really packs a punch - there is absolutely nothing subtle about the flavour.

I use “Ices” by Liddell/Weir as a base when making ice cream.

Here we go again. We are barely through with the SOA hype (See SOA-or-DOA or Does my bus look big in this) before the norwegian colored it press is starting to hype “software as a service” (SaaS). Again.

Old news apparently becomes good as new when big companies decide that they are looking into changing their licensing model. It seems that Microsoft is looking hard at the online office productivity tools made by Google.

The journalist then added some stuff of his own. In just a short while, all our applications are going to be online applications. We won’t store stuff locally any more, and we will rent our applications. Hosting your own mail server is going to be a thing of the past. If only I hadn’t heard this before. There are still lots of problems that need to be overcome before we end up leasing our productivity applications - even if I am writing this in an online editor in my blog tool, which incidentally is also free.

The hype has been here before under a different acronym. In 2001, ASPs were going to take over the market. I guess the half-life of a journalist in the colored IT press is shorter than six years, which means we now get a new round of the same hype with different acronyms attached.

There are still no critical questions being asked. And it seems that we like it. Someone has to buy these rags.

Whatever happened to critical journalism? Why are there no magazines out there that can actually provide me with pre-chewed overview information I can trust? The only IT magazines I have been able to lay my hands on seem to be tabloids with all the journalism removed.

In the non-IT world, there are some exceptions. The New York Times seems to give a fairly balanced view on american issues. The local newspaper “Morgenbladet” also seems to employ journalists that are worth something. So far, in the IT area, I have found nothing. Nada.

I am afraid this is how things are going to continue. We are still going to be subjected to waves of hype whenever some journalist wannabe wants to sell more copies. I would love to be proven wrong, though.

Today, I got my preconceptions challenged. As such, it has been a good day.

It concerns someone on the periphery of the people I know, who I am afraid I had a bad feeling about when I met her a couple of years back.

People usually form a very firm impression of people when they first meet someone. You have ten minutes, and if you haven’t measured up in that time, it is likely you never will unless you work very long and hard to fix whatever happened. Our brains love to do pattern matching, and this seems to be one of the uses that have been honed by evolution over the years. Will you trust someone you just met, or do you need to keep your distance so you don’t get burnt? Can you hunt with this individual, or will whatever you bring down just get stolen?

There is a long, long history if preconceptions like these through the ages. Humorism has had a long and fruitful life. The idea is connected to the idea of there being four elements - fire, earth, water and air. Therefore, or so the idea seems to go, there must also be four bodily fluids - blood, yellow and black bile, and phlegm. Or so the story goes. With these four humors, four personality types go: Sanguine persons, who are courageous, hopeful and amorous, had more blood. Those with more black bile were thought to be melancholic, and therefore suffer from sleeplessness, irritability and being despondent. But not to worry, if you eat right, you can balance things out. It almost seems like the precursor to the stuff you find in the glossies in the dentists waiting room. Would you buy something from the melancholic?

Thoughts like these have been with us all the way. We have tried to analyze and categorize what I think are the results of our first impressions. Some have even tried to ascribe personal characteristics based on measurements of the face and skull - and this is not too long ago.

These disciplines are mostly considered defunct today. I guess recruiters may still use some of this based on more modern psychological indicators like the Myers-Briggs test. I was subjected to a test like this by a recruiter when I applied for my first job, along with some other tests. I didn’t know the result of the test, but I had just finished my draft service and wanted to start a new job. In my final interview, I was told I would get an offer, asked to be allowed to read it and signed on the spot. Apparently, my profile read “has difficulty making decisions”. It was the last time we used that recruiting agency.

Ideas like these have been eagerly used by people with agendas in the past. In the not too distant past, Norway sterilized people who were in some way thought to be in the shallow end of the gene pool. There are lots of other examples. The idea that you can easily divide people into good and bad is a very seductive one. It takes away all the hard questions.

So, I don’t like these stereotypical models. I feel they simplify and cheapen the people around me into flat, cardboard versions. How can a one-hour test summarize the personality that has been developed over the course of many years?

And this is where I meet myself, having tried to work against my own stereotypical preconceptions against someone which are based on their looks and some short conversations. My preconceptions have now been proven right twice - she really didn’t follow up on her promises. Our trustworthiness detector works at least once in a while, and I can’t explain how easily.

What to do about it?

Nothing at all. Business as usual. I will probably still have to size people up on the spur of the moment once in a while, and run the risk of getting it wrong. If you have to, you have to. I will also continue to dislike stereotypical labels on persons.

But there may be a middle ground which just became a bit clearer.