Geir Hedemarks blog

I have a guy from Sweden on my team. He is an accomplished developer, and also a very reflected guy.

A couple of weeks ago, we had a discussion about cultural differences. He came up with an insight I have been thinking about for a while: norwegian developers are harder to lead than swedes. As I understood him, we tend to rebel against being constrained.

The kind of constraints I am talking about is the good kind. The kind where the product owner tells you that he needs some feature before some date for it to make sense. Can you do it? No? Is there something you can do to make it happen? I am not talking about the other stuff.

And here the rebellion lies. Norwegian developers tend to question the need to have a given feature out by some date. We question the need to save money – a couple of hundred thousand of whatever the local monetary unit happens to be can’t make a difference, can it? We spend so much already? We tend to question the need to cut corners instead of describing the horrific consequences which enable someone else to make the decision.

Mike Cohn has just written about removing team members. In his post, he describes something he calls the CDE model. The CDE model describes a container that is placed on a self-organizing team.

The container are the boundaries the team is to operate within. I view constraints on feature dates, which features to add, and cost to be the container we are placed in by the product owner. The product owner owns the budget, the resulting product, and all the problems that come of cutting corners. It isn’t really up to a developer to “adjust” this container. A good developer will tell the product owner what problems he will end up with, but the decision to actually suffer the consequences in order to achieve something else lies with the product owner.

I think this may be to do with the negative view of non-techies that is pervasive in some environments in Norway. If you have a background in economy, chances are that some techies will look down on you, using the somewhat derogatory term “blåruss” to describe your position. Yet I never saw a company succeed without some kind of economic control.

I am worried that norwegian developers are being described as what I view as container-averse. I don’t think we will get agile projects – or any other kind – to become successful products if this is actually correct. I hope this very insightful developer is wrong, or has been unlucky in his coworkers.

I have been fooling around with Scala, Lift and Spring the last couple of months.Why? I want to see if this combination of libraries and languages can make me more productive than my current preferred application stack. I also like the prospect of having to write less linear text (stuff like getters and setters) in order to accomplish something.

Underlying this is a wish to kill off the factory (and, up to a point, the singleton) pattern. When I first started using Spring a number of years ago, a new world opened in terms of maintainability of my code. Inversion of Control (IoC) was the enabler of this leap forward. I think I have only written a handful of factories since.

I am basically looking for a stack that

• has some kind of inversion of control that enables regression testing outside of a container/in an IDE without requiring deploying test structures in production, or changing code to run tests.
• has declarative transaction control, along the style of Spring’s @Transactional annotation. Many of the Scala/Lift examples I see has little or no transactional demarcation.
• has most of the infrastructure stuff I need already present – I do not want to even hear people talking about how easy it is to roll your own. I want stuff to be mainstream and recognizable. None of the companies I work for make money off of doing infrastructure like ORM. It might be symptomatic that Scala so far has two ORM frameworks (Mapper and Record), and three Actor implementations. I expect this flurry of library creation to die down after a while, though. Java wasn’t much better around 2001.
• enables me to separate my applications along vertical layers, and which helps me separate presentation and business logic. This does not mean I want nonfunctional wrapper layers.
• is efficient and bug-free.

It turns out other people is thinking along the same lines as I do. A good discussion can be found in this thread. This is a discussion between java/spring based people like me and the authors of libraries like Lift. One side wants IoC, the other says not to bother and use something like Lift’s Boot class instead. And at a superficial level, this isn’t too bad.

The boot class used by Lift looks like a stripped-down Spring XML bean definition list: “Class A needs to have an instance of class B”. I mostly don’t do these kinds of configuration items any more. I tend to declare my services using an annotation (@Component or @Service), and telling beans that use then that they need something injected (@Resource). I think this is cleaner since all the information about the wiring of my application is right there in the code, where it is easy to find for other developers.

My Spring config files deal with transactional support, databases, security, ORM – and test support. I also really, really like the pre-made test libraries that come with Spring. They make it so much easier to get people started with regression tests. None of these are available in Scala, and it seems to be somewhat less than trivial to retrofit these parts of Spring to Scala.

I haven’t decided yet, but I do know that I don’t like my production Boot class knowing about my test structures. Need to think about this for a bit.

Developers are strange. We tend to spend hours discussing which feature is the best, what language to use, which color our laptops should be.

One of the longest and most polarized discussions is wether strong or weak typing is the better. Then we go on to select a language based on our preference.

Both camps have points. Weak typing allows you a more flexible domain object tree, where you very easily can add new stuff. If you can keep control of your design, this is clearly good. Strong typing allows for more error detection in your IDE – if you use an IDE. You do, right?

I could go on and on, but the problem we tend to create through these discussions is that one tool is not best for all applications. The domain I am going to write about today is integration.

If you come from the weak typing camp, you are probably used to the idea that you can accept any object that conforms to the interface you actually need. If you need version 7 of an interface, and the producer guarantees backwards compatibility, you are probably quite happy to accept even version 10 of the interface. In cases where very good designers have been around, I have seen the client ask the producer wether it can produce data that is compatible with the version it can accept. This very seldom happens in the software world. This is mostly the domain of hardware designers. My macbook is able to detect wether an external display only has a resolution of 800×600, and adjust accordingy. This type of negotiated versioning is my personal favorite – but then again, I have a hardware background.

Other camps accept whatever they get, store it away, alert someone, and then hope someone upgrades the application before the information that is available in the new version is needed. The benefit of this is that the problems gets pushed forward in time, and does not create problems until the very last moment when the information that is unparsed is actually needed. There are benefits and drawbacks to this solution, but it has worked very well for those who are able to view their integration points as a document.

Imagine adding strong typing proponents to this mix. Strong typing means knowing exactly what your objects can look like. My macbook would only accept the 30″ apple-branded displays, making it unusable when I want to present at a conference with an unknown projector. This is where 95% of the integration platforms out there are. To me, it seems like this branch of integration platforms has grown from the language-supported method call interfaces we use internally in our applications, through CORBA, then we did the same thing again only with XML and got web services, which have now evolved into JBI, where stuff looks like documents, but are very strongly typed. These technologies seem to cause teams to make only token efforts towards governance. I have found none who are forward-compatible.

I come from a company where people have been thinking about this for some time. These people have now moved on, but the ideas are still around. One of the ideas brought forward by this bunch of people is to do away with most of the web services stack and use xpath to pick out the bits of a message you need. If elements you don’t recognize and don’t need appear in the message, the extra information is ignored. More information from these people can be found at the Cantara wiki.

So – where am I going with all this? I am a proponent of strongly-typed languages. I have seen too many projects fail because most developers lack the discipline necessary to avoid abusing a weakly typed language. That is why I prefer Scala to Ruby.

I am not a proponent of strongly typing everything, though. Strong typing is a good idea inside an application. Anything that is outside the application is going to be less under the control of the developers than whatever is inside. This is a fact. Nothing to do about it. Therefore a different approach is needed in order to avoid hideously large development/maintenance costs. This other approach is going to cost the team more, but the cost is going to be lower than trying to treat the outside world as something that can be controlled. I am not even sure I would want to do that – the implication is that the slowest team in the outside world is going to control the speed of your team, too.

This is where my gripe lies. I want to be able to selectively select how weakly an interface should be typed. When I assemble interfaces, the typing should affect how they interact. Right now, there does not seem to be a language that allows this. All languages have a one-size-fits-all approach to typing which is going to be wrong sometimes. The best compromise I have been able to find so far is the cantara or spring integration approach with the strongly-typed Java.

Perhaps in ten years…

I have been getting more and more questions from both managers and techies about scala lately. Is this the time to switch to scala. This time, I am going to address the management/architect side of the question.

“Should I switch to scala” usually is the wrong sort of question. A better question is “what can I change that will improve my situation”. Techies usually have a habit of jumping the gun when it comes to technology-related issues. We dive right into the things we know best – the technology.

A lot of organizations are troubled by this. Why does the question come up at all? What does not work out for your organization? Very often, the question is a variation on “we need to get stuff done faster, and once it has been delivered, it needs to continue working”. Fixing the things that will give the biggest improvement first is usually the way to go. So where in all this does Scala – or any other “cool” technology – come into the picture.

This is my preferred list of todos before starting to fiddle with the technology stack, provided there is a workable technology stack around:

• QA and testing. Many organizations do not have a workable regression test suite around, or have a regression test suite that gets in the way of changing the product. This will give a test workload that increases with the square of the number of features in the product. This will work out for the first couple of months, but the cost of not having a workable, automated regression test suite will quickly cause management or customer attention.
  There will still be a need to test every release. Regression tests are about ensuring that the stuff that worked in the last release is still operational. Anything that has been done in the current release, and anything that is of a visual nature, will have to be tested manually. Regression tests are there to reduce the cost of the tedious testing – the stuff that needs to be repeated every last release, and which gets bigger and more complex for every release.
• Requirements capture. Many organizations are not very good at selecting which requirements should be implemented, and in what order. I think this is a sign of office politics – someone is out to increase their dominion without regard to the health of the organization. Mental visions of big, flaming eyeballs, orcs in their thousands, and tall, forbidding, black gates are appropriate here. The organization will never have any chance at being remotely efficient before the evil dominions have been reduced to a size where the overlord has to explain why there is a need to have big, black, forbidding gates inside of the organization, until the orcs turn out to be kids with poor complexions and butter knives, and the overload has to pay for the effect his black gates have on the organization from his pre-existing budget.
  Of course, the evil overlord may be the head of IT. Nothing makes an evil overlord like an IT guy. We are too well read in the science-fiction and fantasy worlds not to have picked up a thing or two. There needs to be a balance – read the Recluce books by Modesitt for a fantasy introduction to balance, otherwise have a go at “User stories” by Mike Cohn.
• Knowledge  management. This has nothing to do with ITIL processes. This is about one part of the organization knowing what the other does. It is about developers understanding how tasks are performed by users of the system. It is about one developer being able to pick up where another left off. It is about management being willing to listen to arguments in non-managerial language.
  This is in direct conflict with being “more efficient” in the short term, and is therefore a problem with some current managerial trends which tend towards optimizing short-term profits to pump stock prices, even at the expense of long-term profits. Developers should get time out to clear up costly “technical issues”, they should work tightly with the users of the system so they understand what is needed, and so on. They are not creating new functionality while they are doing this, and will therefore be “less productive” and give less impressive results in the next financial report.
• Lifecycle management of the software is something almost no development organizations think much about. I have had people yell at me when I asked what the life expectancy of a system was projected to be. The system was rewritten from scratch two years later. Switching technologies is not an interesting question compared to throwing away all the code every two years. So – how long is the system meant to live, and what should it cost? Should it be possible to make it into a product?
• Personnel issues are a recurring problem in our profession. We had a visit from Uncle Bob a month or so ago, and he said something about not having weapons in the same room as two senior developers. Sadly, he was right – some of us are hard to work with. These issues can overshadow any benefits gained from moving to a new technology.

Any organization that does not have good control of all of these issues, or who have taken steps to get them under control, should not be thinking about adopting Scala. Fixing any of the above issues – if they apply to your organization – is going to have a much bigger impact than Scala ever will. So – make sure that introducing a new technology is a good idea.

Moving to Scala is going to require investments in new knowledge, there will be technical issues that will have a direct cost both in time and money, there may be downtime. Scala is also a very immature platform compared to Java or .NET. The benefits and risks should be carefully weighed before going for any new technology. This is my own list of pros and cons for Scala right now:

• Pro: The language is “more orthogonal” than Java. This makes for a language that may require fewer workarounds and sticky-tape solutions.
• Con: Developers have tendency to abuse new and “cool” technology in an attempt to show off. Unreadable or unmaintainable code is a problem whatever language it is written in.
• Pro: Scala can reuse existing libraries from the java world. This should improve efficiency, especially if there is existing knowledge of Scala libraries in-house.
• Con: There is very little knowledge about what bugs may be caused when using Scala with these old libraries. This, sadly, is a also developer-dependent. The organizations that are already using Scala will have solved their own teething troubles – but this does not mean your organizations will not create the same kinds of problems.
• Pro: The designers of Scala have created a language that is less verbose than Java. This is significant because Java applications tends to get unmanageable by sheer codebase size.
• Con: The development tools available for Scala are horrible. The best one while I am writing this is IntelliJ, with Eclipse in second place. I don’t like the compilation model of IntelliJ, so I am sticking to Eclipse. Netbeans is not a contender until the Oracle noise has been resolved.
• Pro: Developers want to use Scala. Hiring talent is probably going to be easier, depending on what the previous technology in use was.
• Con: Scala might not make it. What happens if Scala turns out to be a fringe language in two years? The risk of having to do another rewrite in another language should be factored into the decision to move to Scala. Personally, I believe in Scala, but there are no guarantees.

Now, don’t misunderstand me. I am going to try to find the time to write a small application in Scala during the next half year or so. I like the language. I am looking forward to getting rid of the non-functional fluff that is irritating me in Java. I just think organizations wanting to move to Scala should have a clear idea of why they are doing it, and what they will have to sort out along the way. Having a strategy or at least an explanation to the challenges posed here is a way to actually get to experience the benefits made possible by Scala.

Oh, and have a happy new year.