Archive for the ‘Software’ Category

« Older Entries

TDD, evolution and architecture

Sunday, November 15th, 2009

TDD has had a rollercoaster ride since Kent Beck first coined the term in 2002. This is nothing to wonder about – TDD really has the power to be a significant power for change for a team of developers.

There are basically two camps, who don’t agree 100% on what the last “D” is. Kent Beck originally called it “Test Driven Design”, making the tests an integral part of the design process. More than half the people I talk to today (counting myself) is having a problem not calling it “Test Driven Development”, where the tests are not as strong a design driver.

Today, I am going to write about the “tests provide good design” camp. I am in good company, too. Uncle Bob and James Coplien have both debated this, along with many, many others. I am now going to weigh in on this, too. What arrogance. Oh, well. Here goes:

TDD, in the original meaning of the acronym, is a technique where tests are written first. Since the test is new, it will fail. The minimum amount of code that will make the test run is then written, and the process is repeated. At some point, the developer may decide to clean the code up, in which case the tests must run successfully both before and after the cleanup. This refactoring of the code gradually improves the quality and the architecture of the solution.

Testing the wrong thing

A couple of weeks ago I was told that a team had testing well in hand. They had 97% test coverage. I immediately became very, very skeptical since I know the bug count for this team.

We developers are techies. This is what enables us to create very good applications. It is also one of our biggest blind sides. We tend to treat everything as a technological problem. If this is transferred to testing, we get nonsensical tests with names like “testConstructor”.

This is a really, really big problem. Your customer is never going to care about your constructors. What he cares about is the behavior he expects from the solution. This kind of problem was described by Dan North when he coined the term BDD, where the tests are explicitly modeled on customer-oriented statements. So – tests should test something the customer cares about, not the implementation. We may want to change the implementation in the future.

Your tests should be connected to your business model

A side effect of the above statement is that you, if you are using DDD (Domain-driven design, are going to have tests that nestle up to your domain model very intimately. Your customer is going to use some nouns, those nouns are going to become parts of the model, and your tests will probably express themselves using the same nouns. It is imperative that your tests can handle changes to the “domain language” your customer is using.

The weakness of refactoring

Refactoring is a process where you improve your system by getting your tests running right before and after you make a nonfunctional improvement. The first book I saw on the subject was by Martin Fowler. I recommend this book heartily to anyone who hasn’t heard of the term before. Since TDD makes heavy use of refactoring, it is important to understand the limitations of refactoring.

I really hate the “refactoring” menus in IDEs. This is not refactoring, in the meaning Fowler used. These are automatic transformations of your code, and disconnected from your test cases. This tool is on a lower level than refactoring is.

Refactoring can only help you make changes in situations where you have tests that cover the behavior that is affected by the code you want to touch. There are lots of implications to this simple statement, but the most important one is that you cannot change the interfaces your tests use using refactoring. You would need to change your tests too, which would place the operation you are undertaking squarely outside of the “refactoring” paradigm. Tool support may help reduce the risk of the change, but the change is not a refactoring operation.

I know lots of very competent developers who are trying for tests that have sub-second runtimes, which means they are usually unit tests where large parts of the application has been disabled. The danger inherent in this approach is that your test boundaries should be very carefully chosen – and should not evolve significantly. A sub-second test runtime is nice, but is not a requirement the customer necessarily cares about if it costs him significantly along some other dimension.

Creating APIs is something developers think of as fun – and it is. This does not mean your application should be littered with hundreds of shoddily designed APIs. Keep the interfaces you expose (to tests or other clients) to a minimum. This is necessary since these bits cannot be allowed to change very often.

I think tests should be running against a system that is as close to the finished architecture as possible, and test against fairly large functional areas. These areas will become your “published” APIs.

Evolution of architecture

TDD in its pure form is an evolutionary process where the developer decides which code structure is the “fittest”. He uses the microscope created by his test suite to do this.

As I have already mentioned here, I have just finished reading “The greatest show on earth” by Richard Dawkins. He devotes a whole chapter to the design shortcomings of evolution. In his case it is a rebuttal of creationist theories, but I am going to pinch his argument here to point out the known problems to trying to evolve an architecture.

On page 360, I found the following paragraph:

During the evolution of the mammals, however, the neck stretched (fish don’t have necks) and the gills disappeared, some of them turning into useful things such as the thyroid and parathyroid glands, and the various other bits and pieces that combine to form the larynx. Those other useful things, including the parts of the larynx, received the blood supply and their nerve connections from the evolutionary descendants of the blood vessels and the nerves that, once upon a time, served the gills in orderly sequence. As the ancestors of mammals evolved further and further away from their fish ancestors, nerves and blood vessels found themselves pulled and stretched in puzzling directions, which distorted their spatial relations one to another. The vertebrate chest and neck became a mess, unlike the tidy symmetrical, serial repetitiveness of fish gills. And the recurrent laryngeal nerve became more than ordinarily exaggerated casualties of this distortion.

Again – go read this book!

So – the recurrent laryngeal nerve. What am I on about? Well, this is a nerve that is routed from the brain, down into the chest to hook around a major blood vessel, then to return to the larynx. An engineer would lop it off and reroute it to save several metres of nerve in the case of a giraffe, but this is something that we cannot completely trust evolutionary strategies to do.

Ok, what is the solution?

I believe in self-organizing teams where there are incentives to choose the best designer to go off and design the APIs that need to be good from the beginning. I believe in using tests to help get the rest of the design good enough.

This won’t help much, though. Some teams are not heavy on design expertise, or worse yet, may think they are. I have had colleagues with two years of work experience refer to their extensive experience while talking to people with double-digit career spans. I am sure I did it myself ten years ago, and I don’t think there is a single developer out there who is not overestimating his knowledge in one or more areas. Teams who could use more design expertise should do their design up front, or at least to find the parts of the solution which needs to change.

Other teams are able to get things right straight away because most of the teams are experienced designers. If your team has multiple Kent Becks on board, the best thing for an architect to do is probably to shut up and let them get on with it.

There is absolutely nothing I can say from this end to help find this balance, since the balance is team specific. But if you are a person who in some way or other has an interest in the ability of a team to deliver change and quality fast, I suggest you make sure they get a steady stream of domain model changes – and measure them on their ability to get things into production. That way, you will always know what to expect in terms of speed of change.

If they get it wrong – let them stop and sort it out.

If they suggest not releasing to production as often (it is too costly!), get outside help! Your team is in effect telling you they can’t cope, and the problem may be you.

Posted in Project management, Software, Testing tools and information | No Comments »

Agile projects don’t need architects?

Saturday, November 7th, 2009

The past couple of days, people have talked to me about architects. There are a couple of statements I have heard:

- agilists don’t do design

- we (my architect team) are supposed to work as a team with the developers as a self-organizing team. How can we do that, and still tell the teams that they need to do something differently?

This is all based on a couple of misunderstandings, I think. I will try to straighten them out here.

“Self-organizing teams can decide everything by themselves”. This is obviously wrong. If this is the case, then what is the product owner for? The team decides how to make something, but not what to make.

Mike Cohn wrote someting about this yesterday on his blog. There is obviously at least one proponent of agile – with a proven track record – who also thinks about up-front design, which means the teams around him are not all reactive. YAGNI, but you just might want to think about things for a bit anyway.

The product owner is one thing. Something similar applies to the scrummaster or the agile coach. People seem to think these characters don’t wield any power. This is also bunk. The scrummaster controls the direction of the team, almost in a tayloresque. The major difference is that the team is allowed to set their own pace. The same thing applies to the scrum coach – whoever thinks coaches have no influence is obviously not one.

I have to admit I am not a purist. I think there is a need to have a technical function – something like a product owner – to handle technical alignment and nonfunctional stuff at the system level in large projects. I don’t really care what this character is called as long as both the development teams and whoever is exercising the function are in agreement on what this function can and cannot do. I will continue to call the function “architect”, even if this will lose me some hardcore agilists.

Whoever is in a function like this should take care not to become just another developer. This is the first mistake any developer who is tasked with a different job makes. Once the problems start cropping up, it is very easy to fall back into the job you know how to do. I have seen this happen many times – and I did it myself, back when I started out as an architect/techie. The primary task of an architect is to ensure the teams are aligned. Life will become hard if this task is left unfulfilled.

This is where life gets hard. I don’t believe it is possible to get this technical alignment to happen without being a hardcore techie. I absolutely abhor non-developer architects. Still, I am saying techies who get this kind of mandate need to keep at least partly away from being a developer. Feel conflicted yet? You should. This is a very hard balancing act, which is compounded when teams do not agree and there is a need to be a part of all teams and none to be able to get things moving forward.

A coach achieves this by actively not delivering opinions. There are several ways of trying to handle this conflict, ranging from becoming “an opinionated asshole” who never delivers anything of value, going for the more military imperative style, through a coach-like style. Not everyone can adopt all of these styles, and I am not sure any of these styles are right in all situations either. Most architects lack psychology knowledge, and are ill-equipped to do their task.

I am reading “the greatest show on earth” by Richard Dawkins. On page 217, he muses on the building of cathedrals in the context of evolution:

An architect designs a great cathedral. Then, through a hierarchical chain of command, the building operation is broken down into separate departments, which break it down further into sub-departments, and so on until instructions are finally handed out to individual masons, carpenters and glaziers, who go to work until the cathedral is built, looking pretty much like the architect’s original drawing. That’s top-down design.

Bottom-up design works completely differently. I never believed this, but there used to be a myth that some of the finest medieval cathedrals in Europe had no architect. Nobody designed the cathedral. Each mason and carpenter would busy himself, in his own skilled way, with his own little corner of the building, paying scant attention to what the others were doing and no attention to any overall plan. Somehow, out of such anarchy, a cathedral would emerge. If that really happened, it would be bottom-up architecture. Notwithstanding the myth, it surely didn’t happen like that for cathedrals. (Footnote: My medieval historian colleague Dr. Christopher Tyerman confirms that this was indeed a myth that was invented in victorian times for idealistic reasons, but that there was never a scintilla of truth in it.)

This is a wonderful book – go read it.

So, it seems that our profession is now creating the same myth that was created a long time ago by (parts of?) the building industry. I am guessing whoever built cathedrals in victorian times did not perpetuate this myth, but builders of smaller things – things requiring architecture that could easily be handled by a small handful of people – could have used this myth to cut the architects out of the loop. You can still buy a small house without getting help from an architect. I wonder if there are any failed cathedral projects out there where a bunch of masons went wild?

Our problem is that the development business is very new. We don’t know how to define an architect role in a way that works yet, and we don’t know when we need to have this role around. I have a hunch that most who get this right are doing what they are doing by the seat of their pants.

I would also guess the definition of what a software architect should do is situation-dependent. Some teams produce useful installation instructions without being prompted. Others are completely at a loss and need a bit of guidance. Some teams are whipped into stopping their QA processes in favour of producing more functionality, and are never allowed the time to correct the shortcoming.

My own shortlist of candidate areas for guidance is:
- System-level QA
- Performance
- Versioning
- Deployment
- Technology platform (as in “all subsystems should use the same OS and middleware to reduce operating costs”, not “use junit 4.3.1″)
- Arbitration of interfacing conflicts
- Tracking of delivery by external parties, if applicable
- Moderation of time-to-market demands by showing the cost of skimping on structural quality
- Establishing an expectation of the lifespan of the software

I am writing an architecture mandate right now, where I am trying to find the balance between owning nonfunctional requirements, letting the teams be agile (or not, if that takes their fancy), and still opening the door for support if they need and/or want it. If I go too far in one direction, the mandate turns into a big ball of mud which won’t help anyone. If I go too far in the other direction, there won’t be room for change if there turns out to be a need for it. I am going to try to get reviews of the mandate every three months to try to get the whole thing right.

If you have any thoughts on this, please feel free to leave a comment.

Posted in Project management, Software | 5 Comments »

X is dead. Long live X.

Monday, October 19th, 2009

The last couple of weeks, there has been a flurry of “X is dead” blog posts. Have your pick: Java, ORM and a lot of other things have been declared dead. Rest in peace. Pushing up the daisies. There is a new technology on the block now.

The inevitable answers are usually not far behind. X is not dead after all. You insensitive clods. And I happen to think the coffin should be black, thank you.

I wish people would take half a step back. Yes, technologies become outdated. Then they die, or linger on, doomed to a Cobol-esque existence in projects without the funding and need to evolve. Get over it. The new technologies will also become obsolete, I guarantee it. Struts is now uncool. I can remember how nice it seemed when it appeared.

The interesting question is how your project can get rid of “dead” technologies, which usually means technologies that are now “uncool” because the in-crowd are not using them any more. Can you do that in your projects? What technologies can you swap out for something different? How often would you want to do that? What would the costs and the benefits be? How can you reduce the costs of getting a project to become dynamic and less technology-bound?

Ave artis nova, simili artis seneca.

Posted in Project management, Software, Testing tools and information | No Comments »

Intelligence and hiring practices

Sunday, October 18th, 2009

I found a weekly in my mother-in-law’s flat today, with a relatively long article on intelligence (good), and why having an IQ of 180 didn’t necessarily mean you were able to get a good job (presumably bad, depending on your definition of “good”).

I started interviewing potential developers for positions around six years ago. Since then, I have seen a lot of resumes, and I have talked with a lot of people. The reason why I am writing this is that a friend of mine, who is a top-notch developer without any managerial experience told me he envied me because I have an insight into how an interview works from the employer side. Most people tend to get this wrong, so I will try to explain how my head is bolted together when interviewing. Hopefully, other people will think the same way.

Norwegian schools and intelligence

The backbone of the article I referred to seemed to be an assumption that very smart people get bored in the norwegian school system, and that Mensa is a kind of support group for the poor people who are cursed with an above average intelligence. There were quotes along these lines, at least, but I will try not to tar all members of Mensa with the same brush. The idea was that school work was so ridiculously easy that being interested in it was a bit beneath these people.

When I was at university, we had to jump through a number of hoops in order to graduate. Some of these hoops were knowledge-based (I remember a course called MA-105 with fondness). These courses were designed to be harder than adequate, prompting most students to have some thoughts about whether they wanted to go through with their studies, or to find something else to spend their time on.

The other kind of test put into the coursework was a test of willpower. Many students spent a lot of time finishing their M.Sc. thesis. Part of the reason was that the earlier grade of Cand. Real. was twice as long, and faculty had not completely caught up with this fact. Another reason was that students were mostly left to their own devices. Finishing the thesis was a different kind of filter. If you hired one of these people, you knew that you were getting a reasonably smart person with a will towards doing something.

Keeping students around indefinitely is a costly proposition, so these studies have now been restructured to provide much more guidance, forcing the students through the system faster. The downside is that candidates have not been sieved the same way as they were before. Employers have had to compensate for this.

This is why I think there is a broken premise in the article I read. The article is based on the premise that extremely intelligent people can do whatever they want. This is garbage. Intelligence is only one part of the equation. There are persons out there who are both highly intelligent and simultaneously too lazy to work at something for a significant amount of time. I don’t believe there is any connection between these two personality traits.

What does this mean for my interview?

There are literally thousands of pages out there explaining how to put a sensible CV together. The goal of these pages is to get you to an interview. From there on out, you are on your own. What happens in the interview is mostly a private affair.

Putting a good CV together is a very, very important task. It is not common for senior developers to read through 250 CVs. The contents of the CV is the first toll gate that gets you closer to an interview. If an employer gets 250 CVs, only three or four people get an interview. The other 245 people are – hopefully – not right for the job.

Sometimes, the sieving process means your CV will not get read. A local employer is known to have divided a bunch of CVs in two, binning one half while saying that he didn’t need unlucky employees. In other words, not getting an interview does not mean you are unsuitable. You may just be unlucky.

There are ways to grab attention. A girl who was applying for a position five years back tried to bribe her way into an interview by offering a hot balloon ride. It was all done with humour, and I still remember the CV to this day since she managed to tickle my interest. This is a bit dangerous since it depends on the interviewer sharing the same kind of humour as the candidate. Oh, and by the way, she didn’t get an interview, and I didn’t get a balloon ride, but it was a very close call.

Another guy in the same bunch of CVs was not that lucky. I was given a 5 cm wad of CVs. About 25% of that thickness turned out to be one CV. This bloke had added copies of everything – even a copy of his boy scout knots award. I also remember this CV for much the same reason – it gave me a laugh, even if it was more of an exasperated one.

The final CV I want to mention is the electrician with a six week course in building PCs who applied for a senior developer position. Take care to understand what the position is, and don’t apply if you are clearly not eligible. You will be seen as a time waster.

I will look at your positions, I will look at the technologies you claim to know. I will have a look at your cover letter – if you say you are interested in switching pastures, I may still have room for you. The ones I think best match the skill set I need, who will be willing to work for the money I have, and who seems to fit socially with the team I want to build will get the interview. It is that simple, and that hard. Prospective employees usually have no idea how I want my team to be built.

There are other ways to get my attention. If you know someone I trust, you should get them to give you an introduction. If the introduction is good and I have good experiences with referrals from this person, you will almost certainly get an interview.

So, let us say you have been able to get an interview. Now what?

Time for the interview

Norwegians are usually an informal lot. I usually wear a sweater, t-shirt and jeans to work. I may wear a suit if I feel like it, but I usually don’t. It would not be playing the part of techie hereabouts. I want you as a candidate to dress much the same as I do, but a little neater. Business casual is good. A three-piece suit will earn you a couple of questions to try to figure out what is happening. Turning up smelling of sweat, and possibly stale beer, is a surefire way of losing the interview on the spot. Talks with employees about the need for personal hygiene are not experiences I cherish.

When the time comes to meet, we will probably meet on neutral ground, which means the reception or somewhere like it. I will offer you coffee, water or something else, and try to put you at your ease. If you are very nervous, skip the coffee and take water instead. When you are in the meeting, try not to use your mug or glass of water as a lifeline. Many people – myself included – grab onto something for comfort when they are under pressure. That is all fine, but you may be holding onto the cup when you should have been pulling out some paper to draw something for me.

Most people are very, very nervous. There is no need to be. My job is not to kill or maim. My job is to find out if we should work together. I really won’t bite. So try to relax, yeah?

There are two ways we might start. If nobody has done it before, I will present the company and what it does. If someone else has already done that, I will pare it down, and just describe myself and what I do.

I think there are two ways the remainder of the interview can work out. If you are talking to a senior techie who has lost his touch, and is willing to do full-time administration, you will probably breeze through the interview. I would ask myself if I really wanted to work for an architect like that.

The why…

Now, here is a dirty little secret: I don’t care if you know the contract of the equals method in the java Object class. I may ask, but the contract is not the endgame. You can look the contract up if you need it. My goal is to get you talking about contracts, equality, possibly cross-references to Haskell, ruminations on how hard it is to implement equals, why you are fed up with ORM, have I tried the new eclipse plugin that does this, that or the other, …

The details are not interesting. When I ask someone why they prefer to develop on Ubuntu, I don’t care about their reasons. I care about the reasoning that lead them to the decision. A developer who says he prefers jetty over tomcat probably has some reason for it – even if the reason only is that he has never seen a reason to fix something that works, and haven’t been able to prioritize toying with other containers. I want you to show me that you are not just parroting what other people are doing, but that you understand the reasons behind what you do.

The how…

I will probably also ask quite intricate questions about stuff you say you are proficient with. Sadly, the term “expert” is very variable. “Expert” from one CV means beginner-level, while another CV might hide world-class knowledge. I have had people claiming to know unix well not be able to tell me what services were running on one of my boxes. I need to know this stuff in order to map your skill set against my expectations, but it doesn’t mean much either way. Competent people are able to pick up technology fast. I will need to know if you need teaching.

I will also try to figure out how you learn. Do you read books? Blogs? Go to conferences? Do you participate in something that gives you knowledge? What do you do on your time off? For norwegians – are you comfortable reading english books? Speaking english? How about italian?

… and how it all works together

There is still a relatively large amount of developers out there who do code-and-fix. I need to know how structured and disciplined you are. I will try to draw you out on this, and you should let me. Most of the people I interview have absolutely no idea how their code production process works. How do you handle bugs? Have you got any personal little quality improvement procedurettes you use? How many years were your projects meant to live?

All of this goes towards giving me an idea of how structured you are. I think this is much more important than knowing technologies. I can teach technologies, but how do you teach being structured? How do you make a kid tidy his room? And do I want to teach a 30 year old developer how to write tidy code? Probably not.

After a slightly extended hour or so, I will probably start winding down. I will progressively give you a looser and looser rein in case there is something you want to tell me – stuff you know, and which I haven’t been able to drag out of you.

Don’t lie by omission

A while ago, I was in an interview with a bloke that waited until this last moment to tell me he had already signed a contract with a different company. He couldn’t get out of this contract. I cannot for the life of me understand why he didn’t call me before the interview to tell me this. His CV was good, and he would probably have gotten an interview when his contract ran out after a couple of months. He left me doubting whether I can trust him. Don’t do that. It can’t help your chances. Be honest.

– oh, and if you are looking for a job in the Oslo area as a senior java developer, my company is hiring right now.

Posted in Project management, Software | No Comments »

Web application security

Wednesday, February 25th, 2009

A couple of people on my team are heavily into security, and I have also been part of several security reviews in the past.

It is high time I had a look at OWASP, which is why I am going to my first ever OWASP meeting in 15 minutes.

For those of you who have not had a look already, OWASP is a community focused on improving application security. Nice project. These people have a comprehensive reference about all things security related.

There is a very real need for this kind of project. I am still seeing SQL injection problems caused by people assembling SQL by concatenating SQL and user-supplied data instead of using prepared statements. I would have hoped we were rid of this kind of problem years ago, but this is not the case.

So – if you haven’t already, have a look at OWASP. There surely is something to learn there for everyone.

Posted in Software | No Comments »

Building deployable applications

Sunday, August 3rd, 2008

(Edited 2008-09-14 to include information from comments)

I have been working with a hosting company and seen what we developers pass off as applications.

These are some musings on what a developer needs to do in order to keep applications deployable.

The installation guide

All developers really, really should create an installation guide. They should then try to forget everything about their application and try to follow the guide.

Did it work? If not, it needs to be fixed. Modern development environments thrive on being able to have massive numbers of deployments. For a far better reasoning behind this than I can ever give in a short blog article, have a look at Refactoring databases by Ambler and Sadalage. Every project, team and developer should isolate themselves from errors made by someone outside their control. Virtualization technologies (or, if you are like me, I prefer to use my own workstation) have made this possible – but only if it is easy enough to deploy the application that people actually bother.

This is somewhat the same problem as testing. Architects need to make it very easy to write good tests, otherwise people won’t bother. So write that deployment guide, and try to make it as easy as possible to get from a source code repository to a running version of the system for someone who does not know the system.

Configuration should be split

A lot of developers seem to think that it is ok to place the configuration inside your application. Since I am a java developer, this often means inside some kind of java archive.

This is a really bad proposition for keeping an application deployable. Often, people will start to check in configurations for the test environments. There never seem to be provisions for configuring the production environment (or, when it comes to that, developer environments) without doing silly stuff.

It is perfectly ok to have all the static configuration inside the application. Whatever changes between installations – database logins and the like – need to go somewhere else. These are some guidelines I try to follow:

  • Production passwords must never be checked into a developer repository. Developers should not have access to production passwords without a human giving it to them. Letting everyone tamper with the production database is a very good way to increase the risk of stability problems.
  • The hosting organization must not be required to unpack an application, rewrite the configuration, and repack the application. Repacking leads to several different versions of the application floating around, with who knows what bugs. The release manager builds the application archive, period. Nobody messes with it afterwards.
  • There must not be defaults for the installation-critical configuration bits that point to the test rig. If you do this, you are placing yourself in a situation where the only protection against accidentally putting production data into a test rig is a routing table entry. If the entry disappears, the best thing that can happen is a very costly reconciliation between the test and production data sets. Worst case, data is lost when someone tries to test the stuff that is in production – the stuff which by now is not behaving as expected – and clears the test database before the test starts. Please don’t write this off as far-fetched. I have seen it happen.
  • If installation-critical configuration does not exist, the application needs to stop. This is one of the oversights of the JEE specifications. There is no way for an application to tell the application server that it wants to stop because continuing is not an option. This functionality usually needs to be written inside the application or be application server specific. If this condition is not met, people trying to install the application are going to spend lots of time reading logs, trying to find the log statement saying that it wasn’t possible to connect to the database. This statement should perferably be the last statement before the “I am stopping” statement in the application log.

I usually make some kind of provision for providing the name of a configuration file using either JNDI or command-line options to JAVA. Both are usually acceptable to hosting organizations.

A nice side effect of not hardcoding the location of the configuration file is that several instances of the application may suddenly run on one computer. This does away with a lot of virtual machines.

Use a production-like environment during development

The closer the system the developer uses is to the production enviroment, the fewer the problems.

How do you know that you are close to the production environment before this enviroment has been built? Specify how you want it to look. Lots of projects get into trouble because different team members think something is self evident – like how the production environment looks. It usually isn’t at this level. Knowing how the production environment needs to look also makes it easier to select a hosting provider.

Specifying how the application enviroment should be, seen from the development side, is the first step of starting a constructive information exchange with the hosting provider. A good, professional hosting provider will wish to make changes to the environment of the application in order to reduce the costs associated with hosting your application, and there is absolutely nothing a developer should attempt to do about this. A competent hosting vendor is the hosting technology expert, and needs to be able to exercise this kind of control.

Whatever changes to the application environment is agreed on also needs to be fed back into the development and test environments. As the first paragraph of this section started – the closer the system the developer uses is to the production environment, the fewer the problems.

Share the pain!

All developers should install a local copy of the application. If this is not possible, something is wrong. If it is too cumbersome to do so, only the developers are in a position to fix the problem – and hosting personnel is not much cheaper than developers, so it is better to fix the issue than to let the problem persist for the lifetime of the project.

Next: Installing deployable applications… When I get the time, at least. Shout if I should prioritize it.

Tags:
Posted in Software | 2 Comments »

Evolving domain models and processes

Thursday, May 1st, 2008

During a talk with a colleague this week, I think I hit upon an insight into why SOA and domain-driven design seem to be causing so much pain in organizations out there.

First of all, I am writing about businesses where there is a fair bit of money on the line, and with a nontrivial complexity. In other words, I am talking about borderline “enterprise” businesses.

When a business first starts out, there is uncertainty about the business and marketing plans. As a result, the first couple of years of development in an emerging market is a hunt to find the right domain language, and how to best put it into software systems that help create value. During this period, the domain model is very plastic, and usually expands every week. However, since the business is still finding its feet, there are no IT-based processes to speak of – or they are IT-based, but not very complex. There are experts around to handle the processes.

Any business entering an unknown business area has a very strong economic motivation that pushes them towards finding a business model that works – quickly. If you don’t have a working business model, you are going to get outpaced by your competitors, or not get enough business to stay afloat. In other words, anything goes as long as the business is able to hit a sweet spot in the market. Who cares if there are people handling stuff manually if you aren’t getting any customers?

After the business has found one or more of these sweet spots, the game changes. At some point, someone is going to find that the business plan is good enough – it is time to cut costs and eliminate unnecessary manual steps. This is where business process modeling starts to make an appearance, and with it, SOA-style technologies. There is usually a fair bit of pain involved in introducing these kinds of technologies. They are costly, and in some cases hard to understand. Sometimes, there are consultants around to help with choosing the right path – and there is still a substantial risk of doing it wrong, even though significant funds are poured into finding the best expertise out there.

The good news is that a successful SOA project makes it easier to change the business processes. This is the upside which makes all of the pain bearable. However, making the business processes pliable usually also makes the business model rigid. It is no longer as easy to change the underlying domain model. This is a function of having exposed services, which by now have an unknown number of clients, and which are part of a set of processes which usually get complex very fast. Anytime something in the underlying data model needs to change, there is a significant risk of major pain or administration happening.

Now, imagine that the enterprise wants to expand into a new market, or if something major happens to the market. This is a clear change to the business model – new domain objects will usually be introduced, or at the very least new services.

This may be part of the reason why large organizations have a hard time keeping up with smaller, younger organizations when the market changes. They may be trying to fit a changing domain model into an existing environment geared towards efficient operations instead of building a “domain model nursery” where the focus is evolving the business model as fast as possible.

What to do about this? Not much, seen from the development point of view. Developers are generally not fond of having two ways of doing what seems like one thing. This is probably very hard to change.

From the management side, there may be lots of things that can make a difference. The new market may be handled by a separate group, possibly even by a separate organization. The differences between the markets may be communicated to the developers as a precondition before the work starts, possibly even accompanied by a strategic plan for how the system is expected to evolve.

This may seem obvious, but the idea that published services may fossilize your domain model in this way was not something I had been able to put in writing until now. I hope someone gets some mileage from this…

Posted in Software | 1 Comment »

This is why macs are the java developer machines of choice

Thursday, April 3rd, 2008

I have just figured out why Apple seem to have cornered the market for java development hardware.

It has nothing to do with speed, ease of use, or glitzyness.

It has everything to do with annotations. People are starting to use annotations for loads of stuff now. The most extreme example is probably qi4j, where the annotations are running the show with bits of java showing through.

Around here, Spring have cornered the market for application infrastructure. Spring has also started to do annotations that are very useful.

This is where Apple comes in. They have put the “@” symbol on a separate key. That has to be good since it feels like java code now is consisting of close to 50% ampersands.

I really, really wish Sun had taken a different approach with annotations. My code is starting to look like perl, with the symbols you get by pressing shift and the upper row on a PC keyboard eating more and more attention.

At least I don’t have to press shift to get the @ symbol. It almost seems like Sun are also using Macs.

Posted in Software | No Comments »

Is SOA coming of age?

Friday, March 21st, 2008

Finally, something seems to happen to SOA.

My take on SOA is that it is one of the most over-hyped bandwagons the software community has seen since the three-tiered “enterprise model” appeared around 2000. Everyone seems to have written a SOA book, often only slapping the “SOA” label on a set of technologies before setting them free in a cruel world. There is no consensus on what a SOA architecture is, how to talk about one, much less hos to make one successfully. People mostly talk about what SOA is not.

The same thing happened to the three-tier application model, where an application is split into a presentation layer, a business layer and a data storage layer, with each layer being remoted. The remoting (once again, a technology push) was there to enforce separation of presentation and business logic.

After a while, real-world considerations caught up with this model. Things like latency requirements and usability pushed the application of the model backwards to a middle point where all requirements were taken into account. The presentation layer would still be separate, but the remoting between the presentation and business layers was removed because it didn’t carry its own weight.

I was very glad this morning when I read Jabows is the enemy of enterprise SOA. I wholeheartedly agree with the statements in this blog posting – SOA, or any architecture for that matter, is not primarily a function of technology. It is mostly a function of people and business needs.

Hopefully, this is all going to lead to a set of best practices for creating SOA services that are workable, and which are not vendor- or technology-centric. There are people who know how to do this today – I work with a couple of them right now – but up until now, people have learnt by trial and error. Perhaps someone is going to step forward and create these guidelines now.

Perhaps SOA is growing up.

Happy easter.

Geir

Posted in Software | No Comments »

Composite oriented programming to the rescue?

Friday, December 28th, 2007

Some people, counting Richard Öberg, have come up with the latest in a long series of attempts at solving a long-standing OO problem – how to create classes that can take part in several different functions.

An example: Suppose you are a developer working for a telco. You may have heard about something called a “terminal”, which is… well… a phone. Sort of. Only it can do much more. No, really. A telco guy would know what I meant. (And, presumably, agree with me about the concise meaning of the word, which I can guarantee would not happen. Another blog post waiting to happen right there.)

If you have been persuaded by Eric Evans or Martin Fowler, you will quickly notice that terminal is a noun, and therefore may be a domain object. If you have not been persuaded by the gurus that a domain language or a domain model is a good thing, this post is not for you.

A terminal has several different natures. You can think of it in terms of a logistical unit – something that sits in a warehouse, gets mailed to a customer, then possibly returned.

It is also something that provides a dial tone to the customer. Note how a dial tone has very little to do with logistics. This is something separate, which happens once the customer has gotten hold of the terminal. Both of these functional areas are relatively complex behaviors which are going to need many lines of code to describe.

I can think of at least two more major functional areas terminals are involved in – and this is typical of a domain model. Your model is probably going to contain several hundred domain objects. A handful of them are going to be model hot spots which will, if you don’t watch your step, grow into spaghetti monsters with several functional areas smeared out over a single object. This is a problem on many levels. The simplest problem to describe is the sheer size of such a spaghetti mess – several thousand lines of code that is not one coherent unit, located inside one class/file is very hard to understand. Hard-to-understand code is not a good thing. It is a specialized case of writing your system using a single object.

Many have tried to solve this problem before. C++ uses multiple inheritance. AOP solves this problem, as well as many others. So far, no single solution has “won”. If you want to read more about these solutions, I suggest getting Filman et al, “Aspect oriented software development”. Oh, and while you are at it, Clarke/Beniassad: Aspect-oriented analysis and design might be a good companion book for when you want to draw your domain model out on a whiteboard. UML is not going to cut it alone.

Qi4j seems to be a rework of parts of AOP with this problem in mind. I haven’t been able to look at it much yet, but so far it looks more promising than AOP did. I think AOP has lost. I can’t really see any programming shop invest in teaching AOP to its junior developers if they haven’t done so already. The required “delta skill set” to use AOP is just too large compared to the benefits, and it is also “old technology”, which for some reason is ungood in the software business.

Qi4j looks like a much lighter-weight approach to this problem. I am going to be interested to see what happens to it in the coming months. I hope I can find the time to look much harder at it.

Geir

Tags:
Posted in Software | 3 Comments »

« Older Entries