My book “Secrets of a Buccaneer-Scholar” will be published in September by Simon and Schuster. It’s a book about how I approach self-education, personal branding, and original thinking. It’s the story of how I got started and how I got along all these years without any formal education or certification. It’s not a book about testing, but it will be highly relevant to testers. It will be relevant to any knowledge worker who wants to compete better in a down economy.

The book is finished and delivered, it just takes a long time to get through production and sell into book stores. While I’m waiting and writing the next buccaneer book in the series, I’ve started a new site at buccaneerscholar.com and a new blog. There I will talk about the more general topics of thinking and learning that are not specific to testing.

I’m also creating a class to teach self-education (not as ironic as it sounds… but it does sound ironic, eh?).

This blog will continue to deal exclusively with testing and issues directly related to the software development industry.

Sometimes I hear people react to context-driven methodology with a shrug. “Yeah, everything depends on context. So what?”

Here’s the so what: If all practice depends upon context, then the competent practitioner must know how to invent, evaluate, criticize, and modify practices. In other words, focus must shift from merely copying and memorizing practices to developing skill in the craft.This is a profoundly different focus.

Wouldn’t it be absurd for someone to claim that the best way to commute to work, for everyone in the software industry, is to take a horse and buggy? Wouldn’t it still be absurd even if they added the words “but you should take context into account and do what’s right for you” at the end of such a strange claim? When consultants talk to people from different practical contexts, and they treat context variables as a trivial concern, they do their audience little service. That’s the trouble with “best practices.”

To say that context matters but that “almost always X is the right thing to do” is to speak like a parent dictating to a child. I fear that many testing bloggers and consultants are happy to do just that. But if what we want is a meeting of minds and sharing of wisdom, we need to practice talking about context-dynamics and the dynamics of practices that serve those contexts.

The most recent time I practiced this, myself, was in writing the previous paragraph. I almost wrote that you should not, as a methodologist, speak to your clients as a parent dictating to children. But then I noticed that very statement is just such a dictum. I don’t want to be like that. So I had to add “But if what we want is…” to at least acknowledge that even here there are different ways to be, depending on context.

A Suggestion

A great way to automatically avoid the perils of best practice talk is simply to talk about your own experiences and preferences, rather than make general prescriptions. This is why, in peer conferences such as LAWST, we focus on experience reports (actually we called them “war stories” until our terminology was attacked by bloodthirsty pacifists) rather than advertisments for good practice.

Cem Kaner has just posted a clarification of context-driven testing. I contributed to it, and it also represents my point of view.

One reason we’ve come up with this new description is that several people (we don’t really know who) have posted or copied descriptions of context-driven testing that bear little resemblance to the vision of the founders of the school. These descriptions typically claim that context-driven is a flavor of Agile. No, no, no, no, no, it isn’t. If context-driven testing is a flavor of anything, it’s a flavor of problem-solving.

In a previous life I took a music degree in Bass Trombone, which is a discipline that’s even more geeky and with a worse gender balance than Software (see this meeting if you don’t believe that’s possible). Over the course of the degree, I raised my bar from Enthusiastic to Not-too-embarrasing-to-appear-in-public. In the spirit of this famous article on Pair Programming, here are a few things I learned there that (if I squint) seem to apply to software.

Quality follows a power law Every time I practiced and got to play with better ensembles, the next level up was an order of magnitude improvement, not just linear. I could imagine being as good as the players one level above me, but two levels was a huge jump. The standard of serious professionals nowadays is just astonishing. As an example, one of our ensembles worked on an Edgar Varèse (obscure but influential modernist composer) piece. For the première in the 1920s, Varèse complained bitterly that he only had 100 rehearsals, we did it in about 10 rehearsals, our conductor performed it with a group of freelancers after working on it for an afternoon.

Quality is fractal To put it another way, good performances requires attention to detail at all levels: from the conductor’s management of the overall structure, to players’ phrasing of their individual lines. The better the ensemble, the more levels just work. It was quite a shift for me when I started joining groups that played in tune, a whole area of insecurity just disappeared and I could now use the effort I’d spent on compensating for inaccurate tuning for something more important.

Play for the audience They’ve paid to hear you. They don’t want to hear your technique, they want to hear music. One of my teachers liked to point out that anyone can make great music work, but good players can make bad music sound better. At the other end of the scale, I once heard a scaled-down version of The Rite of Spring where the Bass Trombonist dominated the orchestra; as a practitioner, I was impressed but it was ugly and self-indulgent.

Don’t take it up unless you mean it The perfoming arts are tremendously rewarding if that’s what you want do. If not, it’s a hard trade involving lots of stress and effort—and there’s a limit to how long you can stand discussing mouthpiece diameters. I met several older professionals who hated the business but had nowhere else to go, and the story goes that one of the trombonists in Toscanini’s NBC orchestra wrapped his instrument around his music stand at the end of his last day before retirement.

The section takes the blame (and credit) together Brass playing, particularly at the low frequency end, is a collaborative activity. You spend much of your time playing chords, so no-one can tell that you’re better than the rest of the section. Your best hope is to try to raise everyone’s standard. When it works, it’s just fantastic.

Some people’s abilities are just obvious I met a few players who were clearly here on Earth to play their instrument, that’s just who they were. These are the kind of people who get top-rank jobs before they’ve finished college. Our Head of Brass had been a child virtuoso, he didn’t understand why people played wrong notes. Why would they want to do that? At the other end, there were some who should have had their instruments confiscated.

Sometimes people take a while to shine Other players are not so obvious. I was lucky enough to meet Ed Anderson (Cleveland Orchestra) who was one of the best. He told me that at college, he’d played in the opera orchestra because he didn’t get enough sessions with the (higher prestige) concert orchestra.

Reality, deal with it In a performing discipline there is nowhere to hide, everyone knows how good you are all the time. I had a bit of a crisis in my second year when a new Tuba student took the time to point out my shortcomings; the message wasn’t pleasant to hear but it worked, I was much better by the end of the year. Later a one-off lesson with Arnold Jacobs, who spent a lifetime researching the mechanics of wind playing, changed my playing life because he could show me what was happening to my breathing and how it needed fixing.

How do you get to Carnegie Hall? Practice! It turns out that this topic has just come up on Tim O”Reilly’s blog.

What’s the dynamic range of a bass trombone? On or off

Copied wholesale from D-Squared

In business circles, particularly among a certain kind of aggressive American businessman (or consultant, or banker, or politician, they’re fairly interchangeable), there is a favourite proverb about a pig:

“When you have bacon and eggs for breakfast, you’ve got your breakfast from a chicken and a pig. The difference between them is that the chicken is ‘involved’ but the pig is committed“

which is of course, true. It should also be noted, however, that when you go out to get your next few breakfasts over the course of the rest of the month, the chicken will have laid another egg every day, but the pig will eventually run out of bacon

We’re just a few days away from 2009 and it’s that time when we all start looking back at the last year and set goals for the coming new year. I felt like sharing some of my thoughts on how I’m aiming to approach the new year.

Historically, I’ve never been a huge fan of New Years Resolutions because my attempts were always too big to successfully measure. The goals themselves weren’t poorly thought-out, it’s just that it’s really easy to make a list of personal targets, without putting a lot of emphasis on how you’re going to achieve them. The biggest trouble that I’ve had with goals is allocating enough mental energy for implementation planning. (if only I had someone to and wireframe my life…)

Due to this, New Years Resolutions haven’t been a huge success for me. I’ve found it much too easy to pass the buck onto the usual suspects, which consist of: lack of time, energy, too much work, general life changes, health, etc.

So, for 2009… I’m going to try something different by focusing on a set of best practices that I can use on a daily-basis. I suppose that my main goal is to not place too much emphasis on any specific targets and instead place the responsibility on myself to follow these best-practices and see what good (or bad) comes of it.

By rephrasing my internal conversation from, “What did I achieve this last year?” to “Am I doing things the best that I can?” I am confident that the answer will usually be, “not likely.” I do believe that through this subtle change in context, I’ll be better apt to self-evaluate how (and why) I am doing the things that I do and refactor accordingly. If we’re not consistently Refactoring ourselves (as we do with our code), we’re going to retain a lot inefficiencies in our personal and work lives, which make it difficult for us to quickly respond to changes and opportunities.

Our life (personal and work) is just another project that we manage. Much of methodologies that we spend learning about and adopting can easily be translated to these other areas of our lives.

So as I brace myself for 2009, I find myself asking, How can I lead a more Agile life?

I’d love to hear how you’re adopting best-practices inspired by Agile methodologies in your life and I promise to share mine over the coming year.

Related Posts

• Agile development begins within…
• Agile development begins within… continued
• Those that Tend the Store need Dialogue

We've all heard the conventional wisdom about good work habits. Many of us have attended time management classes, participated in workshops and have been advised to "work smarter, not harder."


Work habits that might seem less productive will make you more effective, say experts.

Some ideas, however, appear at first glance to be unusual or even counterintuitive.

I gave an interview at the STAR West 2008 conference about agile testing. They’ve just put it online. Here it is! Once again I give answers that are too long. I’ll never make it as soundbite artist.

In other news today:

Emmanuel Gaillot just blogged about our upcoming French Refactoring Training.

January 28 and 29 at the office of Octo Technology in Paris. This training will be in French. We ran it together in-house (also in Paris) and it worked quite well . The training was good fun, we did even more things dojo-style (including demos) than I normally would, and we made time for hands-on systems thinking. I think with the experiments we’ve done this year, that we’ve finally found a good way to do systems thinking with programmers. It seems to be a bit more intuitive for managers and coaches,  but presented in the right way, it turns out developers find it eXtremely valuable. Surprisingly (not ) the topic the developers chose for their diagram was not entirely technical: how too much labour turnover was hindering their productivity and maintenance.

In other other news: the QWAN newsletter is out. It has some instruction on how to do your own Temperature Reading, conference reports and the public courses agenda. Enjoy!

Now it is time to relax after a  busy and period. Maybe I’ll finally get around to posting photos from all the conferences we’ve been to since october…

In my previous post, Working Hard, or Hardly Working?, I mentioned how it was possible to tell the wheat from the chaff by observing.  Alistair Cockburn rightly suggested that the manner of observing that I suggested was more appropriate for a team leader than for upper management.  Yes, he’s quite right about that.  He had in mind a Vice President who, when told that the development team had estimated something would take two weeks, asked him,

How can I tell if it really should take that long, or they’re just padding to make their life easier?

How would you answer that question?Alistair says he had no answer.  I have a question.  What does “should” mean?

At one level, it’s a tautology.  If they’ve already done the work and it took two weeks, then “bearing in mind that there are many factors of which I am unaware, since the event has already occurred, it is obvious that” this work should have taken two weeks.

Or it’s a self-fulfilling prophecy.  For these developers in this situation, it’s going to take at least two weeks if they’ve decided that.

Now it could be they’ve estimated poorly, and they’ll deliver it in one week to everyone’s surprise.  So it shouldn’t take that long and it didn’t.  If there’s padding, it’s probably to prevent disappointment rather than to avoid work, else they’d have stretched the work out for the full two weeks.  You’re likely to see this behavior where estimates are treated as commitments.  You’re also likely to see this when developers have become accustomed to such treatment elsewhere.

It could be that some other set of developers could easily do this work in one week, but the best that this set can do is two weeks.  Does that mean this set of developers should be able to do it in one week?  Should they have the experience with this sort of problem, with this language, or the skills to work faster?

It could be that these developers could do the work in one week if it weren’t for all the impediments in the environment.  Maybe the computers are old and slow and have too little memory.  Maybe there are delays waiting for information or confirmation or projects on which this one depends.  The developers should be able to do it in one week, but due to circumstances beyond their control they can’t.  Should the circumstances be different?

It could be that these developers could do the work in one week, but they think it implies other work that isn’t part of what’s being asked of them.  If it were discovered that they thought this extra work were part of the request, then they could be disabused of this notion and complete the work in one week.  But it’s not discovered, so the developers do some unwanted speculative work and it takes two weeks.  Should they have noticed and done it in one week by doing just what was wanted?

It could be the developers feel that they’re treated poorly.  They feel demoralized and it’s hard to get out of bed to come to work, much less to focus on the work they have to do.  If they weren’t so depressed and disorganized, perhaps they could get it done in a week or less.  But everything seems so hopeless to them that it actually takes two weeks.  Should they be able to shake off their depression and get it done more quickly?

It could be the developers are lazy SOBs who are out to do as little work as possible while drawing their salary.  They take frequent coffee breaks where they talk about sports.  When at their desks, they spend hours playing Quake or surfing random websites.  Should they work harder on the company’s business when on the clock?

These are all quite different scenarios.  The world “should” seems to mean very different things in each of them.

Should someone who has risen to the level of VP be about to tell the difference between these scenarios?  Well, of course.  But this use of “should” has as many meanings as the one concerning the developers.

Should she have some comparable experience to use as a sanity check?  This would also be nice.  But maybe it’s a new industry.  And given all the possible situations listed above, how can she now if it’s comparable?

Should she take the time to talk with each developer, one on one, to learn how they view their work and the company?  If the question is important to her, then I think this is the way to go.  Having these conversations can, if handled well, contribute to mutual trust.  And the very wording in the question indicates to me that in this situation, trust is not as strong as it might be.

But the direct benefit is insight.  Some of the questions in my head would be

• What is it about this job that makes it take two weeks?
• How sure are you that it will take two weeks?  What are the chances it might take one week?  Or three weeks?
• What are the unknowns about this work?
• What similar work has been done by this team?  How long did that take?  How does this work differ?

I might not ask these questions directly, but instead talk with each developer until the answers seemed clear to me (or that the answer would never be clear).  If Ms. VP hasn’t practiced this, she might benefit from something like the Unearthing the Data You Need session I attended at AYE 2008.

And, if it really is just a case of a goldbricking team, then it’s unlikely that all of them are equally skilled at pulling the wool over our VP’s eyes.

One of the tricky things about writing about external DomainSpecificLanguages is that I'm walking through territory already heavily tracked by the programming languages community. Programming language research has always been a popular area of academic activity, and I'm the first to admit that I don't have anywhere near the depth in this topic as many people who've been studying in this space for years. So inevitably the question comes up as to why such a noob as me thinks he can write a book in this well trodden ground?

The primary reason is that nobody else has written a practitioner-oriented book on DSLs. I like topics like this that are well-trodden but not well written about. However as I've spent time walking these pathways I think there's another factor in the works.

There's a lot of work on programming languages out there, but almost all of it has concentrated on general purpose programming languages. DSLs are seen as a small and simple subset of general purpose programming thinking. As a result people think that what's true for general purpose languages is also true for DSLs (with the implication that DSLs are too small to be worth thinking much about).

I'm increasingly of the opposite conclusion. The rules for DSLs are different to the rules for general purpose languages - and this applies on multiple dimensions.

The first is in language design. I was talking with a language designer who I have a lot of respect for, and he stressed that a key feature of languages was the ability to define new abstractions. With DSLs I don't think this is the case. In most DSLs the DSL chooses the abstraction you work with, if you want different abstractions you use a different DSL (or maybe extend the DSL you're using). Sometimes there's a role for new abstractions, but those cases are the minority and when they do happen the abstractions are limited. Indeed I think the lack of ability to define new abstractions is one of the things that distinguishes DSLs from general purpose languages.

Differences also occur in the approach that you use for implementing the tools that go with languages. A constant issue for general purpose languages is dealing with large inputs, since realistic programs will have thousands or millions of lines of code. As a result many tools and techniques for using them involve aspects that make parsing harder to follow but support these large inputs. DSL scripts tend to be much smaller, so these trade-offs work differently.

In my work I've put a lot of emphasis on using a DSL to populate a Semantic Model, using that model as the basis for any further processing: interpretation, visualization, or code generation. Lots of language writing I've seen tend to emphasize code generation, often generating code directly from the grammar file. Intermediate representations are not talked about much, and when they do appear they more in the form of an Abstract Syntax Tree rather than a semantic model. Serious compilers do use intermediate representations, such as program dependence graphs, but these are seen (rightly) as advanced topics. I think Semantic Models are a really valuable tool in simplifying the use of a DSL, allowing you to separate the parsing from the semantics.

Since DSLs are less expressive, you can design a simpler language for them. Much of the language community's writing talks about how to handle the difficulties of a complex general purpose language, while the challenge of DSLS is to write a language that is readable to the intended audience (which may well include non-programmers) and also should be easy to parse (to simplify the maintenance of the parser). Not just does this lead to different decisions on the design of a language, it also means that you only really need a subset of the features of parser generators.

A consequence of this is DSLs are written with the expectation that each individual DSL won't solve the whole problem at hand and often you need to combine DSLs. Traditional language thinking hasn't explored the idea of composable languages that much, but I think this topic is very important as DSLs develop. Thinking about composable languages should have significant effects on both language design and language tools.

So I'm increasingly coming around to the thinking that DSLs inspire some seriously different ways of thinking about programming languages. It may also lead to developing different kinds of parsing tools that are more suited for DSL work - usually tools that are simpler. I hope the increased attention that DSLs are getting these days will lead to more people treating DSLs as first class subjects of study rather than a simplistic form of general purpose languages.

The unknown in all of this is that if a team have a velocity of 6 how do you tell if they should have a velocity of 8 i.e. they are underperforming. It gets complex. If they have a velocity of 16 are they doing well or have they estimated at the higher scale of story points.

I first heard this joke way back when, at my first real job–I was a TV repairman when I was 14.  It may generate a polite chuckle when asked between peers, but it’s serious business when the boss asks the worker.  It’s also been a topic of conversation over on the Scrumdevelopment yahoogroup, where Graeme Matthew described the difficulty of determining this using velocity.

The unknown in all of this is that if a team have a velocity of 6 how do you tell if they should have a velocity of 8 i.e. they are underperforming. It gets complex. If they have a velocity of 16 are they doing well or have they estimated at the higher scale of story points.

I agree with Graeme that this is one of the difficulties with using velocity to measure performance.  I agree with Alistair Cockburn when he says

There is NO good measure of “programmer productivity”.

earlier in the same thread.  Yet when you work with people, you generally know who’s working hard and who isn’t.  It’s an interesting conundrum, isn’t it?

Alistair challenged me on this statement, asking

I guess I’m interested in the “who” in that sentence — what are your presuppositions about the “who” who is working with them. My son can’t tell, so there’s at least one constraint you are assuming.

and I came up with this list:

1. Being close to the observed subject for enough time.
2. The time when the subject is observed is time when the subject is doing the desired work, rather than time when the subject is engaged in creating a good impression on the observer.
3. The observer knows enough about the work to be able to tell good work from bad.
4. The observer knows enough about the work to be able to tell slow work or hasty work from an appropriate speed.

Is this list sufficient?  Or have I missed something?

Alistair points out that this list is fairly easy for a technical lead, but more difficult for managers.  A manager has less time to spend working with the person being evaluated, and generally has moved away from technical competence, if he indeed came from that background.

The fact is, though, that there are managers who seem to keep their bogosity meters working pretty well, even though they’re no longer technically competent to do the work. And there are others who immediately, or even sooner, buy every lie and distrust every truth.  Even though the vast majority are somewhere in between, it gives evidence that there is probably something that a manager can choose to do that will help them continue to make reasonably good value judgements.

I cannot articulate exactly how to determine who is wheat and who is chaff by observation, but I gave a list of some conditions I thought would be sufficient to enable one to do so. It’s almost certainly not the only set of sufficient conditions, because there exist very good managers who manage to make sufficiently accurate observation apparently without meeting the conditions I listed.

What advice might we give these managers?  For the project manager, I would suggest reading Johanna Rothman’s and Esther Derby’s excellent book, Behind Closed Doors.  What advice would you give?  How about for the upper manager, several levels removed from the software developer?

In any event, my point was that the determination of who’s working hard and who’s hardly working can be made without those troublesome productivity metrics.

Just for some fun I thought I would toss together a couple of top ten lists for 2008 (hey it works for Letterman, why not me?). There are three lists here: Top Ten Posts (my picks), Top ten items I wrote for InfoQ and Top Ten by Traffic.

My Picks

in chronological order

• Tools, Tools, Tools As software developers we have this innate belief that another tool will solve all our  problems. To that end many agile practitioners search for a tool to track their projects for them.
• Minimally Agile What is the minimum it takes to be Agile?
• Agile/Scrum Smells an initial catalog of common things that go wrong on Agile projects. It needs your help to make it better.
• Minimalist Coding Style when I sit down with you to pair, I'm likely to make some comments about your code - now you know what is going on my brain.
• Coaching Self Organizing Teams - one of the best sessions I attended at Agile 2008
• TDD Coding Dojo (Randori Format) our first attempt at a coding Dojo.
• Advantages of TDD
• TDD Coding Dojo (Randori Format) Round II - where we apply the lessons applied the first time.
• Scrum Case Studies - given the frequent requests I hear for Scrum Case Studies I gathered a bunch up.
• Misconceptions with Test Driven Development - sensing a theme? I hear a lot of objections saying TDD can't work in XXX case or it won't scale.

InfoQ

In a random order

• Can Product Owner and Scrum Master be Combined?
• Scrum Certification Test
• Kanban as Alternative Agile Implementation
• Rewards to Improve Team Habits?
• Performance Reviews Banished
• Questioning Servant Leadership
• Co-location Transition, Tips and Concerns
• What are the Qualities of a Good Test?
• Well Formed Teams: Helping Teams Thrive, not just Survive
• Touchy Feely Impediments to Agile Adoption

Google Analytics Picks

These prove that the world cares more about photography and vista woes than Agile Software Development. Obviously a list like this will favour older posts.

1. Aperture vs. Lightroom - best comparisons
2. Agile/Scrum Smells
3. Nikon D3 vs D300
4. Driver woes on Vista? Sonic Solutions DLA problem solved
5. Scrum in a Nutshell or 5 minutes to learn scrum BTW the presentation here is a little dated, it needs a good rewrite based on what I'm learning about changing brains.
6. Customer Retention Department - Vonage Customer Service Sucks
7. Scrum Case Studies
8. Lightroom Tip: How to Successfully Import your Photoshop Elements Catalog
9. Minimalist Coding Style
10. Writing Clean Testable Code
11. Multiple Returns from a Single Method

Because 11 is better than 10.

Finally just for pure ego's sake some stats for the year:

• 70,000+ page views this year
• 42,000+ visitors with half the visitors from three countries: US, UK and Canada
• 1250+ readers (according to feedburner and that doesn't the readers who found me through Agile Planet), up from ~450 on Jan 1.

Thanks for reading in 2008.

If you enjoyed this post, subscribe now to get free updates.

I wrote a teaser post about iterative and incremental rebranding of eXperience Agile in September… It’s been four months and almost as many newsletters since then . So time to de-tease the blog…

Marc suggested to do part of the upcoming newsletter as a Temperature Reading, so besides getting some information, our readers also learn a technique, and it helps us structuring our own reflection over the past year. It’s been about a year since Marc, Rob and yours truly sat together after xp days london to discuss closer collaboration with courses and coaching (but still as loosely coupled as possible, because we like our independence).

A Temperature Reading has five questions, the order of which you can vary, depending on the context (see Temperature Reading for explanation and the default order, I’m going to do something different here).

1 New Information

We proudly presented QWAN = Quality Without A Name

We started handing out these rubber ducks at CITCON Amsterdam, for those cases when we are not around to pair (program, brainstorm, reflect, be innovative) with. Talking out loud helps, and ‘rubber ducking’ is a well known technique for this (e.g. Kathy Sierra writes that talking to pets works even better. We came across it the first time in the Pragmatic Programmer ). We chose QWAN as a name, because we are still inspired by Christopher Alexander’s early work on patterns (Timeless Way Of Building) most notably. QWAN is a label for stuff done by Rob Westgeest, Marc Evers and yours truly, often in co-operation with others.

QWAN co-created a bunch of new courses and workshops in 2008: eXperience Refactoring (also with Octos’ Emmanuel Gaillot), Unit Testing Masterclass, responsibility driven design with mock objects, dirty jobs (with Tjakko Kleinhuis), pimp my retrospective (with Topic’s Nicole Belilos), political games (with Emmanuel Gaillot), story testing with Rspec, and a revamped rightsizing your unit tests (and then some that I forgot. This collaboration seems productive, many more than last year it seems. We supported conferences in one form or another, e.g. by sponsoring, as co-organizers or ‘just’ running a session: CITCON Amsterdam, several Agile Opens, XP Days Benelux and London, ESSAP summerschool, Agile2009, AgileHolland.

2 Appreciations

We’d like to thank everyone who participated in our courses, conferences and workshops. Your enthusiasm and feedback helps us to improve and, not least, have fun in our jobs. Everybody who provided encouraging feedback when we published our brochure earlier this year. Especially Nynke Fokma, Becky Winant and other keys for giving detailed suggestions for improvement. We’ve had several great customers (you know who you are), who created amazing conditions for us to focus on our work. Thank you!

3 Complaints with recommendations

We’ve made some progress in supporting ourselves with a Customer Relationship Management system. There are still some improvements that would make our work easier (usability) and that of our customers (sign-up and notification of new courses). (Marc wishes for a more sustainable pace. I’m careful what I wish for - a more sustainable pace would be nice, but I’m also quite happy with our productivity, passion and income at the moment)

4 Puzzles

We made progress in selling open enrollment courses, and are still puzzling on better ways of selling those (as well as making it easier on our customers to plan and buy participation). The context for courses seems to be shifting at the moment - working more effectively has value, but many large corporations have the knee-jerk reaction of putting all training on hold when sales slow. At these corporations people have little time for training when things are good - too busy, and no budget for training when things are bad. Luckily we also have customers with a long term perspective, who invest in training, education and hiring continuously.

5 Hopes and wishes

We hope the ‘interesting’ economy will mark the breakthrough of lean and agile ways of working in more places. We hope to carry out more integrated on-site coaching … With integrated we mean doing whatever works best in a given context : a whole systems approach using combinations of lean, scrum, extreme programming with ’soft’ skills inspired by Satir and systems thinking.

We wish you a healthy, fun and prosperous 2009, and look forward to collaborating with many of you.

(meta-appreciation: thanks to Marc for coming up with the idea for a Temperature Reading, and giving feedback while listening to the items while writing them. Thanks to Marc & Maroesja’s kids Maris, Marden and Marijn for providing the necessary distractions )

I was chatting with Keith about what the Software Craftsmanship event should really be about, in the context of a discussion of whether some of Jason Gorman’s list of guitar heroes are actually worth listening to (you still with me?) ¹.

The next obscure piece of evidence is Maurice Murphy’s YouTube masterclass. For those who don’t know, Maurice Murphy has been Principal Trumpet in the London Symphony Orchestra for years (nerds without an interest in orchestral music will nonetheless have heard him on the Star Wars sound tracks). What’s interesting about the video is that what they emphasise most is being musical, making sense of the melody. They have no interest in pyrotechnics and assume that the student knows the basics.

The (barely sustainable) link with master-level coding and performance, we realised, is that both are about being expressive. The master coder/performer has a point of view and they use their considerable technical skills to get it across. The code that really impresses me is almost entirely about the domain, it just reads well. Most of the code most of us see, however, is stuck a level below, I have to read through the implementation to get to the meaning.

Years ago an old friend of mine said that when she first started going to conferences she wouldn’t be impressed by some of the big names because what they said was obvious. Later, she learned that being obvious can be really hard.

A while ago I was chatting with a post-doc on his way to an academic career. He was asking me about research topics wanting my input as he felt I could inform him on what would be research of practical use. I wasn't very helpful, but I did mention that the best way to do this would be to spend some time in industry to get a feel of how software development works in the wild and what problems could do with some research effort.

His answer to this thought was very troubling. He said he'd be up to do that, but if he spent time in industry that would ruin his chances of getting a job in academia. Competition for academic jobs is high, and what they look it is your publication history. A year or two in industry would create a gap in your publication history that would be lethal to your job prospects.

The divide between academia and industry has always been an awkward one for software (as indeed for other professions). My contacts with academia have been stilted at best. The academics I respect are, I'm told, not highly regarded within academia because the things that I count as useful are usually dismissed by the academic community.

A good example of where this came to a head is the patterns community. Those involved in the patterns world were keen to look at practice to discover, package, and document techniques that had been proved through experience. But this is in direct opposition to academic standards which consider value to lie in novel things. My work, for example, is generally dismissed because all I do is write about stuff that is old hat (at least to some).

I think this is a terrible shame, not because I'm looking for an academic post, but because I think there is huge value in mining effective techniques from the experience of software development. To me it seems that trying to draw lessons from our experience is a very worthwhile academic activity. By devaluing it the academic world is ignoring a fruitful avenue to improve the capabilities of our profession.

If my opinion counted, I'd argue that any academic department worthy of note should include a group of faculty with a long experience of the day-to-day of industrial software development. They would be valued on how they had reflected on this experience and drew from the lessons to inform their teaching and research. I'd like to see a regular rotation of people from the academic to the industrial world, where it's common to see people spend several years in industry, then academia, then industry again, and so on.

This problem isn't only in software. A friend of mine had the chief engineer role in one of the most challenging engineering projects in the world. He fancied a stint in academia, but was only able to get a second-class position reserved for people who weren't considered to be real academics, certainly not something that was tenured or would lead to tenure. I find it hard to believe that students wouldn't gain an enormous amount from being taught by people with a long and thoughtful experience in the profession they are entering.

It's always frustrating to see communication gaps between different groups within the same profession. I've become a big fan using Rotation to help open up communication channels, as people are the key to good knowledge transfer. Being tolerant of academic rotation, indeed encouraging it, could do a great deal to make academia more aware of where industry needs help and industry more aware of where academics can improve practice.

Pramod Sadalage is one of the leading innovators of evolutionary database design.

Our profession seems to be constantly hampered by the communication barriers we erect for ourselves. For enterprise systems one of the most annoying barriers is the one between application developers and database people. Although much of my early years involved databases and data modeling, my involvement with object-orientation cast me firmly into the application development side. As a consequence I haven't spent much time talking to people from the database community.

On January 9 I get a rare opportunity to fix that as I'll be speaking at my local chapter of DAMA with my colleague Pramod Sadalage. Pramod played a central role in the development of evolutionary database design techniques in ThoughtWorks as a DBA who closely works with development teams. His also written valuable books on Refactoring Databases and Continuous Database Integration. So, as you might have gathered, he knows all the material backwards and I'm glad to be invited along for the ride.

We'll be talking about database refactoring and techniques for evolutionary database design. It will be interesting to see how this talk is received. I'm told that these topics are still seen as highly controversial in many parts of the data community, yet these are techniques that are so usual for ThoughtWorks that they are just part of the furniture of our development projects.

Looking further out - I'll be appearing in London again for QCon. I'll update this page later with more details about what I'll be doing there.

Will DSLs allow business people to write software rules without involving programmers?

When people talk about DSLs it's common to raise the question of business people writing code for themselves. I like to apply the COBOL inference to this line of thought. That is that one of the original aims of COBOL was to allow people to write software without programmers, and we know how that worked out. So when any scheme is hatched to write code without programmers, I have to ask what's special this time that would make it succeed where COBOL (and so many other things) have failed.

I do think that programming involves a particular mind-set, an ability to both give precise instructions to a machine and the ability to structure a large amount of such instructions to make a comprehensible program. That talent, and the time involved to understand and build a program, is why programming has resisted being disintermediated for so long. It's also why many "non-programming" environments end up breeding their own class of programmers-in-fact.

That said, I do think that the greatest potential benefit of DSLs comes when business people participate directly in the writing of the DSL code. The sweet spot, however is in making DSLs business-readable rather than business-writeable. If business people are able to look at the DSL code and understand it, then we can build a deep and rich communication channel between software development and the underlying domain. Since this is the Yawning Crevasse of Doom in software, DSLs have great value if they can help address it.

With a business-readable DSL, programmers write the code but they show that code frequently to business people who can understand what it means. These customers can then make changes, maybe draft some code, but it's the programmers who make it solid and do the debugging and testing.

This isn't to say that there's no benefit in a business-writable DSL. Indeed a couple of years ago some colleagues of mine built a system that included just that, and it was much appreciated by the business. It's just that the effort in creating a decent editing environment, meaningful error messages, debugging and testing tools raises the cost significantly.

While I'm quick to use the COBOL inference to diss many tools that seek to avoid programmers, I also have to acknowledge the big exception: spreadsheets. All over the world suprisingly big business functions are run off the back of Excel. Serious programmers tend to look down their noses at these, but we need to take them more seriously and try to understand why they have been as successful as they are. It's certainly part of the reason that drives many LanguageWorkbench developers to provide a different vision of software development.