Sunday, March 18, 2012

Not all TDDs are created equal.

There is little or nothing that has not been said about TDD. As you probably know if you are a practitioner, there are lots of articles describing step by step the whole process behind it and arguably also the state of mind that TDD aims to promote. However, there is something that I find again and again when I go to Coding Dojos, events like the Global Day of Code Retreat, or even when I am pair programming or mentoring someone.

There seems to be, at least from what I have seen, two main ways to approach TDD. I'm going to call them Holistic and Reductionist approaches. Both of these have their advantages and disadvantages and since it is something that comes up a lot with beginners and seasoned professionals alike it seemed like a good idea to pay some attention to it and write this post. 

So in order to understand it better and just like when you go to practice yoga or music or anything else you want to get really good at, I started paying attention to the way I was doing things. Particularly, behavior and thought processes I followed when practicing TDD from both of these perspectives.

Note: The rest of the post addresses a lot of abstract concepts, and I'm terrible at explaining myself. If you feel like this post would benefit from more examples, or needs some editing, let me know and I will add them or change it.

The Holistic approach.
What do I mean by "Holistic approach"? Holism is described as the idea that natural systems should be seen and studied as a whole and not as a collection of their parts. The underlying principle is that the whole is not just equal to the sum of its parts. 

This is mainly a top to bottom approach when dealing with solving the issue.

When we normally take this approach, what usually happens is that you tend to focus on solving the problem at hand directly instead of the sub problems. 

With this approach you think of things like: outputting all prime numbers in the sequence of the n first natural numbers starting at 2, instead of thinking first of solving how to finding out if a given number is prime or not, or outputting all numbers which satisfy a condition to the screen.

What I have seen is that since you tend to focus on the functionality and results of the principal issue instead of the sub problems, you get results that you can use faster. Most of the time they are partial or incomplete results, but the bottom line is you get more "deliverable" value from the start. Things like: I can create a user, but the process doesn't check all fields that need to be validated, etc.

However, as a side effect, what usually happens is that one tends to add "extras" to the code to help him test. I am talking about things like object properties to inspect inner state in your unit tests, having virtual methods that could be marked as final because you need to override them to test, etc.

Another common side effect that I have detected is, that the frequent mantra of "the emerging architecture", although still true, usually happens at the end, when you think you have a solid grip over your main problem and then you start refactoring your code. 

The inconvenience I see with this is, that if you don't refactor as aggressively and often as you should, this "architecture" may not emerge at all, or it may be deficient. I am not sure if this is good or not, but I personally like to have at least "some" architecture than no architecture, or an "emerging" architecture. Mainly because in both cases it could turn out to be a recipe for disaster if everyone on your team is not on the same page. Like when you have a junior developer on your team or not everyone  has the same skill level regarding TDD or refactorization.

One last thing is that as you are developing, you tend to prevent your target problem's corner cases, but not the sub problems' corner cases. This may lead to other bugs when the functionality gets refactored out or when it gets used by others, since they will tend to assume that it works as expected, not just in the context of the main issue, but in the context of the sub problem. 

To give an idea, following the prime numbers example, if another coder wanted to use your is_prime() function, it would be reasonable for him to expect false when it gets passed a 0 or 1, but if that wasn't part of the main problem's constraints, the developer may not have considered those exceptions, leading in turn to new bugs.

The Reductionist approach.
Reductionism on the other side is sometimes considered the opposite of Holism. In this case you would try to understand the whole by looking at its parts and their interactions. Inherently this is a bottom up approach when doing TDD.

You focus on sub-problems of the problem and when you have solved them, you focus on the interactions. Finally, you refine the interactions so that the results combined provide the solution you are looking for.

Contrary to holism, in this case, architecture emerges from the start, driven by the necessity to decouple particular sub problems. For me this approach makes it easy to loose focus on what your general aim is. It requires a great deal of self control and experience to know when is it enough and you should move on and continue with another area.

However, code gets tested more thoroughly. Corner cases for each sub problem are more evident and they tend to be addressed from the start. So the problems found when the clients of your API made assumptions about how the code worked diminish a great deal.

Nevertheless, focusing on the sub problems usually narrows your thinking. As a consequence you need to do more integration tests in order to validate sub parts and the way they are supposed to be used. You tend to write a more developer-friendly API because you are always thinking as the client of your API when you are writing the unit tests. 

This way of doing things seems to work better when you happen to have a roadmap or well-defined plan of how the pieces fit together. For instance, when you are implementing algorithms, which are usually decomposed in very specific sub problems.

Well, this is most of what I have observed myself and with a lot of input from the conversations I have had with friends. In the end, to me at least, no one method is applicable to every problem. In my day to day job I usually jump from one mode to the other depending on how it "feels". I'm very interested in hearing what others have to say about this subject so please leave a comment below. 

Until next time and happy coding!!

Tuesday, March 6, 2012

Give me some code52. No wait, make it code365.

Have you heard of code52?

No? Well, me neither, at least not until a couple of weeks ago, when browsing through the Github trends I found that these guys had a lot of activity going on and decided to have a look. I was amazed. Not by the code, which has great quality, but by the idea. And what a great idea it is. If you didn't click on the link, don't worry, I want to share with you what's so special about it.

In a nutshell, and as publicly expressed by the current organizers, these are some of the goals for the code52 project:
  1. Introduce developers to open-source projects and help them learn the ropes
  2. Contribute to projects across a range of technologies
  3. Let contributors have a say in what they do each week
But, how? Well,  Andrew Tobin, Brendan Forster and Paul Jenkins decided to try and create a brand new open source project, each week

If you work on software development, you know how difficult completing and actively working in one project can be. Open sourced or not, it demands a lot of energy and dedication. You can imagine then that attending to and starting one each week successfully, is a monumental task. Even more if you already have a day job. However, this guys are pulling it off and that's just astonishing.

The reason why I am so excited about this idea, is that it's a conscious and continuous well organized effort, to establish a mindset of contribution and collaboration among developers around the world. Let me explain what I mean.

While there are certainly lots of open source projects out there, most of them  are born in a totally random way. That meaning, usually, someone has and itch they need to scratch, and that's how they come to be. This, as I see it, has a couple of disadvantages. 

First, there is no systematic way of giving birth to projects, and while that in itself is not bad, it definitively doesn't help the growth and quality either. 

Second, the fact that they get created in a disconnected way, most of the time means redundancy in effort and resources. By that I mean that lots of projects with the same ambitious objectives come to life, but due to lack of synergy or momentum, they ultimately die or don't get the attention they need to become a great project. Instead they just turn out to be an average one. If this effort and resources where willingly and consciously put together, that would probably not be the case.

Third, there are lots of great ideas out there from people who are not developers or simply don't have the time to put it in. Those ideas will take longer to crystallize, or probably, will never see the day of light because there was no one there to listen or pick up the ball. This is one of the best things about code52, they listen to people.

Yes, there is the Apache Software Foundation, others like it that provide an ecosystem by allowing projects into their incubation facilities. But I wouldn't call them friendly in the same way as these guys do, but more like an enabler architecture for more mature projects. Yes, they provide you with tools etc, however, to join the Apache Labs it is required that the podlings have an established and working codebase. It is then, that through the incubation process they are expected to grow and improve their communities. 

This for me, addresses other areas of open source that are not directly related to development, which is more like what code52 tries to work on.

Fourth, we need more developers helping out to create and evolve open source software projects. Although there are initiatives like Google's Summer of Code and others, they are product, participants and community centered. What happens to the rest of the people who want to contribute the rest of the year?

Well, they don't know where to start. By they I mean, I wouldn't know where to start. I can see people having trouble finding a project they would be motivated to contribute to, because everything is so spread out and there is no way to easily sort and find that info. 

Even if they do, people like me, who have never actively worked on an open source project, wouldn't be too clear on how they are supposed to join the production process. Add to that the lack of meaningful and updated documentation from which a lot of open source projects suffer. You have a recipe for alienation and disengagement. 

Well... It is only reasonable to think then, that the easiest way to get over that, would be to start your own project, which suffers from the problems already mentioned.

Another thing that I like about the code52 initiative is that you can join their channel any time of the day and there will always be someone willing to put you up to speed, provide some guidelines, or useful insights. That, is just priceless. They have taken advantage of a platform of freely available communication a collaboration tools that support each other and make the development experience more fluent.  

Also, the projects are small enough and targeted that it would be easy for someone with no expert knowledge to join in an contribute. Like they say, just grab one of the posts in the Trello board, and start coding or pop into the channel and provide some insight.

Although, as much as I am in love with the idea,  there are some things that I can see people flagging as caveats. For instance, the technology most of the projects are being worked on (c# on the Microsoft stack). I think this is more of a by product of the background the main contributors have than anything else, and that the solution for this is simple, more contributors. 

They have expressed several times that they would be willing to try other languages and platforms if there is enough interest, support and contributors. So if you are a hard-core coffeescript, ruby, python or *insert language here* developer, give it a try. Join in.

Another warning is that most projects being worked on so far, have a very hands on intention. Mostly tools with particular purposes. So right now I don't envision lots molecules simulation frameworks being born there.

Above all, code52 is a call to action and training ground that turns what-ifs into reality. The door, to who knows which other wonders like this one. Who knows, maybe I will get to see a code365. 

Kudos to everyone behind it, supporting and helping out. I'm definitively looking forward to participate in this initiative! Are you?

Saturday, March 3, 2012

Promoting change...

I have heard a couple of friends, developers and  IT mostly, complaining about the situation they are in at their workplaces. Some say they have a boring job, others say they don't get paid enough, others are simply not motivated. My question to most of them is mostly the same: are you doing something about it?

Let's face it. We spend a lot of time sitting in front of a computer. In uncountable cases, that time is more than 40 hours a week. If you are going to spend that much time doing something your entire life, you better make sure you like doing it. If you don't, maybe what you are doing is not your thing. Maybe there is something else that would make you happier and that would be more gratifying to you. Something you can also get paid for... *insert joke about prostitution here*

But what if you do like what you are doing? What if your are good at what you are doing but, somehow, because of external factors you are still not enjoying it the way you think you should. Maybe you are even, starting to hate it... Well... that's why I am writing this post.

The truth of the matter is, most of the people I ask this, haven't even tried to solve the problem. They haven't even recognized their problem! They are just complaining, and they are not even complaining about it to the right person!

Whether you are the owner of a small software consulting firm, a project manager, or just another developer, you can always do something about it. There is no such thing as no options when it comes to promoting change in your work environment or organization.

This post is based on personal philosophies, experiences that I have had and some that I haven't, but I have received first hand. It is not intended to be a guide but more of an example on how it is possible to use what you have to improve your current situation and somewhat, a call to action. If you are willing to take the step, you will be surprised of how simple it is sometimes.

The post was written after the fact, so a lot of the things I am talking about here may sound very planned, when they weren't and happened organically. 

Also, it is focused on how we dealt with and approached introducing agile software development to the company I am working for. Some of the challenges, the solutions we found (and are still finding). I imagine the concepts are the same for any kind of change. So if you think: - "Ok it's just for agile" - I urge you to reconsider and take the time to see what you can get out of it that is not Agile Practices or Software Development related. 

Finally, it's an open invitation to anyone in the same situation to share their experiences in the subject. I would love to hear about them.

A little background
Some time ago others and I in the team were really burnt out about how bad our code base was. It was impossible to work with ! Fixing bugs became this (to misquote Dr. Sheldon Cooper) un-unravelable web  of changes all over the place, and as if that were not enough, there was no other way of testing any of the changes worked but to deploy the application and manually test it. We work with Eclipse and a really old version of JBoss, etc.

You see what I'm getting at... Let's just say I was really unhappy about how things were going on that front. Being the new guy on the block there, it was expected that I would find some resistance to making changes from some of the guys in the team, as in fact I did.

However, taking a chance, I started using TDD on my new projects. This saved a couple of hours on testing and bugs fixing. Then with those and the help of another team member, we set up a continuous integration server, a Maven repository, and some other tools for metrics and knowledge sharing. Afterwards, every project he and I started working on, diminished the amount of friction required for set up, working on them, deploying and testing.

Soon, others started asking how they could get to use it on their own projects. It became "mainstream" and eventually the rogue continuous integration server passed from being hosted on the slowest computer ever (the only one we could use...) to become a virtual machine on the company's servers.

The moral of the story being: if you want change, you have to start yourself. How are you going to ask others to change if you are not willing to do what it takes? Sometimes a spark is all you need to set a whole forest on fire. Check out this Derek Sivers TED video

Let's talk about change
Change is a strange animal. Almost like a cat. We pat it and care for it. From time to time, it comes over and wants to play with you. However, like a cat, it barely listens when you are calling him. So how come it is so hard to "change" things when you want to, instead of it just happening to you.  

What I have come to realize is that affecting the organization is a matter of knowing how to approach change. Whether it is getting the team to consider a new technology, tool or methodology, it boils down to dealing with change. Why? Because change creates fear. Change creates discomfort. Finally, most changes involve transforming ourselves and the way we see things, and that is just plain scary.

I would say there are mostly four main things you have to consider when you are trying to change something. I am not saying they are the only things you have to consider, but for what I have seen they are the ones that I personally value the most. I will explain them in detail later, but for now, those four things are:

First, the desire to change. Whether you, or the team, or the organization wants to change. Then the ability to change. Are you ready? Do you have what is needed? Followed by the permission to change. In case you need it of course. Then, last but not least, the resistance to change.

Each of these with their own unique traits. I think we could agree that you will find them wherever there is someone defying the status quo.

The Desire
Why would you like to change? If you think about it, your answer will be different than mine, or the person sitting right beside you. But somehow the bottom line behind it is: dissatisfaction, yours or others'. Why? That desire to change, comes from the pain, the discomfort this dissatisfaction causes. With that in mind, to generate change you first need to generate this discomfort with the status quo, or create awareness of that pain. 

Don't get me wrong, I am not talking about making other people's lives miserable or anything like that. In fact, if you want to change, chances are the situation is already there, whether it is caused by external factors, or internal. It may only affect you, or it may affect others too, but you want those problems to surface and be dealt with. Make them. 

Usually when extending agile through a company, the idea comes from the people feeling the pain. Those being us, developers. When that happens it is referred to as bottom-up change. This is where the idea and action starts with the people at the bottom of the structural pyramid.

On the other side, if the change comes from the top of the pyramid, it is called, yes you guessed right: top-to-bottom. I have never seen this type of change first hand myself, but I know a few guys who have. In this case, when the guys generating the change are not directly the ones who have to change, things can get a little complicated.

Top to bottom, bottom up, in any case, everyone involved needs to be on the same page and moving towards the same goal or it won't work. Creating commitment is hard. You may not know how to do it. However, one thing is for sure, whatever you do, be sure not to undermine it, or people's engagement in it.

More about some ways in which you can create awareness below.

The Ability
Now comes the next stage. Suppose you already have that desire to change. You want to move forward with using whatever flavour of  agile you want, or TDD or whatever. But the question is: are you ready?

Just wanting to do something is not enough. Yes, it's a big part of it. I personally think it's the most important part. But sadly, the successful undertaking of anything, rests on top of the ability to execute.

I always remember this great quote when I hear someone ask for something impossible. It's from a children's book.

Being ready to change, is not just about the mental state. You need to have the environment. Here are a couple of things that we have particularly been looking out for: skills, change of beliefs, and barriers.

In my experience, embracing agile comes from two things, a mind set, and an ability to focus on what's important. Ok, I can hear you saying: - but everything is important! - I like to rephrase it like this sometimes: ability to see the things that are not important, to make them not interfere with what is important. Sometimes that involves focusing on them first.

In our case, we didn't have that ecosystem that would allow us to start doing so. No continuous integration server, none or little experience or knowledge about agile, SCRUM, TDD. Lots of uncertainties about business, project and technical constraints, etc. We needed to address those, and more.  However, not all of them before we moved to applying, and not applying all of them at the same time.

You want developers to be writing code, not looking for libraries. You want your newcomers to get in line with the program and start contributing as soon as possible. You definitely don't want them wasting time on things that a machine could do and you want them to join the mindset and leave behind the baggage from years of working differently.

Improve your skills. Which skills? Well, talk to people! Ask them what problems do they have. How can you help? Why aren't things moving forward? I will talk in more detail about what you can do about skills below.

Beliefs are hard to change, but it's not impossible. Having external influence is a great help. You can talk to someone who has already gone through the same process to come and share their experiences with the team. You can also find and recognize success case studies inside the company so that people know what is expected of them. Initiative, inventiveness, resourcefulness, self-motivation are all qualities you want the team to have.

One last thought. Simplify the change process. Avoid bureaucratization or it will collapse on its own weight. 

The Permission & Support
Well, my question here was: to what extend do we need that? I definitely don't know about your particular case, but in our case, we are very result driven. That worked great for us. By that I mean, if you get work done, they won't bother in controlling every little detail, because at the end of the day they trust us to do the job. 

That's great! It is a trait of all of the agile flavors, trust people, focus on them. They are the ones that make the magic happen. But, we also have to make a living. That is a part that, as regularly paid developers (not freelancers), we usually forget. 

A lot of the time when I talk to developer friends about that, I find some disconnection between what they would like and reality. Reality being the company may want to change, they may even think it is a good idea and see the benefits, but at the end of the day it is a business, and they want to make money. You can't forget that. They need to keep the engine running and the way to make both things happen is to put your feet on the ground, and level with it.

Things we looked out for were:
Do they see the benefit? Personally, I think the best way is to show them and use their own language. It may sound harsh, but if they talk numbers, you should talk numbers. Estimate improvement. Get real data, code coverage, defects, estimated time for completion, money saved, money gained. 

Run trial projects and change within the confines of your space, they will help validate your assumptions. If you need it, ask for permission, although I personally prefer to give an apology than to ask for permission. But, be smart about it.

Again, find people who are in their same positions and went through the process, who are willing to exchange their experiences and insights with you so you can gain better understanding.

This and this are nice examples of where you can find that kind of people in Barcelona. 

How does the agile process fit into the commercialization process? Does it have an impact or are you going to keep selling the same way you have and work your magic on the background?

How much of an investment are they, and you, willing to make in the change? Will they pay for training? Will they give you time to work on what needs to be worked on but does not directly produce any income? On your side, will you for instance, be willing to go on your own budget to receive this training or spend a couple of hours after work?

What is it that they do or don't like? Ask, and communicate. 

Lots of other ideas come to mind. The list is huge. Just put yourself in their shoes and ask and answer the questions you would ask.

If you do need permission, there are ways in which you can get it. Negotiate it. You can't expect a big company to change from dawn to sunrise. Set your company's expectations and yours clearly, so that there will be no misunderstandings. Again, create awareness. Do your homework. 

Prepare a presentation for management and deliver it. Selling agile to management, can be hard, so prepare yourself. Give yourself the best chance. Get data. Find success stories. Find people in the community who are going through the same process or who already undertook the change and get them to come and talk to your guys and explain to them the benefits. Explain them yourself to your co-workers. 

Set goals and timings for the gradual implementation. It is easier to sell them a particular plan that you can refine and adjust to their concerns, than to sell the idea of change alone which leaves too many open ends.

The Resistance
There is always resistance to change. It is hard-coded in the laws of the universe. But the way we have found to deal with it, is understanding where it comes from. To do that, you have to listen. Chances are people already want to tell you. Be aware of the symptoms. Fear, discomfort, lack of communication. Listen to what people tell you.

This is where the communicative skills come in handy. Adapt to your interlocutor, put yourself in his/her shoes. Ask the question: what is it that you don't like about it or you are having problems with? And wait for an answer. Finally, think about what the person said and act accordingly, don't just rush to deliver a speech on how good agile or TDD is. If you don't put any thought and heart into it, it will just come out as fake.

Where do you start?
Start at the beginning! Where are you right now? What is it that is bothering you? What is it that you want to change about your current situation? This is simple most of the time. You already know what it is that is bothering you... Solidify your thoughts about it... Most of the time this is usually a symptom of an underlying problem. Use the 5 whys technique to find out what that underlying cause is. Have a clear vision for yourself and others to understand and follow. For instance: what does it mean to have self organized teams? What does it mean that the team or employees are participating?

Next in line is figuring out where is it that you want to go. What is it that you want to achieve? A more interesting job? Better work dynamics? Better salary? Career advancements? Find out what it is, or at least, what it is that you don't want. 

If there is something that I have learned over these past few years, it's that if you know where you want to go, you already have 5o% of the problem solved.

Finally, plan and execute the way to get there! This is tricky because you may at first not be able to see how to change the situation. You may think that some of the solutions are too hardcore for you to consider. They are not. Like I heard someone say not so long ago in a talk: think of it this way, you know how your work life has been the past three days, the past three months, maybe even the past three years... If you don't do anything about it, you already know how it's going to be in the next three days and the next three months... etc.

I am confident by now, you get my point.

Don't get paralyzed.
Now, a word of advice, don't get paralyzed! Yes, plan, but don't fall into the trap of paralysis by analysis. Break the inertia by doing something.

For instance, start playing around with new technologies at home or in your spare time. Maybe even incorporate them into a new project. The important thing here is for you to start doing something and don't stop once you have. Keep thinking about what your objectives are and increase your efforts on getting there.

Create awareness.
Create awareness sounds pretty simple, however, I sometimes hear people complaining they don't know what to do. Ok, here are some tips.

You can, for instance, send management articles about the competition to incentivize them to compete or compare with them.

Invite your colleagues or boss to webminars or seminars. In our case one of the things we did was inviting people to come over to a free seminar for a master in agile project management at laSalle. The presentation was so well delivered that the guys that went there were bursting with energy and enthusiasm about making the change!

Ask for feedback from customers and others in the team or the company. Talk about what works and what doesn't. Take a look at the current process and identify what is not working or could be improved.

Ask, what do you think will happen if you or your organization doesn't change?

Get others involved.
It's sometimes easier if you are undertaking a particular challenge with someone else. If you want to get someone else involved and you think that would help you, go ahead. It may be as simple as asking them! If you are feeling the pain, chances are someone else is walking in the same shoes. It is a great idea!

In our case I had the luck of having the support and help of great guys that were the fuel behind it all. They, more than anyone made everything possible. I'm really glad about that.

Find the barriers.
There will always be barriers and constraints. We need to know about them so that we can lift, and work around them. The base for this is communication. 

Almost the first thing we did after deciding that we wanted the change, was getting together and under suitable circumstances do some debugging of the processes we were following. What works, what doesn't. What were the options and the constraints we were aware of.  

We got together and made a list of all the problems that were affecting us, divided them in recognizable groups, and tried to find distinguishable roots for the problems, some technical, other managerial, personal, etc. so that we would better understand how to change them. 

Improve your skills.
Hone your skills, not just as a developer, but as a communicator. It will be useful.

Learn whatever it is that you need and when you know what you are doing, teach it to others. They say you remember only 15% of what you studied but 90% of what you taught.

Again, in our case, we organized a set of talks on the different subjects that we were interested in like: TDD, SOLID, Designing for test-ability, etc. 

We also sent links of free webminars and other free training sessions and resources like Coding Dojos, etc. to everyone interested.

We started helping  people to interact more with the agile community here in Barcelona, to promote the experiences and information exchange. Also, we used pair programming as a way of shortening the learning curve.

In general, we tried to create a friendly environment for learning. By that I mean, we got books, digital or not and made them visible and available to everyone who needed or wanted them. We also tried to allow time for learning, whether it is after hours (like in our case) or not.

Well, I have other stories from the trenches, but I think it is enough for now. Until another post and don't forget to start doing something about itAbove all things: have fun doing it! I know I am.