Iterative Development – An Introduction

February 5, 2010 · Filed under 1 · Tagged , , ,

Iterative development is a pretty basic concept.  You just develop iteratively…  Hmm pretty sure i read once that you shouldn’t use the word you’re defining in the definition.  Lets try again.

Iterative development is the practice of developing software in small defined chunks of time.   This is in contrast with the old-school method of starting a project and working on it over the course of months or years in one long chunk, or iteration.

Lets just dig in to the meat of why it’s important.  Imagine you wanted to get a house built.  You line up a contractor who has experience building houses.  You tell him you want a 1200sq ft house with lots of windows and a double garage.  He asks you what layout you want, where should the windows be, do you want a deck, what colors for the interior and exterior etc.  You answer him by saying “I don’t know, I hired you because you’re good at building houses, so you just make sure it’s 1200sq ft and has lots of windows”.  Can you see where this is going?

Six months later he calls you and says the house is done.  You walk in and while the house is impressive, it’s not how you had envisioned it, and with the amount you are paying him, you want it to look how you want.  Unfortunately now that all the work is already done, it will cost a LOT more to make changes.

Software is the same way, if somebody comes and says to you “I want a system that does X” but they don’t have any details as to how it should look or work, it would be foolhardy to believe that they are going to just be happy with whatever you deliver to them.

Iterative development helps with this by allowing you to break up the project in to small iterations, the smaller the better, ideally 1 or 2 weeks.  This way you can show what you have accomplished during each iteration to the customer and see what they think, and get their feedback on what they would like changed.  Even if they customer had no idea what they wanted ahead of time, you can bet that they will after they see what you have built.  The sooner you get that feedback from them, the cheaper it will be to ensure the project turns out how they want.

One of the other big advantages of dividing work up in to short iterations is that it’s a lot easier to estimate what you can get done in 2 weeks than in 2 months.  Once you have the project divided up in those iterations it also becomes a lot easier to see if you’re falling behind or not as well.

That should give you a basic idea of some of the advantage iterative development provides.  I’ll go in to more detail in future posts of some of the other advantages.

by: Chris Dagenais

Leave a comment »

Presenting at the Prairie Developer Conference June 2nd and 3rd

February 4, 2010 · Filed under 1 · Tagged , , ,

The Prairie Developer Conference is the first of it’s kind in Saskatchewan, a 2 day event covering a wide range of topics including things such as cloud technologies, agile development practices, database performance tuning, etc. 

I submitted three proposals for sessions to the conference and two of them were accepted!

  • Agile Development Practices, brief introduction and how they help your business
  • Iterative Development, How it works, Why it works, Who it helps.

I’m very excited to get the chance to present on these topics to a reasonably large group of people from around my own area, as well as raising the profile of Point2 in Regina.

By: Chris D

Leave a comment »

Point2 Speaks at CIPS Luncheon

January 22, 2010 · Filed under Point2 - Process and People · Tagged , ,

Last Wednesday Point2 had the opportunity to speak at a CIPS (Canadian Information Processing Society) Saskatchewan luncheon hosted at the Potash Corporation of Saskatchewan tower in Saskatoon. The topic that Joel Loewen, Melanie Cey, and I presented at their request was IT Recruitment Issues Unique to the Saskatoon Area.

Join Melanie, Joel and Hemant as they discuss their particular challenges and success factors. Hear how their shop differs from a typical IT support organization and what they are looking for when they hire IT professionals at Point2, and other HR challenges.

Once we arrived at PCS we had a chance to mingle with the thirty or so guests that were in attendance. It didn’t take long to realize we would be speaking to quite an eclectic bunch ranging from students to development managers to HR recruiters. After lunch was served it was time to get down to business.

Joel started it off with introductions and then handed it over to Melanie who gave a brief summary of Point2, discussing the evolution of the company over the last fourteen years. She gave an overview of the two major product lines in regards to heavy equipment and real estate, and the steady pace at which the company continues to grow.

I was next on the agenda and spoke about the unique development shop we have put together in Saskatoon. I briefly discussed how we used to develop software and why it had to change if we wanted to continue to be a progressive company. I got into the meat of my talk when I illustrated our move to Agile development practices including Scrum and XP, how we were starting to look at software development as a craft, our open work environment, and the culture of learning we have created by embracing professional development.

Joel followed up with a detailed look into the IT department. He spoke about the challenges we face as our business and data needs grow, and the complexities that we have to overcome by hosting our systems on-site. His explanation of our move to virtualization was eloquent and illustrated the level of expertise in our systems department.

The idea was to lay down the the unique technology environment that has emerged at Point2 before getting into the challenges it creates regarding hiring. This opened up about a forty-five minute question period around our recruitment process, our technology stack, and the Agile development practices that we have adopted.

All-in-all the presentation was well received and it was great to have a chance to highlight all of the great things that are happening at Point2. On behalf of Joel, Melanie and myself I would like to thank CIPS for inviting us to present at their luncheon. It was an honour to be involved in important discussions regarding the challenges that Saskatchewan based technology companies are facing.

By Hemant J. Naidu

Comments (2) »

Why NOT to avoid (or forget) “Walking Skeletons”

January 8, 2010 · Filed under Point2 - Technical · Tagged , , , , , , , , , ,

At Point2, we have recently embraced using the concept of a “Walking Skeleton” as the first development work we do when starting a new project/module/bundle. This approach allows us make sure all of the overhead of a new project is accounted for and functioning properly before our project becomes too complex to allow for. We create the walking skeleton as part of our first sprint which ensures we have some deliverable by the end of our first iteration. By the time our walking skeleton is complete we have worked all of the kinks out of our CI, deployment, and testing strategies which are very straight forward still at this point.

Unfortunately, when my team recently started its current project, we dropped the ball when it came to finishing the walking skeleton end-to-end before starting on more complex tasks. It didn’t occur to as at the time that this was a bad thing because we were still making visible progress. In retrospect though, we acknowledged the difficulties and extra tasks we had created for ourselves by neglecting to help the skeleton take its first steps.

One issue we encountered was not being able to easily demonstrate new functionality to the product owners for sign-off. Because we had no certain way of running data through our application end-to-end (not even “Hello World”), we ended up fudging steps in the process just to see the desired results. This didn’t allow the business to try out the features without first having knowledge of the internal mechanics of the product. It also made it difficult, if not impossible, to properly functionally test the system in a true black-box fashion.

Another speed bump we ran into was the extra refactoring we found ourselves doing because the interfaces between components in our system were still evolving. We had not pushed data through each moving part so this meant that the way the parts fit together had not been clearly considered and defined. As the pieces came together we realized different interfaces were more appropriate and with interface refactoring comes unit test refactoring. Now I normally encourage a healthy dose of refactoring to every piece of code but not when I just finished the code for the task earlier that morning. Had we actually pushed something all the way through the pipe, we might have realized earlier that our initial architecture was not appropriate and, in fact, didn’t even make sense.

So why did this happen in the first place? After recognizing what happened, my team agreed we should perform a root cause analysis. This activity produced the following Ishikawa diagram:

Walking Skeleton Fail

The causes we came up with were interesting but not surprising. We decided the main reason we forgot about finishing the walking skeleton was we were just too excited to get started on the new project, using new technology.

We came up with two action items to address the issues we came up with:

  1. Simply remember to plan for a walking skeleton next project.
  2. Blog about our experience to help others avoid the same problems. ;)

By: Jesse Webb

Leave a comment »

A Positive Root Cause Analysis

January 8, 2010 · Filed under Point2 - Process and People · Tagged , ,

At the end of every development iteration we always do a retrospective and a root cause analysis meeting. In our retrospective we cover what went well, what did not go so well, and come up with action items for things to try in our next sprint. In our root cause analysis meetings we usually pick some problem that occurred, try to figure out why it happened, then understand how we could prevent this problem from happening again in the future. Our root cause analysis meetings are usually always on a negative topic. However, during our recent project we decided to change things up and do a root cause analysis of the following question:

Why did our last project go so well?

This may seem like a strange question to ask, but often in root cause analysis meetings we tend to focus only on the mistakes made during a project. When problems occur you need to identify the root causes of those problems to prevent them from happening again. While in a project that goes well it is just as important to recognize what we did right this time to ensure we understand “why”. What did we do different this time that worked? What were the crucial decisions made that kept the project on track? In our root cause analysis we found some key decisions, that in retrospect were important but at the time did not seem critical.

Failing fast when a decision starts to become problematic
For our project we thought we had a clear, straight forward design to work with from the beginning. However, after spending even just a day spiking some ideas our design immediately started to show cracks. Our design that on the surface looked simple, turned out to be far more complicated to implement than we had imagined. A large part of the reason for this problem was that our project was to make changes to an existing application about which no one on our project had any previous knowledge. We immediately had a team huddle, called a “Just In Time” design meeting and corrected our course. As a result, we lost a day, instead of a week or a month going down the wrong path.

Consulting experts early
When we started our project we knew of a few ways to accomplish our task, we had received suggestions that sounded fine, but we really were not sure if our approach was the best solution available. Fortunately we have some very experienced people in our company that have spent many years contracting and as a result have an incredible variety of experiences from which to draw upon. So we called a quick design meeting with one of these experts, showed them what we were thinking of doing and just picked their brain for ideas. It turned out our expert was able to come up with an approach to our problem that not only would allow us to complete the task within the time-line given to us by our business team, but at the same time would allow us to implement a cleaner solution.

Keeping code ownership high
We had no one person on the team that if they were sick for a day it would prevent a task from being completed. During our project we made sure every line of code was written with a pair (We always try to pair program every line of code) and switch pairs regularly. Because of this knowledge sharing we did not have any “Experts” on any one area of the application. We always had at least 2-3 members of the team who were knowledgeable enough on any given area of the application be able to bring another developer up to speed.

Break all stories into small tasks with a clear definition of “Done”
The stories we work on during a sprint always show the business value we are adding, but from a developers perspective there are usually multiple tasks required to complete each story. At the start of each sprint we held a task breakdown meeting for breaking each story down into a set of small tasks. Our team found that having a set of clearly defined tasks for each story was very important to keeping the project on track. With any story we receive from the business team their will be questions and as a result we found that doing this task breakdown meeting helped flush out many of those questions at the start of the sprint, as opposed to after development had already began, which in the past was usually what happened. It made it clear to our team lead and business analyst exactly what work was being done, who was doing it, what tasks had been completed, and what tasks had not yet been started. Also this gave our business analyst and team lead a better idea of when to expect demos since they could see how many tasks were remaining before a story would be completed.

Demo to business team often
We started our project doing a fairly poor job of demoing but this was corrected after one of our sprint retrospectives. Business analysts need to see the work being done and often will think of something that was missed, or see something that perhaps spawns another story. One of the easiest ways to ensure that what is being developed matches what the business team wants is to keep them in the loop and an excellent way to do that is through frequent demos.

Quick feedback from business team
During development there are always going to be cases in business logic spotted by developers that were missed during the initial planning phase. When a developer spots a missed case and brings it up to the business team, quick feedback from the business team can play a major role in keeping the project on schedule. In our last project this turn around time was often hours, if not shorter in most cases.

Keep the systems team involved from the start
The people who will be deploying the application and hosting it should be involved right from the start of the project. Your systems team has experienced many deployments and also know the pain of hosting a problematic application. Allowing the systems team, who will be responsible for the application after development has been completed, to be involved in key decisions can greatly improve the chances of a successful deployment and potentially reduce the cost of hosting the application.

Conclusion
Having your team do a positive root cause analysis can be very useful. It sometimes seems like after a problem in one sprint, we focus in the next sprint so much on improving in that one area that we sometimes slip in areas where we were previously doing well (for our team it was demos). On previous projects I have worked on, we definitely tried to follow each of these best practices outlined in our positive root cause analysis. However, since we moved to Agile almost two years ago, this was the first project where everything just “clicked”.

by Brian Richardson

Leave a comment »

How to Avoid Auto Deployments Behaving Badly

December 18, 2009 · Filed under Point2 - Technical · Tagged , , , ,

A couple of posts relating to development/ops gaps, and access control reminded me how very far we have come with our environments over the past few years.

Environment management in general comes with its own set of complications, but what I am interested in knowing is how other Systems/Operations departments are handling auto deployments. Several of the points in the articles above are moot for us and have been for some time. Access to production systems is heavily limited to our Systems Administrators and many of our products are deployed with scripts triggered by team city builds. Code or configuration files are not deployed manually to our end to end test environment, and when they are, generally an auto-deployment script needs to be touched to alter the outcome rather than a human touching a file. Don’t get me wrong, there are still blunders that occur from one environment to another “oh I forgot that we needed to add that file so I didn’t add it to the production release… oops”. However, my main concern is what to do when an auto deployment script behaves badly.

First, let us discuss what auto deployment is to our teams. When code is ready to release to a test or production environment, an auto deployment script will be executed to place the code on the correct server(s). In the case where code is going to production, a Systems Administrator is assigned a ticket to take a few preparation steps, then run the script to deploy the code. Depending on the environment and product, one or all of the following things could happen during an auto-deployment run:

  • Disable/enable monitors
  • Disable/enable nodes in a cluster
  • Run database scripts
  • Svn checkout or switch a directory or file
  • Copy files from one location to another
  • Start/stop services
  • Start/stop web containers

The first time I ran an auto-deployment release, I felt naked. I was so used to knowing every piece of what was actually required for a product to become live that pressing a button felt completely foreign. I worried that something would be missed, and I just wouldn’t know. So what should you do in a case like this? How do you mitigate the risk of the system being compromised? You’re running a script that you know little about other than you just typed several passwords into a config file on a secure server and it’s using your permissions to execute. However, because you have done this manually before, you know what the behavior and outcome of the script should be, so verify it.

The first few times we were vigilant. Check to make sure the node is out, review all the database scripts prior to release, check log files during and after the release, etc. Over time though, we started to become more comfortable with the release runs, and this is when the problems started to arise.

Auto-deployment scripts in recent months have been known to produce any one of the following symptoms:

  • Didn’t run the database scripts, or ran the incorrect database scripts
  • Didn’t remove the node from the cluster or didn’t put it back in
  • Deployed the wrong the version
  • Deployed the wrong application
  • Failed because of a spelling mistake
  • Showed “all green” in our monitoring system, but still caused a serious underlying problem

All these problems are compound, they are the responsibility of both the development and systems teams to realize and mitigate. The real root of the solution is to treat the auto-deployment script as exactly what it is: code. What do you do with code? You test it, you unit test it, and you make sure that it throws exceptions and failures when it doesn’t do what it is supposed to do. Most of the above problems could easily be avoided if the script was intelligent enough to know when it has made a mistake.

So, this is my list of remedies for saving an operations team from feeling naked when learning to run auto deployment scripts.

  • Get involved, have the script demoed to you and understand what it does before you run it. Ensure that every time it changes, you receive a new demo.
  • Don’t be naive, yes, maybe one day this script will run without someone watching over it, but before that happens, sys admins and developers should work out the kinks.
  • Review database scripts with a developer before the script is executed then check to ensure that one of the intended operations occurred.
  • Any time a problem is encountered post-release that is found to be the fault of the script, have the script altered to create a FAILURE state if it ever happens again. Additionally, review the script for other possible failures and have those adjusted too.
  • Check log files, review the script output and review the logs of your application containers.
  • Ask developers to check log files too, you will not always catch an error.
  • Consider having the script pause at critical stages (i.e. between nodes, etc) so that things can be inspected before they go too far.
  • Improve your monitoring systems. Create monitoring situations for the errors that get caused by the releases.

One day, after you have executed a script 10 times and it has not made a mistake, you will trust it. The important thing to remember is that the scripts will continue to evolve, so stay involved and help test every change. After all, as Systems Administrators or Operations staff, it is your job to ensure that the system is operational and take responsibility or alert the appropriate team when it is not.

Melanie Cey
Systems Administration Team Lead

Comments (4) »

Is Tardiness Contagious?

December 11, 2009 · Filed under Point2 - Process and People · Tagged , ,

Meetings are a regular occurrence in all office environments, and it is no different for an agile software development team. With the majority of of our teams following Scrum, the Daily Scrum, estimation, and retrospectives are just some of those many meetings. Even though my particular team has become quite efficient at running these meetings, there remains one problem that is common amongst almost all of them – people are always showing up late to them. This very problem was the focus of our last root cause analysis meeting (and yes, people showed up late to that one as well :P ) as we hoped to identify reasons for meeting tardiness.


After drilling down into the problem, my team identified four categories that played a role in tardiness amongst members of our software development team.

Focus

  • Since we do 100% pair programming on our development team, a developer feels bad if they need to leave for a meeting that their pair is not in.
  • A pair is usually in some sort of flow while working on a problem and they often find it difficult to leave abruptly.

Complacency

  • It is generally okay for meetings to run long, so people start to think it is alright to show up late.
  • Meeting invitation notes (agendas, goals, etc.) are not read until a few minutes prior to the meeting start time, resulting in people being late.
  • It has just become the norm.  We have started accepting people being late as business as usual.

Scheduling

  • Coffee and bathroom detours are made on the route from desk to meeting location.
  • Meetings scheduled on the hour/half-hour are not disruptive enough to garner full attention.

Software

  • Since developers are pairing most of the time, they are rarely at their own desk to get the meeting reminders.
  • The snooze function and meeting reminders in Outlook and Entourage are not reliable.

It was no secret that this was a problem faced by all teams in our department (I can’t speak for other departments) so we came up with some action items that we could start to implement (on our particular team, at least).

  • Simply round people up prior to the meeting. This may only work if you are all located in the same area.
  • The organizer of the meeting should be the first person to show up, and welcome the attendees as they arrive. Many times people show up to a meeting room and no one is there, so they too wander off.
  • Identify the meetings for the day at the Daily Scrum.
  • Just start the meeting on time, regardless of whether everyone is there or not. People who show up mid-meeting will get the point.
  • A more extreme method which will not work all of the time is to simply cancel the meeting if everyone is not there on time.

It’s been a week since we’ve started tackling this problem and I can say that there has already been improvements made. Even though there is still a ways to go, we are confident that tardiness will soon become a thing of the past. The idea that we have become complacent to people showing up late just means that more and more people will follow this trend. If we can just turn that attitude around I think we can successfully make punctuality just as contagious as tardiness.

By Hemant J. Naidu

Comments (2) »

Movember

December 10, 2009 · Filed under Point2 - Process and People · Tagged , , , , ,

Ah what can I say about Movember, it’s definitely the hairiest time of the year around here at Point2. For those of you who have never heard of it, Movember is the renaming of November to signify the thousands of men all over the world who are growing mustaches for the fight against prostate cancer. Most people at Point2 learned about it last year when our now CTO asked us to join him in growing mustaches to raise money for this cause. Soon after that many of the normally clean shaven men of Point2 were proudly displaying hair where there was no hair before. Again this year we have decided to put the razor away and focus on growing the baddest mustaches our faces will allow.

The development team that I am currently on has four brave men willing to put forth their best mo for a good cause. Mustaches can be very powerful tools, helping to fight against prostate cancer, and even helping our team reach new heights of greatness. During our retrospective last week it was found that our mustaches had a large role in helping us achieve our velocity. We did so well that we decided to hold a Root Cause Analysis to find out how our teams Velocity was so much higher than the previous week. In that meeting our mustaches were cited as an extra member of the team helping complete many tasks, they also helped to bring morale higher than usual. I believe that I was not the only one who came to the conclusion that it was in fact the ‘Mo’ that not only helped us achieve our velocity goal but also to surpass it.

However growing a sweet mustache is not all glory, for most people it takes time filled with ups and downs. Some people may look you at in disgust however you may find that the longer your mustache grows the more attractive you become. Most men usually go through the 5 stages of  ‘Mo’.

The 5 Stages of ‘Mo’:

1.  Denial – Thinking this year I’m not going to look like a porn star from the 70’s
2.  Anger – Grow faster and thicker!!
3.  Bargaining – I’ll grow twice the mustache next year if you just let me shave this thing off
4.  Depression – Oh, my wife won’t kiss me, and everyone looks at me like I’m insane
5.  Acceptance – Well we’ve come this far it’s finally looking like something, and I think that hot blond just gave me the eyes.

Now that the month is over and the mustaches have moved on, things feel different, almost less masculine. However Our Point2 team did raise almost $3000 this year for Prostate Cancer, and I know that everyone who participated feels stronger from the experience. We now know that growing a good friend only takes a month. Well mustaches and Movember go together like bacon and eggs, but all good things must come to an end.

Click here to donate to our team

By: Dean Gaudet

Leave a comment »

Test Driven Development with a Widescreen Monitor

December 9, 2009 · Filed under Point2 - Technical · Tagged , , , , , ,

My Point2 workstation recently received a monitor upgrade; the old pair had had served me faithfully for many years, but they were worn out. So I am now the proud user of a pair of shiny new widescreen monitors.

The new monitors are much nicer than the previous monitors, and they are also a much higher resolution (1920 x 1080 vs 960 x 1280). Using them is a very enjoyable experience, but at first it seemed like the widescreen aspect ratio was wasting a lot of horizontal space:

Before: my widescreen monitor with a single file open

Before: my widescreen monitor with a single file open

Then a few days ago, in the middle of a pairing session, I discovered IntelliJ’s split screen feature. In my opinion, the combination of a wide screen monitor and split screen is a killer feature for TDD. If you assign your test code in one pane, and your production code to another pane, you can see your test and production code at the same time:

After: my widescreen with two files open simultaneously

To activate split screen choose “Split Vertically” from the Window menu, or right click on a document tab and choose “Split Vertically”.

IntelliJ works very nicely with this split screen configuration; intentions, “Go To Declaration”, and refactoring shortcuts jump nicely to the appropriate screen. Being able to read test and production code at the same time without clicking a button is especially valuable when pairing, as each half of the pair can be reading a different file. One important caveat: this only works correctly if each file is only open in one pane, and IntelliJ doesn’t enforce that for you.

Some of you might wonder what the second monitor is used for, and the answer is pair programming. We pair program all the time at Point2, so I run the two monitors as mirrors. I’ve also added an extra keyboard and mouse, and the combination allows each member of the pair to see exactly what’s going without craning their necks or stretching. It’s a really nice feature for a pairing station.

My "Pairing Station" - a pair friendly workstation

My pair-friendly workstation. My keyboard and trackball are on the left, and the developer pairing with me has their own keyboard, mouse, and monitor on the right. Either of us can drive without any inconvenience.

By: Sean Reilly

Leave a comment »

Let’s Learn from Each Other: BarCamp Saskatoon 2009

December 4, 2009 · Filed under Point2 - Process and People · Tagged , ,

Last weekend members of Saskatoon’s technical industry congregated at Louis’ Pub on the University of Saskatchewan campus for the annual BarCamp, a technical conference “for the rest of us.” For those of you who are not familiar with BarCamp, here is a little history as explained on the website.

The idea of BarCamp came from Foo Camp where a bunch of the best techie minds in the world got together to talk about new technologies, and to create opportunities for cross-fertilization between people and technologies.

Since only the top minds from O’Reilly, Google, Yahoo, Intel, Apple, etc. were invited to attend, two people in California decided to make a Camp for the rest of us. Now, there are BarCamps somewhere in the world almost every day.

This was the first year that I attended BarCamp so I wasn’t quite sure what to expect. I did understand that it had a laid back atmosphere which is quite apparent since it is held in a bar, but I also knew that the caliber of the talks would be high. The idea behind BarCamp is that everyone who attends has to participate in some way ranging anywhere from helping set up to leading a presentation.

As to be expected there was a high number of Point2ers in attendance, and impressively four of the nine presenters were from the office. Dave Kellow gave a spirited talk on Agile Modeling explaining why it is important, and discussed ideal ways of approaching the subject. Nate Heagy gave a brief, but extremely informative and entertaining presentation on Why You Should Get Good at Javascript. He used this as an opportunity to showcase an iPhone app he wrote using this technology as leverage. Chris Powell teamed up with his buddy Mark Poppen to give an information session on Starting a Web-based Business. Sean Reilly then wowed the crowd with his extensive knowledge of effective Functional Testing techniques, and why they are so important.

The time between all of the great presentations provided an opportunity for the attendees to mingle and find out what each other was up to. Being in a relatively small market, Saskatoon has a limited number of events like this so it is rare for members of the industry to get together and just hang out. Even though many different companies were being represented the atmosphere was of collaboration,  not competition. Everyone was there to learn from each other in hopes of bettering themselves professionally.  It’s events like these that will have a positive effect on Saskatoon’s technical community moving forward.  On behalf of Point2 I would like to thank everyone who made this event possible.

So you may be asking how I contributed at this year’s BarCamp. Well I didn’t help set up or lead any presentations, but I did shoot a lot of photographs to document the event. You can view the shots from the day here. The local media was also there to cover the conference and you can read the resulting article here.

By Hemant J. Naidu

Leave a comment »

« Previous entries
Next Page »