5.10.2013

Successful software development is about creating value, not software

In my last article “Why objectives and value are important in software development” I gave examples to show that it's important to diagnose before prescribing technical solutions. But what does this entail?

Ask why

Diagnosis can be as simple as asking "why". When someone presents what, ask why. Sometimes the response is more of a what, in which case you may have to dig deeper, keep asking why! This can be like peeling the layers of an onion. At some point you find the center, the core objective.

Successful business objectives create value

Businesses exist to create value. Objectives are guiding principles to create value. For example, most businesses value profit. One objective to create profit may be to expand the customer base. Value is often intangible, for example employee morale. An objective to increase employee morale may be to pursue equitable workload distribution. Objectives are a means to the value (end). Value and objectives are numerous, focusing on them will hone the skills necessary to identify and create value, another reason to ask why.

Assess value

Eventually after peeling back enough layers, you will arrive at a rather fundamental objective.

Let's say someone asks, "Can we generate a PDF of X every night and email it to Y?" Follow up with "Absolutely, but first can we talk about what this will accomplish?" If other people are affected by or invested in the request, involve them too. Make sure to involve the person that will write the check. A typical response might be "Right now John creates this by hand and it consumes a significant amount of time." So we've identified a possible objective, saving time. Again, objectives are a means to an end, in this case it's probably about money.

Objectives often create more than one type of value. Now that you have an understanding of the objective, use it to ask further questions to assess related value. Go grab John and include him too! These are some things you may consider:

  • "How often are there mistakes because it's done by hand?"
  • ask John "Does this drain on your morale?"
  • "How much time will be saved? Over what duration?"
  • ask John "What could you accomplish if you didn't have this burden?"

Involve everyone in this exploration and expand upon what value is desired, quantifying and qualifying it as you go.

Identify investments

With a shared understanding of value, everyone can brainstorm investments in software or otherwise. These options can be analyzed to quantify the level of value they may create and the potential cost. Neither the value nor the cost can be quantified exactly in advance, that's the nature of making investments and taking risk. However, an estimate of a solution without determining if it will create value puts everyone in a much riskier situation. Assessing the value reduces risk.

With enough practice, this approach will hone your ability to make successful investments. Who wouldn't want that?

Also, I'd recommend throwing out any notion of the original request and working from the value to identify solutions. Sadly, the original request is going to bias your thinking, just try not to let it narrow your vision. There are many ways to accomplish an objective and to create value. Keep an open mind to maximize the value.

Further reducing risk with measures

With clearly identified value, the team can find ways to measure how much value is created. In the example above, find a way to measure how much time is saved, count historical mistakes, ask John how we can measure the time or if he already measures it. Measure historically if possible, starting now and going forward. Monitor how things improve (or not). Keep these metrics simple and estimate what it will cost to measure them. Make sure it's worth the investment to measure. A few good measures can help the team improve future value analysis: maybe the value was less than expected, maybe it was more, maybe there was unidentified value.

Maximizing investments

If you don't do this analysis, how can you compare one business investment to another? Some investments are more lucrative than others. Who wouldn't want to tackle the most lucrative of investments first? If you have several pending investments and you've done a value analysis of all of them, you can easily prioritize based on the value!

Since we have a more accurate measure of value and cost, we can prioritize based on ROI. Think about this: if the people who will make the investment have determined that the value they think is possible is low relative to the cost or potentially a negative ROI, why would you make that investment? Likewise, if they have identified a high ROI investment, why wouldn't that be at the top of the list of priorities?

A better approach: objectives first

Though asking why can help you start this process, it's best to start with objectives first. Goals and problems are plentiful, encourage people to track them and bring them to the team as objectives and not as requests. This will eliminate the bias in a particular solution and will eliminate the need to backtrack to the objective.

Summary

When doing business, we should assess objectives first, thoroughly ask why, and determine what value we hope to create. Identifying technical solutions should come after, along with an assessment of their value and cost. Successful software development is about creating value, not software.

5.8.2013

Why objectives and value are important in software development

I need...

Developers get these requests all the time:

  • I need a table of X that I can sort any which way and export to Excel, can you build that?
  • or I need to generate a pdf of Y, can you build that?
  • or I want a system to reconcile Z, can you build that?
  • or I want a query to pull X, can you build that?

All of these request have a common form, a want/need for a specific technical solution.

Absolutely we can do that

Developers will typically reply absolutely, I'll get started right away! The developer will expend considerable time to fulfill the request. Upon completion, or perhaps earlier if a developer practices iterative releases, the customer realizes what they asked for won't meet their objective. It won't solve their problem or achieve their goal.

Scope creep

Naturally, the customer makes subsequent requests to tweak or replace the original request. Eventually they may arrive at a solution that meets their objective. They may also run out of time and/or money. If they are paying by the hour, the cost keeps rising. If they have a fixed bid, the developer is suffering and/or disputing the changes. Either way someone isn't happy, and that's no fun!

Self prescribing

The above scenario would be akin to being sick and choosing your own medication before going to see your doctor. Then, going to the doctor and saying "I need an antibiotic, can you give me that?" The doctor would administer the drug. Several days later you're still sick, so you go back and ask for a different medication. This repeats until you get better, but maybe you end up dead!

Or, say your hot water stops working in your bathtub. You call a plumber and inquire "I need my water heater replaced, can you do that?" They replace it and that night you still have no hot water. You call them out again and have them replace the faucet in the bathtub. That night, still no hot water!

Maybe these things happen at times, but it's not the norm.

Diagnosis first

Patients rarely will ask a doctor for a particular prescription to their ailment, and even if they do, the doctor will step back and discuss symptoms first. Same thing with a good plumber, they'll ask you why you need your water heater replaced. Either way, the professional is diagnosing the problem before prescribing a solution.

If diagnosis comes first in software development the developer can bring a variety of viable solutions to the table. Discussing objectives first can save a lot of time and money.

Why does this happen?

There are plenty of reasons, chief of which is we all like to be problem solvers. Most of us will try to diagnose our medical and plumbing problems too, we just defer to the doctor/plumber to make the final call.

I think developers encourage this too, we're avid problem solvers and we love a new challenge. Instead of listening first, we prescribe.

Sometimes solutions are repetitive, so naturally customers see patterns and offer suggestions.

Sometimes in the middle of a development effort, a suggested feature seems simple, and it may be, so why not add it?

It also happens when customers are charged by the hour. Instead of paying the developer to diagnose the problem, they can do it themselves and "save money."

Objectives first

No matter what the cause, it's clearly beneficial for developers to practice listening to the customer's objectives first. Developers should thoroughly understand the objectives before prescribing solutions. Customers need to participate by describing their objectives until everyone understands. Once the diagnosis is complete, then everyone can feel free to suggest ideas, but I would recommend deferring to the developer, they are supposed to be the expert after all.

5.3.2013

Monitoring Tests - When This Runs Over

Martin Fowler made a reference to Monitoring Tests in his review of Gap Inc.'s SCMS PO System - testing. I really like the idea of formalizing monitoring as a type of testing. Monitoring involves an assertion like any other test. Why not automate the validation of that assertion as the system is running?

Premature optimization is wasteful

I prefer to avoid premature optimization and fussing over performance of systems. In my experience systems run magnitudes faster than necessary 99% of the time. Sometimes I have a section of code that I don't expect to take more than a specified amount of time and I use that assumption to design simplified, more reliable code. If that assumption is violated, it would be nice to know about it.

Reacting to disaster

I often find there is a threshold above my assumption that can be crossed and still not cause any problems. At that point though, it might be a good idea to start talking about optimization. In the worst case these assumptions are absent from code, in the best case they may be in a comment:

// Processing settlements shouldn't take more than 10 minutes, it could be an issue if it gets close to 30 minutes because we try to email statements an hour after processing starts.
ProcessSettlements();

Even in this case, going over an hour probably isn't a big deal. There's probably defensive coding in the statement module to abort or defer sending statements if settlements aren't complete. But who wants to wait an hour to find out there might be a problem?

Being Proactive - When This Runs Over

Why not step it up a notch and turn the comment into an executable assertion?

using (WhenThis.RunsOver(Settings.Default.SettlementsWarningDuration).Then(SendSettlementsWarning))
{
    ProcessSettlements();
}

private void SendSettlementsWarning(TimeMonitor monitor)
{
    // put whatever code to send the warning here, this just shows how you can use the constraints of the monitor to format the message
    var message = new
        {
            Subject = "Settlements Are Taking Longer Than Expected",
            Body = "Process started at " + monitor.Started + " and is expected to complete in " + monitor.NotifyAfter + " but it's " + DateTime.Now " and they are not yet complete.",
            //Recipients
        };
    // message.Send();
}

The duration is configurable so I can adjust it to avoid being pestered, but I can find out well before an hour if I have a problem!

Here's the full example with some tests.

Monitoring Tests defined

I would define Monitoring Tests as any test that validates a system as it runs and provides automated feedback to those that maintain the system.

Using AAA syntax, the components of these tests are:

  • Arrange: this already exists, it's the system itself!
    • ie: inputs to the ProcessSettlements method
  • Act: this is the scope we want to validate, which also already exists!
    • ie: the ProcessSettlements method
  • Assert: this is the only missing piece, it requires identifying missing assumptions and/or assumptions in comments and making them executable
    • ie: the consumer of the ProcessSettements method has constraints for how long it should take because it has to run statements later

Summary

There is immense value in making assumptions in a system executable and providing automated feedback. Time tracking, exception management, logging and performance counters are ways we already use this. Monitoring Tests are about having your system tell you about problems as soon as possible, preferably before they occur! Dip your toe in the water, find a framework to help, use a service online.

The value of formalizing monitoring as a type of test is that we begin to think explicitly in terms of the AAA syntax and methodically engineering what of this we automate and what we don't. Like other tests, we can evolve these to meet ever changing monitoring needs.

4.23.2013

The Value of Using ReSharper with Visual Studio

ReSharper is an absolutely fantastic complement to the Visual Studio development experience that provides immense value. It's a requisite in my day to day work with Visual Studio. In many ways ReSharper features form the basis for the criteria I use when evaluating a development environment for any platform.

ReSharper helps me reduce bugs.

Many bugs can be avoided with good automated testing up front. Testing also helps prevent recurrence of bugs. One barrier to testing is the wide variety of tools and the lack of integration with development environments. This barrier decreases the dividends testing can provide and justifiably in some cases costs more than it is worth, or at least mentally seems to.

ReSharper lowers this barrier by providing integration of a consistent test runner. Out of the box it supports NUnit (one of the oldest .net testing packages) and MSTest from Microsoft. Plugins can provide coverage for other frameworks. Recently ReSharper even added support for testing JavaScript via QUnit and Jasmine frameworks! The testing tools are one of my absolute favorite features. I can run a test or set of tests with a keystroke and get results in an integrated, visually appealing test runner. Running tests with this level of granularity and integration is a very rare feature in testing platforms. It allows me to thoroughly test components before assembling and testing the entire product.

If I have a bug in existing code, features like Stack Trace Explorer make it easy to track down and identify the issue.

Lowering the barrier to testing makes automated testing more economical versus manual verification. At the end of the day I can work faster and the resulting product is more reliable. In my opinion, this testing integration alone covers the cost of the license. I save at least 2 hours a week using automated testing with ReSharper over F5 debugging. It also leads to at least one less production bug a month and gives me immense confidence in what I do allowing me to work even faster.

ReSharper helps me maintain code and avoid "re-writes"

Poorly maintained code is difficult to change, difficult to understand and in some cases difficult to read! This is often referred to as spaghetti code. These design problems lead to stagnant progress evolving a software product to the ever changing demands of business. Writing, understanding, maintaining and evolving software is hard work and is often tedious.

ReSharper reduces repetitive and tedious tasks evolving code by providing a plethora of automated refactorings. It's challenging enough to decide WHAT refactoring to apply, let alone to carry it out. Most beneficial refactorings can be automated and ReSharper does just that with integration and shortcut keys! Having these tools available at a keystroke means I don't wait to change code that is troublesome. ReSharper allows me to avoid the need for a "re-write" by continuously evolving the code to meet the needs of business. Paired with easy automated testing and I have confidence that my refactorings aren't introducing bugs!

Maintainable code means I don't have to worry about re-writes, ReSharper can even help tackle the worst of spaghetti code and evolve it to something more maintainable over time. The costs and risks in a "re-write" are phenomenal. Day to day refactorings save me at least an hour every week manually refactoring my code and it expands my ability to do so. Continuous refactoring also reduces the likelihood of a delay from code that is difficult to change.

ReSharper helps keep my code consistently formatted

Style preferences aside, most developers agree consistency is a good idea. Each project often has a particular style that is an artifact of the time it was created in. Often there's no reason to change the style, but it can be extremely difficult to enforce it when I adopt new styles.

ReSharper allows me to setup shared settings for multiple machines or when I work on teams. They are applied hierarchically so I can have whatever level of consistency I desire. These settings allow decisions in the past to be enforced via simple Code Cleanup commands and don't require me to spend time worrying if I am following consistent rules for any particular project. Instead, I can focus on the content and get consistency for free!

I save at least 30 minutes each day, I know this because I use the silent cleanup ctrl+e,f immediately followed by ctrl+s and find myself doing it in applications that don't support it :) If I stumble on code that is inconsistent, I setup rules and apply cleanup as needed to get a better idea of what it does.

ReSharper helps me comprehend code faster

As systems grow, so does the code behind them. This can become difficult to organize and even more important to navigate. Folders and projects only go so far and the Solution Explorer is a very crude way to get where you need to go, it's not very keyboard friendly! I spend a significant amount of my time reading code, so having a barrier to navigation is extremely unproductive.

ReSharper provides a bevy of features to ease navigation. A single Navigate To shortcut with contextual support can get me where I am going quickly. The Go to Type, Go to File and other Go to features make searching and jumping to any place in code lightning fast. ReSharper even supports integrated decompilation and navigation with dotPeek. Paired with refactoring and testing, superb navigation further allows me to evolve and comprehend code.

ReSharper also makes identifying unused code with solution analysis a simple task and helps remove it with Safe Delete. Unused code just leads to confusion and begets more unused code!

Every day I navigate to different places in my code 100s of times, ReSharper provides features that shave multiple seconds off each navigation, the amount of time is incomprehensible, at least 30 minutes every day, and that's only estimating about 4 seconds saved per navigation, it takes longer than that to do anything with the solution explorer and a mouse.

ReSharper helps me type less

Even the best of languages can be verbose, C# is not immune and wow is all I can say about Visual Basic. Why should I spend time typing syntax when I could be focused on content?

ReSharper provides many completions to make writing code less burdensome. It also supports several levels of templating and generation to reuse common patterns. These templates are customizable and can be shared with other team members with ease. One of my favorite templates is a test method snippet. When I want to add a test I don't have to type the scaffolding, I can focus on the content. I write more fine grained tests because of this which increases maintainability of my tests! I also thoroughly enjoy code generation of method declarations so I can focus on what the method does!

Less typing saves me even more time every day, at least an hour a day versus manually typing everything out, the biggest savings is in avoiding compilation and other mistakes that shift my focus from content.

Summary

Of the above, which are by no means comprehensive of the value I get out of ReSharper, I've enumerated a savings of at least 13 hours per week. That's time I re-invest in content instead of tedium. This is likely a conservative estimate as I feel my productivity is more than doubled with ReSharper.

If you are a developer, what would these savings mean to you? If you fund development teams in your company, what would these savings mean for your company?

4.19.2013

Chicago Node.js - Control Flow Techniques

I'm excited to share the recording of this talk from the 4/16 Chicago Node.js meetup.

Abstract

Verbose asynchronous programming in javascript can be frustrating. This talk focuses on alternatives to the pyramid of doom. We'll look at await/defer CPS transforms with IcedCoffeeScript/TameJS and streamline, control flow libraries, backcalls and promises. We'll cover some evaluation criteria for the solutions to this problem, including how the new C# 5 await/async syntax fares. Hopefully this talk can help keep a few more hairs on your head as you explore the wonderful world of asynchronous javascript programming!

Video

Code

The video has guiding slides that match up to the code samples below:

pyramid of doom - javascript

pyramid of doom - coffeescript

async package - coffeescript

promises with Q package - coffeescript

iced-coffee-script

TameJS

streamline - coffee

streamline - javascript

backcalls - coffeescript (proposed)