Posts in  vcs

4.12.2012

Commit Review Questions

Note: in this article when I refer to a commit, I mean the commit you plan to share with the rest of the team, if you have local commits that you plan to amend/combine, I am referring to the final result.

In time you will find these easier to do as you develop, however, all of these are valuable before checking in! The pre commit review is a nice time to polish what might have been several hours of intense work, during which these things were the last things on your mind! If you are concerned about losing your work in the process of responding to these questions, first do a check-in and amend it as you go (assuming you are using a tool such as git that supports this), rolling the result into one nice commit for everyone else.

Did you review your commit, change by change, with a diff utility?

  • If not, this is a list of reasons why you might want to start!

Did you test your changes?

  • If the test is valuable to be automated, is it?
  • If it’s a manual testing scenario, did you at least try the basics manually?

Are the additions/changes formatted consistently with the rest of the project?

  • Lots of automated tools can help here, don’t try to manually format the code, that’s a waste of time and as a human you will fail repeatedly.
  • Are these consistent: tabs versus spaces, indentation, spacing, braces, line breaks, etc
  • Resharper is a great example of a tool that can automate this for you (.net)

Are naming conventions respected?

  • Did you accidently use abbreviations, unless you have a good reason to use them?
  • Does capitalization match the conventions in the project/language?

Are files partitioned?

  • Sometimes we add new code in existing files in a pinch, it’s a good idea to split these out if they don’t belong
  • ie: are new classes defined in new files, if this is something your project values?

Is there commented out code?

  • If you are removing an existing feature, get rid of it, that is why we have VCS
  • If it’s not done yet, then why are you checking it in?
    • Perhaps a stash commit (git)?

Did you leave debug or unnecessary changes?

Do you understand all of the changes?

Are there spelling mistakes?

  • Including your commit message!

Is your commit message concise?

Is there follow up work?

  • Are there tasks you didn’t write down that you need to follow up with?
  • Are readability or reorganization changes needed?
  • This might be amended into the final commit, or it might be future work that needs added to the backlog.

Are there other things your team values that you should review?

vcs 
2.6.2011

Things I've noticed with DVCS

Things I encourage:


Frequent local commits

This way you don't have to be bothered by changes others are making to the central repository while working on a handful of related tasks. It's a good idea to try to work on one task at a time and commit all changes at partitioned stopping points. A local commit doesn't have to build, just FYI, so a stopping point doesn't mean a build point nor a point that you can push centrally. There should be several of these in any given day. 2 hours is a good indicator that you might not be leveraging the power of frequent local commits. Once you have verified a set of changes works, save them away, otherwise run the risk of introducing bugs into it when working on the next task.

The notion of a task

By task I mean a related set of changes that can be completed in a few hours or less. In the same token don’t make your tasks so small that critically related changes aren’t grouped together. Use your intuition and the rest of these principles and I think you will find what is comfortable for you.

Partial commits

Sometimes one task explodes or unknowingly encompasses other tasks, at this point, try to get to a stopping point on part of the work you are doing and commit it so you can get that out of the way to focus on the remainder. This will often entail committing part of the work and continuing on the rest. Leverage Git's index (if you are using Git) to group and commit related changes while leaving the remaining work uncommitted.

Outstanding changes as a guide

If you don't commit often it might mean you are not leveraging your version control history to help guide your work. It's a great way to see what has changed and might be causing problems. The longer you wait, the more that has changed and the harder it is to test/debug what your changes are doing! This is a reason why I am so picky about my VCS tools on the client side and why I talk a lot about the quality of a diff tool and the ability to integrate that with a simple view of everything that has changed. This is why I love using TortoiseHg and SmartGit: they show changed files, a diff (or two way diff with SmartGit) of the current selected file and a commit message all in one window that I keep maximized on one monitor at all times.

Throw away / stash commits

There is extreme value in being able to throw away a commit (or stash it) that is getting out of hand. If you do not commit often you will have to isolate the work you want to commit from the work you want to throw away, which is time consuming, not fun and error prone. I find myself throwing away commits about once a week, especially when doing exploratory re-factoring. It's much easier if I can just revert all outstanding changes.

Sync with the central repository daily

The rest of us depend on your changes. Don't let them sit on your computer longer than they have to. Waiting increases the chances of merge conflict which just decreases productivity. It also prohibits us from doing deploys when people say they are done but have not merged centrally. This should be done daily! Find a way to partition the work you are doing so that you can sync at least once daily.


Things I discourage:

Committing single files

Committing single files might indicate one waited too long and no longer understands all the changes involved. It may mean there were overlapping changes in single files that cannot be isolated. In either case, the suggestions above should help avoid this.

Committing frequently does not mean committing frequently right at the end of a day's work :)

It should be spaced out over the course of the day and several tasks.

4.28.2010

How I do VCS

After years of dabbling with different version control systems and techniques, I wanted to share some of what I like and dislike in a few blog posts. To start this out, I want to talk about how I use VCS in a team environment. These come in a series of tips or best practices that I try to follow.

Note: This list is subject to change in the future.

Note: I edited this to make it more friendly, I was too opinionated when I first wrote this.

  1. Always use some form of version control for all aspects of software development.
    1. Development is an evolution. Looking back at where we were is an invaluable asset in that process. This includes data schemas and documentation.
    2. Reverting / reapplying changes is absolutely critical for efficient development.
    3. The tools I use:
      1. Code: Hg (preferred), SVN
      2. Database: TSqlMigrations
      3. Documents: Sometimes in code repository, also SharePoint with versioning
  2. Always tag a commit (changeset) with comments
    1. This is a quick way to describe to someone else (or your future self) what the changeset entails.
    2. Be brief but courteous.
    3. One or two sentences about the task, not the actual changes.
    4. Use precommit hooks or setup the central repository to reject changes without comments.
  3. Link changesets to documentation
    1. If your project management system integrates with version control, or has a way to externally reference stories, tasks etc then leave a reference in the commit. This helps locate more information about the commit and/or related changesets.
    2. It’s best to have a precommit hook or system that requires this information, otherwise it’s easy to forget.
  4. Ability to work offline is required, including commits and history
    1. Yes this requires a DVCS locally but doesn’t require the central repository to be a DVCS. I prefer to use either Git or Hg but if it isn’t possible to migrate the central repository, it’s still possible for a developer to push / pull changes to that repository from a local Hg or Git repository.
  5. Never lock resources (files) in a central repository!
    1. We have great merge tools now, merging sucked a long time ago, it doesn’t anymore!
  6. Always review everything in your commit.

    1. Avoid committing without reviewing the changes in each file.
    2. If you leave to make changes during a review, start the review over when you come back. Never assume you didn’t touch a file, double check.
      1. This is another reason why you want to avoid large, infrequent commits.
    3. Requirements for tools
      1. Quickly show pending changes for the entire repository.
      2. Default action for a resource with pending changes is a diff.
      3. Pluggable diff & merge tool
      4. Produce a unified diff or a diff of all changes. This is helpful to bulk review changes instead of opening each file.
    4. The central repository is not your own personal dump yard.
    5. If you turn on Visual Studio’s commit on closing studio option, I will be very sad :(.
  7. Commit (integrate) to the central repository / branch frequently
    1. I try to do this before leaving each day, especially without a DVCS. One never knows when they might need to work from remote the following day.
  8. Never commit commented out code
    1. If it isn’t needed anymore, delete it!
    2. If you aren’t sure if it might be useful in the future, delete it!

      This is why we have history.

    3. If you don’t know why it’s commented out, figure it out and then either uncomment it or delete it.
  9. Don’t commit build artifacts, user preferences and temporary files.
    1. Build artifacts do not belong in VCS, everything in them is present in the code. (ie: bin*, obj*, .dll, .exe)
    2. User preferences are your settings, don't override other team member preference files! (ie: .suo and .user files)
    3. Most tools allow you to ignore certain files and Hg/Git allow you to version this as an ignore file. Set this up as a first step when creating a new repository!
  10. Be polite when merging unresolved conflicts.
    1. Count to 10, grab a stress ball and realize it’s not a big deal. Actually, it’s an opportunity to let you know that someone else is working in the same area and you might want to communicate with them.
    2. Following the other rules, especially committing frequently, will reduce the likelihood of this.
    3. Don’t blindly merge and commit your changes. Make sure you understand why the conflict occurred and which parts of the code you want to keep.
    4. Apply scrutiny when you commit a manual merge: review the diff!
    5. Make sure you test the changes (build and run automated tests)
  11. Become intimate with your version control system and the tools you use with it.
    1. Avoid trial and error as much as is possible, sit down and test the tool out, read some tutorials etc. Create test repositories and walk through common scenarios.
    2. Find the most efficient way to do your work. These tools will be used repetitively, so inefficiencies will add up. Sometimes this involves a mix of tools, both GUI and CLI.
      1. I like a combination of both Tortoise Hg and hg cli to get the job efficiently.
  12. Always tag releases
    1. Create a way to find a given release, whether this be in comments or an explicit tag / branch. This should be readily discoverable.
    2. Create release branches to patch bugs and then merge the changes back to other development branch(es).
  13. If using feature branches, strive for periodic integrations.
    1. Feature branches often cause forked code that becomes irreconcilable. Strive to re-integrate somewhat frequently with the branch this code will ultimately be merged into. This will avoid merge conflicts in the future.
    2. Feature branches are best when they are mutually exclusive of active development in other branches.
  14. Use and abuse local commits

    ,
    at least one per task in a story.
    1. This builds a trail of changes in your local repository that can be pushed to a central repository when the story is complete.
  15. Never commit a broken build or failing tests to the central repository.
    1. It’s ok for a local commit to break the build and/or tests. In fact, I encourage this if it helps group the changes more logically. This is one of the main reasons I got excited about DVCS, when I wanted more than one changeset for a set of pending changes but some files could be grouped into both changesets (like solution file / project file changes).
  16. Avoid committing sensitive information
    1. Especially usernames / passwords

There is one area I haven’t found a solution I like yet: versioning 3rd party libraries and/or code. I really dislike keeping any assemblies in the repository, but seems to be a common practice for external libraries. Please feel free to share your ideas about this below.

-Wes

vcs 
3.4.2010

Adopting DBVCS

Identify early adopters

Pick a small project with a small(ish) team. This can be a legacy application or a green-field application. Strive to find a team of early adopters that will be eager to try something new. Get the team on board!

Research

Research the tool(s) that you want to use. Some tools provide all of the features you would need while some only provide a slice of the pie. DBVCS requires the ability to manage a set of change scripts that update a database from one version to the next. Ideally a tool can track database versions and automatically apply updates. The change script generation process can be manual, but having diff tools available to automatically generate it can really reduce the overhead to adoption. Finally, an automated tool to generate a script file per database object is an added bonus as your version control system can quickly identify what was changed in a commit (add/del/modify), just like with code changes.

Don’t settle on just one tool, identify several. Then work with the team to evaluate the tools. Have the team do some tests of the following scenarios with each tool:

  1. Baseline an existing database: can the migration tool work with legacy databases? Caution: most migration platforms do not support baselines or have poor support, especially the fad of fluent APIs.
  2. Add/drop tables
  3. Add/drop procedures/functions/views
  4. Alter tables (rename columns, add columns, remove columns)
  5. Massage data – migrations sometimes involve changing data types that cannot be implicitly casted and require you to decide how the data is explicitly cast to the new type. This is a requirement for a migrations platform. Think about a case where you might want to combine fields, or move a field from one table to another, you wouldn’t want to lose the data.
  6. Run the tool via the command line. If you cannot automate the tool in Continuous Integration what is the point?
  7. Create a copy of a database on demand.
  8. Backup/restore databases locally.

Let the team give feedback and decide together, what tool they would like to try out.

My recommendation at this point would be to include TSqlMigrations and RoundHouse as SQL based migration platforms. In general I would recommend staying away from the fluent platforms as they often lack baseline capabilities and add overhead to learn a new API when SQL is already a very well known DSL. Code migrations often get messy with procedures/views/functions as these have to be created with SQL and aren’t cross platform anyways. IMO stick to SQL based migrations.

Reconciling Production

If your project is a legacy application, you will need to reconcile the current state of production with your development databases. Find changes in production and bring them down to development, even if they are old and need to be removed. Once complete, produce a baseline of either dev or prod as they are now in sync. Commit this to your VCS of choice.

Add whatever schema changes tracking mechanism your tool requires to your development database. This often requires adding a table to track the schema version of that database. Your tool should support doing this for you. You can add this table to production when you do your next release.

Script out any changes currently in dev. Remove production artifacts that you brought down during reconciliation. Add change scripts for any outstanding changes in dev since the last production release. Commit these to your repository.

Say No to Shared Dev DBs

Simply put, you wouldn’t dream of sharing a code checkout, why would you share a development database? If you have a shared dev database, back it up, distribute the backups and take the shared version offline (including the dev db server once all projects are using DB VCS). Doing DB VCS with a shared database is bound to cause problems as people won’t be able to easily script out their own changes from those that others are working on.

First prod release

Copy prod to your beta/testing environment. Add the schema changes table (or mechanism) and do a test run of your changes. If successful you can schedule this to be run on production.

Evaluation

After your first release, evaluate the pain points of the process. Try to find tools or modifications to existing tools to help fix them. Don’t leave stones unturned, iteratively evolve your tools and practices to make the process as seamless as possible. This is why I suggest open source alternatives. Nothing is set in stone, a good example was adding transactional support to TSqlMigrations. We ran into situations where an update would break a database, so I added a feature to do transactional updates and rollback on errors!

Another good example is generating change scripts. We have been manually making these for months now. I found an open source project called Open DB Diff and integrated this with TSqlMigrations. These were things we just accepted at the time when we began adopting our tool set. Once we became comfortable with the base functionality, it was time to start automating more of the process. Just like anything else with development, never be afraid to try to find tools to make your job easier!

Enjoy

-Wes

db vcs 
3.4.2010

Database version control resources

In the process of creating my own DB VCS tool tsqlmigrations.codeplex.com I ran into several good resources to help guide me along the way in reviewing existing offerings and in concepts that would be needed in a good DB VCS. This is my list of helpful links that others can use to understand some of the concepts and some of the tools in existence. In the next few posts I will try to explain how I used these to create TSqlMigrations.

Blogs entries

Three rules for database work - K. Scott Allen

http://odetocode.com/blogs/scott/archive/2008/01/30/three-rules-for-database-work.aspx

Versioning databases - the baseline

http://odetocode.com/blogs/scott/archive/2008/01/31/versioning-databases-the-baseline.aspx

Versioning databases - change scripts

http://odetocode.com/blogs/scott/archive/2008/02/02/versioning-databases-change-scripts.aspx

Versioning databases - views, stored procedures and the like

http://odetocode.com/blogs/scott/archive/2008/02/02/versioning-databases-views-stored-procedures-and-the-like.aspx

Versioning databases - branching and merging

http://odetocode.com/blogs/scott/archive/2008/02/03/versioning-databases-branching-and-merging.aspx

Evolutionary Database Design - Martin Fowler

http://martinfowler.com/articles/evodb.html

Are database migration frameworks worth the effort? - Good challenges

http://www.ridgway.co.za/archive/2009/01/03/are-database-migration-frameworks-worth-the-effort.aspx

Continuous Integration (in general)

http://martinfowler.com/articles/continuousIntegration.html

http://martinfowler.com/articles/originalContinuousIntegration.html

Is Your Database Under Version Control?

http://www.codinghorror.com/blog/archives/000743.html

11 Tools for Database Versioning

http://secretgeek.net/dbcontrol.asp

How to do database source control and builds

http://mikehadlow.blogspot.com/2006/09/how-to-do-database-source-control-and.html

.Net Database Migration Tool Roundup

http://flux88.com/blog/net-database-migration-tool-roundup/

Books

Book

Description

Refactoring Databases: Evolutionary Database Design

Martin Fowler signature series on refactoring databases.

Book site: http://databaserefactoring.com/

Recipes for Continuous Database Integration: Evolutionary Database Development (Digital Short Cut)

A good question/answer layout of common problems and solutions with database version control.

http://www.informit.com/store/product.aspx?isbn=032150206X

db vcs