Mighty Moose a/k/a ContinuousTests

Introduction

At my current place of employment, I was introduced to NCrunch, a continuous testing solution for .NET and Visual Studio. It’s a great tool that I really enjoy using at work. However, for at home use, I can’t justify paying their asking price for a personal license. Don’t get me wrong, I really love the tool, and if I were doing some serious development on my own (i.e. creating products for sale as a side job), I’d probably pay for it, because it’s fast and shows exactly where an exception occurred or a test failed.

So I really like this concept of continuous testing and went on an Internet search to see what was available out there for free. I wanted something that would run in Visual Studio 2012/2013. A lot of the “free” tools use external test runners which display their output in either the browser (as an XML/HTML page) or in a console window. I don’t like those tools because they interrupt my context, having to switch between Visual Studio and, say, a console window. Plus, then I have to go digging for the file and line number when a test fails.

Enter Mighty Moose a/k/a Continuous Tests

During my search, I found Mighty Moose a/k/a ContinuousTests, by Greg Young. Mighty Moose has a slightly different philosophy about continuous testing. Instead of a code coverage margin in Visual Studio, they have a risk margin. The risk margin shows a colored circle next to a method under test with a number inside. The number inside tells you how many tests are covering the method. The color: red, green, or yellow, indicates the risk of changing that method.

How is risk calculated? It’s based on how close the method was called from the test. According to Greg, while code coverage tells you whether or not the code was executed by a test, that information alone is not enough to classify how risky it is to change the code. What if the code that’s “covered” was called from a method 10 frames down the call stack? By changing that code, you could be affecting not just the method under test, but all 8 other methods in the call stack.

In addition to the risk margin, Mighty Moose contains graph “overlay” graphics (for those of us who can’t afford Visual Studio Ultimate ;) ) that show the call graph of all methods called during the test. This can help you gauge “how far away” a method was invoked from the test, and in general, just help you see the entire call graph, which in itself, could be enlightening.

Getting Mighty Moose to Work with Visual Studio 2013

Mighty Moose is currently at version 1.0.47. It officially only supports Visual Studio 2008, 2010, and 2012. However, there’s an excellent question over on Stack Overflow that I contributed to which shows you how to get Mighty Moose to work with Visual Studio 2013.

AllMargins 2012 v2.8.x Releases

On 1 September 2013 I released version 2.8 of the AllMargins 2012 Visual Studio extension. Since that release, there have been two additional "dot" releases. Briefly, here's what v2.8.x has to offer:

  • Visual Studio 2013 RC and RTM support

  • All the registry value names for the extension's settings have been renamed to prevent naming conflicts with the Productivity Power Tools for Visual Studio 2012 and beyond (more on that below)

  • Fixed the registry value name for method separator line color

  • Fixed the StructureAdornment extension so that method separator lines are drawn using the specified color

  • Added C/C++ union support

Extension Option Registry Value Name Clashes

David Pugh (MSFT) ported a lot (but not all) of the functionality of the original AllMargins extension into the Productivity Power Tools for Visual Studio 2012. The extension option registry value names were left the same. So, when I re-released the AllMargins extension as AllMargins2012, the two extensions would step over each other in terms of their settings for the common extensions. Starting with v2.8, the registry value names have changed to prevent this. As a result, those who installed v2.7 and subsequently customized the extension options by editing the registry, upon installing v2.8, will see that the extension is using the default values for all settings—it appears as though your customizations were changed. They are not. They are still present in the registry. However, new settings are now used by the AllMargins 2012 extension. You need only copy the setting value from the v2.7 registry value to the new v2.8 registry value. I apologize for the inconvenience.

For more details about what the various registry settings are, and their old and new names, see the description tab for the extension at the Visual Studio Gallery.

What to expect in the next major release

Many have spoken, including myself, about what you would like to see in the next major release. Work is underway; here is a quick rundown of what to expect (of course, this is subject to change—nothing is set in stone unless otherwise noted):

  • The ability to set the extension's options from the Visual Studio Tools|Options dialog. This is a top priority, non-negotiable item for the next major release. I have been wanting this very feature myself since David Pugh originally wrote this extension. Now that I'm maintaining this code base, this task falls on me.

  • The ability to specify the line widths displayed in the editor for the StructureAdornment extension

  • Add an option to show/hide the StructureMargin preview tool tip

The items above are what I can reasonably expect to fit into the next major release, though only the first item is guaranteed to make it through. Whatever doesn't make it through will be in a follow-up release. One item that is a planned feature for a future release is the availability to select a color theme for the StructureMargin and StructureAdornment extensions. As many have noted, and I agree, the "default" colors don't work well with Visual Studio 2012's dark theme. In addition, I plan on changing the default color for the class structure adornment from black to some other color, because with the Visual Studio light theme the black line can cause exclamation points to appear to not be present in C# and C/C++ preprocessor statements (e.g. #define !SOME_PREPROCESSOR_VAR), as one user pointed out to me. Black was the original color used by David Pugh—but due to this potential problem, it's best if this color is changed.

If there are any other features you'd like to see, please feel free to leave a comment here on my blog or on the Q AND A tab at Visual Studio Gallery. Thanks again for all of your comments, feedback, suggestions, and ratings!

Announcing the Release of AllMargins for Visual Studio 2012

In August of last year, I wrote about upgrading Visual Studio 2010 extensions so that they will work with Visual Studio 2012. In particular, I discussed upgrading the AllMargins Visual Studio extension, which was quite popular.

For some time now, David Pugh of Microsoft has pulled this extension off of Visual Studio Gallery. It is only available if you happened to have downloaded it and saved it to your hard drive or have downloaded the source code (which you can find the link to in the article I wrote back in August). The problem with the source code is that it's a few versions behind the last release that was available on Visual Studio Gallery and contains a few bugs and is missing support for things like C#'s class initializer syntax.

The extension was pulled because David ported most of the functionality into the Visual Studio Productivity Power Tools. However, I'm not a big fan of the particular implementation of the enhanced scrollbar in VSPPT and prefer the AllMargins scrollbar instead.

Today, I'm announcing general availability once again of the AllMargins 2012 extension on Visual Studio Gallery. I modified the available source code, bringing it up to parity with version 2.6, the last available release on Visual Studio Gallery. In addition, I modified the extension so that it will install in Visual Studio 2012. The extension is now at version 2.7.

Thanks to David Pugh at Microsoft for giving the Visual Studio community this wonderful extension and releasing the source code for it a while back!

As I have time, I hope to make a few additions to the extension. Some planned enhancements include providing the ability to change various colors used for drawing the lines in code files delineating various code blocks or for drawing various markers in the overview margin (e.g. drawing a mark for all occurrences in a document of a word under the cursor in the overview margin) right from the Visual Studio Options dialog.

A few notes about AllMargins 2012. First, because the original extension is linked to David Pugh's Windows Live ID, I had to change the identifier of the extension. (This is good, afterall, you wouldn't want Microsoft to allow people to hijack author's extensions possibly replacing them with malicious code, now would you?!) Since the ID, well, identifies the extension, it's essentially a different extension. So I renamed it AllMargins 2012 while I was at it. What this means is that if you want to use this extension in Visual Studio 2010, you must uninstall David Pugh's original AllMargins extension before installing this extension. I don't know what would happen if you have both installed (Double overview scrollbars? Crash Visual Studio?)

I hope that you will continue to find this extension as useful as I have! The Read Me for the top-level AllMargins 2012 extension is presented below.

AllMargins 2012

Introduction:
This is a compilation of the OverviewMargin and several other extensions that use the OverviewMargin. The OverviewMargin shows
a margin on the right side of the editor that logically maps to the entire file (similar the the vertical scroll bar). Unlike
the scroll bar, it maps to the entire file and can contain other margins that provide more information about the file.


Included extensions:
CaretMargin
ErrorsToMarks
MarkersToMarks
OverviewMarginImpl
StructureAdornment
StructureMargin

BlockTagger The definition of an API to get the structure of a code file.
BlockTaggerImpl An implementation of the BlockTaggerAPI for C/C# and VB files.

OverviewMargin The definition of an API for creating margins that map to the entire file.

SettingsStore The definition of an API to load and save editor options across sessions.
SettingsStoreImpl An implementation of the SettingStore API that uses IVsSettingsStore to access the system registry.

Usage:
See other extensions.

Options:
None.

Notes:
The full source for this extension is at http://code.msdn.microsoft.com/OverviewMargin/Release/ProjectReleases.aspx?ReleaseId=3957
The source code is not yet available for download.

History:
v1.0 David Pugh 2/26/2010
Initial release
V1.1 David Pugh 3/4/2010
Updated included extensions.
V1.2 David Pugh 4/19/2010
Updated included extensions.
V1.3 David Pugh 4/20/2010
Updated included extensions.
V1.4 David Pugh 4/22/2010
Updated included extensions.
V1.5 David Pugh 4/27/2010
Updated included extensions.
V1.6 David Pugh 4/28/2010
Updated included extensions to fix VB parser error.
V1.7 David Pugh 4/28/2010
Updated version number to work around install issue.
V1.8 David Pugh 4/28/2010
More VB parse fixes (Public, REM, Interface, Structure).
V1.9 David Pugh 4/28/2010
Updated CaretMargin, VB, C# and C parsers.
V2.0 David Pugh 5/03/2010
Bumped version number to fix installation problem.
V2.1 David Pugh 6/10/2010
Updated OverviewMargin, BlockTagger, StructureAdornment & StructureMargin.
Changed namespaces and DLL names to Microsoft.VisualStudio.Extensions.....
V2.2 David Pugh 6/29/2010
Updated StructureAdornment.
V2.3 David Pugh 6/30/2010
Picked up fix for VB parser, tweaks for structure adornment & margin.
V2.4 David Pugh 7/08/2010
Updated StructureAdornment.
V2.5 David Pugh 7/12/2010
Updated included BlockTaggerImpl to handle C/C# array initializers.
V2.6 David Pugh 7/31/2010
Updated included CaretMargin to fix problem with margin not updating correctly.
Updated included OverviewMarginImpl to fix scrollbar mapping problem.
V2.7 Craig E. Shea 1/16/2013
Added Visual Studio 2012 support.
Removed the attribute MaxVersion="4.0" on the SupportedFrameworkRuntimeEdition element.
Changed the author and description to indicate who is now responsible for the extension and that it is a re-release of a previously available extension.

Updating a Visual Studio 2010 Extension for Visual Studio 2012



UPDATE - 16 January 2013

The contents of this article have been updated to correct a problem that existed at the time of publication. In addition, I have uploaded this updated Visual Studio extension to the Visual Studio Gallery.

Hello everyone. It's been way too long. It seems I'm always doing this and that and never have time to write down anything, anywhere—whether that be on this blog, or on Facebook. But today, I have something special for those of you who have that favorite Visual Studio 2010 Extension that they just can't live without, but it hasn't been updated for Visual Studio 2012.

In order to update the extension, you will need access to the extension's source code; so these instructions will have limited impact for most of you out there unless you can get your hands on the source code for that favorite extension.

Fortunately for me, the author of my favorite extension, AllMargins, did post his source code, especially since this extension is no longer available from the Visual Studio Gallery.

Install the Visual Studio 2012 and 2012 SDKs

Get Visual Studio Professional 2012 with MSDN from the Microsoft Store

In order to author extensions for Visual Studio, one must have the Visual Studio SDK installed. The same is true for modifying them. If you don't already have it installed, go out and download it and install it.

Prepare the Source Code for Visual Studio 2012

If you want to maintain backward compatibility for the extension, it would be best not to open the original Visual Studio 2010 solution in Visual Studio 2012. That's because Visual Studio Extension projects for Visual Studio 2010 are not compatible with Visual Studio 2012. Well, that's not exactly true. The real truth is that the VSIX manifest XML schema has changed between Visual Studio 2010 and Visual Studio 2012. The old schema still works for Visual Studio 2012, but Microsoft would have you use the new schema going forward. The problem is, if you use the new schema, you won't be able to use the extension in Visual Studio 2010. Don't worry though, Visual Studio 2012 will not upgrade the schema, just the project file(s); but in doing so, the projects will no longer open in Visual Studio 2010.

Why should you care? Well, for one, testing. You can't simply instruct VsixInstaller to install the extension to an experimental instance of Visual Studio, unfortunately. That really makes it hard to test the extension for more than one version of Visual Studio. Even so, you really should be testing your extension on a virtual machine with the installed version of Visual Studio to test it with and which does not have the Visual Studio SDK installed. If the Visual Studio SDK is installed on your test machine, your extension will always work but, you may find that it doesn't work users who don't have the Visual Studio SDK installed. Your extension should not explicitly require the Visual Studio SDK be installed (unless you're writing a Visual Studio Extension project template! Lol...).

If you want to be able to develop and test in both Visual Studio 2010 and Visual Studio 2012 on the same machine, here's my suggestion.

  1. Create a new empty solution called <Your Extension>.2012 in the same place as the original solution.

  2. Open Windows Explorer to the location of one of the projects that are in the Visual Studio 2010 solution.
    1. Select the project file and copy and paste it into the same folder. The file will be renamed 'Copy of <Your Project File>.csproj' (or .vbproj, if you use Visual Basic). Rename the file to <Your Project File>.2012.csproj.

    2. Back in Visual Studio 2012, add <Your Project File>.csproj to your solution.


  3. Add all the files that are part of the Visual Studio 2010 project to the newly created Visual Studio 2012 project.

  4. Repeat these steps for each of the other projects that were part of the original Visual Studio 2010 solution.

Now you can work on the source code in both Visual Studio 2010 and Visual Studio 2012, opening the respective solution for the version of Visual Studio you're using to make changes. When you build the Visual Studio 2012 solution, it will be deployed to the Visual Studio 2012 experimental instance. The same is true for 2010, the extension will be deployed to the Visual Studio 2010 experimental instance. Also, it allows you to work on a single code-base that should work in both environments so long as you don't use features exclusive to Visual Studio 2012.

While upgrading this extension to work with Visual Studio 2012, I discovered that there is an enumeration value in the VSIX XML schema for specifying the Express editions of Visual Studio. However, the only "extensions" that Express SKUs support are project/item template extensions.

We're now ready to start modifying the various files to make this extension compatible with Visual Studio 2012!

Update the *.vsixmanifest file

The first order of business in updating the extension is to modify the *.vsixmanifest file. Here's an example of one of the files from the AllMargins.vsix extension:

<?xml version="1.0" encoding="utf-8"?>
<Vsix xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.0.0" xmlns="http://schemas.microsoft.com/developer/vsx-schema/2010">
<Identifier Id="SettingsStore.dpugh.ce3ce74a-4966-4a66-b9ad-a3adf4f9e940">
<Name>SettingsStore</Name>
<Author>dpugh</Author>
<Version>1.2</Version>
<Description xml:space="preserve">This defines an API for persisting IEditorOptions across sessions.</Description>
<Locale>1033</Locale>
<MoreInfoUrl>http://code.msdn.microsoft.com/OverviewMargin/Release/ProjectReleases.aspx?ReleaseId=3957</MoreInfoUrl>
<SupportedProducts>
<VisualStudio Version="10.0">
<Edition>Ultimate</Edition>
<Edition>Premium</Edition>
<Edition>Pro</Edition>
</VisualStudio>
</SupportedProducts>
<SupportedFrameworkRuntimeEdition MinVersion="4.0" MaxVersion="4.0" />
</Identifier>
<References />
<Content>
<MefComponent>|%CurrentProject%|

</Content>
</Vsix>

You need to update the SupportedProducts and SupportedFrameworkRuntimeEdition elements as follows:

<SupportedProducts>
<VisualStudio Version="10.0">
<Edition>Utimate</Edition>
<Edition>Premium</Edition>
<Edition>Pro</Edition>
</VisualStudio>
<VisualStudio Version="11.0">
<Edition>Utimate</Edition>
<Edition>Premium</Edition>
<Edition>Pro</Edition>
</VisualStudio>
</SupportedProducts>
<SupportedFrameworkRuntimeEdition MinVersion="4.0" />

By removing the MaxVersion attribute for the SupportedFrameworkRuntimeEdition element, we're telling Visual Studio that so long as the machine supports the .NET Framework 4.0, the extension can be used.

Also for good measure, I updated the Version value (e.g. 1.3) and updated the Description element noting that I changed the VSIX package and it is not the original. As an aside, every time you make a change to the extension that you wish to distribute, you must increment the value for the Version element.

Now I ran across a tricky problem with updating the AllMargins extension. This extension has several "sub-extensions" that are contained within the AllMargins extension itself. And some of these "sub-extensions" have yet other "sub-extensions". The long and short is, during testing, Visual Studio had a problem locating some of the required DLLs.

What I found out is that some of the sub-extension projects contained an additional file—a *.pkgdef file. This is an older file that was used with VSPackages back in the VSI days of Visual Studio 2005/2008. This file is not much unlike a *.reg file, though with a slightly different syntax.

This file for the SettingsStore sub-extension (for which the VSIX manifest file was shown above) looks like this:

[$RootKey$\BindingPaths\{8ADD56CC-E512-4CB0-AF58-33ECCC0B4912}]
"$PackageFolder$"=""

This creates the GUID under HKCU\Software\Microsoft\VisualStudio\<Maj. Version Num>_Config\BindingPaths key. $PackageFolder$ expands to the folder where the extension was installed by VSIXInstaller.

Apparently, in Visual Studio 2010, just placing this file within your VSIX package was enough for Visual Studio to go ahead and create the requisite registry keys and values. However, this does not work with Visual Studio 2012 and attempts to use extensions relying on this extension result in a System.IO.FileNotFoundException. In order to resolve this issue, you must place some additional content in your VSIX manifest file as shown below.

<References>
<Reference Id="Microsoft.VisualStudio.MPF" MinVersion="10.0" />
<Name>Visual Studio MPF</Name>
</Reference>
<!-- Any other references you may have -->
</References>
<Content>
<MefComponent>|%CurrentProject%|</MefComponent><!-- Expands to the assembly name -->
<VsPackage>SettingsStore.pkgdef</VsPackage><!-- Not required by VS2010, but is required by VS2012 in order for the file to be processed -->
</Content>

With all of that out of the way, save the file and we're done with the *.vsixmanifest file. If there are several VSIX extensions that make up the extension you're updating (as is the case for AllMargins.vsix), you will need to do this for each *.vsixmanifest file in the solution.

At this point, at the original date of publication (22 August 2012), I had you retarget the projects in the solution and replace some DLL references. If you want to maintain compatibility with Visual Studio 2010, this is not necessary. So I have removed that section of the original blog entry.

Recompile a Debug Build

You should now recompile the solution using a Debug build. This will automatically install the extension(s) into the Visual Studio experimental instance(i.e. it won't muck up your Visual Studio that you use for everyday coding). Make sure that everything works correctly. If all is well, you're ready for the final step.

Recompile in Release Mode

If your testing above went OK, then change the build mode from Debug to Release and recompile. Then simply navigate to the bin\Release directory under the project folder for the extension and double-click the VSIX file to install it.

Now, in this case, I was not required to change a single line of code. This may not be the case for every extension you wist to upgrade. YMMV.

Enjoy using Visual Studio 2012

That's it! Enjoy using Visual Studio 2012 with your beloved extension.

Put feedlink markup here.