Associate VS2017 with JSON files

Double-clicking a JSON file to try and open it in Visual Studio 2017 Professional doesn’t work; VS2015 worked fine. This link explains how to fix for the Community edition. The same principle applies for VS2017.

Using regedt32 first find the Visual Studio magic number:

Key: Computer\HKEY_CLASSES_ROOT\.json\OpenWithProgids
Value: VisualStudio.json.a8eb385c

2017-10-19 08_32_54-Registry Editor

 

Then find the associated shell key and create a new sub-key ‘Command’ with the path to devenv.exe as the default value:

Key: Computer\HKEY_CLASSES_ROOT\VisualStudio.json.a8eb385c\shell\Open\Command
Value: "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\devenv.exe"

2017-10-19 08_34_27-Registry Editor

 

Right-clicking a JSON file and selected Open with now looks like this:

2017-10-19 08_37_22-Sphere

 

Advertisements

VMWare Windows 10 expand/partition problem

Intro

I had a Windows 10 VM, managed using VMWare Workstation Pro 12. The VM was originally created with the default 60GB hard disk.

I needed to expand the disk, so I shutdown the VM, removed all snapshots, expanded the virtual HD to 120GB, and rebooted the VM. The plan was to use Windows 10’s disk management tool to expand the original partition and merge in the new partition.

But the recovery partition was sandwiched between the original and new partitions, and couldn’t be deleted using the Disk Management tool:

Untitled picture.png

I found the basics of how to fix this on the VMWare knowledge base. I’m adding my procedure here because it includes some useful screenshots.

Steps

I ran diskpart to work with partitions. (I ran as admin, but don’t think it was required.)

1Untitled picture

From the DISKPART shell, I then used the following to select and then remove the unwanted partition:

DISKPART> list volume

Volume ### Ltr Label Fs Type Size Status Info
 ---------- --- ----------- ----- ---------- ------- --------- --------
* Volume 0 D DVD-ROM 0 B No Media
 Volume 1 C NTFS Partition 59 GB Healthy System
 Volume 2 NTFS Partition 450 MB Healthy Hidden

DISKPART> select volume 2

Volume 2 is the selected volume.

DISKPART> delete partition override

DiskPart successfully deleted the selected partition.

The Disk Management window showed the new partition layout:

2Untitled picture.png

Next, I right-clicked on the C: partition and chose ‘Extend Volume’:

4.png

567

At last, a single 120GB partition.

8

🙂

 

C# TimeSpan TypeConverter and UITypeEditor

Code for this post is on GitHub.

I have an application that presents various TimeSpan properties to a user. The default string conversion isn’t great, in fact for anything other than hh:mm:ss it isn’t intuitive.

A TimeSpan of 1 day, 2 minutes, 3 hours, 4 seconds, and 5 milliseconds is shown in the example below:

Dev10.png

After a little noodling I found some articles that helped me put together something better (at least for me!).

The first feature is the presentation of a TimeSpan instance as a string:

dev10

The second feature is the ability to convert back from a string. For example, entering a value of 1h, 5s:

Dev10.png

… becomes…

Dev10.png

And finally, the property can present an interactive editor via a dropdown:

Dev10.png

Here’s an example of how the new classes are used as attributes on a TimeSpan:

[TypeConverter(typeof(TimeSpanStringConverter))]
[Editor(typeof(TimeSpanUIEditor), typeof(UITypeEditor))]
[DefaultValue(typeof(TimeSpan), "1.02:03:04.005")]
[DisplayName("Custom 1")]
public TimeSpan A { get; set; } = new TimeSpan(1, 2, 3, 4, 5);

VS2015 Update 2 VC.DB git mayhem

So today, after updating Visual Studio 2015 to Update 2, I was committing some changes to a project. Nestled amongst my own changes, deleted files, and new files, was a new VS2015 file, AOX3.VC.db (AOX3 is the name of my solution). This file is the new database engine and replaces (I believe) the SDF file. In Update 1 this was an experimental feature, but as of update 2 it’s official:

The new SQLite-based database engine is now being used by default.

I inadvertently added the file to my local git repository and then committed the changes.(I did the same thing on a couple of other projects, and made several more modifications and commits afterward).

The problem is, this file shouldn’t be in git, especially because it is around 150MB !

I only realised I had a problem when I tried to sync to my online (remote) repository, hosted by VSTS – luckily the attempt failed. The Synchronization panel had the following error message:

Failed to push to the remote repository. See the Output window for more details.

The output window had the following details:

Error encountered while pushing to the remote repository: Error while copying content to a stream.

Inner Exception:

Unable to read data from the transport connection: The connection was closed.

After a bit of googling I found I had to resort to the nuclear option to fix the problem.. deleting the file from the local git commit history.

Here’s what I did:

1. In VS2015 find the first commit that included the file. To do this I went to the Sync panel on the Team Explorer page and found the outgoing commit, then double-clicked it to see the commit details:

 

Screen Shot 2016-03-30 at 20.25.24.png

Screen Shot 2016-03-30 at 20.27.43.png

2. Note the parent commit ID of the first commit that included the new file (shown above, 645faa8c), then close VS2015.

3. Run a git bash shell in the solution folder. Then use the filter-branch command to delete the file from the last two commits. This required specifying a range of commits from the parent of the first commit all the way up the HEAD (the latest commit). This means noting the parent commit ID of the first commit that included the new file – as shown in the above screenshot. Here’s the git session:

jon@DESKTOP-0J37S2C MINGW64 /c/dev/Projects/Amber/AOX3 (dev_jon)
$ git filter-branch --tree-filter 'rm -f AOX3.VC.db' 645faa8c..HEAD
Rewrite 2fa87507296744092c2ac8b0c300b8c5973276ef (1/2) (0 seconds passed, remain
Rewrite 2917e9beea5ec0f546538345e33bd01586d359fb (2/2) (15 seconds passed, remaining 0 predicted)
Ref 'refs/heads/dev_jon' was rewritten

4. Open VS2015 and check the commits.. now the added/modified database file has disappeared from the list of changes:

Screen Shot 2016-03-30 at 20.35.52.png

5. I then edited my .gitignore file and added *.VC.db, then committed this mod and finally checked everything synchronized.. all worked as expected !

 

I guess the moral of the story is: don’t be flippant when adding new files, especially those you don’t recognize as your own. As the filter-branch documentation states:

This occurs fairly commonly. Someone accidentally commits a huge binary file with a thoughtless git add ., and you want to remove it everywhere.

This thoughtless programmer signing off, time to sleep 😉

 

 

OpenCV3, VS2015

Intro

So I finally got myself a clean Windows 10 VM for the next couple of years development, installed Visual Studio 2015, downloaded OpenCV, migrated my VS2013 OpenCV C++ test app, and.. and.. of course it didn’t work 😦

This was only an issue because I was using the static OpenCV libraries rather than the pre-built binaries. It looks like VS2015 is quite happy using the VC12 OpenCV binaries.

The current release of OpenCV 3.0.0 (04/06/2015) comes with pre-built static and import libraries for VS2013 and VS2010, but not for VS2015. I found an excellent article on the Colorado School of Mines (!) WebSite on how to build VS2015 libraries. But I needed some other tweaks beyond a basic build of OpenCV. So the goal of this note is to show how I built OpenCV for use with Visual Studio 2015 with the following configurations:

  1. 32- and 64-bit support.
  2. Static and Shared OpenCV libraries.
  3. Debuggable static libraries by having the correct combination of source code, libraries and PDBs.

The basic procedure I’m using (after several iterations) is:

  • Clone version 3.0.0 of OpenCV to a local development folder. I’m going to keep these source files handy because the debugger can use them in Debug builds
  • Install CMake, a build configuration tool.
  • Generate 4 VS2015 solutions
    • x86, static
    • x86, shared
    • x64, static
    • x64, shared

Clone OpenCV

This assumes you have VS2015 and the latest Git for Windows.

The trick, at least for me, is to clone into a folder that will be used to keep the OpenCV source code. This folder won’t get deleted during the steps below. I’m also only interested in the official latest release, which at the time of writing is version 3.0.0.

I have a standard development folder structure which is currently based on c:\dev, and I have a standard sub-folder structure for all third party libraries, c:\dev\ThirdParty. So I’m going to put OpenCV into c:\dev\ThirdParty\OpenCV3.0.0.

Run a git bash shell (from any folder) and clone version 3.0.0. Note the use of forward slashes in both paths !

Here’s my bash session:
jon@DESKTOP-0J37S2C MINGW64 /c/dev/ThirdParty/OpenCV3.0.0 ((3.0.0))
$ git clone --branch 3.0.0 https://github.com/Itseez/opencv.git C:/dev/ThirdParty/OpenCV3.0.0/sources
Cloning into 'C:/dev/ThirdParty/OpenCV3.0.0/sources'...
remote: Counting objects: 183285, done.
remote: Compressing objects: 100% (52/52), done.
remote: Total 183285 (delta 22), reused 6 (delta 6), pack-reused 183224
Receiving objects: 100% (183285/183285), 419.28 MiB | 3.54 MiB/s, done.
Resolving deltas: 100% (124890/124890), done.
Checking connectivity... done.
Note: checking out 'c12243cf4fccf5df7b0270a32883986b373dca7b'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

Checking out files: 100% (4656/4656), done.
Now I have the following folder structure:
471B7A0B-A24A-43FB-966C-14CB3E80C70F.png

Get CMake

There isn’t a VS2015 solution bundled with the OpenCV download so you must create one. This is made easy by using CMake, a cross-platform build utility. Download and install from here – I used the Windows (Win32 Installer) ‘cmake-3.4.1-win32-x86.exe’, and kept the default installation options.

Create the VS2015 Solutions

The basic procedure with CMake is:

  • Set the source and build folders
  • Click Configure
  • Choose the generator (compilers)
  • Tweak the options
  • Click Generate

I’m going to place each solution in a unique build folder and will start with:

  • 32-bit (x86)
  • Shared OpenCV_World300.dll
  • Static C runtime (CRT)
  • No test libraries (because they don’t build cleanly for me, and I want a clean build for the moment)

Run CMake and set the source code folder. Then choose a folder for the build. Make sure the build folder doesn’t already exist. E.g.

W10_Dev.png

The click Configure, accept any request to create the build folder, and then choose the generator for the project. For x86:

W10_Dev.png

The first pass at the configuration takes a minute or so and may download some IPPI files. It also ends up looking quite bad but the red items are just new values (which initially means everything):

W10_Dev.png

Use the Search box to find and verify/update the following settings:

  • BUILD_opencv_world: True
  • BUILD_SHARED_LIBS: True
  • BUILD_WITH_STATIC_CRT: True
  • BUILD_PERF_TESTS, BUILD_TESTS: False

For example, to find and update the OpenCV_World option:

W10_Dev.png

 

Click Generate. Then check you have a new VS2015 solution (but don’t open or build yet):

  • C:\dev\ThirdParty\OpenCV3.0.0\build_x86_shared\OpenCV.sln

For the remaining 3 VS2015 solutions first change the build path, click Configure, apply the settings, and click Generate.

  • x86, static
    • Build path: C:/dev/ThirdParty/OpenCV3.0.0/build_x86_static
    • Generator: Visual Studio 14 2015
    • BUILD_opencv_world: False
    • BUILD_SHARED_LIBS: False
    • BUILD_WITH_STATIC_CRT: False
    • BUILD_PERF_TESTS, BUILD_TESTS: False

 

  • x64, shared
    • Build path: C:/dev/ThirdParty/OpenCV3.0.0/build_x64_shared
    • Generator: Visual Studio 14 2015 Win64
    • BUILD_opencv_world: True
    • BUILD_SHARED_LIBS: True
    • BUILD_WITH_STATIC_CRT: True
    • BUILD_PERF_TESTS, BUILD_TESTS: False

 

  • x64, static
    • Build path: C:/dev/ThirdParty/OpenCV3.0.0/build_x64_static
    • Generator: Visual Studio 14 2015 Win64
    • BUILD_opencv_world: False
    • BUILD_SHARED_LIBS: False
    • BUILD_WITH_STATIC_CRT: False
    • BUILD_PERF_TESTS, BUILD_TESTS: False

 

Build x86 static

Close CMake and open VS2015, then open the x86 static solution:

  • C:\dev\ThirdParty\OpenCV3.0.0\build_x86_shared\OpenCV.sln

Give it a minute or two for the parsing stage to complete, select the Debug|Win32 configuration, and build the solution.

W10_Dev.png

I got the following output:

Build: 14 succeeded, 0 failed, 0 up-to-date, 5 skipped

Next build the CMakeTargets/INSTALL project – this will put the output files in an install folder which we’ll use later on:

W10_Dev.png

You can check you have the opencv_world300 debug DLL:

W10_Dev_and_Open.png

Now switch to the Release|Win32 configuration, build the solution, and build the CMakeTargets/INSTALL project. This will add the opencv_world release DLL to the install folder.

Now open the x64 shared solution and repeat the above to get the debug and release opencv_world300 DLLs. Note the configurations are:

  • Debug|x64
  • Release|x64

 

Open the x86 static solution: before build we must fix a glitch with the PDB file names. For some reason the debug PDBs were not configured to have a d appended to the filename. This means the debug and release PDBs will all use the same filenames. For example:

  • IlmImf.lib -> IlmImf.pdb
  • IlmImfd.lib -> IlmImf.pdb

There is probably a quick way to fix this using the CMake tools, but I don’t know how to ! Instead, do the following:

  1. Multi-select all the projects in the 3rdparty groupW10_Dev.png
  2. Open the Property Pages (right-click and select Properties).
  3. Set the Configuration to ‘All Configurations’
  4. Navigate in the tree-view to C/C++ / General
  5. Change the ‘Debug Information Format’ to ‘C7 compatible’
    W10_Dev.png
  6. Hit OK to apply the changes and close the Property Pages.

Repeat the above procedure for all projects in the following groups:

  • applications
  • modules
  • object_libraries

Why do this ?

  1. Instead of producing .lib and .pdb files we now have the PDB information merged into the .lib.
  2. It solves the problem of the debug build PDBs not having different names from the release equivalents.
  3. The CMakeTargets/INSTALL project doesn’t copy the PDBs, meaning a further manual step would be required to find the PDBs and copy them to the install folder.

Is there a better way – I’m sure there is, but this is quite simple and quick and if it works then why not ? (One good reason to not use this approach is if you’re going to work on the OpenCV source code; using the C7 option takes away certain compiler and linker options that could make for much faster builds). There’s plenty of discussion on the web about this, for example on StackOverflow.

Repeat the previous build procedure to get the Debug and Release builds and INSTALLs done. Then repeat again for the x64 (remembering to modify the Debug Information Format settings first).

Consolidating the build

I want to replicate the folder structure of the official release of OpenCV as much as possible. So first, create a new build folder:

  • C:\dev\ThirdParty\OpenCV3.0.0\build

Then copy the contents of each of the install folders into the above build folder. Skip over duplicate files.

  1. build_x64_shared
  2. build_x64_static
  3. build_x86_shared
  4. build_x86_static

My final folder structure is around 1.2GB and looks like this

 

W10_Dev.png

All four of the build_XXX folders can now be deleted – for me they take up around 10GB. Alternatively you can reload each solution, select Batch Build, Select All, then clean. This drops the folders down to around 2GB. For me I’m going to ZIP and archive them, in case I made a mistake !

Testing the build

One of my requirements is to make sure colleagues can work with these libraries, even if they use a different folder structure to me for their own projects. The only prerequisite is that there’s a ThirdParty\OpenCV3.0.0 folder; but ThirdParty can be anywhere.

My first trick is to make a User Macro in VS2015 to set the ThirdParty folder location. This is another example of something that could probably be done better, but I don’t want to change the system PATH and I need something simple. My reference for User Macros is from this MSDN page.

  1. Open or create a C++ project.
  2. View the Property Manager:
    W10_Dev.png
  3. Select the Microsoft.Cpp.Win32.user node of any project, right-click and select Properties. Then select the User Macros setting and add a Macro called ThirdParty:
    W10_Dev.png
  4. Apply the changes and close the dialog, then close the Property Manager.

The macro can now be used in projects which may be distributed among other developers; each developer just needs to have their ThirdParty folder (with OpenCV3.0.0 inside) and this macro.

As an example, I have a 32-bit project using the static OpenCV library. The key settings are:

Additional Include Directories:W10_Dev.png

Additional Library Directories:W10_Dev.png

Additional Dependencies:W10_Dev.png

The list of dependencies for my Release build is:

IlmImf.lib
libjasper.lib
libjpeg.lib
libpng.lib
libtiff.lib
libwebp.lib
opencv_calib3d300.lib
opencv_core300.lib
opencv_features2d300.lib
opencv_flann300.lib
opencv_hal300.lib
opencv_highgui300.lib
opencv_imgcodecs300.lib
opencv_imgproc300.lib
opencv_ml300.lib
opencv_objdetect300.lib
opencv_photo300.lib
opencv_shape300.lib
opencv_stitching300.lib
opencv_superres300.lib
opencv_ts300.lib
opencv_video300.lib
opencv_videoio300.lib
opencv_videostab300.lib
zlib.lib
ippicvmt.lib

And for my debug build it is:

IlmImfd.lib
libjasperd.lib
libjpegd.lib
libpngd.lib
libtiffd.lib
libwebpd.lib
opencv_calib3d300d.lib
opencv_core300d.lib
opencv_features2d300d.lib
opencv_flann300d.lib
opencv_hal300d.lib
opencv_highgui300d.lib
opencv_imgcodecs300d.lib
opencv_imgproc300d.lib
opencv_ml300d.lib
opencv_objdetect300d.lib
opencv_photo300d.lib
opencv_shape300d.lib
opencv_stitching300d.lib
opencv_superres300d.lib
opencv_ts300d.lib
opencv_video300d.lib
opencv_videoio300d.lib
opencv_videostab300d.lib
zlibd.lib
ippicvmt.lib

To make sure it’s all looking good I build the Debug configuration and check there are no spurious errors or warnings. (For example, before I switched to the C7 debug-info setting I had all sorts of warnings about missing PDBs.)

Then try setting a breakpoint on some OpenCV code and stepping into it: if all goes well you’ll be able to step into and around the OpenCV source code.

Note: if you move your OpenCV distribution to another folder then the debugger will prompt you for the location of the OpenCV source files. This only happens once (as far as I can tell).

To use the DLL version of OpenCV change the following:

  • Linker/General/Additional Library Directories:
    $(ThirdParty)\OpenCV3.0.0\build\x86\vc14\lib;%(AdditionalLibraryDirectories)
  • Linker/Input/Additional Dependencies:
    opencv_world300.lib

Note: if you run a shared DLL version without making the DLL available then you’ll get something like this:

‘The program can’t start because opencv_world300.dll is missing from your computer.’

W10_Dev.png

To avoid changing the system PATH I added a post-build event to copy the DLL to my project’s output:

W10_Dev.png

A good sample program to start with is Load and Display an Image, on the OpenCV website.

That’s it – I’m done ! After at least 10 variations of these builds, and two or three attempts to write up how I did it, I’m leaving it here.  At least until I figure a better way to do it, or an update is issued from OpenCV, or I figure out how to try and contribute directly to OpenCV, etc.  🙂

 

Python 2.7 stack, OpenCV 3, Visual Studio 2013

I’ve started working with Python (2.7 and 3.x) and OpenCV 3 recently, and the IPython Notebook way of working with inline plots is awesome. But the debugging, editing and some other features are not great when compared to working in a Visual Studio environment.

So now I’m trying Visual Studio 2013 as the IDE, OpenCV 3, and Python 2.7 via the numerical and scientific WinPython stack.

I extracted OpenCV and WinPython to:

  • My Documents\OpenCV
  • My Documents\WinPython-32bit-2.7.10.2

From VS2013 I then installed Python Tools for Visual Studio (PTVS).. I did this within VS2013 using the Extensions and Tools dialog.

Finally I configured a new Python Environment:

W10_Dev

From an Interactive Window I can then check everything works as expected. First, Open CV3:

Python interactive window. Type $help for a list of commands.
import cv2
cv2.__version__
'3.0.0'

Next up, NumPy and a MatPlotLib chart:

import numpy as np
import matplotlib.pyplot as plt
x = np.linspace(0, np.pi * 2, 100)
y = np.sin(x)
plt.plot(x, y)
plt.xlabel('Radians')
plt.ylabel('sin(x)')
plt.show()

This gives us a popup (model) window:

W10_Dev

Now to make it a bit more interesting – inline plots.. ! The Python Interactive Window has options for the Interactive mode:

W10_Dev

The default Interactive Mode is Standard but we can change this to IPython:

W10_Dev

After restarting VS2013 we can now go inline…

W10_Dev

And lastly an OpenCV 3 test:

In [4]: import cv2
In [5]: img = cv2.imread(r"C:\Users\jon\Pictures\Samples\eye.jpg", 0)
In [6]: plt.imshow(img, cmap='gray')
Out[6]: <matplotlib.image.AxesImage at 0x5b8e510>

W10_Dev

In [34]: blur = cv2.medianBlur(img, 91)
In [35]: plt.imshow(blur, cmap='hot')
Out[35]: <matplotlib.image.AxesImage at 0x1edea7d0>

W10_Dev

The great thing is I can now move this code into a Python project, get it into git, distribute among colleagues, and use the excellent intellisense and debugging powers of PTVS ! (Obviously not this actual code !)

JSON editing in Visual Studio 2013 – tip!

Just installed VS2013 on a clean Windows 10 virtual machine, noticed I wasn’t getting the same JSON editing experience I had on my previous VS2013 installation. The colour coding and auto-completion features were missing.

Solved the problem by modifying the installation and including the Microsoft Web Developer Tools:

W10_Dev

Now the editor uses colour coding, auto-completes the various brackets, and highlights errors.