TechNote

macOS, Visual Studio Code, Python 3.7.5, OpenCV4

It took a few attempts to get a compatible Python and OpenCV library running under Visual Studio Code on macOS Catalina using a virtual environment. I made a video to show how I got this going – this post just adds some more details.

There is also an excellent tutorial from Microsoft:

Getting Started with Python in VS Code

Note: virtual machine rendering problem

Visual Studio Code running on a virtual machine may have problems rendering the interface. This seems to be related to the underlying Electron framework and GPU acceleration. I made a quick video to show how I got around this:

Fix rendering problems for Visual Studio Code running on a virtual machine

Install Python 3.7.5

A virgin Mac comes with Python 2.7 installed – this is not recommended and V3.7.5 works with OpenCV4 on a Mac. V3.8 does not work at the time of writing (although since I started writing this post it looks like it now does). Download the installer from the main python website by selecting Downloads, Mac OS X, and then selecting the 64-bit installer:

Run the installer – I used all default settings.

Install Visual Studio Code

Download the installer from Visual Studio Code and immediately move the downloaded file to the Applications folder. (This is the actual application, not an installer). Try to run once – macOS will refuse due to security:

Close the message, open System Preferences, and select the Security and Privacy settings. Then select “Open Anyway” to allow VSC.

Visual Studio Code should now start:

Configure Python

Open a folder by selecting Open folder and then add a new file. Save the file using the .py extension:

Visual Studio Code immediately offers to install the Python extension, select Install:

On a virgin Mac there will now be a prompt to install command line developer tools, so click Install if prompted and allow the installation to complete before returning to Visual Studio Code.

The status bar will show the selected interpret if everything has gone well:

Install the linter (pylint): this helps analyse the code for bugs and style issues. It also might not work first time but we can fix shortly…

If the terminal window suggests upgrading pip, the Python package manager, then go for it by running the following in the terminal window:

python3 -m pip install --upgrade pip

Make a virtual environment

A virtual environment is a self-contained directory tree that contains a Python installation for a particular version of Python

https://docs.python.org/3/tutorial/venv.html

Each project can use its own virtual environment to ensure any modules it requires don’t clash with modules in other projects.

From the terminal create a virtual environment:

python3 -m venv .venv

Visual Studio Code will detect this new environment and offer to select it for the current project folder – select Yes:

Because this is a new Python environment you may need to install the linter again:

Now – the bit that confused me… the project is now using the .venv virtual environment:

However, the terminal session has so far only created the environment, it has not activated it for itself. The shell identifier says:

jon@Jons-MacBook-Pro Python %

There are two ways to fix this. First, using the source command in the terminal window:

source .venv/bin/activate

Second, by creating a new Terminal session using the command palette. (Select View, ten Command Palette):

Now the terminal shows that it’s using the virtual environment:

Install OpenCV

At last we can install OpenCV. Using the terminal session in the virtual environment we can first search for OpenCV packages:

python3 -m pip search opencv

When called with -m module-name, the given module is located on the Python module path and executed as a script

https://docs.python.org/3/using/cmdline.html

We see results like this:

opencv-utils (0.0.2) – OpenCV Utilities
ctypes-opencv (0.8.0) – ctypes-opencv – A Python wrapper for OpenCV using ctypes
opencv-wrapper (0.2.3) – A Python wrapper for OpenCV.
opencv-cython (0.4) – An alternative OpenCV wrapper
dajngo-opencv (0.3) – Django Opencv integratio
opencv-python (4.1.2.30) – Wrapper package for OpenCV python bindings

For this test I’m using opencv-python. The details on version 4.1.2.30 can be found on the Python Package Index site. Interestingly this version was only released a few hours ago and says it supports Python 3.8 😬 I guess I’ll try this on a virtual machine first to check it’s all ok!

Install OpenCV using pip:

python3 -m pip install opencv-python

Write some code and fix the linter

First test: import the OpenCV module and print the library version.

import cv2
print('Using OpenCV version {0}'.format(cv2.__version__))

After running this output is shown in the terminal:

But – there’s a problem. In the editor the linter is suggesting that cv2 is not a known module:

This has been seen before on the pylint GitHub issues page. For me, the solution is to edit the .vscode settings. Using ⇧⌘E (shift+command+E) to view the explorer page, expand the .vscode file and click settings.json:

Add a comma to the end of the line of the existing setting, then add the following new setting:

"python.linting.pylintArgs": ["--generate-members"]

My settings now look like this:

And now the red squiggle has gone from cv2.__version__ 😀

All that remains is to learn Python and OpenCV which will surely lead to great things!

Hope this helps.

3 useful keyboard mods for better editor support on Windows 10 using Parallels

Tested with Parallels Desktop 15 Pro Edition on macOS Catalina.

Problem 1: using Visual Studio or notepad++ or any similar multi-document application in Windows 10 on a Mac using Parallels, ⌘W (Command+W) maps to Alt+F4. This means To close just an editor page you have to revert to the original Windows shortcut of Control+F4 which is a minor pain on a Mac with the Touch Bar instead of function keys.

Solution 1: change the Parallels preferences to remap ⌘W to Ctrl+F4. ⌘Q will still close an application but ⌘W will now close an internal editor window; this is the same behaviour used in Safari to close the whole application (⌘Q) or close just one page (⌘W).

Problem 2: Control+Tab and Shift+Control+Tab don’t switch between editor windows.

Solution 2: a recent update to Parallels resulted in these shortcuts being used for Parallels tab switching. They don’t get passed on to the VM. Just unchecking these shortcuts fixes the problem.

C# from clause vs nested foreach loops

Short story: writing a unit test for an image processing function, I had the following key parameters for each test:

enum Algorithm {  A, B, C }; 
enum ImageFormat {  Gray, Color }; 
enum ImageSize {  Small, Medium, Large };

I wrote a core test function that worked on just one combination of the 3 parameters. E.g.

void Test(
    Algorithm algorithm, 
    ImageFormat imageFormat, 
    ImageSize imageSize)
{
    Console.WriteLine($"Testing algorithm {algorithm} for " +
        {imageFormat} image with {imageSize} size");

    // the test...
}

Next, I wrote a utility to make iterating the values of an enumeration a little easier.. still not sure why this isn’t part of .Net yet:

class EnumHelper<T>
{
    static public T[] Values
    {
        get { return (T[])Enum.GetValues(typeof(T)); }
    }
}

Then, I wrote the nested loops that built each combination and sent them for testing:

void TestAllV1()
{
    foreach (var algorithm in EnumHelper<Algorithm>.Values)
    {
        foreach (var imageFormat in EnumHelper<ImageFormat>.Values)
        {
            foreach (var imageSize in EnumHelper<ImageSize>.Values)
            {
                Test(algorithm, imageFormat, imageSize);
            }
        }
    }
}

Now there’s nothing really wrong with the above, but it looked like something that should be able to be written more simply. So I came up with this:

void TestAllV2()
{
    var tests =
        from algorithm in EnumHelper<Algorithm>.Values
        from imageFormat in EnumHelper<ImageFormat>.Values
        from imageSize in EnumHelper<ImageSize>.Values
        select (algorithm, imageFormat, imageSize);


    foreach (var test in tests)
    {
        Test(test.algorithm, test.imageFormat, test.imageSize);
    }
}

The use of the from clause seems better, mainly due to the reduced level of nesting. The Visual Studio 2019 code analysis metrics are interesting:

MemberMICycC
ClsC
TestAllV1() 76 4 7
TestAllV2()69 2 18

Where:

  • MI: Maintainability Index
  • CycC: Cyclomatic Complexity
  • ClsC: Class coupling

So the foreach approach is (allegedly!) more maintainable, while the from clause method has a lower cyclomatic complexity. This latter metric reinforces the idea that this is slightly simpler than the foreach technique.

It’s also quite easy to add specific filtering inside the tests generator. For example, to quickly stop testing the B algorithm:

var tests =
    from algorithm in EnumHelper<Algorithm>.Values
    where algorithm != Algorithm.B
    from imageFormat in EnumHelper<ImageFormat>.Values
    from imageSize in EnumHelper<ImageSize>.Values
    select (algorithm, imageFormat, imageSize);

Food for thought 🙂

Edit: actually found another way to do this using the Linq SelectMany method, but I’m not keen on this:

void TestAllV3()
{
    var tests =
        EnumHelper<Algorithm>.Values.SelectMany(
            algorithm => EnumHelper<ImageFormat>.Values.SelectMany(
                imageFormat => EnumHelper<ImageSize>.Values.Select(
                    imageSize => (algorithm, imageFormat, imageSize))));

    foreach(var test in tests)
    {
        Test(test.algorithm, test.imageFormat, test.imageSize);
    }
}

TimeSpan.FromMilliseconds rounding!

Today’s fairly brutal gotcha: TimeSpan.FromMilliseconds accepts a double but internally rounds the value to a long before converting to ticks (multiplying by 10000).

For example, using C# interactive in VS2017:

> TimeSpan.FromMilliseconds(1.5)
[00:00:00.0020000]

> TimeSpan.FromMilliseconds(1234.5678)
[00:00:01.2350000]

Using .FromTicks works as expected:


> TimeSpan.FromTicks(15000)
[00:00:00.0015000]

To be fair this is the documented behavior:

The value parameter is converted to ticks, and that number of ticks is used to initialize the new TimeSpan. Therefore, value will only be considered accurate to the nearest millisecond.

But really, it isn’t expected since the input is a double!

This all came to light because a camera system I’m involved with started overexposing –  the integration time was programmed as 2ms instead of the desired 1.5ms. Hmmph!

So a little alternative:

> TimeSpan TimeSpanFromMillisecondsEx(double ms) =>
    TimeSpan.FromTicks((long)(ms * 10000.0))

> TimeSpanFromMillisecondsEx(1.5)
[00:00:00.0015000]

 

Note: the FromMilliseconds method delegates to an internal Interval method, passing the milliseconds value and 1 as the scale:


private static TimeSpan Interval(double value, int scale)
{
    if (double.IsNaN(value))
    {
        throw new ArgumentException(Environment.GetResourceString("Arg_CannotBeNaN"));
    }
    double num = value * scale;
    double num2 = num + ((value >= 0.0) ? 0.5 : -0.5);
    if ((num2 > 922337203685477) || (num2 = 0.0) ? 0.5 : -0.5);
    if ((num2 > 922337203685477) || (num2 < -922337203685477))
    {
        throw new OverflowException(Environment.GetResourceString("Overflow_TimeSpanTooLong"));
    }
    return new TimeSpan(((long) num2) * 0x2710L);
}

 

 

VS2017 and NuGet for C++/CLI

At the time of writing it still isn’t possible to use the NuGet package manager for C++/CLI projects. My workaround is to:

  1. Add a new C# class library project to the solution.
  2. Add any NuGet packages to this new project.
  3. Configure the C# project so it always builds in Release configuration.
  4. Use the Build Dependencies dialog to ensure that the new C# project is built before the C++/CLI project.
  5. Add to the C++/CLI project a reference to the NuGet packages by using the output folder of the C# project.

Example

Create a new solution with a C++/CLI class library…

Add a C# class library (.Net Framework), delete Class1.cs, then go to the solution’s NuGet package manager:

2018-09-05 12_19_13-.png

Install the Newtonsoft.Json package for the C# project:2018-09-05 12_29_01-Solution3 - Microsoft Visual Studio.png

Change the C# build configuration so that the Release configuration builds for both Debug and Release:2018-09-05 12_31_24-Configuration Manager.png

Then delete the unused Debug configuration:2018-09-05 12_31_42-Configuration Manager.png

2018-09-05 12_32_14-Configuration Manager.png

Make C++/CLI project dependent on the C# project:2018-09-05 12_34_09-Solution3 - Microsoft Visual Studio.png

(Note: I use the above for this dependency rather than adding a reference to the project to avoid copying the unused C# project to the C++/CLI’s output folders.)

Build the solution.

Add a reference to the Newtonsoft library by using the Browse option in the Add References dialog and locating the C# project’s bin/Release folder:

2018-09-05 12_38_57-Select the files to reference....png

Build the solution again. The Newtonsoft library will now be copied to the C++/CLI build folder:

2018-09-05 12_40_59-Debug.png

First test: add some code to the C++/CLI class to demonstrate basic JSON serialisation:

#pragma once

using namespace System;

namespace CppCliDemo {

	using namespace Newtonsoft::Json;

	public ref class Class1
	{
	private:

		String^ test = "I am the walrus";

	public:

		property String^ Test
		{
			String^ get() { return this->test; }
			void set(String^ value) { this->test = value; }
		}

		String^ SerialiseToJson()
		{
			auto json = JsonConvert::SerializeObject(this, Formatting::Indented);
			return json;
		}
	};
}

Then add a simple C# console app, reference just the C++/CLI project, and test the class:2018-09-05 12_50_22-Reference Manager - CSharpConsoleTest.png

static void Main(string[] args)
{
var test = new CppCliDemo.Class1();
var json = test.SerialiseToJson();
Console.Write(json);
}

 

The output – nicely formatted JSON 🙂

2018-09-05 12_51_48-C__Users_Jon_Source_Repos_Solution3_CSharpConsoleTest_bin_Debug_CSharpConsoleTes.png

Second test, make sure a clean rebuild works as expected:

  1. Close the solution
  2. Manually delete all binaries and downloaded packages
  3. Re-open solution and build
  4. Verify that the build order is:
    1. CSharpNuGetHelper
    2. CppCliDemo
    3. CSharpConsoleTest (my console test demo app)
  5. Run the console app and verify the serialisation works as before

 

 

Raspberry Pi Desktop virtual machine

This weekend I discovered that there is a Linux distribution, Debian Jessie, that now has the Raspberry Pi Desktop, download from here.

So I installed it as a virtual machine on my Windows 10 PC using VMWare Professsional Pro 14. The only difficulty was getting VMWare Tools working to allow automatic screen resizing and file sharing. Found some good info on the VMWare communities site (search for vmware tools debian jessie). From this thread there is a great perl script to help install VMWare Tools. The only modification I made was to delete –default from the line that runs the VMWare installer, without which the script will suggest an alternative option and abandon the installation.

I made a YouTube video of my virtual machine installation and VMWare tools configuration:

 

The script for VMWare tools:

#!/bin/bash 
 
sudo apt-get update 
sudo apt-get upgrade 
echo "Do go and mount your cdrom from the VMware menu" 
echo "press any key to continue" 
read -n 1 -s 
mount /dev/cdrom 
cd /media/cdrom0 
cp VMwareTools-*.tar.gz /tmp 
cd /tmp 
tar xvzf VMwareTools-*.tar.gz 
cd vmware-tools-distrib/ 
sudo apt-get install --no-install-recommends libglib2.0-0 
sudo apt-get install --no-install-recommends build-essential 
sudo apt-get install --no-install-recommends gcc-4.3 linux-headers-`uname -r` 
sudo ./vmware-install.pl 
 
sudo /etc/init.d/networking stop 
sudo rmmod pcnet32 
sudo rmmod vmxnet 
sudo modprobe vmxnet 
sudo /etc/init.d/networking start

 

 

 

VMWare Windows 10 expand/partition problem

Intro

I had a Windows 10 VM, managed using VMWare Workstation Pro 12. The VM was originally created with the default 60GB hard disk.

I needed to expand the disk, so I shutdown the VM, removed all snapshots, expanded the virtual HD to 120GB, and rebooted the VM. The plan was to use Windows 10’s disk management tool to expand the original partition and merge in the new partition.

But the recovery partition was sandwiched between the original and new partitions, and couldn’t be deleted using the Disk Management tool:

Untitled picture.png

I found the basics of how to fix this on the VMWare knowledge base. I’m adding my procedure here because it includes some useful screenshots.

Steps

I ran diskpart to work with partitions. (I ran as admin, but don’t think it was required.)

1Untitled picture

From the DISKPART shell, I then used the following to select and then remove the unwanted partition:

DISKPART> list volume

Volume ### Ltr Label Fs Type Size Status Info
 ---------- --- ----------- ----- ---------- ------- --------- --------
* Volume 0 D DVD-ROM 0 B No Media
 Volume 1 C NTFS Partition 59 GB Healthy System
 Volume 2 NTFS Partition 450 MB Healthy Hidden

DISKPART> select volume 2

Volume 2 is the selected volume.

DISKPART> delete partition override

DiskPart successfully deleted the selected partition.

The Disk Management window showed the new partition layout:

2Untitled picture.png

Next, I right-clicked on the C: partition and chose ‘Extend Volume’:

4.png

567

At last, a single 120GB partition.

8

🙂