WCF HttpListenerException

A problem I’ve had for a while:

Running an application from Visual Studio 2013 without administrator privileges and trying to start a WCF service results in a HttpListenerException.

My service host’s URI was http://localhost:8000.

W8x64_2014

The problem goes away when Studio is started with Administrator:

W8x64_2014

But that’s a pain for me for a variety of reasons.

I googled and found lots of information on stackoverflow. I tried to use the developer-reserved Design_Time_Addresses solution on port 8732, and then on 8731, but to no avail.

So then I figured how to look for this URL on my PC. From a command shell run:

netsh http show urlacl

Then I spotted the design time addresses URL:

W8x64_2014

Port 8733 !

So I changed my URI to http://localhost:8733/Design_Time_Addresses and everything worked.

This new URI is only for use when running the service in a debugged session via Visual Studio. For normal runtime use I still use the original URI of http://localhost:8000.

Update: I’m now using the following property to get the URI at runtime:

static public string ServerPath
{
    get
    {
        string serverPath = "http://localhost:8000";

        if (System.Diagnostics.Debugger.IsAttached == true)
        {
            serverPath = "http://localhost:8733/Design_Time_Addresses";
        }

        return serverPath;
    }
}

OS X, Python 3.3, LiClipse

Yesterday I tried to get Eclipse and PyDev installed on my MacBook Pro (running Mavericks). After a couple of hours of struggling I gave up and discovered LiClipse which is a highly recommended system for working with Python and many other languages from the Eclipse framework.

I used a clean OS X Mavericks virtual machine to get my installation nailed down, especially important since I wanted to work with Python 3.x.

Downloads

Download and install:

  1. Python 3.x.
  2. Java 7.
  3. LiClipse: click the Google Drive link, select the latest version (currently 0.9.6), click and download the OS X file (currently liclipse_0.9.6_macosx.cocoa.x86_64.dmg), then install using all default options.

Configuration

Eject all download package (DMG) folders on the desktop.

Use Finder and go to Applications, open the liclipse folder, then drag the LiClipse app to the dock.

Run LiClipse. If a software update prompt appears for Java SE 6 then install it.

Accept all defaults when starting for the first time and restart the app.
 
Select File / New project, then select PyDev/PyDev Project:
ad45e741fc7fadfa78252bb606d55228

Hit Next, then set a project name and click the link to configure an interpreter:

489ed184ec9e6e8b074feccf177fcb78

Select the advanced configuration option:

ff2c2bbde406a0db49eae0d0d91f25e7

Then select python3.3:

32ca7a2249fdaa5c7ab1905a39b37dbb

On the next screen make sure all folders are selected:

378876332975796581d1a1bba04b66ee

Back on the new project screen make any other changes – I opted not to configure the Python path – then click Finish:

04b122222be512f58fe0f7fa062a8801

The Open Associated Perspective prompt may appear – I chose to associate the project with the PyDev perspective:

0a8180c02287ff3b076f068ffd8b3004

These Python and perspective settings will be used the next time LiClipse is started for Python projects.

Quick Python test

Following on from the above new I added a new Python file, entered a couple of lines of code, and ran it:

5b8f1d79fa162facf4e384b1394d0a99

132a82e84ba8b7066edba59927094d74

4a997671f4a90eee79e6c84d6342f354

638b29041e54c6d81427455cf5407130

feb22a04d91362315bae7493317c90ec

Debugging

If the debugger is used, for example by stepping through the code, there’ll be an option to switch to another perspective – this should be used for debugging:

a8a26c331c032792e467e96787890a55

This presents more window areas for examining variables, viewing the editor and console output, etc:

a9b4613df5986bd7bd9915660c715f15

After the debugging session is complete the Debug perspective will still be active. To switch back to the original perspective use the menus:
  • Window | Open Perspective | Other, then select PyDev.

Automated Dynamic DNS updating on Raspberry Pi

I use a Raspberry Pi to host SVN and Git repositories but don’t have a static IP with my BT Infinity account. To support remote connectivity I use a dynamic DNS service from Dyn.

The only feature missing was the ability to detect when the IP address had changed, and then to automatically update it.

So with a bit lot of Googling I’ve put together a Python 3 script which is run hourly via crontabThe script uses Dyn’s IP address utility to find the current IP address. This is compared to the last known IP address which I store in a text file. If the address has changed then Dyn’s Perform Update utility is used to sent the new IP address. Any changes or errors are emailed back to me.

Here’s the script:


#!/usr/bin/env python3

import re
import urllib.request
import os
import sys
import smtplib

# -----------------------------------------------------------------------------
# variable: lastIPAddressFileName
#
# The name of a file used to store the last committed IP address.
# -----------------------------------------------------------------------------
lastIPAddressFileName = '/var/tmp/last_router_ip'

# -----------------------------------------------------------------------------
# function: mailInfo
#
# Emails a message to one or more recipients. This should just be used for
# errors and successful changes of IP. (I.e. not used for regular no-change
# messages).
# -----------------------------------------------------------------------------
def mailInfo(info):
	#print('Sending email [{0}]'.format(info))
	from email.mime.text import MIMEText

	msg = MIMEText(
	"Hello from the Raspberry Pi DNS IP update utility.\n"
	"I have a new message for you:\n\n{0}".format(info))

	msg['Subject'] = 'CDS.RPi DSN IP Utility'
	msg['From'] = 'CDS RPi'
	msg['To'] = 'jon@carpediemsystems.co.uk'

	s = smtplib.SMTP('smtp.gmail.com', 587)
	s.ehlo()
	s.starttls()
	s.ehlo()
	s.login('jon@carpediemsystems.co.uk', '******')
	s.sendmail('jon@carpediemsystems.co.uk', ['jon@carpediemsystems.co.uk'], msg.as_string())
	s.quit()

# -----------------------------------------------------------------------------
# function: findRouterIP
#
# Returns the IP of the router by scraping the results of the DYN DNS check
# IP web site. None is returned if the address cannot be found.
# -----------------------------------------------------------------------------
def findRouterIP():

	ipAddress = None
	webAddress = 'http://checkip.dyndns.org/'
	siteData = urllib.request.urlopen(url = webAddress).read()
	siteAsText = str(siteData)
	ipAddresses = re.findall('\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}', siteAsText)

	if(len(ipAddresses) == 1):
		ipAddress = ipAddresses[0]

	return ipAddress

# -----------------------------------------------------------------------------
# function: dynDnsPerformUpdate
#
# Updates the IP address of a DynDNS host using Dyn's Perform Update request.
# The resulting web page is scraped and the function will return True if one
# of the valid results codes is seen (meaning the IP address was either updated
# or was not changed).
# -----------------------------------------------------------------------------
def dynDnsPerformUpdate(username, password, hostname, ipAddress):

	url = 'http://{0}:{1}@members.dyndns.org/nic/update?hostname={2}&myip={3}&wildcard=NOCHG&mx=NOCHG&backmx=NOCHG'.format(username, password, hostname, ipAddress)
	fancyOpener = urllib.request.FancyURLopener()
	siteData = fancyOpener.open(url).read()
	siteAsText = str(siteData)

	acceptableResults = ['good', 'nochg']
	isOk = len([r for r in acceptableResults if r in siteAsText]) > 0
	return isOk

# -----------------------------------------------------------------------------
# function: checkForNewIPAddress
#
# Finds the current router IP address and compares it to the last address
# saved to the lastIPAddressFileName file (if it exists).
#
# Returns a tuple:
#	Current IP address (string)
#	Is this a new and valid IP address (boolean)
# -----------------------------------------------------------------------------
def checkForNewIPAddress():
	lastIPAddress = None

	if(os.path.exists(lastIPAddressFileName) == True):
		with open(lastIPAddressFileName, 'r') as f:
			lastIPAddress = f.read()

	currentIPAddress = findRouterIP()
	isNewIPAddress = ((currentIPAddress != None) and (currentIPAddress != lastIPAddress))

	return currentIPAddress, isNewIPAddress

# -----------------------------------------------------------------------------
# function: saveNewIPAddress
#
# Sends the new IP address to Dyn DNS. This is also saved to the
# lastIPAddressFileName file if the web operation succeeded.
#
# Returns True if the operation succeeded.
# -----------------------------------------------------------------------------
def saveNewIPAddress(newIPAddress):
	username = '******'
	password = '******'
	host = '******.dnsalias.net'
	didUpdate = dynDnsPerformUpdate(username = username, password = password, hostname = host, ipAddress = newIPAddress)

	if(didUpdate == True):
		with open(lastIPAddressFileName, 'w') as f:
			f.write(newIPAddress)

	return didUpdate

# -----------------------------------------------------------------------------
# Script main
# -----------------------------------------------------------------------------

try:
	currentIPAddress, isNewIPAddress = checkForNewIPAddress()

	if(currentIPAddress == None):
		mailInfo('ERROR: failed to get current IP address')
	elif(isNewIPAddress == False):
		print('IP address is unchanged - nothing to do')
	else:
		saveNewIPAddress(currentIPAddress)
		mailInfo('IP address updated to {0}'.format(currentIPAddress))
except:
   mailInfo("ERROR: An unexpected error was detected in the script:", sys.exc_info()[0])

Note: the scripting presentation in this post was done by using WordPress.com’s sourcecode widget, as described here.

The script was saved to:

 /bin/DNS_IP_Util.py

and made executable with:

sudo chmod +x DNS_IP_Util.py

It was scheduled to run hourly by editing root’s cron file. The command to run the editor is:

sudo crontab -e

The cron file was edited to include the following two lines:

# DNS update, every day, hourly
0 * * * * /bin/DNS_IP_Util.py &

OmniGraffle 6: changing default settings

Changing the default settings in OmniGraffle 6 is straightforward – once you know the magic trick !!

The toolbar icons mainly have three states. Using the line tool as an example:

1: Unselected

20131104-141405.jpg

2: Selected (after clicking once) – in this mode the default tool settings will be used. Changes made via the inspector will apply only to the currently selected shape or line, etc.

20131104-141413.jpg

3: Defaults mode (after clicking twice) – in this mode changes made in the inspector will become the new defaults.

20131104-141418.jpg

OS X Pages 5.0 graphics in header

The latest version of Pages for OS X and iOS is great for iCloud users who want to edit documents on both desktop and mobile with maximum compatibility. But Pages for OS X has lost many features to attain this compatibility.

Yesterday I tried to create a new document with a company logo in the header, so that it would repeat on every page. The process wasn’t intuitive!

First off the Pages 5.0 manual is easiest to find via Pages, rather than Googling for it:

Fullscreen_01_11_2013_11_43-2

Then the trick is discovering master objects.

You can add text, watermarks, logos, or other images that appear in the same place in the background of every page of your document. These repeated elements are called master objects.

So here’s my sequence of events for getting an image into the document header so that it repeats on every page:

  1. New (blank) document.
  2. Turn on Layout View (⇧⌘L). This makes it a little easier to see where the header and body areas of the document are.
  3. Drag and drop the logo to the document body.
  4. Resize the logo and drag to the appropriate location in the header. Caution: if the resized logo fits within the header cell boundary it seems difficult or impossible to re-select it. I made mine slightly larger so that the mouse cursor could just click it.
  5. With the logo selected use the menu Arrange | Section Masters | Move Object to Section Master to make the logo a master object.
  6. If the logo needs to be repositioned use the Arrange | Section Masters | Make Master Objects Selectable to allow this and any other background objects to be selected.

Shapes for header/foot borders

This technique can also be used with shapes to allow the header and/or foot to have a full width border. For example, after setting up the header text try adding a simple shape to the document:

Untitled-2

Resize the shape and add any effects, move it below the header, use the menus again to make it a master object:

Untitled-2

When viewed as a PDF in Preview:

Untitled__1_page_

iPad

This is all possible on the iPad, give or take a few frustrating finger taps and Googling!

The user manual is here.

Subversion, per-repository access

These are my notes on how I configured per-user access rights for my Subversion repositories. There is an excellent article here to help with this.

I run three repositories (for three companies) and need access to all of them. Other users must have access only to the repository for their respective company.

1. Create an access control file called svn_access_control:

jon@raspberrypi ~ $ sudo nano /etc/apache2/svn_access_control

Enter information like this:

# jon belongs to all groups, paul and nigel to one group each
[groups]
TeamName1 = jon, paul
TeamName2 = jon, nigel

# Only admin has access to everything
[/]
admin = rw

# Setup access to a specific repository
[repos_1:/]
@TeamName1 = rw

# Setup access to a specific repository
[repos_2:/]
@Teamname2 = rw

Note: the user names in the above file are case sensitive.

Then edit the dav_svn.conf file and include a reference to the new access control file:

<Location /svn>
 DAV svn
 SVNParentPath /home/jon/svnrepos
 AuthType Digest
 AuthName "Subversion Repo"
 AuthUserFile /etc/apache2/dav_svn.htdigest
 AuthzSVNAccessFile /etc/apache2/svn_access_control
 Require valid-user
</Location>

The following command must be used to enable the new access control:

jon@raspberrypi ~ $ sudo a2enmod authz_svn

Restart the apache service to activate the changes.

Subversion Server on Raspberry Pi, part 3

Automated backup to a Dropbox folder

In parts 1 and 2 I documented how I setup a Raspberry Pi to provide Internet-based access to a Subversion server. The task here is to write a Python script that creates a hot copy of a repository, zips it, and sends to an appropriate place for archiving.

Note: after writing my own scripts I found some example scripts including hot-backup.py. This has more goodies than mine, including cleaning up excess (old) backups.

Python scripting notes: to allow a Python script to be ran directly from the command line set the first line to:

#!/usr/bin/env python

and then make it executable with:

jon@raspberrypi ~ $ sudo chmod u+x your_filename.py

The #! text on the first line is called the shebang and tells the shell what type of interpreter to use for the rest of the script. We’ll need to make system calls from the Python script – this site has lots of cool examples.

Dropbox via Python on the RPi

Note: using the Dropbox API requires a Dropbox account.

An application key is required to use the Python API. Go to the Dropbox app console, agree with Terms and Conditions, and select the following options:

D1
Click Create app. The following screen will appear and includes the application key and application secret strings which we’ll need in our Python scripts:

D2
Then download the Dropbox Python SDK:

jon@raspberrypi ~ $ wget https://www.dropbox.com/static/developers/dropbox-python-sdk-1.6.zip
 --2013-08-28 14:09:38-- https://www.dropbox.com/static/developers/dropbox-python-sdk-1.6.zip
 Resolving www.dropbox.com (www.dropbox.com)... 199.47.217.170
 Connecting to www.dropbox.com (www.dropbox.com)|199.47.217.170|:443... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 812016 (793K) [application/zip]
 Saving to: `dropbox-python-sdk-1.6.zip'
 100%[=======================================================================================>] 812,016 118K/s in 8.3s
 2013-08-28 14:09:53 (96.1 KB/s) - `dropbox-python-sdk-1.6.zip' saved [812016/812016]

Unzip the archive and cleanup:

jon@raspberrypi ~ $ unzip dropbox-python-sdk-1.6.zip
jon@raspberrypi ~ $ rm dropbox-python-sdk-1.6.zip 

Install the Python SetupTools package, then the Dropbox Python SDK :

jon@raspberrypi ~ $ sudo apt-get install python-setuptools
jon@raspberrypi ~ $ cd dropbox-python-sdk-1.6/
jon@raspberrypi ~/dropbox-python-sdk-1.6 $ sudo python setup.py install 

The Dropbox Core API page is useful in developing upload scripts. For this backup we need two scripts:

  1. A temporary script that requires operator interaction to generate a Dropbox access code. This access code will be used by the main script for authentication and is the key to writing non-interactive Dropbox scripts.
  2. A backup script that provides non-interactive repository archiving (via a hotcopy), compression, Dropbox upload, and caretaking.

First the access token: create a new script, db_get_access_token.py (and insert the actual app_key and app_secret strings). (For my scripts I created a new folder, svn_pi, underneath the home folder).

#!/usr/bin/env python
import dropbox
# Get your app key and secret from the Dropbox developer website
app_key = '*** insert ***'
app_secret = '*** insert ***'
 
flow = dropbox.client.DropboxOAuth2FlowNoRedirect(app_key, app_secret)
authorize_url = flow.start()
print '1. Go to: ' + authorize_url
print '2. Click "Allow" (you might have to log in first)'
print '3. Copy the authorization code.'
code = raw_input("Enter the authorization code here: ").strip()
access_token, user_id = flow.finish(code)
client = dropbox.client.DropboxClient(access_token)
 
print 'linked account: ', client.account_info()
print 'Access token ['
print access_token
print ']'

Run the script, copy the URL from the terminal window into a browser, then allow the new app to have access:

D3

D4

Copy the code and paste into the terminal session prompt: ‘linked account‘. If everything works as expected the ‘access token‘ information will be displayed. Otherwise something similar to dropbox.rest.ErrorResponse: [400] u'invalid_grant' will be displayed. Copy the access token string – this is needed to allow dropbox file transfers without operator interaction.

Then create the main backup script, svn_hotcopy_db.py. This will process all the repositories, for each one creating a hot copy, compressing it to a zip file, coping the zip to Dropbox, then finally cleaning up. Replace any *** string sections in the script with the appropriate information before running!

#!/usr/bin/env python

import datetime
import subprocess
import os
import dropbox
import shutil

homePath = '/home/***/'

def makeDateTimeStamp():
	now = datetime.datetime.today()
	nowAsString = now.strftime('%Y_%m_%d_%H_%M_%S')
	return nowAsString

def backupRepository(repoName, db_client):

	# Step 1: make hotcopy
	pathToSourceRepository = homePath + 'svnrepos/' + repoName + '/'
	pathToHotCopyRepository = homePath + 'svn_py/hotcopy/'
	cmdMakeHotCopy = 'svnadmin hotcopy ' + pathToSourceRepository + ' '  + pathToHotCopyRepository
	print cmdMakeHotCopy
	subprocess.call(cmdMakeHotCopy, shell=True)

	# Step 2: zip the hotcopy
	zipFileNameWithoutPath = repoName + '_' + makeDateTimeStamp() + '.zip'
	zipFileName = homePath + 'svn_py/' + zipFileNameWithoutPath
	cmdMakeZip = 'zip -r ' + zipFileName + ' ' + pathToHotCopyRepository
	print cmdMakeZip
	subprocess.call(cmdMakeZip, shell=True)

	# Step 3: send zip file to Dropbox
	print 'Sending zip file to Dropbox'
	with open(zipFileName, 'r') as zipFile:
     	db_client.put_file(zipFileNameWithoutPath, zipFile)

	# Step 4: cleanup
	print 'Cleaning up'
	os.remove(zipFileName)
	shutil.rmtree(pathToHotCopyRepository)

# Configure dropbox access
db_access_token = '***'
db_client = dropbox.client.DropboxClient(db_access_token)

# Setup list of repositories to dump/zip/backup
repositories = ['***', '***', '***']

# Run !
for repository in repositories:
	backupRepository(repository, db_client)

The backup can be scheduled to run automatically. First edit the schedule for the current user:

jon@raspberrypi ~ $ sudo crontab -e

Edit the file and include the following line:

0 2 * * * /home/jon/svn_py/svn_hotcopy_db.py &

Notes:

0 2 * * *: Run at 02:00 every day.
/home/jon/svn_py/svn_hotcopy_db.py: script to run
&: run in background