Gert Lombard's Blog     About     Archive     Feed
My passion is code quality, automated testing, software craftsmanship.

Keep it simple

I was browsing around on the Quantified Self website and saw these nice easy tips / lessons learned on QS, but clearly these are 100% applicable to software development (and practically any aspect of life). They are pretty obvious points, but worth reminding yourself of now and then, especially when you're as easily distracted as I am :-)
  1. Something is better than nothing. Engaging yourself in some experiment, no matter how flawed it may be, is better than never starting. The best way to learn is to do. So go out and do something!
  2. When you decide to start something try and do the simplest thing that you think might give you some insight. It’s great to have ambitious ideas, but keeping it simple ensures your experiment is manageable.
  3. Mistakes are worthwhile. Some of our best knowledge comes from learning from our failures so don’t be afraid of failing. By keeping it simple you also keep the mistakes small and manageable.
  4. Seek help from others.

Ireland English dictionary for the non-Irish

Random musings on living in Ireland:

As a South-African who's been living in Ireland for over 4 years, let me show you some of the Irish lingo (I think) I've learned so far. Please feel free to add/correct as necessary.

First the Irish-English phrase and then the "international" English phrase would be understood by South-Africans and Americans alike (I would hope).
  • It's half eight = It's half past eight.
    • This one I especially used to find confusing because in Afrikaans the direct translation "half agt" actually means the equivalent of "half to eight" i.e. "half past seven".
  • What's the craic? = Hey, what's up?
  • What's the craic and all? = Hey, what's up? (as said by an Indian living in Ireland - hi Ramesh :-)
  • Just call in to pick it up. = Just come in and pick it up. (I still keep falling for this one and actually CALL people on the phone, only to discover I didn't have to call but only have to call in, duh!)
  • evening = afternoon
    • example: Yesterday evening I worked on...
    • translation: Yesterday afternoon I worked on...
  • Did you enjoy that dinner? = Did you enjoy that lunch? (I think 'supper' is reserved for 'dinner'..?)
  • Do you want to come over for tea? = Do you want to come over for dinner? (Not sure about this one...?)
  • Runners = tekkies (South-African), sneakers (American)
  • Pants = underwear / underpants
  • Cheers! = Thanks! (Not 'bye'!)
  • So I told yer man = So I told the guy
  • Jumper = Jersey
  • Are you dosed? = Do you have the flu?
  • I'm wile dosed = I'm very sick
  • Bap = bun (bread)
  • At the minute = At the moment
I admit that some of these might not necessarily be purely Irish words/phrases but may be influenced by British-English, which is possibly slightly closer to South-African English, but not quite. I'd say South-African English is, when spoken, perhaps the closest to US-English (but closer in spelling to UK-English).

Also, just a silly story to add:

I've been in love with Ireland my whole life, so on previous occasions (before moving to Ireland) I've bought some Irish wool caps and a very nice (and expensive I want to add) jersey. Then we moved to Ireland in 2009. I got a little book about Ireland and Irish culture. In this book I learned about the concept "More Irish than the Irish"... I realised that if I wore my Irish-made wool cap, which I bought in the same county that I live in now, but years ago (around 2000 or so), then I'll just seem like an idiot tourirst who's trying to look Irish. For the same reason I'm reluctant to wear any Irish-branded stuff (like a nice Ireland t-shirt I also bought years ago) for the fear of looking like a silly foreigner trying to look Irish :-)

I must admit, occasionally when I visit a neighbouring town in Donegal, and a shop-keeper doesn't understand my weird accent, it is convenient to pretend that I'm just visiting from SA. It does get weird when you keep pretending to be a visitor from SA every time you visit the same store though (like that from the lovely Mr. McGinty's in Donegal Town - hi Mr. McGinty, I love your shop, but I've bought lots of stuff from your shop over the years and I expect a discount next time... haha ;.)

Sometimes when people in Letterkenny don't understand this crazy unfamiliar accent, they assume that I'm Polish. Not sure why. In the US people thought I'm either Scottish or Australian...?

I hope this post doesn't cause any offense to the occasional Irish or Western-European visitor :-)

The correct way to read a text file in Python

When you search for examples of reading a file on the web, most examples you see show:

with open('file') as f:
text = f.read()

But it's quite hard to find examples of the "correct" or "proper" way to read text files in Python. Of course correct/proper is a matter of opinion and depends on what you're trying to accomplish. In many cases you really want to slurp the full file into memory as above, but in my case I more often need to process one line at a time (perhaps in conjunction with a generator), in which case this is what I need:

with open('file') as f:
for line in f:
yield line

Not to mention that you'd deal with binary files in a different way as well...

Automated install of PowerShell on Windows XP

The problem: I want a batch script to perform an unattended/silent install of PowerShell on Windows XP.

We need to download the PowerShell installer from Microsoft, but it needs to be downloaded from an HTTP server. I'm not aware of an easy way to download files from the command-line or script in Windows XP, other than the FTP command. But what we can do easily, is to download cURL via FTP first.

So we need to download 4 things from the batch script:
  1. We need to get 7-Zip to extract cURL.
  2. We need to get cURL (or wget) to download PowerShell.
  3. Download and install .NET Framework 2 since it's required by PowerShell.
  4. Finally we download PowerShell itself from Microsoft via HTTP.
Step 1: download 7-Zip archiver's command-line version (7za.exe) from an FTP server. You need to search for any FTP server that hosts 7za.exe.

This FTP script would work to get 7za:
open ftp.cadwork.ch
user
anonymous

cd /DVD_V17/CADWORK.DIR/COM/
bin
get 7za.exe
Step 2: download curl from one of the official FTP mirrors if you can. I didn't have any luck with the official mirrors so I just searched around until I found an older version.

This FTP script would work to get curl 7.19.3: 
open ftp.gr.freebsd.org
user
anonymous

cd /pub/net/ftp/curl/
bin
get curl-7.19.3-win32-nossl.zip
Now we can use 7za.exe to extract the curl zip file.

Step 3: download .NET Framework 2.0 and install it silently:
curl -O http://download.microsoft.com/download/c/6/e/c6e88215-0178-4c6c-b5f3-158ff77b1f38/NetFx20SP2_x86.exe
start /wait "Installing .NET" NetFx20SP2_x86.exe /qb
Step 4: download PowerShell for Winfows XP and install it silently:
curl -O http://download.microsoft.com/download/E/C/E/ECE99583-2003-455D-B681-68DB610B44A4/WindowsXP-KB968930-x86-ENG.exe
start /wait "Installing PowerShell" WindowsXP-KB968930-x86-ENG.exe /passive /log:powershell-install.log
The final batch file to put it all together is here.

If you're curious why I'd be using Windows XP: I sometimes need to test old software. In this case, I want to install Visual C++ 6 to see if some old C++ code compiles without warnings before I start the process of converting it to a newer Visual Studio.

Note: I used FileWatcher.com to search for FTP servers that host the 7-Zip command-line tool (7za.exe) and cURL.

Automating software installs

This is my favourite software development topic: automating installs.

I'm a lazy programmer, in the sense that Larry Wall talks about when he says laziness is one the three great virtues of a programmer: that means I hate doing things manually over and over again. I'd rather spend an hour writing a script to download and install a utility, then doing it manually many times on multiple machines. Automating the installation of tools also has some other benefits: (a) it ensures the consistent replication of environments and (b) it serves as documentation of exactly what was done for future reference.

Nowadays I make a lot of use of virtual machines (mostly using Hyper-V recently) for my development on Windows. This allows me to easily delete VMs and start over as often as I like. It also means I can keep my development environments separate: e.g. one VM for Java/Eclipse, one for an older version of Visual Studio, one for playing with Ubuntu, etc.

When you install a lot of development environments over and over, you need to be aware of the different tools available to help automate things. In the Linux world, automated/unattended installs are the norm (using things like apt-get), but in Windows it takes a bit more effort to automate the installation of applications.

Here's a few ideas for installing utilities/tools from the command-line (i.e. from scripts):
  • Chocolatey - very cool tool to install lots of popular applications with a simple command like: cinst notepadplusplus
  • Ninite - Pick the applications you want to install and it generates a single installer to package them all.
  • DISM.exe - a Windows tool to Enable/Disable Windows Features from the command-line. Use this to enable things like IIS or Hyper-V from a script.
  • WebPICmd.exe - Microsoft Web Platform Installer - install certain products like SQL Server Express from the command-line.
  • In some cases it's easy enough to just write your own customer silent installer for a product. A lot of applications use InstallShield or other well-known installers and a lot of them behave the same way, so it's usually possible to figure out which command-line parameters to pass to the installer to do an unattend/silent install.
In addition to the above tools, you obviously need to be comfortable writing scripts of some sort. In the old days, I used to use Perl for all of my scripting needs. In recent years I've done more plain Batch files and some PowerShell. PowerShell is great because it's so ubiquitous on Windows systems, but it does have a steep learning curve in my opinion. I'm currently trying Python as my primary scripting language instead of plain old Batch files.

Using Chocolatey:

As an example of using Chocolatey on Windows, let's install the first two tools every developer needs on their machine: cURL and the 7-Zip Command Line tool for extracting archives from the command-line:

First, install Chocolatey itself (requires .NET Framework 4.0 to be installed) and then we use cinst to install curl and 7za:
C:\> powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%systemdrive%\chocolatey\bin
C:\>
C:\> cinst 7zip.commandline
C:\> cinst curl
And that's it!

Using Web Platform Installer (webpicmd):

Let's install webpicmd with Chocolatey and then we'll install IIS Express using webpicmd:
C:\> cinst webpicmd
Now list all packages matching "IIS":
C:\> webpicmd /List /ListOption:Available | findstr "IIS"
From the above we see that we're looking for the package called just "IISExpress", so install it:
C:\> webpicmd /Install /Products:IISExpress /AcceptEula
Using PowerShell to install utilities:

Now, for contrast, let's say you've got a clean Windows 7 machine and Chocolatey won't work yet (because the .NET framework ins't installed), then it's easy enough to roll your own automated installs of 7-Zip and cURL using PowerShell. Once we have 7za and curl, this bootstraps everything: next we can easily script the download and install .NET Framework and Chocolatey...

To download 7-Zip command-line version (7za) and extract it using PowerShell:
function Unzip-File
{
param ([string]$zipFile, [string]$destFolder)
New-Item -ItemType Directory -Force -Path $destFolder
$shell = New-Object -Com Shell.Application
$zip = $shell.NameSpace($zipFile)
$shell.NameSpace($destFolder).CopyHere($zip.Items(), 16)
}

Write-Host "Installing 7-Zip Command Line (7za)..."
$dest = "$env:ProgramFiles\7-Zip"
$destFile = "$env:ProgramFiles\7-Zip\7za.exe"
if (!(Test-Path $destFile))
{
Write-Host "Downloading 7za..."
$wc = New-Object Net.WebClient
$url = 'http://downloads.sourceforge.net/sevenzip/7za920.zip'
$file = "$env:TEMP\7za920.zip"
$wc.DownloadFile($url,$file)
Write-Host "Unzipping to: $dest"
Unzip-File $file $dest
Remove-Item $file
}
else
{
Write-Host "7za already installed."
}
Next, let's download cURL and copy it into the Windows directory (because it's a single file and then it's already in the PATH for easy use):
function Unzip-File
{
param ([string]$zipFile, [string]$destFolder)

$7z = "$env:ProgramFiles\7-Zip\7za.exe"
if (Test-Path $7z)
{
$info = New-Object Diagnostics.ProcessStartInfo
$info.FileName = $7z
$info.Arguments = "x -y -o""$destFolder"" ""$zipFile"""
$info.Verb = "runas"
$proc = [Diagnostics.Process]::Start($info)
$proc.WaitForExit()
}
else
{
$shell = New-Object -Com Shell.Application
$zip = $shell.NameSpace($zipFile)
$shell.NameSpace($destFolder).CopyHere($zip.Items(), 16)
}
}

Write-Host "Installing cURL..."
$wc = New-Object Net.WebClient
$url = "http://www.paehl.com/open_source/?download=curl_732_0_ssl.zip"
$file = "$env:TEMP\curl.zip"
$wc.DownloadFile($url,$file)
Write-Host "Unzipping to: $env:windir"
Unzip-File $file $env:windir
Remove-Item $file
Now download the .NET Framework 4.5.1 using cURL (of course you can download it with PowerShell as well, but I like to use cURL):
C:\temp> curl -L http://go.microsoft.com/fwlink/?LinkId=322116 -o c:\temp\NDP451-KB2858728-x86-x64-AllOS-ENU.exe
C:\temp> NDP451-KB2858728-x86-x64-AllOS-ENU /q
Resources: