Experience
March 2012 - present: Software Developer, Redgate Software
Redgate makes software tools, mainly for SQL Server database administrators. It is also by far the best team environment I have worked in, with a strong emphasis on being reasonable which encourages honest feedback within the company.
I've worked in teams of 2-12 people on many Redgate products and services over the years, with my role as a developer growing to include setting the technical direction of the team. I write C# every day but also PowerShell, T-SQL and occasionally C++.
I am also leading the company toward automated releases, so our teams can release products more frequently and with fewer errors during the process. I led a team of 10 developers to write the original implementation, which has now grown and been adopted by most product teams within the company.
Products I worked on at Redgate
SQL Compare and Data Compare engine
Redgate's flagship product is the basis of half the business: the ability to safely change the schema of a database is used not just by the desktop tool, but by SQL Source Control and dozens of our other products. Reliablilty is extremely important because an error risks destroying a customer's mission-critical data.
Unfortunately, Compare is very difficult to test because it works by creating a script that is run by another piece of software, Microsoft SQL Server. Unit tests can verify that the script matches our understanding of what should be done, but you can't be sure the script will succeed without actually running it. This led to massive integration test suites that ran overnight on dozens of machines, with hundreds of random failures per night.
We came up with a solution we called Record/Replay, which allowed us to reuse the values returned from a series of database queries to mock out the state of the database, so that these scripts didn't have to be run every time. We reduced the test run time to 20 minutes, breating new life into the company's staple product.
- A Lightning talk I gave at the end of the project
- Featured on Simple Talk
SQL Compare and Data Compare desktop apps
One consumer of SQL Compare Engine is of course SQL Compare UI. I helped rewrite the user interface for version 12 of the UI, and have been on the team maintaining it ever since.
SQL Source Control
Another consumer of SQL Compare Engine, this tool allows a user to store their database schema in version control. I worked on integrating SQL Lock into the product, a feature which helps users to avoid stepping on each others' changes.
SQL Backup
Rumours of its death had been greatly exaggerated back in 2012. Microsoft was starting to offer comparable database backup features that should have made SQL Backup obsolete, so product management had reduced the team expecting the product to die. But demand stayed high, and we were brought in to perform the resurrection. The product is still going strong today, and I write bug fixes for it from time to time.
SmartAssembly
{SA} has some tricky bugs to diagnose, because the product itself can complete without error messages but produce an executable with corrupted obfuscation. We only had a week to work on this product, but managed to make a noticeable dent in the number of bug reports, in addition to modernising the version control, build and test infrastructure.
I have since ended up maintaining some of the server-side error reporting infrastructure.
.NET Reflector
Reflector's integration into Microsoft Visual Studio means that any error reports are sent to Microsoft, not directly to Redgate. Unfortunately we only see a tiny minority of these reports: we rushed out a release in November as soon as a batch of reports were forwarded to us, and will release again when we can get the remaining information we need.
ANTS Memory Profiler
Why is my program using so much memory? This tool will help you find out. The UX angle is really interesting here, because .NET memory management is one of those very complex things you usually don't have to worry about. Collecting the data is only half the problem: you also have to explain what is preventing an unneeded object from being "garbage collected".
Another issue is the large number of crashes people experience when profiling. Many of these are Out of Memory errors, which is to be expected since most of our users are diagnosing such problems in their code! But we wanted to know more, so we added unmanaged error reporting to the C++ profiler cores, which allowed us to fix many of the crashes.
Web Technologies Team
Redgate is supporting Glimpse, an open source tool to visualise the performance of your web application from within the browser. We produced a prototype that used the Performance Profiler to show individual methods running on the webserver, so that the user could drill even further into what happened during a request.
ANTS Performance Profiler
This awesome tool just makes my jaw drop every time I use it. Profile even the simplest "Hello World" application and you see the millions of methods your computer ran to display that simple phrase. The profiler core was my first major C++ project, and I learned about IL and .NET internals as well as the balance between performance and readability as we added support for profiling incoming and outgoing HTTP requests. I also learned WPF as we rewrote the user interface.
Cloud Services
I joined Redgate as a Software Engineer with no previous experience of professional coding, or development in a Microsoft environment. I had to learn the technology stack fast:
- C# and ASP.NET MVC projects using Visual Studio and ReSharper
- NInject and Unit Testing with Moq
- JSON and JQuery
- Database manipulation with SQL, as well as with Redgate's own database tools such as SQL Compare and Data Compare
- Cloud APIs, mainly Amazon and Azure.
Down Tools Week Projects
I've also worked on many "Down Tools Week" projects:
- {SA}Crunch: Many products have a long list of unsolved crash reports. {SA}Crunch highlights every line of code that appears in the stack traces in the bug database, using git blame to account for differences between the version in the editor and the version that crashed.
- Nuget Updater: SQL Compare has a very deep set of project dependencies: Data Compare UI uses Compare UI which uses ToolsUI, which uses Compare Engine which uses the AST Parser... you get the idea. Each of these packages is versioned, which creates a dependency hell where two dependencies can depend on different versions of a third dependency. The nuget UI was failing to resolve these problems, so we wrote our own updater utility.
- Test slot results history: To investigate and demonstrate the severity of the testing problems in SQL Compare, we scraped the TeamCity logs and created a huge bitmap of test results sorted by fixture, branch and date. The result is a sort of tartan pattern, with horizontal red lines for consistently failing tests, vertical red lines for nights when the infrastructure fell over and a red and green haze for random failures elsewhere. This result provided some of the justification for the Record/Replay project.
- SQL MultiScript: This long-neglected product is basically a for loop that runs the same script on multiple servers. Simple though the concept is, the user interface was in need of some bugfixes and minor features.
- Cloud dev environment: A php-based prototype of a hosted development environment with continuous deployment. Whenever your unit tests passed, your changes were immediately deployed to production. The concept worked remarkably well, although the paradigm meant that any feature without excellent test coverage was almost certain to break in production.
- Why is threading hard? Our profilers have a design flaw that means various UI components tend to throw ObjectDisposedException when ending a session. I spent a week looking for answers, and another week writing a ReSharper plugin to perform static analysis on the codebase and highlight the issues. While this adequately explained the problem, I never did find a good solution.
- Computer Says No: In the second week of my job, I participated in a project to automate the recruitment process! The idea was that a prospective developer would submit some code, which would be assessed automatically by running some tests against it, with the amusingly-named "tactful delay" before an invitation or rejection letter was sent to the applicant. Needless to say this project was canned, partly due to the ethical concerns and partly because the only technical recruiter left HR, leaving nobody with time to maintain the system.
November 2006 - March 2012: Graduate Assistant to Stephen Hawking
For Professor Stephen Hawking, I spent 5 years as the in-house engineer, supporting his unique and demanding lifestyle by updating and maintaining the wheelchair and speech system, and planning most of his international travel.
- I designed, built and fitted a new computer to the wheelchair.
- I redesigned his Infrared Cheek Switch to be smaller and lighter than the previous model, and to compensate for varying ambient light levels.
- In order to reverse-engineer his iconic hardware voice synthesiser, I built Arduino-based boards program and run code on its DSP chip.
- I specified and ordered medical equipment, purpose-built and customised proprietary wheelchairs and a new car adapted specially for Stephen.
August 2010: The New Computer
The design consists of a ThinkPad X61 Tablet in an
aluminium shell, a directional speaker array hidden
in the headrest, and a number of power and USB devices
mounted on the back of the wheelchair.
The Intel machines had generally included an "umbilical" cable, a thick custom-built cable running from the screen to a box on the back of the wheelchair. This cable tended to develop loose connections, and it was very difficult to repair. I therefore decided to use standard parts wherever possible, so that if (for example) a USB cable failed, it could be replaced with another ordinary USB cable. The laptop itself can be swapped out in a few seconds, in contrast to the Intel machines, which had a large number of wires soldered directly to the motherboard.
Intel provided an SSD, which improves performance and makes the unit a great deal more shock resistant. This is very important, as the computer takes the full force every time his 142kg chair is driven into a doorframe!
March 2007: The Infrared Cheek Switch
Stephen originally operated his computer using a hand
switch, but as his disease progressed, the muscles
furthest from his brain became extremely difficult to
control. His movements became slower, and the amount
of involuntary movement began to exceed his voluntary
movement. In 2005 he moved to an infrared-based sensor
made by Words Plus,
which increased his talking speed greatly but had a
number of downsides. The sensor itself was extremely
bulky and heavy, and the weight tended to pull
Stephen's glasses down his face. It also needed
recalibrating whenever the ambient light level changed.
I redesigned the device shortly after I started working for him. My design is far smaller and lighter, and can compensate for ambient light. The sensor itself is actually very homemade: the LED and photodiode are superglued to a paperclip, which is soldered to Stephen's glasses.
May 2010: The Famous Voice
Stephen's famous voice is produced with a hardware
voice synthesiser, of which we have 2 copies: one on
his chair, and one in my office. Modern synthesisers,
which work in software, sound more intelligible
and arguably more natural, but Stephen's voice defines
his identity, and he doesn't want to change it.
I initially tried working with a Cambridge company, which usually produces the synthesisers for computer games, to recreate this distinctive voice in software. I recorded over 7,000 sentences using the hardware voice synthesiser, and they trained a statistical model using that data. The result is very close to the original, but not up to Stephen's exacting standards!
I then started working to reverse-engineer the original circuit, in order to emulate it in software and produce results that would be bit-for-bit identical to what is currently sent to the DAC. I built an Arduino-based board to run code on a DSP chip, so that the vague and often contradictory parts of the datasheet could be clarified.
The project was ultimately succesful thanks to some colleagues who continued the work after I left, but the working system was presented to Stephen only a couple of months before he sadly passed away.
January 2008: The Difficult Task of Choosing a Wheelchair
It is remarkably difficult to find a wheelchair that meets
Stephen's needs.
Standard powerchairs are comfortable and have all the adjustment Stephen needs, but they are heavy (his current one weighs 142kg) and extremely difficult to dismantle. This is problematic for air travel because Stephen needs to take his wheelchair all the way to the gate of the plane; airport wheelchairs simply don't have sufficient neck and back support.
There are lighter chairs which can be dismantled easily, but these have no adjustment features. This means that Stephen must always be in a low enough position that he can enter a car without hitting the ceiling, and that height is unhelpful for a lot of other activities.
The only other alternative is a heavily customised chair, which fulfils these criteria but is less reliable, because a complex one-off machine can't go through as much testing as a standard product.
2006: Data Analysis, Hampstead Garden Suburb Trust
I reconstructed the financial history of the charity, in order
to get to the bottom of of a series of long-lost accounting
mistakes. The resulting spreadsheet retraced 30 years of
conflicting and incomplete records, revealing about £1m had been
mis-spent! In another task, I cleaned up a database of properties,
then wrote a script to look up about 5,000 tax band records
from the council's web interface, before analysing the results.
2002 - 2006: Acoustics Consultancy
- At Peutz & Associates in Nijmegen, Holland, I prepared and tested several twelfth-scale acoustic models of concert halls. Everything was to scale, including the sound waves, so ultrasonic sources and microphones were used. You can see the reports I wrote here.
- At Kirkegaard Associates in Chicago, I constructed computer models of concert venues and set up an ambisonic speaker system.
- At Lee Cunningham Partnership in Reading, I carried out noise surveys, commissioning tests and desktop analysis, including BB93 markup (acoustics in schools) and report writing.