Passwords – for wannabe techies

EDIT: As you will notice a lot of my links point to the works of security researcher Troy Hunt, and I should point out that I have no affiliation with him. I just happen to find articles that make sense and they happen to be written by him or refer to his work. Anyway here is another one, addressing the spirit of this blog post, with horrifying examples.

If you google for “ASP.NET login form” or similar, you will get some hits with really atrocious examples of how to NOT handle peoples’ credentials. Even if you are a beginner writing an app that nobody will use, you should never do user login in a shitty way. You will end up embarrassing yourself and crucially your users, who most likely are friends and family when you are at the beginner stage, when – inevitably – your database gets stolen.

How do you do it properly then, you ask?

Ideally – don’t. Let other people worry about getting hacked. Plenty of large corporations are willing to take the risk.

Either way though,  get a cert and start running HTTPS only.  HTTP is fast and nice for sites that allow anonymous access, such as blogs, but as soon as you are accepting sensitive data such as passwords you need to go with HTTPS.

If you are in the Microsoft sphere, just create a new web project in Visual Studio, register yourself as a developer with Google, Facebook, Microsoft or similar, create apps there, and configure those app credentials in the Visual Studio app – and do make sure you store those app secrets outside of source control – and run that app. After minimal tweaking you should have something working where you can authenticate in your app using those authentication providers. After that –  you can then, with varying effort required, back-port this authentication to whichever website you were trying to add authentication to.

But I insist on risking spreading my users’ PII when I get hacked

OK, fair enough, on your head be it.

Separate your auth storage from your app data storage. At least by database connection user rights. When they hack your website using SQL-injection, they shouldn’t be able to get hashed passwords. That just isn’t acceptable. So, yes, don’t do Windows Authentication on your dev box, define SQL Database Users and SQL Server Logins and make scripts that ensure they are created if they don’t exist.  Again, remember to keeps secrets away from the stuff you commit to version control. Use alternative means to store secrets for production. Azure will help you with these settings in the admin interface, for instance.

Tighten the storage a bit

Store hashed password and the salt. Set your storage permissions such that the password hash cannot be retrieved from the database at all. The database login used to access the storage should not have any SELECT rights, only EXECUTE on stored procedures. That way you write one stored procedure that retrieves the salt for a login so that the application can calculate a hash for the password supplied by the user in trying to log in,  then another stored procedure that takes a username and the hash and compares it internally to the one stored in the database, returning back to the caller whether or not the attempt was successful, without ever exposing the stored hash.

Note that if you tighten SELECT rights  but don’t separate storage between auth and app, every single Entity Framework sample will crash and burn as EF requires special administration to use stored procedures rather than direct CRUD.

Go for a stupidly complex hashing algorithm

Also – you are not going for speed when you are looking for password hashes. MD5 and SHA1 are really nice for checksumming files, but they suck for passwords. You are looking for very slow and complex algorithms.

.NET come with Microsoft.AspNet.Cryptography.KeyDerivation.Pbkdbf2 , but the best and most popular password hashing algorithm is bcrypt.

The hashes generated are of fixed size, so just define your storage to be big enough to take the output of the algorithm and then there is no need to limit the size of the password beyond the upload connection limits defined by the web server due to DoS protection, but for a password, those limits are ludicrously high, so you don’t have to even mention them to the user. Also, don’t mess with copy/paste in the password box. You want to be password manager friendly.

Try and hack yourself

Use Zed Attack Proxy or similar to try and break into your site. It will tell you some of the things you need to change in terms of protecting against XSS and CSRF. The problem isn’t that your little site is going to be the target of the NSA or the Chinese government, what you need to worry about is the tons of automated scripts that prod and poke into every site everywhere and collect vulnerabilities with zero effort from the point of the attacker. If you can avoid being vulnerable to those most basic attacks, you can at least have some self-respect.

In conclusion

These are just the basics, and I am mostly writing this to discourage people from writing login forms as some kind of beginner exercise. Password reuse is rampant still, so if you make a dodgy login form you are most likely going to collect some userid / password combinations people really use at other sites, and those sites may very well be way more important than yours. Not treating that information seriously is extremely unprofessional and bad karma. Please do not build user authentication yourself if you don’t intend to make some kind of serious effort in protecting people’s data. Instead use OAuth solutions and let people authenticate using Google, Facebook, Twitter, Microsoft or whichever auth provider you prefer. That way you will never see any passwords and won’t have anything that can be stolen, which is a much easier life to lead



Passwords. For non-techies

Passwords are a – for the most part – necessary evil in our connected lives. There has to be a way to know that you are you in a transaction, and a piece of information that only you and your counterpart know about seems perfect for that.

You have indubitably been told many times you should never reuse passwords and you should get a password manager. Guess what – that is correct. When you get one, and it proceeds to chastise you about your instances of password reuse and you use the magnificent automated features to reset passwords, you will notice that many sites not only have minimum requirements for length and complexity – they also have maximum limits. This is where alarm bells should ring in your mind.

Passwords shall only be stored as hashes (what people used to call checksums back in my day) because it is mathematically impossible, well, to deduce the original from the hash. This is not enough to be safe, of course, as evildoers quickly figured out that they could hash (using all major algorithms) enormous dictionaries and lists of popular passwords and then had prefab hashes to compare to ones they found in hacked databases. You need to hash several times over and apply salt to the hash and so on in processes that are better described elsewhere, but the point is – it starts out with a checksum. A checksum is always of constant length. So it doesn’t matter if you submit the Holy Bible as a password, the hash will be the same size.

This means, the only reason people put a size limitation on the password is because they are storing it in clear text in the database, or otherwise process it in clear text.

When you complain about it to customer services they will say either of these things:

  1. We are storing passwords encrypted securely in accordance with industry standards
  2. You don’t understand these things – clearly you must be new to computers.
  3. We need to limit password length for security reasons
  4. We take the security of your data very seriously.

All of that is bollocks. The only reason other than storing data in plain text (or storing it encrypted, so that it can be easily displayed in the admin interface which still also VERY BAD, see below for further rant) could possibly be a misguided attempt at usability, where the UI might limit the text field length to prevent the user from getting lost, but I still would say it is 99.999% likely that it is bollocks and they are showing your password to everybody. At least that is the safest assumption. .

So they know my password –  what is the big deal?

OK, so why does that matter? Well, probably it means that tech support or custom service people can see your password, so you can’t use the one that is basically a bunch of rude words strung together or it is going to be awkward on the phone.

Also, it means that if they have one mischievous intern or employee that copies some passwords from the system, they can really do harm to any site which you STUPIDLY used the same password for. Also, if somebody manages to steal one login to the administrative interface they have your password. In a secure system, that level of data breach should take a little more effort.

I have read somewhere that the far biggest problem with weak password security is password reuse, and you can end that today, for yourself, by using a password manager of some kind. Preferably a good one, but I’m no security person so I couldn’t tell you which one to go with. There is though a strong eggs-in -one-basket aspect to this, but given the scale of breaches I don’t see any other way forward.


There are plenty of password jokes out there, like the one with “User enters password ‘docGrumpyHappySleepyBashfulSneezyDopeyAlbany’, requirement was seven characters and a capital” or the classic XKCD password complexity suggestion correcthorsebatterystaple as an example of a secure password that was also easy to remember.

Since early 2000 Windows Servers have been pestering users to use passwords with capitals, numbers and special characters and a minimum password length of 8, and also to change passwords continuously.  

A few things have happened. Graphics hardware and research. Graphics hardware has made it possible for somebody ambitious to buy (or steal, yes) a bunch of graphics cards and use the phenomenal 3D processors to brute-force compare password hashes (yes, even correctly stored passwords) with hashes of dictionary words and popular passwords and common v@r1ati0ns on them. The rate at which these comparisons can be done is only slowed down slightly if you use a very complex hash, but even with that, the enormous compute power these people can muster makes short passwords trivial to reveal. Yes, that includes correcthorsebatterystaple. To just refer back to the case where reversible encryptions are used to store passwords to that customer service/tech support can read them – yes, those encryptions are extremely easy to decrypt with this kind of firepower. Essentially, there is no difference between an encrypted password and a plain text password in terms of security.

The best passwords are the auto-generated ones from a password manager that also can give you variable length, where you might as well crank it up as far as it’ll go, but of course, you’ll quickly find out many shady websites that won’t let you use a secure password.

Also, regarding changing passwords – The more esoteric password complexity requirements are and the more you make people change passwords, the worse their passwords get, because although I’m sure this surprises security people I must mention that normal people have to work too, they can’t spend all their time fiddling with passwords. Of course cheating by using a format that gets incremented as the system makes you change passwords isn’t exactly more secure- in fact a better solution may even be writing the password on a post-it, unless of course you broadcast video from your office and people deface your twitter because they see your password in the background. Of course, again the answer is a password manager. And also, the answer may be to give people more time with the same password, but bring more draconian complexity requirements.

But I though we were all post-password now… I have an iPhone

Good for you. Remember that it is hard to reset your fingerprints. The consumer grade fingerprint readers are far from fool-proof, and once somebody has a decent copy of your prints they have your things.We will see how things develop as the bio-metric identification thing gains more traction, but security people are already concerned about a case where a woman was made to unlock her iPhone because she had TouchID. The coercion did not count as violating the right against self incrimination the way forcing somebody to divulge a password would have. I’m sure further issues will come up. The general advice seems to be, use bio-metric data for identification, possibly, but rely on alternative means for authorisation.

Anyway – the point is, get a password manager if you don’t have one, and don’t forget to name and shame those that limit password length or – heaven forbid – email passwords in clear text. Do disable auto-fill, which is a feature where the password manager automatically adds username and password on a login page as it loads. There are often exploits that tricks password managers and automatically capture username and passwords by tricking the user to browse to evil websites. These vulnerabilities get patched, and new ones get discovered. Just disable auto-fill and you will be OK.

Troy Hunt on passwords and complexity, and how to build a good password reset feature..


Regarding the previous post about running Linux processes on Windows, I have noticed a few common questions and the corresponding answers at various places on the Internet.

The process and fork

The process that runs the Linux binary is a Pico process that comes with a minimal ABI and having all syscalls filtered through a library OS. For the Linux subsystem they have added real fork() semantics to the Pico process which has long been a popular gripe among those in the Linux community trying to port software to Windows. Although the normal case when creating a process is fork() and then lots of cleanup to create the pristine child process you always wanted, in the 1% of the cases where you actually wanted to create a verbatim copy of the current process including open files and memory extents et cetera, doing so in Windows was very complicated and extremely slow. This is now a native feature of the Pico process.


The security incompatibility between the faux Linux universe and the Windows universe, or rather the fact that Linux is unaware of the security settings and unable to affect the in any way has been raised and allegedly improvements are on their way.








After my previous post, Microsoft and Canonical reacted almost immediately – ahem – and on the //BUILD/ conference the Windows Subsystem for Linux and the ability to run the Ubuntu userspace under Windows was presented. If you have the latest fast ring insider build of Windows 10, you can just set your Windows installation into  Developer mode, search for Windows Features in Cortana and activate Windows Subsystem for Linux ~(Beta) reboot and then type bash at the windows command prompt. You will be asked to confirm, and then Ubuntu 14.04 will be downloaded, and off you go.

What is happening? A few things. Instead of the NT subsystem, NTDLL.dll, being loaded, a faux Linux kernel is presented to the process, allowing Linux userland binaries to be executed natively in the context of the current user. The abstraction is leaky at best right now, and as usual Microsoft aren’t even aiming to do this reimagining of the Linux kernel completely exact, so there will always be userland executables that will not work with this subsystem. Their goal is to make the most common dev things work out of the box to make Windows less frustrating for developers to work on. You would be doing people a favour by trying this out and complain loudly about everything that doesn’t work for you.

I just now did Win+R, typed bash and then vim. In SysInternals Process Explorer I see init -> bash -> vim in the process hierarchy. It is completely integrated in Windows, only the filesystem you see from Linux has been organised by Canonical to make sense. Since the context of the WSL is per-user, the Linux filesystem is actually located in the user’s profile folder while all drives on the system are mounted under /mnt/c /mnt/d et c. Because of this you can access “Linux” files from Windows and Windows files from “Linux” in a way that sort of makes sense. I mean, I’ve seen far worse.

I must say I didn’t think Microsoft would take this route, so kudos. What I still feel the need to complain about is that the security situation is complicated with the Windows user always being root in his own faux Linux universe. Server deployment could be magical if there was a way to have a global Linux context and have root access hidden behind elevation or sudo, but developing such a joint security model would be enormously complicated. Running X in Windows, that would be the next dream.

Desktop OS for developers

The results of the latest StackOverflow Developer Survey just came out, showing – among other interesting things – that Windows is dying as a developer OS. Not one to abandon ship any time soon I’d still like to offer up some suggestions.


  • Make the commandline deterministic.
  • Copying files across the network cannot be a lottery.
  • Stop rebooting UI-frameworks
  • Make F# the flagship language

Back in the day, Microsoft through VB and Visual C++ overcame some of the hurdles of developing software for Windows – then the only, effectively, desktop OS in the enterprise. Developers, and their managers, rallied behind these products and several million kilometres of code was written over a couple of decades.

The hurdles that were overcome were related to the boilerplate needed to register window classes, creating a window and responding to the basic window messages required to show the window in Windows and have the program behave as expected vis-a-vis the expectations a Windows user might have. Nowhere in VB6 samples was anybody discussing how to write tests or how, really, to write good code. In fact, sample code, simplified on purpose to only showcase one feature at a time, would not contain any distractions such as test code.

When Classic ASP was created, a lot of this philosophy came a cross to the web, and Microsoft managed to create something as horrible as PHP, but with less features, telling a bunch of people that it’s OK to be a cowboy.

When the .NET framework was created as a response to Java, a lot of VB6 and ASP.NET  programmers came across and I think Microsoft started to see what they had created. Things like Patterns & Practices came out and the certification programmes were taking software design and testing into consideration. Sadly, however, they tended to give poor advice that was only marginally better than what was out there in the wild.

Missed the boat on civilised software development

It was a shock to the system when the ALT.NET movement came out and started to bring in things that were completely mainstream in the Java community but almost esoteric in .NET. Continuous integration – unit testing – TDD – DDD. Microsoft tried to keep up by creating TFS that apart from source code version in had ALM tools to manage bugs and features as well as a built-in build server but it became clear to more and more developers that Microsoft really didn’t understand the whole thing about testing first or how lean software development needs to happen.

While Apple had used their iron fist to force people to dump Mac OS for the completely different, Unix-based operating system OS X (with large bits of NextStep brought across, like the API and InterfaceBuilder) – Microsoft were considering their enterprise customers and never made a clean break with Gdi32. Longhorn was supposed to solve everything, making WPF native and super fast, obsoleting the old BitBlt malarkey and instead ushering in a brighter future.

As you are probably aware, this never happened. .NET code in the kernel was a horrible idea and the OS division banned .NET from anything ever being shipped with Windows, salvaged whatever they could duct tape together – and the result of that was Vista. Yes, .NET was banned from Windows and stayed banned up until Powershell became mainstream a long, long time later. Now, with Universal Windows Apps, a potentially viable combo of C++ code and vector UI has finally been introduced, but since it is the fifth complete UI stack reboot since Longhorn folded, it is probably too little too late and too many previously enthusiastic Silverlight or WPF people have already fallen by the wayside. Oh and many of the new APIs are still really hard to write tests around, and it is easy finding yourself in a situation where you need to install Visual Studio and some SDK on a build server, because the dependency relies on the Registry or the GAC rather than things that come with the source.


As Jeffrey Snover mentions in several talks, Windows wasn’t really designed with automation in mind. OLE Automation possibly, but scripting? Nooo. Now, with more grown-up ways of developing software – automation becomes more critical. The Windows world has developed alternate ways of deploying software to end-user machines than work quite well, but for things like automated integration tests and build automation you should still be able to rely on scripting to set things up.

This is where Windows really lets the developer community down. Simple operations in Windows aren’t deterministic. For a large majority of things you call on the command-line  – you are the only one responsible for determining if the command ran successfully. The program you called from the command-line may very well have failed despite it returning a 0 exit code. The execution just might not have finished despite the process having ended, so some files may still be locked. For a while, you never know. Oh, and mounting network drives is magic and often fails for no reason.

End result

Some people leave for Mac because everything just works, if you can live with bad security practices  and sometimes a long delay before you get some things like Java updates. Some people leave for Linux because if you script everything, you don’t really mind all those times you have to reinstall because thing like a change in screen resolution or a security update killed the OS to the point you can’t log in anymore, you just throw away the partition and rerun the scripts. Also, from a developer standpoint, everything just works, in terms of available tools and frameworks.

What to do about it

If Microsoft wants to keep making developer tools and frameworks, they need to start listening to the developers that engage whenever Microsoft open sources things. They most likely have valuable input into how things are used by your serious users – beyond the tutorials.

Stop spending resources duplicating things already existing for Windows or .NET as that strikes precisely at the enthusiasts that Microsoft needs in order to stop hemorrhaging developers.

What is .NET Core – really? Stop rewriting the same things over and over. At least solve the problems the rewrite was supposed to address first before adding fluff. Also – giving people the ability to work cross-platform means people will, so you are sabotaging yourselves while building some good-will, admittedly.

Most importantly – treat F# like Apple treats Swift. Something like – we don’t hate C# – there is a lot of legacy there but F# is new, trendier and better. F# is far better than Swift and has been used in high spec applications for nine years already. Still Microsoft after years of beta testing still manages to release a JITer that has broken tail call optimisation (a cornerstone of functional runtimes as it lets you do recursion effectively). That is simply UNACCEPTABLE and I would have publicly shamed then fired so many managers for letting that happen. Microsoft needs to take F# seriously – ensure it gets the best possible performance, tooling and templating. It is a golden opportunity to separate professional developers from the morons you find if you google “ login form” or similar.

In other words – there are many simple things Microsoft could do to turn the tide, but Im not sure they will manage, despite the huge strides taken of late. It is also evident that developers hold a grudge for ages.

Salvation through a bad resource

Me and a colleague have been struggling for some time deploying an IIS website correctly using chef. As you may be aware, chef is used to describe the desired state of the configuration of a system through the use of resources, which know how to bring parts of the system into the desired state – if they, for some reason, should not be – during a process called convergence.

Chef has a daemon (or Service, as it is called in the civilised world) that continually ensures that the system is configured in accordance with the desired state. If the desired state changes, the system is brought into line automagically.

As usual, what works nicely and neatly in Unix-like operating systems requires volumes of eloquent code literature, or pulp fiction rather, to implement on Windows, because things are different here.

IIS websites are configured with a file called web.config. When this file changes, the website restarts (the application threadpool does, to be specific). Since the Chef windows service is running chef-client in regular intervals it is imperative that chef-client doesn’t falsely assume that the configuration needs to be overwritten  every time it runs is as that would be quite disruptive to any would-be users of the application. Now, the autostart behaviour can be disabled, but that is not the way things should have to be.

A common approach on Windows is to disable the chef service and to just run the chef client manually when you know you want to deploy things, but that just isn’t right either and it takes a lot away from the basic features and the profound magic of chef. Anyway, this means we can’t keep tricking chef into believing that things have changed when they really haven’t, because that is disruptive and bad.

So like I mentioned earlier – IIS websites are configured with a file called web.config. Since everybody that ever encountered an IIS website is aware of that, there is no chance that an evildoer won’t know to look for the connection strings to the database in that very file. To mitigate this well-knownness, or at least make the evil-doer first leverage a privilege escalation vulnerability,  there is a built-in feature that allows an administrator to encrypt the file so that the lowly peon account that the website is executing as doesn’t have the right to read it.  For obvious reasons this encryption is tied to the local machine, so you can’t just copy the file to a different machine where you happen to be admin to decrypt it. This does however mean that you have to first template the file to a temporary location and then check if the output of the chef template, the latest and greatest of website configuration, is actually any different from what was there before.

It took us ages to figure out that what we need to do is to write our web.config template exactly as it looks once it has been decrypted by Windows, then start our proceedings by decrypting the production web.config into a temporary location. We then set up the chef template resource to try and overwrite the temporary file with new values, and if there has been a change, use notifications to trigger a ruby_block that normally doesn’t execute, but when triggered by the template resource both encrypts the updated config and copies it across to prod.


But wait… The temporary file has to be deleted. It has highly sensitive information (I would like to flatter myself) and shouldn’t even have made it to disk in its clear-text form, and now it’s still there waiting to be read by the evildoer.

Using a ruby block resource or a file resource to delete the temporary file causes chef to record this as a change, and change that isn’t a change is bad. Or at least misleading in this case.

Enter colleague nr 2 “just make a bad resource that doesn’t use converge_by”.

Of course! We write a resource that takes a path and deletes it using pure ruby code, but it “forgets” to tell chef that a file was deleted, so chef will update the configuration when it should but will gladly report 0 resources updated at the end of a run where nothing has changed. Beautiful!

DONE. Week-end. I’m off.


How to hire devs for a small project

Do you have a smallish bit of programming type work you need done? You are thinking of either hiring somebody or getting contractors in? There are several pitfalls that you could experience and not all of them are the fault of the people you hire. I am here to set your expectations.

Know what type of people you hire

By the way I will use hire in this post to mean either employ or contract, you choose.

Historically programming have been done by programmers, or software development engineers. These would throw code together and tell you it is finished. Sometimes within deadline and sometimes not. You would then have QA people write test specifications and test the software and produce a list of bugs that then the programmers would have to fix. if you had a small budget you would forego the QA and try and test it yourself. This is not the way to do things.

If you insist on hiring an old-fashioned programmer, make sure you also hire an old-fashioned QA because otherwise the software you are making will never work. This project of yours is still probably doomed to be over budget.

A better way would be to hire a pair of modern senior agile developers that have learned the skills involved in QA-work and can design correctness into the system by clearly defining acceptance critera and handling edge-cases up front. You can advertise for an agile tester or something, if no developer you find admits to being great at testing, but ideally you are looking for a pair of developers that are close to the complete package.

Yes, by the way, hire at least two developers. As the mythical man-month says you can’t double project performace by doubling the number of resources, but a pair of developers are much more effective than a lone developer to the point that it definitely is worth the upfront cost.

Spend time

You will need to be on hand to look at the software as it is being constructed and make decisions on what you want the product to do. Your developers may give you rough estimates (S/M/L or similar) on how much effort certain things are and suggest solutions that achieve your objectives in an effective way, but you need to be available or else development will halt. This is not a fire-and-forget thing, unless you can fully authorise some person to be product owner for you that you can place next to the developers full time.

The thing you are doing, owning the product, is crucial to the success of your venture. I will post a link below to reading material as recommended by Stephen Carratt, a product owner I work with on a daily basis, that helps you define the work in small, clearly defined chunks that your developers call User Stories.

Work incrementally

Start out with small things and add features iteratively. Start by finishing core functionality first and always only add the most important thing. Don’t get me wrong, do have a roadmap of what you want the piece of software to do so you know where you want to eventually end up, but avoid spending time on specifying details right away. There will be cheaper or better options becoming apparent as you go along. You may be able to release working pre-releases to a subset of the intended audience to validate your assumptions and adapt accordingly.

Read up on working in a modern way

To help you create user stories that will help organise the work Steve recommends User Story Mapping (Patton).

Read about Lean Software Development (M Poppendeick) and Agile Testing (Crispin, Gregory) to know why the old way of making overly precise specifications (either of tests and code) ahead of time is wasteful and how working closesly with the development team is more effective in making sure the right thing is built.

Also, to understand why your developers keep banging on about delivery automation look at  Continuous Delivery (Humble, Farley)  although there are fictionalised books that illustrate all these benefits such as The Phoenix Project (Kim, Behr, Spafford).

But hey – when am I done? And what about maintenance?

Exactly! Why did you develop this product? To sell it? Is it selling? To help sell something else? Is it working? Hopefully you are making money or at least building value. Can you make the product better in a cost effective way – do so, if you can’t – stop. Purely budgetary concerns rule here, but also consider the startup costs included in dismissing your first developers and then shortly aftewards getting two other ones in to evolve the product,  needing to first spend some time learning the codebase from scratch before they can work at full speed.

Tech stuff for non-technical people

This is not a deeply technical post. Even less technical than my other ones. I write this inspired by a conversation on Twitter where I got asked about a few fairly advanced pieces of technical software by a clever but not-that-deeply technical person. It turned out he needed access to a website to do his human messaging thing and what he was given was some technical mumbo jumbo about git and gerrit. Questions arose. Of course. I tried to answer some of the questions on Twitter but the 140 char limit was only one of the impediments I faced in trying to provide good advice. So, I’m writing this so I can link to it in the future if I need to answer similar questions.

I myself am an ageing programmer, so I know a bunch of stuff. Some that is true, some that was true and some that ought to be true. I am trying to not lie but there may be inaccuracies because I am lazy and because reality changes while blog posts remain the same because, again – lazy.

What is Git?

Short answer: A distributed version control system.

Longer answer: When you write code you fairly quickly get fed up with keeping track of files with code in them. The whole cool_code.latest24.txt naming scheme to keep old stuff around while you make changes quickly goes out the window and you realise there should be a better way. There is. Using a version control system, another piece of software keeps track of all the changes you have made to the bunch of files that make up your website/app/program/set of programs, and let you compare versions, find the change that introduced a nasty bug et c. With Git you have all that information inside your local computer and can work very quickly and if you need to store your work somewhere else like on a server or just with a friend that has a copy of the same bunch of files (repository), only the changes you have made get sent across to that remote repository.

There are competing systems to git that do almost the same thing (mercurial is also distributed) and then a bunch of systems that instead are centralised and only let you see one version at the time, and all the comparison/search/etc stuff happen across the network. Subversion (SVN) or CVS are common ones. You also have TFS, Perforce and a few others.

What is a build server?

Short answer: a piece of software that watches over a repository and tries to build the source code and run automated tests when source code is modified.

Longer answer:

The most popular way for a developer to dismiss a user looking for help is to say “That works on my machine”. Legitimately, it is not always evident that you have forgotten to add a code file to your version control system. This causes problems for other people trying to use your changes as the software is incomplete and probably not working as a result of the missing files. Having an automated way of checking that the repository is always in a state where you can get the latest version of the code in development, build it and expect it to work is very helpful. It may seem obvious to you, but it was not a given until fairly recently. Any decent group of developers will make sure that the build server at least has a way of deploying the product, but ideally deployment should happen automatically. This does require that the developers are diligent with their automatic tests and that the deployment automation is well designed so that rare problems can be quickly rolled back.

What is Gerrit?

Short answer: I’ll let Wikipedia answer this one as I have managed to avoid using it myself despite a few close calls. It seems to be a code review and issue tracking tool that can be used to manage and approve changes to software and integrate with the build server.

Long answer:

Before accepting changes to source code, maintainers of large systems will look at submitted changes first and see if it is correct and in keeping with the programming style that the rest of the system uses. Despite code being written “for the computer”, maybe the most important aspect of source code quality and style is that other contributors understand the code and can find their way. To keep track of feature suggestions, bug reports and the submitted bits of code that pertain to them, software systems exist that can help out here too.

I don’t know, but I would think I could get most of that functionality with a paid GitHub subscription. Yes, gerrit is free, but unless you are technical and want to pet a server you will have to hire somebody to set up and maintain the (virtual) machine where your are running gerrit.

Where do I put my website?

Short answer: I don’t know, the possibilities are endless today. Don’t pay a lot of money.

Longer answer: Amazon decided, a long time ago, to sell a lot of books. They figured that their IT infrastructure had to be easy to extend as they were counting on having to buy a ton of servers every month so their IT stuff had to just work and automatically just use the new computers without a bunch of installing, configuration and the like. They also didn’t like it that the most heavy duty servers cost an arm and a leg, and wanted to combine a stack of cheaper computers and pool those resources to get the same horsepower at a lower cost. They built a bunch of very clever programs that provided a way to create virtual servers on demand that in actual fact were running on these cheap “commodity hardware” machines and they created a way to buy disk space as if it was electricity, you paid per stored megabyte rather than having to buy and entire hard drive et c. This turned out to be very popular and they started selling their surplus capacity to end users, and all of a sudden you could rent computing capacity rather than making huge capital investments up front. The cloud era was here.

Sadly, there was a bunch of companies out there that already owned a bunch of servers, the big iron expensive ones even, and sold that computing capacity to end users, your neighbourhood ISP, Internet Service Provider, or Web Hotel.

Predictably, the ISPs aren’t doing too well. They niche themselves in providing “control panels” that make server administration slightly less complicated, but the actual web hosting aspect is near free with cloud providers, so for almost all your website needs, look no further than Amazon or Microsoft Azure, to name just two. Many of these providers can hook into your source control system and request a fresh copy of your website as soon as your build server says the tests have run and everything is OK. Kids today take that stuff for granted, but I still think it’s magical.

Don’t forget this important caveat: Your friendly neighbourhood ISP may occasionally answer the phone. If you want to talk to a human, that is a lot easier than to try and get a MS or Amazon person on the phone that can actually help you.

Getting back into WPF

These last couple of weeks I have been working with a Windows desktop app based on WPF. I hadn’t been involved with that in quite some time so there was some trepidation before I got stuck in.

I have been pairing consistently throughout and I believe it has been very helpful for both parties as the union of our skill sets have been quite large and varied regardless of which colleague I was working with at the time. The app had interesting object life-cycles and has some issues in object creation when viewed from the standpoint of what we need to do today although it was well suited to solve the problems it did at the time it was written.

Working closely with colleagues mean that we could make fairly informed decisions whilst refactoring and the discussions have seemed productive. I tend to always feel that the code is significantly improved as we finish tasks even though we have stayed close to the task at hand and avoided refactoring for its own sake.

Given the rebootcamp experience, I’m always looking to make smaller classes with encapsulated functionality, but I still have room for improvement. As I’m fortunate enough to have very skilled colleagues it is always useful to discuss these things in the pair.  It helps to have another pair of eyes there to figure out ways to proceed – getting things done whilst working to gradually improve the design. I haven’t felt disappointed with a piece of code I’ve helped write for quite a while.

The way we currently work is that we elaborate tasks with product and test people and write acceptance criteria together before we set out to implement the changes. This means we figure out, most of the time, how to use unit tests to prove our acceptance criteria,without having to write elaborate integration tests, keeping them fairly simple trying to wrestle the test triangle the right way up .

All this feels like a lot of overhead for a bit of hacking, but we tend to do things only once now rather than having to go back and change something because QA or PM aren’t happy. There are some UI state changes which are difficult to test comprehensively, so we have had things hit us that were unexpected, and we did have to change the design slightly in some cases to make it more robust and testable, but that still felt under control compared with how hard UI bugs can be to track down.

Whatevs. This is where I am on my journey now. I feel like I’m learning more stuff and that I am developing. At my age that is no small thing.




Simple vs “Simple”

F# has two key features that makes the code very compact. Significant whitespace and forced compilation order.

Significant whitespace removes the need for curly braces or the use of keywords such as begin/end. Forced compile order means your dependencies have to be declared in code files above yours in the project declaration. This gives your F# projects a natural reading order and makes projects follow a natural order that transcends individual style.

Now there is a user voice suggestion that the enforced compile order be removed.

I think this is a good idea. I am against project files. As soon as you have three or more developers working in the same group of files, any one file used to maintain state is bound to become a source of merge conflicts and strife. Just look at C#.

I am sure your IDE could evaluate the dependency order of your files and present them in that order for you, heck one could probably make a CLI tool to show that same information if the navigational benefits of the current order is what is holding you back. Let us break out of IDE-centric languages and allow the programs to be defined in code rather than config.