Finding Proper Scopes for Unit Tests

In my previous rant post on TDD I’ve argued that the majority of the problems many experience doing TDD are caused by testing in too narrow scopes - using classes as units of testability, instead of functional use cases. However, widening the scope of the test too much is just another extreme. So how one finds the sweet spot? In this post I’d like to share the heuristic that I use.

Cyclomatic Complexity

Cyclomatic complexity is a software metric, used to indicate the complexity of a program. We can use this measurement to measure the complexity of a class or a method, and choose a suitable testing strategy.

Tackling Complexity in the Heart of DDD

Let’s do a little experiment: try to explain the gist of Domain-Driven Design to someone who has no clue about it. This, especially doing it succinctly, is not easy. Heck, I struggle with it myself. Bounded contexts, entities, repositories, domain events, value objects, domains, aggregates, repositories… where do you even start?

To find the order in the apparent chaos, I want to analyze the DDD methodology from a rather unusual perspective — by applying Domain-Driven Design to Domain-Driven Design itself. After all, this methodology is intended to deal with complex domains, isn’t it?

Let’s start by identifying the core domain: what is DDD’s main competitive advantage, and what are its means of achieving it?

The Core Domain: Ubiquitous Language

In “Domain-Driven Design: Tackling Complexity in the Heart of Software”(the Blue Book), Eric Evans argues that poor collaboration between domain experts and software development teams causes many development endeavors to fail. DDD aims to increase the success rates by bridging this collaboration and communication gap.

A Quick and Dirty Hack for Interviewing Job Candidates

One simple question can shed a lot of light on one’s competency in a given field: “On a scale of 1 to 10, please rate your knowledge of [enter-name-of-the-field-here]”.

One can assume that the higher the grade, the better. But that’s not the case at all. Why? Science — that’s why. Enter the Dunning-Kruger effect.

The Dunning-Kruger Effect

The Dunning-Kruger effect is a cognitive bias that suggests that one cannot objectively assess one’s own abilities. It’s all about the unknowns — things that you don’t know — and whether you’re aware of them.

The less expertise you have in a given field, the more unknown unknowns you have. You cannot objectively evaluate your knowledge, and, consequently, you are mistakenly assessing your abilities to be much higher than they actually are.

This bias also works the other way around. The more expertise you have in a given field, the more known unknowns you have. This awareness of things you don’t know tricks you into making a more humbling assessment of your abilities.

DDDEU 2016 Impressions

Last month, I had the pleasure of attending the Domain Driven Design Europe conference in Brussels. As I’ve tweeted before, this was the best conference I’ve ever attended. In this post, I’d like to sum the things I’ve learned at the conference.

It’s Not (Only) About Sessions

It was the first time I’ve attended a conference alone. Honestly, I was afraid that my introverted side would take over, and I’d master wallflower imitation techniques between sessions. Fortunately, it didn’t happen. I felt at home the moment I left the hotel for the conference. From that moment on, and up until the very last moments of the conference, I met a lot of like-minded people from all over the world - Belgium, Denmark, Germany, Austria, UK, Poland, Italy, France, USA, Finland, Switzerland, Netherlands, Romania, Bulgaria, and even from Israel.

For me, the social part, alone, was worth the trip. And don’t get me wrong, the sessions were great, but the ability to meet new friends, share experiences and ideas, and get fresh perspectives, was priceless. And I’m yet to mention discussing Star Wars with Eric Evans, discovering that Vaughn Vernon knows Israel better than I do, catching up with Greg Young, and last but not least, drinking beer with Yves Reynhout — it is unbelievable how much I learned from Yves that evening.

Lesson learned: Go to conferences alone and meet new people.

TDD: What Went Wrong…Or Did It?

Test Driven Development has been praised by our industry’s aficionados for a long time. However, lately there have been many harsh words said towards TDD, as it’s being blamed for causing bad software design and not keeping many of its promises. This trend culminated in David Heinemeierhansson’s post “TDD is dead. Long live testing”.

How is it possible, that the same technique, which is so advantageous to so many developers, is so disastrous to others? In this post I want to talk about 3 misconceptions that might explain this phenomenon.

Let’s start with the subtlest and most destructive one.

1. TDD is NOT “Test Driven Design

TDD stands for “Test Driven Development”. Unfortunately, many misinterpret this as “Test Driven Design”. This inaccuracy may sound innocent, but believe me, it isn’t. Let me explain.

Serving Flask With Nginx

Having spent the majority of my career in the Microsoft stack, lately I’ve decided to step out of my comfort zone and to dive into the world of open source software. The project I’m currently working on at my day job is a RESTful service. The service will be running on a commodity hardware, that should be able to scale horizontally as needed. To do the job I’ve decided to use Flask and Nginx. Flask is a lightweight Python web framework, and nginx is a highly stable web server, that works great on cheap hardware.

In this post I will guide you through the process of installing and configuring nginx server to host Flask based applications. The OS I’ll be using is Ubuntu 13.04.

Nginx

To install nginx from apt-get, we have to add Nginx repositories to apt-get sources:

1
sudo add-apt-repository ppa:nginx/stable

Note: If the “add-apt-repository” command doesn’t exist on your Ubuntu version, you need to install the “software-properties-common” package: sudo apt-get install software-properties-common (Thanks to get_with_it for mentioning it in the comments)

Update and upgrade packages:

1
sudo apt-get update && sudo apt-get upgrade

Install and start Nginx:

1
2
sudo apt-get install nginx
sudo /etc/init.d/nginx start

Milestone #1

Browse to your server and you should get the Nginx greeting page: nginx

JSON2CSV

Last week I’ve needed a utility to convert a file containing json data to csv. I found many online solutions, but for some weird reason they didn’t support nested objects and arrays. So I wrote one, this time in python. Grab it here - Github repository.

Usage

1
python json2csv.py "input_file.json" "output_file.csv"

If you pass in the following json file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[
  {
      "id": 1,
      "name": {
          "first": "john",
          "last": "johnson"
      },
      "age": 27,
      "languages": [ "c#", "vb", "python" ]
   },
   {
      "id": 2,
      "name": {
          "first": "scott",
          "middle": "scottster",
          "last": "scottson"
      },
      "age": 29,
      "languages": [ "objective-c", "c++" ]
   }
]

You’ll get the following csv file:

1
2
3
age,  id,    languages_0,    languages_1,    languages_2,    name_first, name_last,  name_middle
27,   1,      c#,             vb,             python,         john,       johnson,
29,   2,      objective-c,    c++,            ,               scott,      scottson,   scottster

Introducing SQL2JSON

If you’ll ever try to generate JSON file using Sql Server’s Management Studio, probably there’ll be lots of pain in your future. If you wanna save the pain, you can use SQL2JSON, an open source project I started last week.

The idea was born out of a need to generate a huge JSON file containing data from MSSQL database. After seeing our DBA struggling with formatting, escaping weird characters and unicode strings, I decided to write a small utility that will do just that - execute a sql query and capture its results as a json file. You can grab it from Github repository.

Usage

Suppose you have a table called “users” with the following records:

1
2
3
4
5
|user_id | first_name | last_name
|--------|------------|----------
|1       | john       | johnson
|2       | scott      | scottson
|3       | paul       | paulson

Unicode File Names in Python 2.7

Today I wrote a small script to find and delete duplicate files. To do this task I needed to iterate over files in a specific folder, and calculate md5 checksum for each file:

1
2
3
4
5
for folder, subs, files in os.walk(path):
    for filename in files:
        file_path = os.path.join(folder, filename)
          with open(file_path, 'rb') as fh:
              ...

If the source folder contains a file or a folder with unicode characters in it, execution of the code results in this bummer:

1
IOError: [Errno 22] invalid mode ('rb') or filename: 'files\\????????? ????? ????????.txt'

Uncoupling Configuration Files

Tight coupling is a known source for inflexible and hard to test code. In this post I want to talk about a rather unexpected source of tight coupling - configuration files. Configuration files are external dependencies. As other external dependencies, its infrastructure may change in the future, and it should be easily mocked for unit testing. Modern software frameworks provide means for easy access to the values stored in configuration files. In the .NET framework configuration files can be accessed using the ConfigurationManager:

1
2
3
4
5
6
7
<?xml version="1.0"?>
<configuration>
  <appSettings>
    <add key="Foo" value="1"/>
    <add key="Bar" value="2"/>
  </appSettings>
</configuration>
1
2
int foo = int.Parse(ConfigurationManager.AppSetttings[Foo]);
int bar = int.Parse(ConfigurationManager.AppSettings[Bar]);

ConfigurationManager makes it trivial to access data in the config file, however in most cases it also introduces various code smells that make the code tight coupled and hard to test. In the next sections I’ll introduce a simple class and will use it to demonstrate the code smells and violations of principles of clean object oriented design. The code will be gradually refactored to a better and cleaner solution.