bookmark_borderDRY is dead

The DRY principle, together with YAGNI, SOLID or KISS, is one of the most popular acronyms which shaped our way of thinking about developing software. It is simple, intuitive and easy to learn even during the early stages of education. However, the principle has been born in completely different circumstances than what we are dealing with today.

Simple idea

I’m not a historian of the software development and I’m not sure how the DRY principle has been born, but I guess it was created during procedural programming age. It stinks with a procedural way of thinking anyway.

The idea is simple. We have some code. The code should be organized. As long as we have some part of the code which repeats here and there we should create a procedure – extract this block of code, give it a name and reuse it.

Time flows

Since procedural programming things have changed. First of all the object-oriented paradigm explosion has happened. The complexity of the software has been growing. The systems for accounting, summing long rows of numbers, generating reports have been already created. The new frontier was internet browsers, instant messaging apps, trading systems for companies and snake for Nokia 3310. Except for the last one – it was quite a challenge.

The DRY principle doesn’t fit OOP as much as the procedural paradigm. Actually, if you think about it – it doesn’t fit at all.

Let’s think for a while – what happens when we try to avoid repetition in object-oriented code? First what comes to our mind is probably inheritance – the beautiful useless idea. The dog has a name, the cat has a name so let’s create a class Animal with property Name. But wait a second. Wild animals don’t have names. Let’s create WildAnimal and DomesticAnimal. Damn! – almost nobody gives names to fish…

Second popular solution for repetition problem is utils or commons.

There’s a secret rule in the software industry – every complex enough project has utils directory or class. Some of them 8k, 16k LoC.

It’s avoidable, it’s possible to properly design object-oriented software without these cancer cells of utils and disastrous inheritance. Please keep in mind anyway that both of them are the result of the DRY principle application. We tried, in the most easy, cheap way, to not repeat ourselves.

Microservices – nail in the coffin

Once upon a time I asked a colleague who worked at Amazon – the company which is a role model, a pioneer of microservice architecture – how do they organize common parts of the project, how they manage reusability, he answered:

We don’t. We do repeat. It’s cheaper and quicker at that scale of the project. 

The enormous size of the systems we are developing nowadays entails a new approach and different rules. The most visible tendency recently is breaking down problems and systems into smaller ones. Actually it is one of the main techniques since the beginning of software development, but recently it becomes more important than ever before. We can spot this trend in front-end frameworks (Angular, React – componentization), as well as in back-end (microservices architecture).

To some extent, we can think of it as a proper way of object orientation, more proper than inheritance. The organisms are similar but not the same. They do not, strictly speaking, share some features. The human eye is not the same as a dog’s or hawk’s eye. Only seemingly, on the level of naming these objects, they’re the same. Implementation details differ greatly. I’m not a genetic scientist but I bet that if we cut out from human DNA the parts which we don’t share with monkeys it will not create a monkey. I guess there are many subtle differences, some small parts of genetic code, few “lines” which make a difference even if most of the code is the same.

What to do?

It seems that the DRY principle became harmful. Should we stop using it? Maybe. For sure we should use it more carefully. In many scenarios, it may bring more harm than good. In some cases repeated code can be signature of failure, in some cases, it may be the best possible solution.

Is it bad when we repeat the identical code twice? If we repeat within the same class – I guess it is; in the same module – maybe; if it’s repeated in the same project, which consists of 100k LoC, and repetition happened in different modules – maybe not.

Is partial code repetition bad? Well, maybe it’s not bad by default, but it depends. Depends on the possibility to create a proper abstraction to avoid it. Quite often we use principles very strictly. Don’t. Don’t follow these rules blindly because they’re merely suggestions.

bookmark_borderWhat happens when the scrum team gets too big?

Participating in an overgrown scrum team is a fascinating experience. It allows us to observe how the framework collapses under its own weight.

Basically, scrum consists of meetings. Daily stand-ups, backlog refinements, sprint planning, review, and retrospective. These meetings, in theory, can consume even 22.5% of developers’ time. That’s a lot. But as always – it depends.

Image result for office sleep

Observation 1 – meetings get longer

Let’s start with a daily scrum. It should last no longer than 15 minutes. As long as the team is just a few developers big – it’s easy. But when we have 15 devs we can give everyone only one minute to speak. It’s pretty impossible. Either we will destroy the idea of the daily meeting – where the team members can properly describe what they worked on and ask for help (by forcing them to speak extremely short) either we’ll extend the time. In the team of 14 developers, I’ve seen daily scrum to grow up to 25 minutes.

Exactly the same is the situation with every other meeting. But in case of refinements or sprint review, it’s getting even worse because these meetings are naturally longer than daily scrum.

Observation 2 – meetings become full of things you don’t need to hear

As the team grows, the scope of work grows. Naturally – as in any big enough group – the subgroups form up. It can be any kind of division – front-end and back-end devs; people who work on feature A, and another group working on feature B; Java, and JavaScript programmers. When they work on the everyday tasks they almost don’t talk to each other unless they need to discuss some API or contract which is necessary for the parts of the system to communicate. But if the scrum team doesn’t split they’re forced to participate in the meetings together.

Then you find yourself in a meeting where half of the time is spent on discussing the things you don’t need to hear. You’re let’s say a JavaScript developer and last 30 minutes of a meeting was a discussion about back-end, or you’re developing database structure while implementation of the UI of the system is estimated.

Observation 3 – productivity plummets

You want to code. That’s why you became a software engineer. You like to focus, you love to create lines of instructions. This is also why, all in all, you’re getting paid. You automate processes, therefore, people don’t need to be employed to make them manually and the company is more effective. The money saved on automation goes to your pocket. Everybody is happy.

Unless you can’t code.

When the length of scrum meetings get longer it consumes your coding time. When the meetings get boring it consumes your mental energy. You become less productive. Not cool.

That’s why Scrum guide recommends a scrum team to be between 3 and 9 developers. Even if it seems difficult to divide – it’s necessary. The alternative is a small crowd of unhappy, ineffective developers.

bookmark_borderMeeting hell

Another meeting which could have been an email – that’s my favorite nag about a meeting hell.

You know it. You know it well. You just want to sit and code, you want to focus for an hour or two and produce some value for the company, for the customers, for the world. Then it shows up. An Outlook reminder. In 15 minutes there will be a meeting. You’re done. You don’t even try to start work anymore because you know it’s impossible to proceed. Time is too short.

The meeting takes five minutes to even begin. Joe comes from another meeting and it took longer than expected so he’s late. Jane is in the toilet. Mark is always late.

During the meeting, you listen to a lot of stuff which doesn’t bother you at all. It’s just a problem which affects two or three people out of eight or ten gathered in a tiny room with not enough air (by the way – this lack of oxygen can affect you).

Finally, you go out of the meeting and it’s lunchtime. After lunch, you’re a bit sleepy and it takes another 30 minutes until the blood will come back to your brain from the stomach.

Half of a day is gone.

Meetings are necessary. Effective organization of them is rare, however. Probably there are some atavistic mechanisms that make us prone to long pointless discussions. As far as I noticed it’s just a lack of emphasis put on the effectiveness of the process of knowledge exchange.

How can we avoid fluff? In some cases it’s tough. But more often than not we can ask ourselves some helpful questions: 
– does the topic require all attendants to be present at the meeting?
– are we prepared for the meeting?
– do we have an agenda?
– is everybody interested and do they need to attend?
– do we need to make a decision quickly or we can use e-mail instead of a meeting room?

Additionally, I would say that boring meetings are a soul-crushing experience. Therefore I would add a few more:
– is the pace of the meeting not too slow, sleepy?
– is the pace of the meeting too fast, too stressful?
– aren’t we too serious?
– aren’t we too much relaxed and offtopic?
– do we take breaks?
– don’t we have too many people in the room?

I believe that minimizing the number of meetings and effectively organizing them can be an amazing improvement in the workplace of a developer. Mental energy sucked by boring, long, nasty meetings organized in crowded rooms is the most precious asset brought by a software engineer to the company. We should take care of this thing. It creates software. It writes code. This energy runs the business. If we drain it we can have huge problems.

bookmark_borderLove-hate relationship with code mess

Have you ever seen a job offer for a software developer with something like your job will be to maintain our legacy system written in [ name of long-forgotten framework ] with over 100k LoC?

I haven’t.

Because in the software world it’s a bit like putting I have HIV phrase in your Tinder description. There are specific people who would be attracted to it (more about them here) but most of us would rather swipe left (or scroll down to the next job offer).

There’s one simple trick however which could change my mind. It’s one word.

Clean up.

People hate mess. Since the beginning of civilization. As human beings we enjoy order. But at the same time, we usually don’t enjoy maintaining order. That’s why we have a washing machine and dishwasher in most homes.

Humanity has a love-hate relationship with the order. On one hand, in most cultures, people who maintain it are perceived as a lower class or at least disliked: domestic servants, cleaners, police, bureaucrats. But at the same time cleaning up the mess is something heroic. The most well-known example is the fifth Labour of Heracles, which was to clean the Augean stables. The stables have been something like ancient 10M LoC legacy software which doesn’t even compile. TL;DR it was 3,000 cows and 30 years of manure (lack of compilation was caused by the immortality of the livestock). Heracles solved the problem, so to speak, by automation (here are the details). Then he became a well-known hero.

Another great ancient figure considered great after cleaning up the mess is Hammurabi. His method was to introduce some collaboration standards. For example, if someone breaks your build you can break his as well (§ 197). Historians say that the mess started to decrease after his code of conduct was published.

To sum up – maintenance: not cool, clean up: cool.

Unfortunately, more often than, nobody is truly interested in cleaning up the mess when it comes to software. What seems to be a goal of the organizations in most of the cases is to keep the mess alive and somehow productive. It’s understandable but dangerous. Middle management comes and goes, software developers also. Everybody wants to deliver results. But there’s a point of no return on this path. It’s a moment when the development of new features is almost impossible and the maintenance of old parts is ridiculously expensive. New developers don’t want to join the company or leave as fast as they can. Old developers are the only people who somehow know what’s going on in the system, but even they are unable to meet the deadlines. Technologies used in the project are so old they aren’t even supported. Everyday work is like a bath in the manure of 3,000 cows.

It can be avoided. By clean up on time.

Personally – I like to clean a mess. It gives me a pleasant feeling of doing something truly important. I don’t want to say I feel like Heracles, who saves the company from bankruptcy. Oh, well. Ok, maybe a bit I do. Anyway seeing every day how something which is a nightmare of future colleagues becomes an organized structure is satisfying.

Exactly opposite is feeling that every day, with every line of code added to the repository I’m feeding the monster which will digest some developers in the future. I feel it’s irresponsible to do it. It’s like allowing the organization I work for to have a bad habit which brings it happiness today but will give it cancer in 10 years. It’s amoral.

The best way obviously is to maintain a healthy lifestyle since the beginning. Coding standards, code reviews, automated tests, well-crafted architecture – all of us know it. Many of us ignore it. An important thing about the mess is that order indicates the development level. When we look at countries we see chaotic streets of India and well-organized streets of Europe. At first sight, we see where people can be more productive, happy and wealthy. We shall not allow the organizations we work for to fall into chaos.

Busy Calcutta (Kolkata), India street traffic.
France

bookmark_borderThe personality of software developer

In Poland, some devs say: I haven’t become a software engineer to talk with people. We, programmers, sometimes like to be perceived as incomprehensible, Heraclitean savants, who stand above the masses of ordinary users. Some of us, look down on junior developers. The internet is full of such memes, showing how dumb junior developers are, compared to legendary seniors.

When we show an old technology to a junior developer

Source: https://thecodinglove.com/when-we-show-an-old-technology-to-a-junior-developer

Among us, developers, there’s a constant battle. The ongoing domination struggle. How many times have I seen refactoring a source code to the shorter version, using unpopular language features just to show off how great our knowledge is… As well as reduced estimations, to prove we’re more efficient. And people starting recruitment processes just for the sake of training. Last but not least – assigning not easy but mundane tasks to junior developers.

In one company I worked for I had a colleague, let’s call him Peter. Peter was a junior developer like me. One day, however, probably due to longer experience, he has been promoted to a regular developer. Two weeks later, when I was working with the source code, I asked my colleagues at the open-space about decimal type in C# which I didn’t know at that time. Peter immediately stopped working and laughed out loudly.

– You do NOT know the decimal!? You’re such a junior dev! – he said

Then some senior developer heard it and laughed out even louder.

– Peter, how dare you to laugh? You asked me the same question one month ago!

Meanwhile, it is cooperation and synergy, not ego and knowledge about esoteric technology features, which creates productive teams that create beautiful, well designed, effective solutions to business problems and make clients happy. After years spent on software development, I’ve understood one thing…

The most important quality of a software engineer is empathy.

Empathy is the ability to understand and share the feelings of another. If a developer can see the world through the eyes of other people, he’ll be able to create software easy and comfortable for ordinary people. If he can think outside of his perspective he’ll be able to write the source code understandable for others. Therefore modifying it will be easier and cheaper. Having empathy, the developer will be able to communicate with the business.

As developers, we should always try to improve our empathy. We should ask ourselves as often as possible – will that be understandable for others, who don’t know what I know? Won’t it be overly complex, will it be easy to use? Do I use the language simple enough?

When we recruit developers we should carefully look at their personality. Self-contained, tyrannical people, even if technically brilliant, can create more problems than they solve. I’d recommend to not focus solely on IQ and technical skills. Most of the projects don’t have as monumental technical challenges as we imagine.

Technologies inevitably fade away and disappear. What will be left is the source code. It’s going to be source code written by the people who were able to understand that someone will come after them and that person will need to understand their work, that this person will probably not have deep knowledge about framework and the programming language, and maybe won’t have as deep domain knowledge as they have. Or it’s going to be written by people who don’t care about their successors but themselves rather. Nobody would like to maintain the source code created by the latter. It will be a hell to recruit people and their work will be slow and ineffective. We don’t want it. Therefore – focus on empathy.

bookmark_borderModified agile

I can’t remember how many times have I heard a word Agile during recent years. We’re agile. We’re doing agile. We’re introducing agile.

It sounds like people dialogues in the kitchen. Yes, I’m going to the gym. Sure, I do cardio. Yeah, I do Crossfit. Of course, I consume supplements. Aero? Every week.

But somehow I can’t see the muscle grow.

The results of this supposed agility I can’t see as well. There used to be chaos and it still is. Estimations have been poor and they still are. The technical debt balance looks like the credit card of a young adult after the summer weekend. Deadlines are scary like cancer and deployments as painful as colonoscopy. And last but not least – the scope is still not flexible at all. But we’re agile!

Are we?

The worst failure of Agile is I believe its promise it can be modified and adjusted. Therefore let’s come back to the analogy of a gym. We are growing muscles. But we don’t take care of the diet because we like to eat. We don’t train our legs because on Fridays we go to the party. We don’t go to the gym when we have a headache.

Then we visit a gym for two years and we still are somehow ashamed owners of a beer belly instead of rock-solid ABS.

Can we be agile if we have a fixed scope? Can we deploy at the end of each sprint if we don’t have the whole organization prepared for it? Can we build better software without showing the client a new version at each iteration? Can we be agile if there’s no client yet, only the budget?

Maybe yes, maybe no.

But for sure if we will modify or remove half of the rules from Agile recommendations we will achieve mess, not benefits. So let’s be serious, responsible developers and not be tempted to eat a cookie when we’re on diet. We can’t eat a cookie and still have it.

bookmark_borderThe fundamental problem of software development

If we would ask the question – what a software developer’s job is about? – probably most of the people would answer – it’s about writing source code. Some of them, more familiar with the software industry would reply – requirement analysis, application design, programming, to some extent also testing and deployment. That’s what can be seen. The operational perspective.

What cannot be seen however, at least at first sight, is much more important and entails a plethora of issues.

The world is complex. Almost infinitely complex. Objects consist of elements, elements of materials, materials are made of chemical substances, those are made of molecules, molecules made of atoms, atoms made of elementary particles, so on, so forth. On the other hand objects aggregate to places, places into cities, then we can think of regions, states, continents, planets, planetary systems, galaxies, groups of galaxies, etc.

Human beings simplify things. When we describe a group of people, we say: a few, several, dozen, crowd. We don’t count and say – 27 people. When we see the colors we say light green, warm, cold, gloomy, happy. We don’t describe the colors as a mathematical concept of red, green and blue values.

Computers do operate on numbers, values. We can’t put into machine the human idea of a car or gasoline. We must create a model of these things, simplification of the real world behind the words we use every day, we must choose the values which will describe them (volume, octane number), we must identify relationships between them (a type of fuel for a car).

People and computers are fundamentally different. Software is a bridge between the two worlds. Building software is creating the proxy, the communication layer between two extremely incompatible realities.

The consequences are enormous. This incompatibility makes software creation difficult, long and complex process. Because of it the software engineers quite often can’t communicate with the clients. Because of this barrier, we don’t have smooth conversations with Siri and we don’t see the robots from sci-fi movies on the streets. Because of this the cars on the streets still are driven by humans.

What draws my attention after years of work in the software industry is the human tendency to minimize effort (see: Law of triviality). A tiny fraction of the general population is eager to put the effort to deeply analyze the problem on which they work. Most of the people will stop at the first reasonable solution they can think of. It’s very clearly visible when software developers try to test their programs. They almost never are willing or able to use their software in a different way than the default scenario. It’s the simplest scenario, the one which is known for them very well. Creation requires effort. This constant mental optimization, this effort reduction is visible also in the designs of the business people in the organizations. Almost always it comes out that some scenarios have been missed during the planning stage. At the moment we even assume, industry-wide, that it’s impossible to analyze everything upfront (fundamentally different philosophy has been applied in the space shuttle program software engineering team, which is very well described here).

Software development, unfortunately, requires precision. At every stage of the project, starting from the analysis, through implementation and finally ending with tests and deployment we need to maintain intellectual rigor. It’s dictated by the nature of software itself. We cannot let ourselves to be lazy. We must always, at every moment of work be as precise as possible. Otherwise we will face consequences – bugs, mistakes, security breaches. Compilers are cruel – bugs are always fault of developers, not computers, and the same can be said about software development in general. Whatever we will ignore – it’s going to hit us back, painfully, sometimes deadly.