Code Deploy Pre and Post Hooks in Java

The Serverless Framework blog has a nice post on how to do blue/green or canary deploys. Most of the work is done in plugins and in the serverless.yml file, but for safe automated canary deploys, you'll want to add pre and post hooks.

As you might infer from their names, the pre-hook runs before the deployment and the post-hook runs once the canary deployment is complete.

If you return a "Failure" from either hook, Code Deploy will automatically rollback your deployment.

The missing piece from the mentioned blog is writing a hook in Java. The only example is in Node.js.

I experimented for a day and now share my results. This should be enough to get you started writing AWS Code Deploy hooks in Java.

public class PreHook implements RequestHandler<Map<String, String>, PutLifecycleEventHookExecutionStatusResult> {
 @Override
 public PutLifecycleEventHookExecutionStatusResult handleRequest(Map<string, string> event, Context context) {
  String deploymentId = event.get("DeploymentId");
  String lifecycleEventHookExecutionId = event.get("LifecycleEventHookExecutionId");
  PutLifecycleEventHookExecutionStatusRequest request = new PutLifecycleEventHookExecutionStatusRequest();
  request.setDeploymentId(deploymentId);
  request.setLifecycleEventHookExecutionId(lifecycleEventHookExecutionId);

                // Do your integration tests here. If they pass, set the status to "Succeeded" otherwise "Failed"
                // I set it to "Failed" to test rollback

  request.setStatus("Failed");
  AmazonCodeDeploy codeDeploy = AmazonCodeDeployClientBuilder.defaultClient();
  return codeDeploy.putLifecycleEventHookExecutionStatus(request);
 }

Like most Java Lambdas, you extend RequestHandler and parameterize it.

The input parameter is a Map of Strings. The map contains two very important pieces of information. The DeploymentId and the LifeCycleEventHookExecutionId.

Both are required to create a PuLifecycleEventHookExecutionStatusRequest.

PuLifecycleEventHookExecutionStatusRequest is required to create the output parameter PutLifecycleEventHookExecutionStatusResult by way of a AmazonCodeDeploy object.

Hopefully the code is enough to get you started. If not, please leave a comment or come find TheDarkSavant at the Serverless forums.

Parameterized Types

I recently had a major epiphany. I always thought Java Generics was about type safety until I realized that they are an implementation of the more general term "parameterized types" and thus are about reuse. Basically, in object oriented programming there are three ways to reuse code: inheritance, composition, and parameterized types. Let's look at how to use parameterized types for reuse.

Let's use the pedantic example of sorting. You've written a new ultra fast O(n) sorting algorithm, and you want to use it to sort all kinds of things, not just integers. In the pre Java 5 world, your sorting class would also implement something like Comparable, so any implementing class would have to override compareTo() for the actual comparison. That's the inheritance way to reuse. But that means your sorting class violate SRP. Now it sorts AND compares. You could have the implementor pass in a Comparator at construction and delegate to it during sorting, that's the composition way to reuse, but you'd rather not have a third object involved for clarity. What you can do is parameterize your sorting algorithm so the type of object that is getting sorted must implement Comparable, and let the sorted object itself decide how to compare itself to other objects of the same type.

public class SuperCoolSorter<T extends Comparable<T>> {

 public T[] sort(T[] collection) {
  for (T t : collection) {
   // secret O(n) sorting
   T other = t;
   t.compareTo(other);
  }
  return collection;
 }
}

So when you want to sort BugBears:
Create a BugBear class that implements Comparable<BugBear>

public class BugBear implements Comparable<BugBear> {
  public int compareTo(BugBear other) {
    // your comparison
  }
}

then new up sorter parameterizing it with BugBears:

SuperCoolSorter<BugBear> bugBearSorter = new SuperCoolSorter<BugBear>>();
bugBearSorter.sort(// bugbear array)

You've now created a sorting class that is independent of how things it sorts get compared, yet is reusable to sort anything that implements Comparable. Of course you could just have sort take in an array of Comparable, but then you'd have to return an array of Comparable then force the client to cast. By using parameterized types, you also control which type gets returned from sort.

Although this is pretty Java Generics specific, it's a usable pattern for any language that implements parameterized types. For example, C++ using templates.

Also, loosely typed languages like Ruby, Groovy, and Smalltalk do not need the extra markup to implement this pattern. Since they are "duck typed" languages, you can just call compareTo() on the passed in objects without the all the extra ceremony.

There you have it. The third type of reuse you may have never known about.

Code Katas I'd Like to See More Of

If you haven't heard of code katas, then you should really check them out. Like their namesake in Karate, code katas are little pieces of code you write over and over to exercise your coding muscles. And as opposed to bench pressing to exercise your coding muscles, katas are designed to fine tune not just strengthen.

I'm not sure who coined the term or idea, but I tend to get most of my katas from Dave Thomas at http://codekata.pragprog.com/

I like the katas listed there, but my experience is that they are mostly just programming contest problems designed to exercise your problem solving skills. That's a good thing, by the way, but there are more than just problem solving skills to master on the road to becoming a software craftsman.

What I'd like to see is more katas like:

Given the follow class:

import java.io.File;
import java.io.IOException;
import java.io.PrintStream;

import junit.framework.TestCase;


public class PolyKata extends TestCase {
 private PrintStream stream;
 enum shape {square, triangle, rectangle}
 
 private void drawPoly(shape the_shape) {
  switch (the_shape) {
   case square:
    stream.print(shape.square);
    break;
   case triangle:
    stream.print(shape.triangle);
    break;
   case rectangle:
    stream.print(shape.rectangle);
    break;
  }
 }
 
 public void test_drawPoly_square() throws Exception {
  PrintStreamFake stream_fake = new PrintStreamFake(shape.square);
  stream = stream_fake;
  drawPoly(shape.square);
 }
 
 public void test_drawPoly_triangle() throws Exception {
  PrintStreamFake stream_fake = new PrintStreamFake(shape.triangle);
  stream = stream_fake;
  drawPoly(shape.triangle);
 }
 
 public void test_drawPoly_rectangle() throws Exception {
  PrintStreamFake stream_fake = new PrintStreamFake(shape.rectangle);
  stream = stream_fake;
  drawPoly(shape.rectangle);
 }
 
 private static final class PrintStreamFake extends PrintStream {

  private shape the_shape;

  public PrintStreamFake(shape theShape) throws IOException {
   super(File.createTempFile("temp", "file"));
   this.the_shape = theShape;
  }
  
  @Override
  public void print(Object obj) {
   assertEquals(obj, the_shape);
  }
  
 }
}

Refactor using "replace conditional with polymorphism". Do this kata five days in a row, refactoring to a different design pattern each time. Then do it one more time only this time focus on taking the smallest steps possible and still keep the tests green. Also find ways to clean up the test code, as there's some duplication that can go as well.

To become a craftsman, you must refine your problem solving and coding skills, but you must also refine your code nose and refactoring skills. You must learn to quickly identify the stench of crap code and perfect your refactoring skills so you can quickly, cleanly, and efficiently replace crap code with clean code.

This is a perfect task for code katas. I only hope more of these come into play. I'll try to contribute my fair share, and if there are katas of this ilk out there in the tubes of the Internets already, please don't hesitate to point them out in the comments.

And I have to add, I had to look up how to do enums and switch statements in Java. That's how much I use them :)

Don't Swallow Exceptions - Ever!

Among lots and lots of good reason not to swallow exceptions, I just ran into a doozy. I'm rewriting a VB.Net application in Java. It has some really complex calculations. One of the calculations involves using a 2D array to setup some default values. The array is in a try catch block. With a catch(Exception e) and nothing else. No logging. Nothing.

Two days of trying to figure out why my calculations are slightly off what's expected. Thank you FitNesse, by the way. Anyway, come to realize that I'm swallowing an IndexOutOfBoundsException. See, VB.Net starts arrays at 1 while Java starts arrays at 0, so my array initialization was barfing and returning a bad value.

I did what most of us do now and then. I saw the empty catch block and thought, "I'll get back to that. I just want to get these calculations right. Then I'll fix those exceptions." Woops.

Handle exceptions as soon as you write the catch block. Handle it as best you can. At the very least print it to standard error, so when it happens you know it happened. Had I done at least that, I wouldn't be two more days behind.

XP is Not a Set of Practices

A recent discussion on the XP Yahoo! group has gotten me thinking about Extreme Programming and Agile Software Development in general. The discussion is around to test or not to test. Kent Beck's blog post provides an over all look into his view at To Test or Not To Test

The discussion started with someone asking for XP Myths and to test or not to test popped up and hijacked the thread. If you are really interested in the gory details of the thread, see http://tinyurl.com/yes8b2u for details. The discussion then stemmed into something like, "well, if it's OK to not test when you are exploring what people will pay for, it might be OK to not do XP". But what does "not do XP" mean?

That got me thinking about "Lean Thinking" by Womak and Jones because I've never looked at XP as a set of practices, well, I did early on but grew out of it. I look at XP and any Agile methodology as a system with certain core values and principles. I prefer to build my XP process based on the same principles expressed in "Lean Thinking:" value, value stream, flow, and perfection. If you want a more detailed list of principles on which to base your agile methodology, see "Lean Software Development" which has, if I recall correctly, 7.

Value

From a lean perspective, value is what you sell to make money. In software that means, well, software. It's why the customer pays you. The customer pays you because the software saves or makes her money. So the main principle is everything you do in your process must create value or it's muda or 'waste'. Whenever you decide to implement a practice ask yourself, "Will this practice help the team create value?" Alternately, whenever you decide to skip a practice, ask yourself, "How will skipping this practice affect the teams ability to create value?"

Value Stream

The value stream is the entire production process. Often it is helpful to map your value stream to see how the value gets produced. In manufacturing it could start at the mine where the raw materials are extracted from the Earth and end where the shiny new product rolls off the assembly line.

In software, your value stream could start with the customer asking for a new feature and flow all the way to delivery back to the customer. To map your value stream, write down every step the team takes while delivering that new feature. Once you have a complete value stream map, you can start looking for muda. Things you do that do not directly contribute to the creation of the desired feature - the value. Also look at all activities the team does that are not in the value stream map. Those activities are primary muda. Secondary muda are activities that contribute in a secondary way but do not directly produce value. An example of secondary muda is source control activities. Using a source control system is a vital component to software development. It keeps track of file changes, enables sharing of code, and allows branching for tracking software versions. All key to developing good software, but none add directly to the value. In other words, a customer will not pay you more money whether you use an SCM or not. In secondary muda cases, the goal is to minimize the time and effort spent on the activity. If the team spends 2 hours a day mucking about with SCM activities, find out why and reduce that time, so the team can spend more time on valuable activities.

Flow

The current buzzword of the agile industry. In manufacturing, flow is how well the value moves through the value stream. It addresses things like inventory and production backlog. For example, if a certain station on the assembly line always has a huge inventory of product stacked up in front of it, and the station down line is always waiting, then there is a flow issue. This is where the Toyota Production System concept of pull comes in to play. Traditional assembly lines thought the plant was most efficient when run at full capacity, but that often produces flow problems. Pull allows each station to pull product from the previous station, so each station can work at its full capacity. Less waste.

The current Kanban movement in software development is based on the concepts of flow and pull. The swimlanes on a Kanban board often map to the value stream map with each swimlane able to pull from the previous swimlane. All of your activities should allow the value to flow through the value stream. Once you have your value stream map, you analyze how the value flows through the system and address bottlenecks.

Perfection

Perfection addresses how the Toyota Production System constantly strove to improve itself. Kaizen is the constant little improvements each team tries to do everyday thinking that small continuous improvements eventually add up to big gains in effectiveness.

In software, retrospectives are often used to identify and address any issues that came up during the previous iteration or sprint. Encourage your team to constantly identify any issues with flow. As the team gets better and better, different problems pop up. Perhaps the team as become very proficient with the new technology and is producing faster than test can verify. Certainly a good problem to have, but a flow problem none the less. According to Theory of Constraints thinking, you move the constraining part ahead of the part that's overwhelming it. Moving test ahead of development, writing automated tests that are executed as the release criteria, now frees test from getting swamped and sets valuable limits on development.

Conclusion

I'm not willing to say what XP is, but I am willing to say it is not just a set of practices. I do believe the best methodologies are built from principles and values and not from a list of practices which are dogmatically executed day in and day out. Each practice you decide to adopt, be it a traditional XP practice, one from another methodology, or one the team creates itself, should contribute to value, value stream, flow, or perfection. Only then will you know it is worth trying. When evaluating whether to dump or skip a practice, analyze how that practice contributes to value, value stream, flow, or perfection to determine the affect dropping the practice will have.

Debt Metaphor is not Technical Debt

Ward Cunningham's coined phrase "the debt metaphor" has caused some confusion. I've seen it mis phrased as "technical debt." This is a mistake because if you watch the linked video, you'll see that Ward is talking about your understanding of the features and the model, and not about going into debt by taking programming short cuts.

Let's take a look at how Extreme Programming does design. Kent Beck expresses the XP practices as taking the good things about development and turning the knob to eleven. This includes design. If design is good, don't relegate it to a phase of development then end that phase and start construction, i.e. big design upfront (BDUF). Do it all the time and allow the design to emerge. This is where the terms emergent design and evolutionary design began.

But if you are going to start writing code on day one, and only do enough design to complete the current story, then you are writing code with a smaller understanding of the problem than you would had you done BDUF. So you are going into design debt in order to allow the design to emerge. As your understanding grows, you can refactor the code to better reflect your new understanding. That's paying off the debt. For that to work, the code needs to be as clean as possible.

I see blogs and email posts talking about technical debt and tracking technical debt so you can pay it back. That's OK that they want to talk about technical debt, but sometimes I see them using Ward's debt metaphor as validation for this practice. Taking shortcuts, like not pairing, not doing TDD, not automating your acceptance tests, or just plane not writing clean code or refactoring to clean code will actually hamper your ability to pay back your design debt because now your code does not express the knowledge you have about the system as well as it should. You are cheating yourself in two ways. You are creating technical debt that the team will have to pay back in the form of refactoring or, more often, bug fixes, and you are severely hampering your ability to pay back the design debt because the code does not clearly express current design knowledge.

What's the point? First, don't take shortcuts. Be honest and firm with your customer and tell them that you are sorry, but she can only have 2 of the 3 must have features for the upcoming trade show. And second, if you think it's OK to go into technical debt to please your customer, don't think that is what the debt metaphor enables. For the debt metaphor to work, you absolutely can not go into technical debt. Period.

Open Letter to Yahoo!

This is off topic for this blog, but I need to get it off my chest. I'll try to be civil. At least more civil than LaGarrette Blount.

Dear Yahoo!

I'm kicking you to the curb. I just can't take it anymore. Here's something I know you won't be able to believe, I'm writing this blog using Firefox on a machine running Ubunut 8.04. Really. Blogger.com actually runs in Firefox on Linux. Cool huh? I don't know how they do either. Well, actually I do 'cause I'm not a Steve Balmer boot boy.

That's right, I'm tired of seeing "This Yahoo! application doesn't work on your browser/OS." Do you know what a 'web application' is? Do you? It's an application that runs on the web, not just on IE in Windows. But if you took any time away from sucking up to Microsoft trying to get them to buy your sorry ass to learn anything about the world wide web, HTML, or standards, you'd already know that.

So go ahead and pander to the Microsoft world. Here's some news, Firefox is the most popular browser in Europe. But what do you care. Europe's only got like 830.4 million people. I'm sure you don't need any of that revenue to maintain your distant second place to Google in search engine revenue.

And here's another little piece of news. YOU ARE THE ONLY FANTASY FOOTBALL SITE THAT CHARGES FOR LIVE UPDATES. CBS Sportline? FREE! ESPN?, FREE! FoxSports? FREE! Maybe if you wrote applications that ran across browsers and OSes, you'd generate enough advertising revenue to offer it for free, but until then, I'm not playing your crappy fantasy games either.

And I'm not using your mail application. Trying to make it look like Outlook doesn't make it better. GMail doesn't look like Outlook, but those hackers over at Google innovate. Really innovate, not the fake innovation your potential bed fellows at Microsoft claim to do. They innovate by doing things new and different. Sorta like in the spirit of what the word innovate actually means. I don't know how I tolerated that spam infested Outlook copying piece of garbage for as long as I did.

And my.yahoo.com? Kicked to the curb. And I'm not using your search engine either. Not that it's a bad search engine, it's just that it's not Google. And you are Yahoo! At least your shenanigans make clear how you picked you name.

So you can count me as just one not Windows using potential revenue source that you no longer have. Never ever will yahoo.com enter my browser in any form what so ever. I'm sure I won't be missed. After all, you'll be able to con enough people into using your crap to keep you afloat. I mean, look at all those people who fall for Nigerian phishing scams. I'm sure you look at them as potential clients. Not that targeting Windows users is a bad strategy. Last I checked it's still the most popular OS. Falling like a rock. But still number one. No, it's that targeting Windows users is the least common denominator. Your stuff feels like Google stole all your real developers, and you have to make due with ASP.NET hackers who don't know a COM object from an ActiveX control (Hint, they're the same thing 'cause when COM/OLE failed, Microsoft rebranded it as ActiveX). I don't believe that, it just feels that way when I use, or used to use, your crap. I trust you're doing the best you can. Your best is just not good enough anymore.

Ciou!

Don't know about you, but I feel better :)