Our Way to Go

Hi there,

Many things have been told about the adoption of programming and languages in software projects. Too many times the criteria to choose some language, framework or tool are not based strictly on rational factors. Many others, the inheritance of the past is really huge and act as a dead weight tied to our feet.

So, in some moment we looked to a different group of languages popular because their performance, versatility and power, the System Languages (C, C++, Go, Rust, etc).  The problem was, we faced the construction of a heterogeneous set of software components, involving regular apps, microservices, functions and so on. After some analysis Go seemed to be the best and most ready option.

In this Techblog was published the post gRPC and an Angular based UI that was based on a gRPC server written in Go and we recommend to see a practical use case with Go. Besides, Docker was written in Go. So, we decided to have a deeper look into this programming language considering certain factors that were important to us:

What performance you need?

The database is the real bottleneck. So why bothering in improving the performance of your software?. I’ve heard this popular sentence too many times I’d like, unfortunately. The performance of our software apps has to be the best as we can. The answer is: I need ALL the performance I can get. Besides, data stores (NoSQL for instance, memory management, etc) have improved dramatically the performance of data storage. You do not have an excuse anymore.

Is my team trained for this language/framework?

It depends on too many factors but usually it is worthy to invest some time and money on staff training. It also helps to keep the engineers motivated and kin to learn new things. I cannot imagine a worst scenario than one where you work for years with the same tools and language. It is good also to retain the talent.

What is the training curve for learning?

Go is easy! It is a system language and some things must be taken into account. However, programming with Go is a smooth and lovely experience. The estimated time period to start producing some useful code is less than a week.

What about the available training materials, courses, documentation?

We’ve found tens of books,articles and so on. Go was released some years ago and the production of materials is mature and extensive, covering many aspects of what is possible to do with Go, from IaC, Microservices, etc. Besides it has an excellent documentation, public tour and sandbox for learning, etc.

What about the technical specification?

You can try at here. It is clear, it is easy and most importantly, any engineer can read it and understand better what is Go. Have you tried to read the Java Specification ? Not exactly the same thing.. 😛 The most exciting thing is, by reading the spec you can really learn Go!

Is there a development community behind supporting the language/framework?

Of course there is a huge community that will help you to solve the problems you can find. The Go Forum is as well available.

Is there a company supporting the language/framework?

Google released Go time ago but the Open Source community is supporting Go without any important intervention from Google.

What IT resources are available?

If you are using Cloud and your resources are scarce or your budget is limited Go is your option. The footprint is extraordinarily low compared to other platforms and languages, specially compared to JVM languages.

What about the Software Development LIfe Cycle? Can be integrated into the same delivery pipelines we are using?

Of course it can. Using Jenkins build pipelines you can integrate your Go projects with minor adaptations. Using dep for management of dependencies and Convey for testing and mocking,  GoLint, etc you can get the information about the quality and status of the code. The Go projects can be integrated to Sonarqube as well, getting a good visualization of the gathered SCA factors.

What tools, IDEs, etc are available?

There are some specific tools for Go as JetBrains like GoLand but it is not free and honestly it is not providing any killer feature to speed up our productivity in Go. Anyway it is always a good option.

We use Visual Studio Code with the Golang plugins to develop our apps and it’s great, it’s free and it works perfectly.

Our Programming Stack

It has been composed to allow the development of the platform components (e.g. API endpoints implementations), business applications and Infrastructure applications, covering all aspects of IT Automation, Automatic tests and so on. That means that the stack has to be small to allow a good ROI of the investment on training, materials, activities, etc but at the same time maintain the polyglotism expected in the Microservices Architectures and taking advantage of the skills of the available talent. Not a bad challenge.

Polyglotism in a Microservices architecture describes how to use the right tool for the right job. If a development engineer thinks that a certain programming language is correct for their service the Cloud IT based on container orchestration and Continuous Delivery pipelines will support this decision. It is recommended an architectural approval to keep a certain level of consistency and that’s all.

Simultaneously, we needed a stack able to provide much better performance and results than the conventional and obsolete stack based on JVM/Spring taking advantage of modern architectures, full adaptation to container orchestration with Kubernetes, full support to High Quality Delivery pipelines and easily tested from the functional and not functional points of view. Besides, Functions and Notebooks used in Cloud and Big Data worlds had to be supported.

So, our technical stack for programming is strongly based on a combination of languages and frameworks:

Why Go?

Just to enumerate some important factors that moved us to adopt Go as one of the main languages in our technical stack.

Scarce Expensive Cloud IT

Low footprint, low consumption of resources, low latency. If you have a small budget or you want to get the maximum performance to your Cloud resources Go is the best option, much better than the JVM languages.

Easy for Programming

In terms of code complexity, the easiness factor does not has a critical priority. However, programming with Go is just a pleasure.

Channels and goroutines have made our work much easier and it’s not just the fact that channels and goroutines are cheaper in terms of resources, compared to threadpool-based Futures and Promises, resources being memory and CPU. They are also easier to reason about when coding.

Microservices?

Well, we found this combination of tools in Go and we think they work incredibly well!

  • Gin-gonic for HTTP support
  • Logrus for log management
  • go-httpclient (default http is not recommended for production)
  • Viper for configuration management
  • Azure tools for support to service bus (queues and topics, event hubs, mongoDB, autorest, and many others

The microservices developed in Go can communicate through HTTP, AMQP, Streams, consume topics, send aggregated logs, etc. Everything we needed!

Analysis

This stuff is usually delegated to JVM monitoring and profiling in the JVM world. However, Go offers a good tooling for performance and resource analysis. Go tools provide text, graph, and callgrind visualization of the profile data using go tool pprof. You can navigate to Profiling Go programs to see them in action.

Exceptional Exceptions

If we talk about try / catch we’re probably talking about something you manage well and understand but… Is not strange that your software flow depends on something called ‘Exception’ ? Should not the exception be something exceptional?

Have the code full of try/catch is something that will never happen in Go, cause you manage ‘errors’, not exceptions, a method could return a value or an error, which makes sense, at least, more sense than return a ‘true/false || exception’.

So, prepare to write your owns errors and be happy with it, if you want some help and add the stacktrace (quite useful) there are some libraries for it.

Lastly, if you are badass and you think you’re perfect, you can omit the error using underscore, notice that this could be used on any variable, is not just for errors!

Excellent Behavior in Container

Do you remember the issues of the JVM to work in containers? Basically the JVM runs in the scope of a Linux container. The usage of resources scaled according to the number of instances of a specific image, multiplying the amount of allocated memory. In fact, the JVM was never designed to run i such a way as containers and the JVM world paid the price by adding tweaks, settings, etc to limit the amount of allocated resources, mainly CPU and memory.

On the other side, Go is a system language. It compiles in a binary that runs everywhere without needing a virtual platform. It is not interpreted!! It’s a language system! So, it does not need any interpreter or engine. That’s why it is know in the Go world how to play with containers. The resulting images are as light and tiny as the compiled binary is. The footprint is as low as the running binary and so on. Scalability is ideal in a Kubernetes cluster where we can deploy a pod of several containers with no additional problems and allocating the same amount of resources as if we run the same number of binaries. In the image below you can compare the size of the first image in the list (Go) with images with regular Java-based microservices.

Below Gist show how to build an image for a Go app by using the golang base image to compile the app and once this is done, creating a super-tiny image with from scratch.

Cloud Support

As you already know we mostly work with Azure for our Cloud based projects but not only. With Azure the support is excellent. Azure maintains a set of tools, SDK, etc to work with Go. The integration to use Go with most of the Azure services and tools is included.

Main Drawbacks

Not a Functional Language?

Functional Programming (FP) is really useful to us and we decided to adopt it. Pure functions make deterministic tests easy, and then a real  TDD approach gained traction (being fully integrated to SDLC workflows) and also spoke to our issues with higher software quality.

Considering FP Scala can be extremely attractive for developers. It allows us to create our own operators or override existing ones, essentially being unary and binary functions with non-alphanumeric identifiers. We can also extend the compiler via macros (user-defined functions that are called by the compiler), and enrich a third-party library via implicit classes. Can we do this with Go?

But in conclusion, Go is not an FP oriented language for now although some FP stuff can be written.

The Java Mindset

Our team comes from the JVM world. This is clear and really common in software teams. But most importantly, the mindset is not really related to JVM but only to the Java language. It is easy to find whole teams using only Java and in a simple way, not taking advantage of the last evolution steps, from Java 8 until now.

This is really a problem because it can frustrate any attempt to not only change to the adoption of new languages but also the adoption of other JVM languages like Scala or Kotlin.

There are not secrets or shortcuts here to change the Java mindset to a something different more aligned to microservices, including system languages and following the principle of the best tool for the best performance of a job.

We need training and hard work.

Proof of work

One of our first encounters with Go was just a small piece of code in charge of migrate Azure Active Directory Users to Kong Consumers, which means that, the software, will  use the Az client (using system shell), filter the users that you want and save it in Kong through Rest.

It’s an easy enough task to start learning Go (the very basic) and getting familiar.

First of all, we defined some data structures where we will store the AD Users.

Easy to follow: `varName : type : jsonVarName` the ‘omitempty‘ is just in here to omit this field if is empty.

So, the next function will login in the Azure client using the system shell.

If we don’t get any kind of error, the login is successful so we can start consuming the users.

First of all, we will get the different user groups and then, once we have them,  we will get the final users, you will see that we have a unmarshal function, it’s in here just for print the error in our way if something bad happens.

Lastly, let check how we will save the AD users in Kong using rest.

As you can see, Go is easy to follow and understand. You just need to take care of errors and different returned variables (yep, functions will return more than one value) and this example was a good start point. You just need to go deeper and deeper until become a guru!

 

Our Plans with Go

We have checked out the power of Go and our current plan is based on three main actions:

  1. Start with Go training and improve average skills in teams
  2. Migrate our platform components to Go
  3. Increase the number of business applications in Go

In conclusion, a decision that has accelerated our development and increased our productivity and quality levels. We strongly recommend to have a look on Go for your next software application.

Just a last thing. A popular question is, can you be a good Go developer without a Go t-shirt? We have not found out the answer yet but we already wear Go t-shirts just in case.

Cheers!

 

Jesus de Diego

Author: Jesus de Diego

Software Architect and Team Lead

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.