Doubling Go's template performance by generating code

Some of the most useful built in-packages in Go are text/template and html/template. They are very easy to use with whatever objects you hand to it, but this flexibility incurs a cost because they interpret the templates dynamically. This means that every time you execute the template, Go has to go through the syntax tree and decide what to do for every different kind of construct, including using reflection to access methods and fields.

I thought it would be a fun challenge to take a stab at implementing a code generator for the built-in template engine. The result is statictemplate. It is a code generator that will analyze your template and produce the equivalent Go-code, which you can then compile into your program. This gives you way better performance! The repository contains an example for a blog, with just a simple index page. The result after putting the templates through the generator uses no reflection whatsoever, still does HTML escaping and is substantially more performant than the built-in template engine. Running a benchmark against it gives me the following numbers:

BenchmarkStaticTemplate-8  20000  65479 ns/op
BenchmarkDynamicTemplate-8 10000 135933 ns/op
PASS
ok      github.com/bouk/statictemplate/example  3.390s

Do note that the performance heavily depends on the complexity of the template, with more complex templates benefitting the most from being compiled. The example is fairly simple, so in practice the numbers are probably better!

Support

I have re-implemented every structure used in the template engine, taking care to support less-used features like being able to call methods that can return an error. The generator is quite extensively tested, but if you find an issue, please report it.

How to use it

While generating the code for your template the generator needs type information to be able to generate the appropriate code for expressions like the following:

{{ .Value }}

The reason is that Value could be either a field or a method, and the only way to figure that out at runtime is through reflection.

To supply this type information, you will need to write some code. You can do something like this:

package main

import (
	"github.com/bouk/statictemplate/statictemplate"
	"os"
	"reflect"
	"text/template"
)

func main() {
	t := template.Must(template.New("template.tmpl").Parse("Hi there {{ . }}!"))
	code, _ := statictemplate.Translate(t, "main", []statictemplate.TranslateInstruction{
		{"Neat", "template.tmpl", reflect.TypeOf("")},
	})
	os.Stdout.Write(code)
}

When run, this will output the following code:

package main

import (
	"io"
)

func Neat(w io.Writer, dot string) (err error) {
	defer func() {
		if recovered := recover(); recovered != nil {
			var ok bool
			if err, ok = recovered.(error); !ok {
				panic(recovered)
			}
		}
	}()
	return fun0(w, dot)
}

// template.tmpl(string)
func fun0(w io.Writer, dot string) error {
	_, _ = io.WriteString(w, "Hi there ")
	_, _ = io.WriteString(w, dot)
	_, _ = io.WriteString(w, "!")
	return nil
}

The Neat function can then be called with an io.Writer and string. Check out the docs for this package.

Code generating code generating code

Remember two paragraphs ago where I said you needed to write some code? I was lying!

The way the generator code should be written is basically the same for most scenarios, so I decided to create a CLI-tool to do this job for you. We are lazy programmers after all. This tool is actually code that generates code, that generates code! It then immediately executes the generates code and outputs the result to a file you specify. It even supports generating a file that can be used while developing, using the regular standard library template package to re-interpret the files on every use, which means you can work on the templates without recompiling between changes. You can install the CLI-tool by running:

go get bou.ke/statictemplate

You then specify what functions you want to generate by using the -t argument. The command used for the example in the repository is as follows:

statictemplate -html -o example/template/template.go -t "Index:index.tmpl:[]github.com/bouk/statictemplate/example.Post" example/template/*.tmpl

This will generate one function named Index that executes index.tmpl using an array of Posts. As you can see you need to specify the full type name including the package name.

Support for functions and HTML templates

Supporting the built-in functions like len required performing some linking acrobatics, as not all the functions are available publicly. The way these functions are accessed is a bit sketchy, but it’s done using the //go:linkname construct. Using this method I created aliases of all the template functions that are accessible.

The same technique was required for adding support for HTML templates, as the method that automatically augments the template with escaping functions isn’t accessible. HTML escaping can be enabled by passing -html to the CLI-tool.

Conclusion

I had a lot of fun with this project, and I hope it’s useful for somebody. Please check out it out, and use it in your own project! I think there’s lots of benefits the Go community can derive from code generators, in both performance and functionality. Compiling the template in like this makes it super easy to distribute the binary, as you don’t need to rsync over any template directory but can just copy over a single binary.

Jan 2017