The State of the Gopher (Oct)

10 October 2014

Brad Fitzpatrick

Gopher, Google

The State of the Gopher

where we've been
where we're going

bradfitz

Work on Go's standard library, builders, etc
Started contributing to Go May 2010
camlistore.org, my big, old Go project: personal data storage & indexing for life

Userbase before November 2009

3-10 people inside Google?

Compilers in 2009

already two: gc and gccgo

Aside: GC vs gc

gc: "Go compiler", based originally on Plan 9's C compiler. The main Go compiler.
GC: "Garbage Collector"

Ports in 2009

linux-386
linux-amd64
linux-arm, just starting to work
darwin-386
darwin-amd64
nacl-386 (first attempt)

With gccgo, more.

weekly releases

"weekly.2009-12-09"
break users weekly, not daily
gofix appears

monthly releases

"r56" ... "r60"
break users monthly, not weekly
still painful to write and use packages (Makefiles)

Go 1.0 (2012-03-28)

the "error" type
delete(m, k)

// before Go 1:
latLong[storeID] = LatLong{}, false
name[userID] = "", false

// new:
delete(name, userID)

stability promise + API check tool:

pkg net/http, const StatusOK = 200
pkg net/http, const StatusOK ideal-int
pkg net/http, type RoundTripper interface { RoundTrip }
pkg net/http, type Server struct, TLSConfig *tls.Config
pkg net/http, var DefaultServeMux *ServeMux
pkg time, method (*Timer) Reset(Duration) bool
pkg unicode/utf8, func ValidRune(int32) bool

the go tool (get, build, test, ...)

Go 1.1 (2013-05-03)

method values

type S struct { once sync.Once; ... }
func (s *S) init() { ... }
func (s *S) Foo() {
    s.once.Do(s.init)

func "terminating statement"

func abs(n int) int {
    switch {
    case n < 0: return -n
    default:    return n
    }
}

64-bit "int", large heaps
30-40% speed improvement, stdlib++
more precise heap GC, usually

Aside: precise vs conservative GC

Conservative GC: if it "looks like" a pointer, treat it like one.

[ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ]
[ ptr? ] [ byte slice (skipped in Go 1.0)                             ]
[ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ] [ ptr? ]

Terrible for 32-bit (e.g. ARM) computers: many integers look like pointers.

Precise GC:

[ int ] [ ptr     ] [ string      ] [ byte slice          ]
[ int ] [ t f t f ] [ ptr ] [ int ] [ float64     ] [ ptr ]

Requires knowing what every position in memory actually is.

no false retention, only look at some memory during GC

Go 1.2 (2013-12-01)

6 month release cycle begins
setting slice capacity with s[i:j:k]

s := make([]byte, 10, 20)
foo(s[10:12:15])

func foo(s []byte) { fmt.Println(len(s), cap(s)) }

preemption in scheduler
go test -cover

Go 1.3 (2014-06-18)

no language changes
GOOS=solaris,dragonfly,plan9,nacl
precise GC of the stack (mostly)
contiguous stacks (mostly)

Aside: Segmented vs. Contiguous stacks

C++/Java: one huge stack per thread
Go <= 1.2: multiple discontiguous 4K stacks, bounce between
Go 1.3: one little stack, grow & shrink as needed

The present (relative to Nov 2009)

More ports

Windows
FreeBSD, OpenBSD, NetBSD
Solaris, Plan 9, DragonFly BSD
NaCl (again)

Better docs

Blog articles,
Community Wiki,
Books

Users: up and to the right

Go user groups, meetups
GitHub commits, stars, forks
Since 2009: golang-nuts@ (1450 → 14134), -dev (37 → 2152)

Go: the emerging language of cloud infrastructure

The Cloud

Docker
CoreOS, etcd, fleet
Juju
NATS, gnatsd, Packer, Heka
dl.google.com, Vitess, Kubernetes
App Engine

CoreOS

increasingly running build.golang.org, each in a Docker container
Go powering Go development!

Better tools

go/parser: godoc, pkg.go.dev, gofmt, gofix →
goimports: automatic import lines
go/types, gofix+types,
go/ast, go/ssa, oracle,
eg, example-based refactoring
gorename, safe global renaming tool

Tooling begets better tooling!

goimports + go/types + go/parser == goreturns (lazy "return err")

Go 1.4 (2014-12-01?)

fully precise GC of the stack + heap
segmented stacks & conservative GC deleted
stack copier & shrinker requires precise stack types too

runtime conversion from C to Go: maps, channels, interfaces, type checks, println, defer, panic, etc.

Garbage collector and scheduler are still in C

Go 1.4

GOOS=android (Elias Naur, David Crawshaw)
GOOS=nacl GOARCH=arm
"internal" packages
"go generate"
syscall package frozen, go.sys
"go tool pprof" converted from Perl (!) to Go
minor stdlib additions, improvements

Go 1.5 (2014-06-01?)

Concurrent GC is main theme
Roadmap: /s/go14gc (GC in 1.4, 1.5, 1.6+)

Go 1.5: ports

GOOS=ios revival? (Minux, David Crawshaw)
GOARCH=ppc64 (minux)
GOARCH=arm64 (aram, dfc)

Go 1.5: removing more C

Didn't make Go 1.4:

cmd/link: linker rewrite in Go
cmd/asm: assembler rewrite in Go
"gc" compiler in Go

"gc" Compiler in Go

lots of C
C → Go translator == kinda ugly Go
refactor that Go with tools: eg, gorename, etc
add packages, tests
unify all 8g, 5g, 6g, 9g
SSA form, new optimizations

See rsc's GopherCon slides, notes, video.

Go 1.5 Tracing

Dmitry's GOTRACE + Chrome trace viewer

Go 1.6

More GC work
Ongoing compiler-in-Go cleanup & optimizations
PNaCl?
NUMA-aware scheduler?
unknown surprises

Who knows.
Find out next year at dotGo Paris 2015!

libraries & tools

HTTP/2, to be merged into net/http
go.tools: goimports, eg, gorename, oracle, ...
go.text: Unicode collation, normalization, ...
go.mobile: Android, iOS, gaming, 3D graphics, event handling
go.crypto
go.net
go.tools
go.syscall

gccgo

keeping up with spec
better in some ways (codegen, OS, arch)
worse in others (precise GC, escape analysis)
work underway to improve

All checking & improving each other:

gc, gccgo, go/types, language spec, unit tests

Notable other compilers

llgo, Go → LLVM → PNaCL, etc
TARDIS Go, Go → Haxe → JS, anything
GopherJS, Go → JS

And all three written in Go, use go/parser, go/types, etc.
We can't wait to join them.

The growth is exciting

things are getting cleaner,
things are getting faster,
things are getting more impressive,
everything is growing on top of everything else,
hard to keep up with golang-dev@, golang-nuts@, bugs, wiki, new github projects, code reviews,

You all are awesome

keep being awesome
keep making awesome things

Thank you

Brad Fitzpatrick

Gopher, Google

@bradfitz

[email protected]

The State of the Gopher (Oct)

10 October 2014