Network scripting using concurrency with Go, Goroutines, and eAPI

Some cool things about Go are that concurrency is built in, and it compiles fast.  Although its possible to create concurrent programs in Python, its much easier to do with Go.  For network scripting this might seem unnecessary, but if you’re running some scripts on a lot of switches, it could make a big difference.  Here’s a simple example I created to illustrate:

Python

First I created a simple eAPI Python script to grab a show running-config from a single switch:

I ran it with the Linux command time to see how long it takes to fetch the config: 0m0.852s

Now I added three more switches, and ran this script:

And the time for this one: 0m3.188s
Roughly 4x the time for one switch, which is what you would expect.

Go without goroutines

Now lets try something similar in Go, first running a single switch:

First I’m going to run it with ‘go run’ which gives this time: 0m1.191s
A little slower than Python, but that includes compilation and running the program! Now we can speed it up a little by first compiling with ‘go build’ then just running the executable: 0m0.791s
Now its slightly faster than Python.

Ok now let’s do four switches:

Timed using go run shrun4.go: 0m3.601s
And when precompiled: 0m3.073s
As expected, we see a 4x increase in time for 4x the switches.

Go with multiple Goroutines

Now for the fun part, adding in concurrency. To do this we’re going to use goroutines. A goroutine is a function that is capable of running concurrently with other goroutines and is very lightweight (lighter than a thread). First I’m going to take the part of the code that gets the config and put it in a function which will become our goroutine:

func configFetcher(url string, cmds []string, format string, c chan eapi.JsonRpcResponse) {
response := eapi.Call(url, cmds, format)
c <- response
}

You’ll notice the variable c which is a channel. I’m not going to go into all the details here, but channels allow for communication between different goroutines, and here we’ve define a channel that will pass a JsonRpcResponse between goroutines. We send this data using the <- operators, so the line: c <- response takes the response and sends it over the channel.

Now in our main function we create four channels to get the data back from our goroutine. There is probably a better way to do this, but this was a simple way for me to conceptualize it:

c1 := make(chan eapi.JsonRpcResponse)
c2 := make(chan eapi.JsonRpcResponse)
c3 := make(chan eapi.JsonRpcResponse)
c4 := make(chan eapi.JsonRpcResponse)

Now we create our goroutines by simple adding go to the front of the function call, and pass in our channels:

go configFetcher(url1, cmds2, "text", c1)
go configFetcher(url2, cmds2, "text", c2)
go configFetcher(url3, cmds2, "text", c3)
go configFetcher(url4, cmds2, "text", c4)

And now I grab the response from the channels:

msg1 := <- c1
msg2 := <- c2
msg3 := <- c3
msg4 := <- c4

Here’s the whole thing in one piece:

Now let’s run it and see what happens. With go run I get: 0m1.330s
Hitting four switches is now about the same as one! And if we precompile: 0m0.865s

Conclusion

# switches Python go run Go compiled go run with goroutines goroutines compiled
1 0.852s 1.191s 0.791s n/a n/a
4 3.188s 3.601s 3.073s 1.330s 0.865s

Having concurrency built-in makes it easier to create programs that make use of the multi-core processors and get some nice performance gains.
While a network script might not seem to need this added complexity, we can see a significant performance boost with just four switches. Imagine if this were for tens or hundreds of switches. It could cut down the time it takes to run a script from minutes to seconds, and thanks to Go, its not that hard to do.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s