Codelab: Writing Web Applications
+ + + + + + +Introduction
+ ++Covered in this codelab: +
+-
+
- Creating a data structure with load and save methods +
- Using the
httppackage to build web applications + - Using the
templatepackage to process HTML templates
+ - Using the
regexppackage to validate user input
+ - Using closures +
+Assumed knowledge: +
+-
+
- Programming experience +
- Understanding of basic web technologies (HTTP, HTML) +
- Some UNIX command-line knowledge +
Getting Started
+ ++At present, you need to have a Linux, OS X, or FreeBSD machine to run Go. If +you don't have access to one, you could set up a Linux Virtual Machine (using +VirtualBox or similar) or a +Virtual +Private Server. +
+ ++Install Go (see the Installation Instructions). +
+ ++Make a new directory for this codelab and cd to it: +
+ +$ mkdir ~/gowiki +$ cd ~/gowiki ++ +
+Create a file named wiki.go, open it in your favorite editor, and
+add the following lines:
+
package main + +import ( + "fmt" + "io/ioutil" + "os" +) ++ +
+We import the fmt, ioutil and os
+packages from the Go standard library. Later, as we implement additional
+functionality, we will add more packages to this import
+declaration.
+
Data Structures
+ +
+Let's start by defining the data structures. A wiki consists of a series of
+interconnected pages, each of which has a title and a body (the page content).
+Here, we define Page as a struct with two fields representing
+the title and body.
+
type Page struct {
+ Title string
+ Body []byte
+}
+
+
+
+The type []byte means "a byte slice".
+(See Effective Go
+for more on slices.)
+The Body element is a []byte rather than
+string because that is the type expected by the io
+libraries we will use, as you'll see below.
+
+The Page struct describes how page data will be stored in memory.
+But what about persistent storage? We can address that by creating a
+save method on Page:
+
func (p *Page) save() os.Error {
+ filename := p.Title + ".txt"
+ return ioutil.WriteFile(filename, p.Body, 0600)
+}
+
+
+
+This method's signature reads: "This is a method named save that
+takes as its receiver p, a pointer to Page . It takes
+no parameters, and returns a value of type os.Error."
+
+This method will save the Page's Body to a text
+file. For simplicity, we will use the Title as the file name.
+
+The save method returns an os.Error value because
+that is the return type of WriteFile (a standard library function
+that writes a byte slice to a file). The save method returns the
+error value, to let the application handle it should anything go wrong while
+writing the file. If all goes well, Page.save() will return
+nil (the zero-value for pointers, interfaces, and some other
+types).
+
+The octal integer constant 0600, passed as the third parameter to
+WriteFile, indicates that the file should be created with
+read-write permissions for the current user only. (See the Unix man page
+open(2) for details.)
+
+We will want to load pages, too: +
+ +func loadPage(title string) *Page {
+ filename := title + ".txt"
+ body, _ := ioutil.ReadFile(filename)
+ return &Page{Title: title, Body: body}
+}
+
+
+
+The function loadPage constructs the file name from
+Title, reads the file's contents into a new
+Page, and returns a pointer to that new page.
+
+Functions can return multiple values. The standard library function
+io.ReadFile returns []byte and os.Error.
+In loadPage, error isn't being handled yet; the "blank identifier"
+represented by the underscore (_) symbol is used to throw away the
+error return value (in essence, assigning the value to nothing).
+
+But what happens if ReadFile encounters an error? For example,
+the file might not exist. We should not ignore such errors. Let's modify the
+function to return *Page and os.Error.
+
func loadPage(title string) (*Page, os.Error) {
+ filename := title + ".txt"
+ body, err := ioutil.ReadFile(filename)
+ if err != nil {
+ return nil, err
+ }
+ return &Page{Title: title, Body: body}, nil
+}
+
+
+
+Callers of this function can now check the second parameter; if it is
+nil then it has successfully loaded a Page. If not, it will be an
+os.Error that can be handled by the caller (see the os package documentation for
+details).
+
+At this point we have a simple data structure and the ability to save to and
+load from a file. Let's write a main function to test what we've
+written:
+
func main() {
+ p1 := &Page{Title: "TestPage", Body: []byte("This is a sample Page.")}
+ p1.save()
+ p2, _ := loadPage("TestPage")
+ fmt.Println(string(p2.Body))
+}
+
+
+
+After compiling and executing this code, a file named TestPage.txt
+would be created, containing the contents of p1. The file would
+then be read into the struct p2, and its Body element
+printed to the screen.
+
+You can compile and run the program like this: +
+ +$ 8g wiki.go +$ 8l wiki.8 +$ ./8.out +This is a sample page. ++ +
+(The 8g and 8l commands are applicable to
+GOARCH=386. If you're on an amd64 system,
+substitute 6's for the 8's.)
+
+Click here to view the code we've written so far. +
+ +Introducing the http package (an interlude)
+
++Here's a full working example of a simple web server: +
+ +package main
+
+import (
+ "fmt"
+ "http"
+)
+
+func handler(w http.ResponseWriter, r *http.Request) {
+ fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
+}
+
+func main() {
+ http.HandleFunc("/", handler)
+ http.ListenAndServe(":8080", nil)
+}
+
+
+
+The main function begins with a call to
+http.HandleFunc, which tells the http package to
+handle all requests to the web root ("/") with
+handler.
+
+It then calls http.ListenAndServe, specifying that it should
+listen on port 8080 on any interface (":8080"). (Don't
+worry about its second parameter, nil, for now.)
+This function will block until the program is terminated.
+
+The function handler is of the type http.HandlerFunc.
+It takes an http.ResponseWriter and an http.Request as
+its arguments.
+
+An http.ResponseWriter value assembles the HTTP server's response; by writing
+to it, we send data to the HTTP client.
+
+An http.Request is a data structure that represents the client
+HTTP request. The string r.URL.Path is the path component
+of the request URL. The trailing [1:] means
+"create a sub-slice of Path from the 1st character to the end."
+This drops the leading "/" from the path name.
+
+If you run this program and access the URL: +
+http://localhost:8080/monkeys+
+the program would present a page containing: +
+Hi there, I love monkeys!+ +
Using http to serve wiki pages
+
+
+To use the http package, it must be imported:
+
import ( + "fmt" + "http" + "io/ioutil" + "os" +) ++ +
+Let's create a handler to view a wiki page: +
+ +const lenPath = len("/view/")
+
+func viewHandler(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ p, _ := loadPage(title)
+ fmt.Fprintf(w, "<h1>%s</h1><div>%s</div>", p.Title, p.Body)
+}
+
+
+
+First, this function extracts the page title from r.URL.Path,
+the path component of the request URL. The global constant
+lenPath is the length of the leading "/view/"
+component of the request path.
+The Path is re-sliced with [lenPath:] to drop the
+first 6 characters of the string. This is because the path will invariably
+begin with "/view/", which is not part of the page title.
+
+The function then loads the page data, formats the page with a string of simple
+HTML, and writes it to w, the http.ResponseWriter.
+
+Again, note the use of _ to ignore the os.Error
+return value from loadPage. This is done here for simplicity
+and generally considered bad practice. We will attend to this later.
+
+To use this handler, we create a main function that
+initializes http using the viewHandler to handle
+any requests under the path /view/.
+
func main() {
+ http.HandleFunc("/view/", viewHandler)
+ http.ListenAndServe(":8080", nil)
+}
+
+
++Click here to view the code we've written so far. +
+ +
+Let's create some page data (as test.txt), compile our code, and
+try serving a wiki page:
+
$ echo "Hello world" > test.txt +$ 8g wiki.go +$ 8l wiki.8 +$ ./8.out ++ +
+With this web server running, a visit to http://localhost:8080/view/test
+should show a page titled "test" containing the words "Hello world".
+
Editing Pages
+ +
+A wiki is not a wiki without the ability to edit pages. Let's create two new
+handlers: one named editHandler to display an 'edit page' form,
+and the other named saveHandler to save the data entered via the
+form.
+
+First, we add them to main():
+
func main() {
+ http.HandleFunc("/view/", viewHandler)
+ http.HandleFunc("/edit/", editHandler)
+ http.HandleFunc("/save/", saveHandler)
+ http.ListenAndServe(":8080", nil)
+}
+
+
+
+The function editHandler loads the page
+(or, if it doesn't exist, create an empty Page struct),
+and displays an HTML form.
+
func editHandler(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ p, err := loadPage(title)
+ if err != nil {
+ p = &Page{Title: title}
+ }
+ fmt.Fprintf(w, "<h1>Editing %s</h1>"+
+ "<form action=\"/save/%s\" method=\"POST\">"+
+ "<textarea name=\"body\">%s</textarea><br>"+
+ "<input type=\"submit\" value=\"Save\">"+
+ "</form>",
+ p.Title, p.Title, p.Body)
+}
+
+
++This function will work fine, but all that hard-coded HTML is ugly. +Of course, there is a better way. +
+ +The template package
+
+
+The template package is part of the Go standard library. We can
+use template to keep the HTML in a separate file, allowing
+us to change the layout of our edit page without modifying the underlying Go
+code.
+
+First, we must add template to the list of imports:
+
import ( + "http" + "io/ioutil" + "os" + "template" +) ++ +
+Let's create a template file containing the HTML form.
+Open a new file named edit.html, and add the following lines:
+
<h1>Editing {Title}</h1>
+
+<form action="/save/{Title}" method="POST">
+<div><textarea name="body" rows="20" cols="80">{Body|html}</textarea></div>
+<div><input type="submit" value="Save"></div>
+</form>
+
+
+
+Modify editHandler to use the template, instead of the hard-coded
+HTML:
+
func editHandler(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ p, err := loadPage(title)
+ if err != nil {
+ p = &Page{Title: title}
+ }
+ t, _ := template.ParseFile("edit.html", nil)
+ t.Execute(w, p)
+}
+
+
+
+The function template.ParseFile will read the contents of
+edit.html and return a *template.Template.
+
+The method t.Execute replaces all occurrences of
+{Title} and {Body} with the values of
+p.Title and p.Body, and writes the resultant
+HTML to the http.ResponseWriter.
+
+Note that we've used {Body|html} in the above template.
+The |html part asks the template engine to pass the value
+Body through the html formatter before outputting it,
+which escapes HTML characters (such as replacing > with
+>).
+This will prevent user data from corrupting the form HTML.
+
+Now that we've removed the fmt.Fprintf statement, we can remove
+"fmt" from the import list.
+
+While we're working with templates, let's create a template for our
+viewHandler called view.html:
+
<h1>{Title}</h1>
+
+<p>[<a href="/edit/{Title}">edit</a>]</p>
+
+<div>{Body}</div>
+
+
+
+Modify viewHandler accordingly:
+
func viewHandler(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ p, _ := loadPage(title)
+ t, _ := template.ParseFile("view.html", nil)
+ t.Execute(w, p)
+}
+
+
++Notice that we've used almost exactly the same templating code in both +handlers. Let's remove this duplication by moving the templating code +to its own function: +
+ +func viewHandler(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ p, _ := loadPage(title)
+ renderTemplate(w, "view", p)
+}
+
+func editHandler(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ p, err := loadPage(title)
+ if err != nil {
+ p = &Page{Title: title}
+ }
+ renderTemplate(w, "edit", p)
+}
+
+func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) {
+ t, _ := template.ParseFile(tmpl+".html", nil)
+ t.Execute(w, p)
+}
+
+
++The handlers are now shorter and simpler. +
+ +Handling non-existent pages
+ +
+What if you visit /view/APageThatDoesntExist? The program will
+crash. This is because it ignores the error return value from
+loadPage. Instead, if the requested Page doesn't exist, it should
+redirect the client to the edit Page so the content may be created:
+
func viewHandler(w http.ResponseWriter, r *http.Request) {
+ title, err := getTitle(w, r)
+ if err != nil {
+ return
+ }
+ p, err := loadPage(title)
+ if err != nil {
+ http.Redirect(w, r, "/edit/"+title, http.StatusFound)
+ return
+ }
+ renderTemplate(w, "view", p)
+}
+
+
+
+The http.Redirect function adds an HTTP status code of
+http.StatusFound (302) and a Location
+header to the HTTP response.
+
Saving Pages
+ +
+The function saveHandler will handle the form submission.
+
func saveHandler(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ body := r.FormValue("body")
+ p := &Page{Title: title, Body: []byte(body)}
+ p.save()
+ http.Redirect(w, r, "/view/"+title, http.StatusFound)
+}
+
+
+
+The page title (provided in the URL) and the form's only field,
+Body, are stored in a new Page.
+The save() method is then called to write the data to a file,
+and the client is redirected to the /view/ page.
+
+The value returned by FormValue is of type string.
+We must convert that value to []byte before it will fit into
+the Page struct. We use []byte(body) to perform
+the conversion.
+
Error handling
+ ++There are several places in our program where errors are being ignored. This +is bad practice, not least because when an error does occur the program will +crash. A better solution is to handle the errors and return an error message +to the user. That way if something does go wrong, the server will continue to +function and the user will be notified. +
+ +
+First, let's handle the errors in renderTemplate:
+
func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) {
+ t, err := template.ParseFile(tmpl+".html", nil)
+ if err != nil {
+ http.Error(w, err.String(), http.StatusInternalServerError)
+ return
+ }
+ err = t.Execute(w, p)
+ if err != nil {
+ http.Error(w, err.String(), http.StatusInternalServerError)
+ }
+}
+
+
+
+The http.Error function sends a specified HTTP response code
+(in this case "Internal Server Error") and error message.
+Already the decision to put this in a separate function is paying off.
+
+Now let's fix up saveHandler:
+
func saveHandler(w http.ResponseWriter, r *http.Request) {
+ title, err := getTitle(w, r)
+ if err != nil {
+ return
+ }
+ body := r.FormValue("body")
+ p := &Page{Title: title, Body: []byte(body)}
+ err = p.save()
+ if err != nil {
+ http.Error(w, err.String(), http.StatusInternalServerError)
+ return
+ }
+ http.Redirect(w, r, "/view/"+title, http.StatusFound)
+}
+
+
+
+Any errors that occur during p.save() will be reported
+to the user.
+
Template caching
+ +
+There is an inefficiency in this code: renderTemplate calls
+ParseFile every time a page is rendered.
+A better approach would be to call ParseFile once for each
+template at program initialization, and store the resultant
+*Template values in a data structure for later use.
+
+First we create a global map named templates in which to store
+our *Template values, keyed by string
+(the template name):
+
var templates = make(map[string]*template.Template) ++ +
+Then we create an init function, which will be called before
+main at program initialization. The function
+template.MustParseFile is a convenience wrapper around
+ParseFile that does not return an error code; instead, it panics
+if an error is encountered. A panic is appropriate here; if the templates can't
+be loaded the only sensible thing to do is exit the program.
+
func init() {
+ for _, tmpl := range []string{"edit", "view"} {
+ templates[tmpl] = template.MustParseFile(tmpl+".html", nil)
+ }
+}
+
+
+
+A for loop is used with a range statement to iterate
+over an array constant containing the names of the templates we want parsed.
+If we were to add more templates to our program, we would add their names to
+that array.
+
+We then modify our renderTemplate function to call
+the Execute method on the appropriate Template from
+templates:
+
+
func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) {
+ err := templates[tmpl].Execute(w, p)
+ if err != nil {
+ http.Error(w, err.String(), http.StatusInternalServerError)
+ }
+}
+
+
+Validation
+ ++As you may have observed, this program has a serious security flaw: a user +can supply an arbitrary path to be read/written on the server. To mitigate +this, we can write a function to validate the title with a regular expression. +
+ +
+First, add "regexp" to the import list.
+Then we can create a global variable to store our validation regexp:
+
var titleValidator = regexp.MustCompile("^[a-zA-Z0-9]+$")
+
+
+
+The function regexp.MustCompile will parse and compile the
+regular expression, and return a regexp.Regexp.
+MustCompile, like template.MustParseFile,
+is distinct from Compile in that it will panic if
+the expression compilation fails, while Compile returns an
+os.Error as a second parameter.
+
+Now, let's write a function that extracts the title string from the request
+URL, and tests it against our TitleValidator expression:
+
func getTitle(w http.ResponseWriter, r *http.Request) (title string, err os.Error) {
+ title = r.URL.Path[lenPath:]
+ if !titleValidator.MatchString(title) {
+ http.NotFound(w, r)
+ err = os.NewError("Invalid Page Title")
+ }
+ return
+}
+
+
+
+If the title is valid, it will be returned along with a nil
+error value. If the title is invalid, the function will write a
+"404 Not Found" error to the HTTP connection, and return an error to the
+handler.
+
+Let's put a call to getTitle in each of the handlers:
+
func viewHandler(w http.ResponseWriter, r *http.Request) {
+ title, err := getTitle(w, r)
+ if err != nil {
+ return
+ }
+ p, err := loadPage(title)
+ if err != nil {
+ http.Redirect(w, r, "/edit/"+title, http.StatusFound)
+ return
+ }
+ renderTemplate(w, "view", p)
+}
+
+func editHandler(w http.ResponseWriter, r *http.Request) {
+ title, err := getTitle(w, r)
+ if err != nil {
+ return
+ }
+ p, err := loadPage(title)
+ if err != nil {
+ p = &Page{Title: title}
+ }
+ renderTemplate(w, "edit", p)
+}
+
+func saveHandler(w http.ResponseWriter, r *http.Request) {
+ title, err := getTitle(w, r)
+ if err != nil {
+ return
+ }
+ body := r.FormValue("body")
+ p := &Page{Title: title, Body: []byte(body)}
+ err = p.save()
+ if err != nil {
+ http.Error(w, err.String(), http.StatusInternalServerError)
+ return
+ }
+ http.Redirect(w, r, "/view/"+title, http.StatusFound)
+}
+
+
+Introducing Function Literals and Closures
+ ++Catching the error condition in each handler introduces a lot of repeated code. +What if we could wrap each of the handlers in a function that does this +validation and error checking? Go's +function +literals provide a powerful means of abstracting functionality +that can help us here. +
+ ++First, we re-write the function definition of each of the handlers to accept +a title string: +
+ +func viewHandler(w http.ResponseWriter, r *http.Request, title string) +func editHandler(w http.ResponseWriter, r *http.Request, title string) +func saveHandler(w http.ResponseWriter, r *http.Request, title string) ++ +
+Now let's define a wrapper function that takes a function of the above
+type, and returns a function of type http.HandlerFunc
+(suitable to be passed to the function http.HandleFunc):
+
func makeHandler(fn func (http.ResponseWriter, *http.Request, string)) http.HandlerFunc {
+ return func(w http.ResponseWriter, r *http.Request) {
+ // Here we will extract the page title from the Request,
+ // and call the provided handler 'fn'
+ }
+}
+
+
+
+The returned function is called a closure because it encloses values defined
+outside of it. In this case, the variable fn (the single argument
+to makeHandler) is enclosed by the closure. The variable
+fn will be one of our save, edit, or view handlers.
+
+Now we can take the code from getTitle and use it here
+(with some minor modifications):
+
func makeHandler(fn func(http.ResponseWriter, *http.Request, string)) http.HandlerFunc {
+ return func(w http.ResponseWriter, r *http.Request) {
+ title := r.URL.Path[lenPath:]
+ if !titleValidator.MatchString(title) {
+ http.NotFound(w, r)
+ return
+ }
+ fn(w, r, title)
+ }
+}
+
+
+
+The closure returned by makeHandler is a function that takes
+an http.ResponseWriter and http.Request (in other
+words, an http.HandlerFunc).
+The closure extracts the title from the request path, and
+validates it with the TitleValidator regexp. If the
+title is invalid, an error will be written to the
+ResponseWriter using the http.NotFound function.
+If the title is valid, the enclosed handler function
+fn will be called with the ResponseWriter,
+Request, and title as arguments.
+
+Now we can wrap the handler functions with makeHandler in
+main, before they are registered with the http
+package:
+
func main() {
+ http.HandleFunc("/view/", makeHandler(viewHandler))
+ http.HandleFunc("/edit/", makeHandler(editHandler))
+ http.HandleFunc("/save/", makeHandler(saveHandler))
+ http.ListenAndServe(":8080", nil)
+}
+
+
+
+Finally we remove the calls to getTitle from the handler functions,
+making them much simpler:
+
func viewHandler(w http.ResponseWriter, r *http.Request, title string) {
+ p, err := loadPage(title)
+ if err != nil {
+ http.Redirect(w, r, "/edit/"+title, http.StatusFound)
+ return
+ }
+ renderTemplate(w, "view", p)
+}
+
+func editHandler(w http.ResponseWriter, r *http.Request, title string) {
+ p, err := loadPage(title)
+ if err != nil {
+ p = &Page{Title: title}
+ }
+ renderTemplate(w, "edit", p)
+}
+
+func saveHandler(w http.ResponseWriter, r *http.Request, title string) {
+ body := r.FormValue("body")
+ p := &Page{Title: title, Body: []byte(body)}
+ err := p.save()
+ if err != nil {
+ http.Error(w, err.String(), http.StatusInternalServerError)
+ return
+ }
+ http.Redirect(w, r, "/view/"+title, http.StatusFound)
+}
+
+
+Try it out!
+ ++Click here to view the final code listing. +
+ ++Recompile the code, and run the app: +
+ +$ 8g wiki.go +$ 8l wiki.8 +$ ./8.out ++ +
+Visiting http://localhost:8080/view/ANewPage +should present you with the page edit form. You should then be able to +enter some text, click 'Save', and be redirected to the newly created page. +
+ +Other tasks
+ ++Here are some simple tasks you might want to tackle on your own: +
+ +-
+
- Store templates in
tmpl/and page data indata/. + - Add a handler to make the web root redirect to
+
/view/FrontPage.
+ - Spruce up the page templates by making them valid HTML and adding some + CSS rules. +
- Implement inter-page linking by converting instances of
+
[PageName]to
+<a href="/view/PageName">PageName</a>. + (hint: you could useregexp.ReplaceAllFuncto do this) +
+