Tech behind Tech

Introduction to Clojure Web Development using Ring, Compojure and Sandbar

Siva Jagadeesan — Wed, 19 Jan 2011 08:36:50 +0000

I gave an introduction to clojure web development presentation to bay area clojure user group. As my slides do not have much information, I am writing this blog so everyone can follow and get started with a sample clojure web application using compojure.

Introduction to Clojure web developmenthttp://static.slidesharecdn.com/swf/ssplayer2.swf?doc=compojure-110119172228-phpapp01&stripped_title=introduction-to-clojure-web-development&userName=sivajag

View more presentations from sivajag.

Setting Up:

 lein new address_book

Add Compojure to Project:

Edit project.clj

(defproject address_book "1.0.0-SNAPSHOT"
  :description "Runnering log"
  :dependencies [[org.clojure/clojure "1.2.0"]
                 [org.clojure/clojure-contrib "1.2.0"]
                 [ring/ring-jetty-adapter "0.2.5"]
                 [compojure "0.4.1"]
                 [hiccup "0.2.6"]
                 [sandbar "0.3.0-SNAPSHOT"]
                 [clj-json "0.3.1"]]
  :dev-dependencies [[swank-clojure "1.3.0-SNAPSHOT"]])

Run

lein deps

Test whether our setup works:

Edit src/address_book/core.clj

(ns address-book.core
  (:use [compojure.core]
        [ring.adapter.jetty])
  (:require [compojure.route :as route]))

(defroutes rts
  (GET "/" [] "Address Book!!")
  (route/not-found "Page not found"))

(def application-routes
     rts)

(defn start []
  (run-jetty application-routes {:port 8080
                                 :join? false}))

Goto http://localhost:8080 and you should see “Address Book”

Interactive Development:

In core.clj, change start function to

(defn start []
  (run-jetty application-routes {:port 8080
                                 :join? false}))

Add Static Folders:

mkdir -p public/{css,js}

Create public/index.html

' My Address book

My Address Book

Try to access http://localhost:8080/index.html. This will fail, as we never told compojure how to handle static files.

Edit core.clj and replace rts

(defroutes rts
  (GET "/" [] "Address Book!!")
   (route/files "/")
  (route/not-found "Page not found"))

Now when you access http://localhost:8080/index.html, it should display correctly.

Ring:

Three components

1) Handlers:
Handlers are main functions that process a request. We define handlers using defroutes macro.

2) Middleware:
Middleware are functions that could be chained together to process a request. Middleware functions can take any number of arguments, but the spec stats that first argument should be an handler and function should return an handler. An example for middleware is logging all requests that comes to your webserver. Ring and compojure comes with some standard middleware. We will see in next part how to create our own middleware.

3) Adapters:
Adapters are functions could adapt our handler to a web server. We are using jetty adapter to tie our handler to jetty server.

Lets create our Middleware:

Create src/address_book/middleware.clj

(ns address-book.middleware)

(defn- log [msg & vals]
  (let [line (apply format msg vals)]
    (locking System/out (println line))))

(defn wrap-request-logging [handler]
  (fn [{:keys [request-method uri] :as req}]
    (let [resp (handler req)]
      (log "Processing %s %s" request-method uri)
      resp)))

Edit core.clj ns form

(ns address-book.core
  (:use [compojure.core]
        [ring.adapter.jetty])
  (:require [compojure.route :as route]
			[address-book.middleware :as mdw]))

Edit core.clj application-routes def

(def application-routes
     (-> rts
	     mdw/wrap-request-logging))

Now when you access your server, you should see request log messages.

Main sourcecode:

Create address_book/address.clj

(ns address-book.address
  (:import [java.util Date])
  (:refer-clojure :exclude (find create)))

(def STORE (atom {:1 {:id :1 :name "Siva Jagadeesan" :street1 "88 7th" :street2 "#203" :city "Cupertino" :country "USA" :zipsourcecode 98802}}))

(defn to-keyword [num]
  (if-not (keyword? num)
    (keyword (str num))
    num))

(defn random-number []
  (to-keyword (.getTime (Date.))))

(defn create [attrs]
  (let [id (random-number)
        new-attrs (merge {:id id} attrs)]
    (swap! STORE merge {id new-attrs})
    new-attrs))

(defn find-all []
  (vals @STORE))

(defn find [id]
  ((to-keyword id) @STORE))

(defn update [id attrs]
  (let [updated-attrs (merge (find id) attrs)]
    (swap! STORE assoc id updated-attrs)
    updated-attrs))

(defn delete [id]
  (let [old-attrs (find id)]
    (swap! STORE dissoc id)
    old-attrs))

Edit core.clj ns form

(ns address-book.core
  (:use [compojure.core]
        [ring.adapter.jetty])
  (:require [compojure.route :as route]
			[address-book.middleware :as mdw]
            [address-book.address :as address]
            [clj-json.core :as json]))

Add this function to core.clj

(defn json-response [data & [status]]
  {:status (or status 200)
   :headers {"Content-Type" "application/json"}
   :body (json/generate-string data)})

Add these routes to core.clj

(GET "/addresses" [] (json-response (address/find-all)))
(GET "/addresses/:id" [id] (json-response (address/find id)))
(POST "/addresses" {params :params}  (json-response (address/create params)))

Replace your public folder with https://files.me.com/sivajag/sh35fq

This folder has needed css and js files.

Now go to http://localhost:8080/index.html and you should see a address book webapp. You can add and view addresses.

Authentication:

Create src/address_book/auth.clj

(ns address-book.auth
  (:use [sandbar.form-authentication ]
        [sandbar.validation]))

(defrecord AuthAdapter []
  FormAuthAdapter
  (load-user [this username password]
             (cond (= username "example")
                   {:username "example" :password "password" :roles #{:user}}))
  (validate-password [this]
                     (fn [m]
                       (if (= (:password m) "password")
                         m
                         (add-validation-error m "Unable to authenticate user.")))))

(defn form-authentication-adapter []
  (merge
   (AuthAdapter.)
   {:username "Username"
    :password "Password"
    :username-validation-error "You must supply a valid username."
    :password-validation-error "You must supply a password."
    :logout-page "/"}))

Edit core.clj ns form


(ns address-book.core
  (:use [compojure.core]
        [ring.adapter.jetty]
        [sandbar.auth]
        [sandbar.form-authentication ]
        [sandbar.validation ])
  (:require [compojure.route :as route]
			[address-book.middleware :as mdw]
            [address-book.address :as address]
            [address-book.auth :as auth]
            [clj-json.core :as json]))

Add this def to core.clj

(def security-policy
  [#".*\.(css|js|png|jpg|gif|ico)$" :any
   #"/login.*" :any
   #"/logout.*" :any
   #"/permission-denied.*" :any
   #"/addresses" :user
   #"/index.html" :user
   #"/" #{:user}])

Add this route to core.clj

 (form-authentication-routes (fn [_ c] (html c)) (auth/form-authentication-adapter))

Change application routes function in core.clj

(def application-routes
     (-> rts
        (with-security security-policy form-authentication)  		 
        wrap-stateful-session
	mdw/wrap-request-logging))

Now when you access http://localhost:8080/index.html it will take you to login page. You can login using “example” and “password”.

That is it folks. A simple web app using clojure.

I am sure this blog could be improved a lot. Please leave comments I will update this blog with your feedback.

Filed under: Clojure Tagged: clojure, compojure, ring, sandbar, web

Leiningen – Adding git submodule to Leiningen project

Siva Jagadeesan — Sat, 11 Dec 2010 21:39:51 +0000

With git, sometimes it easy to use git submodule for dependency management. In Leiningen, for some reason the only “advertised” way of dependency management is using maven. After some poking around, there is a way in Leiningen to use git submodules for dependency management.

Are you ready? ( drumroll …)

It is very simple actually

You have to checkout your submodules to “checkouts” folder. Leiningen will automatically add your submodule’s src directory to your classpath.

That’s it folks

Filed under: Clojure Tagged: dependency, git, Leiningen

Clojure – good coding guidelines

Siva Jagadeesan — Fri, 10 Dec 2010 07:04:22 +0000

Naming :

- only lower case letters separated by hyphens except types and protocols should be named in camelcase
- predicates ends with ?
- destructive functions ends with !
- variables that are meant for be re-binding should have earmuffs
- use “_” for names that will be ignored by the code

Code Structure :

- when there is only “then” clause in a conditional statement use “when” instead of “if”
- put all trailing parenthesis in one line
- for dependency use “:only” for “use” and “require”
- use two spaces for indentation
- destructure arg list only when you want to explain the structure of arg to caller otherwise destructure in first let binding

Coding Style:

- try to solve a problem by using function before trying with macro
- be lazy when possible
- use keywords as keys for map entries
- use keyword-first syntax to access properties on objects
- prefer sequence library functions instead of loop/recurr
- only use anonymous functions for short function definitions that fit comfortably on a single line. Otherwise create a private, named function.
- short functions

Filed under: Clojure Tagged: clojure, guidelines

Clojure defrecord deftype

Siva Jagadeesan — Mon, 15 Nov 2010 18:58:01 +0000

There are different ways to create a struct in clojure.

1) Using Clojure built in data structures

user> (def lotr {:title "Book title"
                 :author {
                          :first-name "first"
                          :last-name "last"}})
#'user/lotr

keyword-access

user> (:title lotr)
"Book title"

nested-access

user> (-> lotr :author :first-name)
"first"

2) Using defrecord

user> (defrecord Book [title author])
user.Book
user> (defrecord Person [first-name last-name])
user.Person
user> (def lotr (Book. "Book Title"
                       (Person. "first" "last")))
#'user/lotr

keyword-access

user> (:title lotr)
"Book title"

nested-access

user> (-> lotr :author :first-name)
"first"

As you can see, accessing map and defrecord is very similar. defrecord provides implementation of a persistent map.

3) Using deftype

PS: bad example ( I will explain why later )

user> (deftype Book [title author])
user.Book
user> (deftype Person [first-name last-name])
user.Person
user> (def lotr (Book. "Book Title"
                       (Person. "first" "last")))
#'user/lotr

Unlike defrecord, deftype does not provide implementation of a persistent map. So when you try to keyword-access , it won’t work

keyword-access

user> (:title lotr)
nil

It provides field access only

user> (.title lotr)
"Book Title"

I mentioned the above example for deftype is a bad example. Let me explain why. In OO, we have two set of classes. One set that defines language constructs like String and other that defines domain like Book. As defrecord provides implementation of a persistent map, it is better to use defrecord for realizing domain. Use deftype for realizing programming constructs.

Filed under: Clojure Tagged: clojure, code, protocols, record, structure

Testing Clojure Code – Awesome “are”

Siva Jagadeesan — Tue, 19 Oct 2010 19:55:56 +0000

test-is a great library for testing clojure. Thanks to Philippe, I recently discover “are” in test-is. This is most underrated/underused macro in test-is library.

Lets take an example, where I want to test a function given a date, it will return end of month.

Without are,

(deftest test-end-of-month
        (is (= "2010-01-31 23:59" (end-of-month "2010-01-03 01:12")))
        (is (= "2010-01-31 23:59" (end-of-month "2010-01-03 21:59")))
        (is (= "2010-02-28 23:59" (end-of-month "2011-02-03 13:01")))
        (is (= "2016-02-29 23:59" (end-of-month "2016-02-03 03:01")))
        (is (= "2011-04-30 23:59" (end-of-month "2011-04-03 00:02"))))

With are,

(deftest test-end-of-month
        (are [expected ts] (= expected (end-of-month ts))

             "2010-01-31 23:59" "2010-01-03 01:12"
             "2010-01-31 23:59" "2010-01-03 21:59"
             "2011-02-28 23:59" "2011-02-03 13:01"
             "2016-02-29 23:59" "2016-02-03 03:01"
             "2011-04-30 23:59" "2011-04-03 00:02"))

The version without “are” has, so many duplications. It is difficult to read as it is cluttered. The version with “are” is easy to read, and it is so easy to add more test cases without much duplication.

Thanks Philippe for introducing me to wonders of “are”.

Filed under: Clojure Tagged: clojure, test-is, testing

Calling java from Clojure

Siva Jagadeesan — Tue, 12 Oct 2010 05:43:26 +0000

Importing Java Class

In repl, when you want to import one Java class you can do

(import 'java.util.Date)

When you want to import more Java classes from a same package you can do

(import [java.util Date HashMap])

(ns com.techbehindtech.java
        (:import [java.util Date HashMap]))

Creating Instances

(import 'java.util.Date)

(def today (new Date))

;; or

(def today (Date.))

Calling Java instance methods

user> (import 'java.util.Date)
java.util.Date
user> (let [today (Date.)]
        (.getTime today))
1286749020847

Calling Java static methods

user> (System/currentTimeMillis)
1286847946813

Sugar sytax:

Doto:

You want to write a function that will return UTC Java Calendar object set at a specific time.

user> (import [java.util Calendar TimeZone Date])
user> (defn utc-time [d]
        (let [cal (Calendar/getInstance)]
              (.setTimeZone cal (TimeZone/getTimeZone "UTC"))
              (.setTime cal d)
          cal))

#'user/utc-time

The let block is ugly. We could use doto to make this code better.

user> (import [java.util Calendar TimeZone Date])	
user> (defn utc-time [d]        
         (doto (Calendar/getInstance)
              (.setTimeZone (TimeZone/getTimeZone "UTC"))
              (.setTime d)))

#'user/utc-time

Dot Dot:

Sometimes in java you want to make calls in chain

Bad way

user> (.length (.getProperty (System/getProperties) "user.country"))
2

Better way

user> (. (. (System/getProperties) getProperty "user.country") length)
2

Even Better

user> (..
       (System/getProperties)
       (getProperty "user.country")
       (length))
2

Avoiding Reflection

By default jvm will be using reflection to identify type. Reflection is slow. But we can give type hints that way jvm does not have to use reflection. For example we can rewrite utc-time with type-hints like this

user> (set! *warn-on-reflection* true)
true
user> (defn str-length [s] (.length s))
Reflection warning, NO_SOURCE_FILE:1 - reference to field length can't be resolved.
#'user/str-length
user> (defn str-length [#^String s] (.length s))
#'user/str-length

Implementing interfaces and extending classes

Let us implement Java Runnable interface

user> (proxy [Runnable] []
        (run []
             (println "running ...")))
#

In clojure 1.2, you could use reify macro to implement. In fact it is better than using proxy.

user> (reify Runnable
             (run [this]
                  (println "running ...")))
#

Filed under: Clojure Tagged: clojure, interopability, java

Clojure Macros Simplified

Siva Jagadeesan — Wed, 29 Sep 2010 06:32:57 +0000

One of the powerful features of clojure ( or any other LISP) is macros. Macros are so powerful that will allow you do things that you can never do in any other language. There is a classic example of implementing “unless” functionality using macros. I am not going to talk about why we need macros and when to use them. There is so much literature out there discussing about Macros. Also checkout Clojure in Action book for in depth look on Macros. I have read both Joy of Clojure and Clojure in Action, when it comes to Macros I will recommend Clojure in Action. The idea behind this blog to simplify macros as much as possible.

As you know in Clojure code is data and data is code. Our code is a clojure List. So we can programmatically create a list and execute it.

user> (def my-code '(println "techbehindtech"))
#'user/my-code
user> my-code
(println "techbehindtech")
user> (eval my-code)
techbehindtech

In above example, we created a list my-code. This list could be created dynamically. And using eval function we could execute that list as a clojure code.

So to create code in clojure, all we have to do is create a clojure list. In clojure, before a code is evaluated we have hook to introduce our own code using Macros system.

In Macros we are basically creating a list of code similar to my-code. Lets create a simple macro that will allow us to write functions with some log message.

(defmacro def-logged-fn [fn-name args & body]
        `(defn ~fn-name ~args
           (println "Calling ...")
           ~@body))

user> (def-logged-fn say[name]
        (println (str "hello " name)))
#'user/say
user> (say "siva")
Calling ...
hello siva
nil

macroexpand and macroexpand-1:

macroexpand and macroexpand-1 are useful functions to know about when using
writing macros. These functions expand our macro forms.

user> (macroexpand-1 '(def-logged-fn say[name]
        (println (str "hello " name))))

(clojure.core/defn
 say
 [name]
 (clojure.core/println "Calling ...")
 (println (str "hello " name)))

You can see that our macro created a fn called “say” that calls println first before the original body. That is pretty cool huh.

The difference between macroexpand-1 and macroexpand is macroexpand-1
will expand only one level of macros and macroexpand calls macroexpand-1
until all macro forms are expanded.

The same macro expanded with macroexpand will look like

(macroexpand '(def-logged-fn say[name]
        (println (str "hello " name))))

(def
 say
 (clojure.core/fn
  ([name]
   (clojure.core/println "Calling ...")
   (println (str "hello " name)))))

As you can see macroexpand expanded defn macro also.

Coming back to our macro, there are 3 reader macros that we have used

- Backtick `
- Tilda ~
- Tilda Ampersand ~@

Backtick ` (syntax-quote):

This is called as syntax-quote. This quotes the whole expression that way you do not have to quote each one of them

Tilda ~ (unquote):

This is called as unquote. This unquotes (substitutes) the value. For
instance, we wanted to replace fn-name with its value, so we tilda
before fn-name.

Tilda Ampersand ~@ (unquote-splicing):

It is easy to show with an example. Lets take our macro that we wrote and change unquote-splicing to unquote before body and let us what happens.

user> (defmacro def-logged-fn [fn-name args & body]
        `(defn ~fn-name ~args
           (println "Calling ...")
           ~body))
#'user/def-logged-fn
user> (macroexpand-1 '(def-logged-fn say[name]
        (println (str "hello " name))))

(clojure.core/defn
 say
 [name]
 (clojure.core/println "Calling ...")
 ((println (str "hello " name))))

You can see the last line looks wierd. It is trying to call output of
println as a function. This will obiviously fail.


user> (def-logged-fn say[name]
         (println (str "hello " name)))
#'user/say
user> (say "techbehindtech")
Calling ...
hello techbehindtech
; Evaluation aborted.

No message.
  [Thrown class java.lang.NullPointerException]

As body is a list ( we are using varible args) when we just unquote it
is putting body inside a single list. To avoid that we have to
unquote-splicing.

Auto-gensym

Inside a macro, sometimes you want to create unqualified symbol to use
in a let block. We can do this easily by appending # in end of symbol.

Filed under: Clojure Tagged: clojure, code, example, functional, macros

Compojure Demystified with an example – Part 6

Siva Jagadeesan — Fri, 17 Sep 2010 06:09:58 +0000

In this part we will start implementing edit and delete functionalities.

We are going to build these services,

Edit address – PUT – http://localhost:8080/addresses/:id
Delete address – DELETE – http://localhost:8080/addresses/:id

1) Adding PUT and DELETE route

Edit routes.clj

(ns address_book.routes
  (:use [compojure.core])
  (:require [address-book.address :as address]
            [address-book.middleware :as mdw]
            [compojure.route :as route]
            [clj-json.core :as json]))

(defn json-response [data & [status]]
  {:status (or status 200)
   :headers {"Content-Type" "application/json"}
   :body (json/generate-string data)})

(defroutes handler
  (GET "/addresses" [] (json-response (address/find-all)))
  (GET "/addresses/:id" [id] (json-response (address/find id)))
  (POST "/addresses" {params :params}  (json-response (address/create params)))
  (PUT "/addresses/:id" {params :params} (json-response (address/update (params "id")  params)))
  (DELETE "/addresses/:id" [id] (json-response (address/delete id )))

  (route/files "/" {:root "public"})
  (route/not-found "Page not found"))

(def address-book
     (-> handler
         mdw/wrap-request-logging))

As you can see it is pretty straight forward to add PUT and DELETE route.

What is up with params?

In previous version of Compojure there were some magic variables, like params, session etc. From version 0.4 there are no more magic variables. Instead Compojure provides a map “params”. “params” map will contain all request information. For some reason, params map have a key :params which contains request parameters. That is why we have do {params :params}. I wish they will rename “params” to something else.

Instead of doing {params :params}, the compojure document states that we could do [params id] and when compojure detects a vector, instead of a map it will assign the parameters to each value of the vector. But for some reason, I was not able to make it work.

Front End code

Checkout from github for front end code. Warning: front end code is bad, do not take this as an example.

In next part we will see how to send flash messages.

Source code is now available at github. Created branches for each part.

Next part is posted.

Filed under: Clojure Tagged: clojure, code, compojure, jquery, REST, web

MQL – A Clojure library for querying Nested Maps

Siva Jagadeesan — Thu, 09 Sep 2010 05:43:18 +0000

I was working on a feature that needed me to query nested map structure. I wanted to do it in a generic way. Amit pointed me to rql, a library for dealing with collections of records in clojure. I thought it will be helpful if I have something similar to rql for querying map, so I created mql.

Given a map

(def m
     {:cid
      {
       :visits {
                :id-1 {
                       :last-ts "1284166000040"
                       :first-ts "1274166000000"
                       :duration "40"}
                :id-2 {
                       :last-ts "1274166000040"
                       :first-ts "1274166000000"
                       :duration "40"}
                :id-3 {
                       :last-ts "1264166000040"
                       :first-ts "1274166000000"
                       :duration "40"}}

       :promo{
              :id-1 {:promo "p2"}
              :id-2 {:promo "p1"}}

       :purchase {
                  :id-1
                  {:order-id "order-id-1"
                   :total-dollars "970.00"
                   :purchase? "true",
                   :merchant-total-dollars "1000.00"}
                  :id-3
                  {:order-id "order-id-2"
                   :total-dollars "1000.00"
                   :purchase? ""
                   :merchant-total-dollars "1000.00"}}}})

Select: simple select

mql.core> (select [:cid :promo] m)
{:id-1 {:promo "p2"}, :id-2 {:promo "p1"}}

Select: with filtering

mql.core> (select [:cid :purchase] (where [* :total-dollars] :gt 980) m)
([:id-3 {:order-id "order-id-2", :total-dollars "1000.00", :purchase? "", :merchant-total-dollars "1000.00"}])

You can currently use :gt,:ge,:lt,:le and :eq as logical operators in where clause. In where clause for key-seq you can use * if a key is dynamic. The last key in where clause key-seq should not be *.

Limitations:

There could be only one * in Where Clause
Select clause cannot have * now

This is an early version of this library. It is doing what I need for now. So if you have any requirements/idea, you can send me a patch or let me know, I will hack when I get some free time.

Please feel free to send me your feedback on syntax, code style … etc

Filed under: Clojure Tagged: clojure, data structure, libraries

Compojure Demystified with an example – Part 5

Siva Jagadeesan — Thu, 02 Sep 2010 07:19:37 +0000

In this part lets write our own middleware.

From part4 you will remember,

“Middleware are functions that could be chained together to process a request. Middleware functions can take any number of arguments, but the spec stats that first argument should be a handler and function should return a handler. An example for middleware is logging all requests that comes to your webserver.”

1) Create a namespace for middleware

Create src/address_book/middleware.clj

(ns address-book.middleware)

(defn- log [msg & vals]
  (let [line (apply format msg vals)]
    (locking System/out (println line))))

(defn wrap-request-logging [handler]
  (fn [{:keys [request-method uri] :as req}]
    (let [resp (handler req)]
      (log "Processing %s %s" request-method uri)
      resp)))

Our wrap-request-logging is a middleware which takes a handler and returns an handler.

2) Add our middleware to our chain of handlers

Edit routes.clj

(ns address_book.routes
  (:use [compojure.core])
  (:require [address-book.address :as address]
            [address-book.middleware :as mdw]
            [compojure.route :as route]
            [clj-json.core :as json]))

(defn json-response [data & [status]]
  {:status (or status 200)
   :headers {"Content-Type" "application/json"}
   :body (json/generate-string data)})

(defroutes handler
  (GET "/addresses" [] (json-response (address/find-all)))
  (GET "/addresses/:id" [id] (json-response (address/find id)))
  (POST "/addresses" {params :params}  (json-response (address/create params)))

  (route/files "/" {:root "public"})
  (route/not-found "Page not found"))

(def address-book
     (-> handler
         mdw/wrap-request-logging))

3) Test our middleware

Now start your server and you can see all requests are getting logged. This is very powerful feature. We can do security etc using middleware. Ring and Compojure comes with some useful middleware. Check them out.

In next part we will implement edit and delete functionalities.

Source code is now available at github. Created branches for each part.

Filed under: Clojure Tagged: clojure, code, compojure, jquery, REST, web

Compojure Demystified with an example – Part 4

Siva Jagadeesan — Tue, 24 Aug 2010 23:25:25 +0000

In this part we will start implementing Add, View and View all functionalities.

The services we are going to build are

View All Addresses – GET – http://localhost:8080/addresses
View single address – GET – http://localhost:8080/addresses/:id
Add Address – POST – http://localhost:8080/addresses

Interactive Development using slime

I mentioned in last part that after every change to our code we need to restart our server for our changes to be reflected. This is a pain and against clojure (lisp) philosophy . It would be great if we could eval our modified buffers in emacs and those changes reflected immediately in our jetty server. Thankfully there is very easy way to do this.

In core.clj we are starting jetty adapter. We can start this jetty server in background using future and reload the namespace that we changed. This way we can do interactive development without restarting our server.

PS: Make sure you have emacs setup with slime. I feel emacs is the best IDE for clojure. Again this is my opinion. There are lot of information on how to do this.

I talked about jetty adapter. It is time for us to look at some what little deeper in Compojure.

Compojure

Compojure is based on a library called Ring. In fact most of clojure web frameworks are based on Ring. So to understand Compojure, it is important to understand Ring.

Ring has three components.

1) Handlers:

Handlers are main functions that process a request. We define handlers using defroutes macro.

2) Middleware:

Middleware are functions that could be chained together to process a request. Middleware functions can take any number of arguments, but the spec stats that first argument should be an handler and function should return an handler. An example for middleware is logging all requests that comes to your webserver. Ring and compojure comes with some standard middleware. We will see in next part how to create our own middleware.

3) Adapters:

Adapters are functions could adapt our handler to a web server. We are using jetty adapter to tie our handler to jetty server.

Refactor code to make it easy for interactive development

Edit core.clj

(ns address_book.core
  (:use [compojure.core]
        [ring.adapter.jetty])
  (:require [compojure.route :as route]))

(defroutes example
  (GET "/" [] "My Address Book!")
  (route/files "/" {:root "public"})
  (route/not-found "Page not found"))

(future (run-jetty (var address-book) {:port 8080}))

1) Add swank-clojure to our project

swank-clojure comes with lein plugin which will allow us to start a swank server and from emacs you can connect using slime. There are lot of documentation about swank-clojure and slime. Let me know if you need more information. If I see enough interest, I could blog about it or at least point to some good resources.

Edit project.clj

(defproject address_book "1.0.0-SNAPSHOT"
  :description "Address Book"
  :dependencies [[org.clojure/clojure "1.2.0"]
                 [org.clojure/clojure-contrib "1.2.0"]
                 [compojure "0.4.1"]
                 [ring/ring-jetty-adapter "0.2.3"]]
  :dev-dependencies [[swank-clojure "1.2.1"]])

and run

lein deps

Now you should be able to start swank server using

lein swank

2) Break core.clj into different namespaces

Current core.clj is doing two things. Setting up routes and starting jetty server. Lets break it into web_server.clj and routes.clj

rm src/address_book/core.clj

create src/address_book/routes.clj

(ns address_book.routes
  (:use [compojure.core])
  (:require [compojure.route :as route]))

(defroutes address-book
  (GET "/" [] "My Address Book!")
  (route/files "/" {:root "public"})
  (route/not-found "Page not found"))

create src/address_book/webserver.clj

(ns address_book.webserver
  (:use [compojure.core]
        [ring.adapter.jetty]
        [address_book.routes :as routes]))

(future (run-jetty (var address-book) {:port 8080}))

3) At last lets create our Address namespace

In this code, I am going to use atoms for persistence. In future I will probably show how we can persist in mysql db using clj-records.

create src/address_book/address.clj

(ns address-book.address
  (:use [address-book.utils number])
  (:refer-clojure :exclude (find create)))

(def STORE (atom {:1 {:id :1 :name "Siva Jagadeesan" :street1 "88 7th" :street2 "#203" :city "Cupertino" :country "USA" :zipcode 98802}}))

(defn create [attrs]
  (let [id (random-number)
        new-attrs (merge {:id id} attrs)]
    (swap! STORE merge {id new-attrs})
    new-attrs))

(defn find-all []
  (vals @STORE))

(defn find [id]
  ((to-keyword id) @STORE))

(defn update [id attrs]
  (let [updated-attrs (merge (find id) attrs)]
    (swap! STORE assoc id updated-attrs)
    updated-attrs))

(defn delete [id]
  (let [old-attrs (find id)]
    (swap! STORE dissoc id)
    old-attrs))

Create utils/number.clj

(ns address-book.utils.number
  (:import [java.util Date]))

(defn to-keyword [num]
  (if-not (keyword? num)
    (keyword (str num))
    num))

(defn random-number []
  (to-keyword (.getTime (Date.))))

PS: I am not writing tests to keep this blog short. But I would advice everyone to write tests.

4) Lets update routes for add, view and view all functionalities


(ns address_book.routes
  (:use [compojure.core])
  (:require [address-book.address :as address]
            [compojure.route :as route]
            [clj-json.core :as json]))

(defn json-response [data & [status]]
  {:status (or status 200)
   :headers {"Content-Type" "application/json"}
   :body (json/generate-string data)})

(defroutes handler
  (GET "/addresses" [] (json-response (address/find-all)))
  (GET "/addresses/:id" [id] (json-response (address/find id)))
  (POST "/addresses" {params :params}  (json-response (address/create params)))

  (route/files "/" {:root "public"})
  (route/not-found "Page not found"))

(def address-book
     handler)

Compojure comes with GET, POST, PUT, DELETE, HEAD and ANY macro to define routes. These macros are self explanatory. Currently we have used GET and POST to define our routes.
Parsing Parameters 
Compojure binds request parameters to params.
To get all parameters from request you can destructure map like we did in
POST "/addresses" {params :params}  (json-response (address/create params)))

To get particular parameters from request you can use Compojure sugar syntax like we did in

[sourceode](GET “/addresses/:id” [id] (json-response (address/find id)))[/sourcecode]
5)  Lets build our front end

PS: front end code is a hack. Please don’t follow these codes as a good practice.
Edit public/index.html
 
 
 
 



	
	My Address Book
        
        
        
        
        
        



	My Address Book

	
          
            Name 
            Street1 
            Street2 
            City 
            Country 
            ZipCode 
          
	  
            
              Name
              Action
            
            
              row 1, cell 1
              row 1, cell 2
            
          
	
	
          
	    
	      New Address
	      
		Name 
	      
              
		Street1 
	      
               
		Street2 
	      
              
		City 
	      
              
		Country 
	      
               
		ZipCode 
	      

              
	    
          
	
	
	  TechBehindTech -- Siva Jagadeesan
	





Create public/css/address.css
body,html {
    margin:0;
    padding:0;
    color:#000;
    background:#a7a09a;
}
#wrap {
    width:970px;
    margin:0 auto;
    background: #fffeff;
}
#header {
    padding:5px 10px;
    background: #054477;
	text-align: right;
	color: #fffeff;
}
h1 {
    margin:0;
}
#main {
    float:left;
    width:580px;
    padding:10px;
	background-color: #fffeff;
}
h2 {
    margin:0 0 1em;
	background-color: #fffeff;
}
#sidebar {
    float:right;
    width:350px;
    padding:10px;
    background-color: #fffeff;
}
#footer {
    clear:both;
    padding:5px 10px;
    background: #fed47f;
}
#footer p {
    margin:0;
}
* html #footer {
    height:1px;
}

form * {margin:0;padding:0;} /* Standard margin and padding reset, normally done on the body */

legend {
	color:#000; /* IE styles legends with blue text by default */
	*margin-left:-7px;
	font-weight: bold;
	font-size: 20px;
}
fieldset {
	border:1px solid #dedede; /* Default fieldset borders vary cross browser, so make them the same */
}
fieldset div {
	overflow:hidden; /* Contain the floating elements */
	display:inline-block;
	padding: 10px;
}
fieldset div {display:block;} /* Reset element back to block leaving layout in ie */
label {
	float:left; /* Take out of flow so the input starts at the same height */
	width:8em; /* Set a width so the inputs line up */
}

Create public/js/address_book.js

$(document).ready(function() {
    $("#address").hide();
    $("#address-list").showAddressList();

    $(".address-link").live("click",function(e){
        $.getJSON($(this).attr("href"), function(json) {
            $("#address").showAddress(json);
        });
        e.preventDefault();
    });

    $('#address-form').submit(function(event){
        event.preventDefault();
        var $this = $(this);
        var url = $this.attr('action');
        var dataToSend = $this.serialize();
        var callback = function(data){
            $("#address-list").addAddress(data);
        };
        var options = {
            success:   callback,
            url: url,
            type:     "POST",
            dataType: "json",
            clearForm: true
        };
        $(this).ajaxSubmit(options);
    });

});

Create public/js/address_list.js
function action_links(data){
    var link = "edit ";
    link += " | delete";
    return "edit | delete";
}

$.fn.showAddressList = function(){
    return this.each(function(){
        var that = this;
        $.getJSON("addresses", function(json) {
            $(that).html("  Name Action ");
            $.each(json,function(i,data) {
                $(that).append("" + data.name + "" + action_links(data) + "");
            });
        });
    });
};

$.fn.addAddress = function(json){
    var data = json;
    var that = this;
    return this.each(function(){
        var that = this;
        $(that).append("" + data.name + "" + action_links(data) + "");
    });
};

$.fn.showAddress = function(json){
    var data = json;
    var that = this;
    return this.each(function(){
        $(that).slideDown('slow');
        $(that).find("#name").html(data.name);
        $(that).find("#street1").html(data.street1);
        $(that).find("#street2").html(data.street2);
        $(that).find("#city").html(data.city);
        $(that).find("#country").html(data.country);
        $(that).find("#zipcode").html(data.zipcode);
    });
};


Download jquery-1.4.2.min.js, jquery-ui-1.8.1.custom.min.js and jquery.form.js to public/js folder.

 
6)  Start the web server
[/sourcecode]lein repl src/address_book/webserver.clj [/sourceode]

Wow that was lot of stuff.  We will look more into writing our own middlewares in next part.
Source code is now available at github. Created branches for each part.
Part 5 is posted.

Filed under: Clojure Tagged: clojure, code, compojure, jquery, REST, web

Name	Action
row 1, cell 1	row 1, cell 2

Compojure Demystified with an example – Part 3

Siva Jagadeesan — Tue, 17 Aug 2010 04:04:27 +0000

In part2 we saw how to setup a skeleton project with Compojure. In this part we will see how to add static files to a Compojure project.

For our application, we will use JQuery as front end and Clojure as a backend. I will concentrate more on Compojure.

1) Create folders for static files.

mkdir public
mkdir public/css
mkdir public/js

It is better to have static files under a seperate directory that way you could use your webserver to serve these files. For now we will use Compojure to serve these files.

2) Create index.html under public folder

My Address book

My Address Book

3) Start your server

lein repl src/address_book/core.clj

4) Access index.html

Go to http://localhost:8080/index.html . You will get “Page not found” instead of “My Address Book”. This is happening because we have not told compojure about our static files folder.

Route helper functions in Compojure

In Compojure we have two route helper functions. We have seen one already in core.clj file.

1) not-found :

This function is used to capture all other routes that are not defined in Compojure. As we still have not defined index.html we got “Page not found”.

2) files :

This function is used to inform Compojure where static files are present.

Lets change our core.clj to handle static files

(ns address_book.core
  (:use [compojure.core]
        [ring.adapter.jetty])
  (:require [compojure.route :as route]))

(defroutes example
  (GET "/" [] "My Address Book!")
  (route/files "/" {:root "public"})
  (route/not-found "Page not found"))

(run-jetty example {:port 8080})

Restart your server (there is a way to avoid restarting using futures. I will show that in a later blog) and access http://localhost:8080/index.html . Now you should see “My Address Book”.

In this part we saw how we can handle static files. In next part we will implement Add , View and View All functionalities.

PS: Using Hiccup you can create HTML and css in clojure. I will try to cover this in future blogs.

Source code is now available at github . Created branches for each part.

Checkout PART4 …

Filed under: Clojure Tagged: clojure, compojure, web

Compojure Demystified with an example – Part 2

Siva Jagadeesan — Mon, 16 Aug 2010 01:49:59 +0000

In part 1 I introduced the address book application that we are going to build.

In this part we will setup our skeleton project with compojure.

1) Install Leiningen

http://github.com/technomancy/leiningen/blob/master/README.md

2) Create a new project using Leiningen

lein new address_book

3) Add Compojure to our project:

a) Edit project.clj

(defproject address_book "1.0.0-SNAPSHOT"
  :description "Address Book"
  :dependencies [[org.clojure/clojure "1.1.0"]
                 [org.clojure/clojure-contrib "1.1.0"]
                 [compojure "0.4.1"]
                 [ring/ring-jetty-adapter "0.2.3"]])

b) Install dependencies

lein deps

This should install all dependencies of compojure

c) Test whether our setup is working

Edit src/address_book/core.clj

(ns address_book.core
  (:use [compojure.core]
           [ring.adapter.jetty])
  (:require [compojure.route :as route]))

(defroutes example
  (GET "/" [] "My Address Book!")
  (route/not-found "Page not found"))

(run-jetty example {:port 8080})

Run the server

lein repl src/address_book/core.clj

Goto http://localhost:8080 and you should see “My Address Book!”

In next part we will start implementing our functionalities.

Filed under: Clojure Tagged: clojure, compojure, web

Compojure Demystified with an example – Part 1

Siva Jagadeesan — Sat, 14 Aug 2010 19:15:25 +0000

In this part, I am going to talk about what application we are going to implement.

Lets build a simple address book web application using compojure. This will be a RESTFUL application. As we go through this example we will see some interesting aspects of compojure.

Functionalities

Add / Edit / Delete Address book
View All Addresses
View a single Address

Domain

Address

Name
Street1
Street2
City
Country
ZipCode

REST interfaces we will implement

View All Addresses – GET – http://localhost:8080/addresses
View single address – GET – http://localhost:8080/addresses/:id
Add Address - POST – http://localhost:8080/addresses
Edit Address - PUT – http://localhost:8080/addresses/:id
Delete Address – DELETE – http://localhost:8080/addresses/:id

Next part 2 we will setup our base project with compojure.

Filed under: Clojure Tagged: clojure, compojure, web

Map, Reduce and Filter in Clojure

Siva Jagadeesan — Mon, 26 Jul 2010 22:00:13 +0000

Map

Map applies a function to each element in a collection and returns a new collection with the results of each function.

user> (map #(+ 10 %1) [ 1 3 5 7 ])
(11 13 15 17)

Reduce

Reduce applies a function on all elements of a collection and returns a value. This value could also be another collection.

user> (reduce * [2 3 4])
24

Filter

Filter applies a predicate function to each element in a collection and returns a new filtered collection.

user> (filter even? [1 2 3 4 5 6])
(2 4 6)

Filed under: Clojure Tagged: clojure, code, functional, snippet

Different tools to help you test your clojure code

Siva Jagadeesan — Sat, 24 Jul 2010 00:51:08 +0000

Problem :

We need to parse a xml string and be able to query using xpath style tag list.

Ex :


  
   Siva

I need a function that can do this,

(get-value xml :person :name)

returns “Siva”

Solution :

To parse and query xml we need to do these following three things in clojure.

1) Convert xml string (file) to Struct Map

Clojure core comes with a build in xml library (clojure.xml http://richhickey.github.com/clojure/clojure.xml-api.html) that has a parse function which takes in InputStream and returns a struct map that represents xml.

(defn get-struct-map [xml]
  (let [stream (ByteArrayInputStream. (.getBytes (.trim xml)))]
    (xml/parse stream)))

user> (get-struct-map xml)
{:tag :friends, :attrs nil, :content [{:tag :person, :attrs nil, :content [{:tag :name, :attrs nil, :content ["Siva"]}]}]}

This struct map is cumbersome to query.

2) Convert Struct Map to Zipper Data Structure

“A zipper is a technique of representing an aggregate data structure so that it is convenient for writing programs that traverse the structure arbitrarily and update its contents, especially in purely functional programming languages.” http://en.wikipedia.org/wiki/Zipper_%28data_structure%29

To make it easy for us to traverse we will change struct map to zipper data structure. Clojure comes with zip library ( it is short form for zipper ) http://richhickey.github.com/clojure/clojure.zip-api.html

We will this zip library to convert xml struct map to zipper data structure.

user> (clojure.zip/xml-zip xml-struct)
[{:tag :friends, :attrs nil, :content [{:tag :person, :attrs nil, :content [{:tag :name, :attrs nil, :content ["Siva"]}]}]} nil]

3) Use Zip-filter library and query zipper data structure.

Now that we have our xml in zipper data structure we could use zip-filter library that is present in clojure.contrib. http://richhickey.github.com/clojure-contrib/zip-filter-api.html

user> (clojure.contrib.zip-filter.xml/xml-> zipper-struct :person :name)
("Siva")

Putting this altogether

(ns com.sivajag.utils.xml
  (:import (java.io ByteArrayInputStream))
  (:require [clojure.xml :as xml])
  (:require [clojure.zip :as zip])
  (:require [clojure.contrib.zip-filter.xml :as zf]))

(defn get-struct-map [xml]
  (if-not (empty? xml)
    (let [stream (ByteArrayInputStream. (.getBytes (.trim xml)))]
      (xml/parse stream))))

(defn get-value [xml & tags]
  (apply zf/xml1-> (zip/xml-zip (get-struct-map xml)) (conj (vec tags) zf/text)))

user> (get-value xml :person :name)
"Siva"

Happy Coding!!!

Filed under: Clojure Tagged: clojure, testing

Clojure futures – A Walkthrough

Siva Jagadeesan — Wed, 14 Jul 2010 04:22:18 +0000

What is Future?

It is a simple mechanism to execute a snippet of code on a background thread.

How to use Future?

We can use Future in two ways.

1) Using macro “future”

	
  (def f
	(future
 		(prn "going to sleep...")
		(Thread/sleep 10000)
		(prn "slept good")))

2) Using function “future-call”

(defn long-process[]
  (Thread/sleep 10000))

(future-call long-process)

Under the shell both these ways are equivalent. The macro is syntax sugar and it gets converted to future-call.

Even though “long-process” function is put to sleep for 10 secs, it will return immediately. It returns a “future” object. We can use this object to inspect whether the function has been executed in background thread.

(future-done? f)
(future-cancelled? f)
(future-cancel f)

Limitations of Future?

As you known clojure comes with a bounded thread pool and unbounded thread pool. Mostly CPU bound operations should be using bounded thread pool and IO operations should be using unbound thread pool. Clojure Future uses unbounded thread pool, so it is good for IO operations but not for CPU bound operations. So for CPU bound operations Future might not be a good choice. But there is a open source project that helps to overcome this limitations. Check out http://github.com/amitrathore/medusa

Filed under: Clojure Tagged: clojure, future, snippet

clj-web-crawler – Web Crawling tool in clojure

Siva Jagadeesan — Wed, 07 Jul 2010 07:22:32 +0000

I was looking for a simple web crawler in clojure and came across clj-web-crawler (http://github.com/heyZeus/clj-web-crawler) . It took me less than 2 minutes to get started. (thanks to Leiningen)

1) In project.clj add these two lines for :dependencies

                 [com.jmeeks/clj-web-crawler "0.1.0-SNAPSHOT"]
                 [org.clojars.kjw/commons-httpclient "3.1"]

2) Run lein deps and you ready to play with clj-web-crawler.

Checkout http://github.com/heyZeus/clj-web-crawler for more information

Filed under: Clojure Tagged: clojure, crawler, libraries

Leiningen Tips

Siva Jagadeesan — Tue, 29 Jun 2010 20:08:49 +0000

Tip 1 : Adding License info

To add your license info, add these following lines in project.clj

:license {:name "Eclipse Public License - v 1.0"
          :url "http://www.eclipse.org/legal/epl-v10.html"
          :distribution :repo
          :comments "same as Clojure"}

Tip 2: Turning off auto delete of lib folder

:disable-implicit-clean true

Tip 3: Turning on reflection warning calls

:warn-on-reflection true

Tip 4: Changing JVM level options

:jvm-opts "-Xmx1g"

Filed under: Clojure Tagged: build, clojure, Leiningen

Parsing XML in Clojure

Siva Jagadeesan — Fri, 25 Jun 2010 13:57:46 +0000

Problem :

We need to parse a xml string and be able to query using xpath style tag list.

Ex :


  
   Siva

I need a function that can do this,

(get-value xml :person :name)

returns “Siva”

Solution :

To parse and query xml we need to do these following three things in clojure.

1) Convert xml string (file) to Struct Map

Clojure core comes with a build in xml library (clojure.xml http://clojure.github.com/clojure/clojure.xml-api.html) that has a parse function which takes in InputStream and returns a struct map that represents xml.

(defn get-struct-map [xml]
  (let [stream (ByteArrayInputStream. (.getBytes (.trim xml)))]
    (xml/parse stream)))

user> (get-struct-map xml)
{:tag :friends, :attrs nil, :content [{:tag :person, :attrs nil, :content [{:tag :name, :attrs nil, :content ["Siva"]}]}]}

This struct map is cumbersome to query.

2) Convert Struct Map to Zipper Data Structure

To make it easy for us to traverse we will change struct map to zipper data structure. Clojure comes with zip library ( it is short form for zipper ) http://clojure.github.com/clojure/clojure.zip-api.html

We will this zip library to convert xml struct map to zipper data structure.

user> (clojure.zip/xml-zip xml-struct)
[{:tag :friends, :attrs nil, :content [{:tag :person, :attrs nil, :content [{:tag :name, :attrs nil, :content ["Siva"]}]}]} nil]

3) Use Zip-filter library and query zipper data structure.

Now that we have our xml in zipper data structure we could use zip-filter library that is present in clojure.contrib. http://clojure.github.com/clojure-contrib/zip-filter-api.html

user> (clojure.contrib.zip-filter.xml/xml-> zipper-struct :person :name)
("Siva")

Putting this altogether

(ns com.sivajag.utils.xml
  (:import (java.io ByteArrayInputStream))
  (:require [clojure.xml :as xml])
  (:require [clojure.zip :as zip])
  (:require [clojure.contrib.zip-filter.xml :as zf]))

(defn get-struct-map [xml]
  (if-not (empty? xml)
    (let [stream (ByteArrayInputStream. (.getBytes (.trim xml)))]
      (xml/parse stream))))

(defn get-value [xml & tags]
  (apply zf/xml1-> (zip/xml-zip (get-struct-map xml)) (conj (vec tags) zf/text)))

user> (get-value xml :person :name)
"Siva"

Happy Coding!!!

Filed under: Clojure Tagged: clojure, code, snippet, xml