Tech behind Tech

Raw information. No finesse :)

Posts Tagged ‘libraries

MQL – A Clojure library for querying Nested Maps

with 8 comments

I was working on a feature that needed me to query nested map structure. I wanted to do it in a generic way. Amit pointed me to rql, a library for dealing with collections of records in clojure. I thought it will be helpful if I have something similar to rql for querying map, so I created mql.

Given a map

(def m
       :visits {
                :id-1 {
                       :last-ts "1284166000040"
                       :first-ts "1274166000000"
                       :duration "40"}
                :id-2 {
                       :last-ts "1274166000040"
                       :first-ts "1274166000000"
                       :duration "40"}
                :id-3 {
                       :last-ts "1264166000040"
                       :first-ts "1274166000000"
                       :duration "40"}}

              :id-1 {:promo "p2"}
              :id-2 {:promo "p1"}}

       :purchase {
                  {:order-id "order-id-1"
                   :total-dollars "970.00"
                   :purchase? "true",
                   :merchant-total-dollars "1000.00"}
                  {:order-id "order-id-2"
                   :total-dollars "1000.00"
                   :purchase? ""
                   :merchant-total-dollars "1000.00"}}}})

Select: simple select

mql.core> (select [:cid :promo] m)
{:id-1 {:promo "p2"}, :id-2 {:promo "p1"}}

Select: with filtering

mql.core> (select [:cid :purchase] (where [* :total-dollars] :gt 980) m)
([:id-3 {:order-id "order-id-2", :total-dollars "1000.00", :purchase? "", :merchant-total-dollars "1000.00"}])

You can currently use :gt,:ge,:lt,:le and :eq as logical operators in where clause. In where clause for key-seq you can use * if a key is dynamic. The last key in where clause key-seq should not be *.


  • There could be only one * in Where Clause
  • Select clause cannot have * now

This is an early version of this library. It is doing what I need for now. So if you have any requirements/idea, you can send me a patch or let me know, I will hack when I get some free time.

Please feel free to send me your feedback on syntax, code style … etc

Written by Siva Jagadeesan

September 8, 2010 at 10:43 pm

Posted in Clojure

Tagged with , ,

clj-web-crawler – Web Crawling tool in clojure

leave a comment »

I was looking for a simple web crawler in clojure and came across clj-web-crawler ( . It took me less than 2 minutes to get started. (thanks to Leiningen)

1) In project.clj add these two lines for :dependencies

                 [com.jmeeks/clj-web-crawler "0.1.0-SNAPSHOT"]
                 [org.clojars.kjw/commons-httpclient "3.1"]	

2) Run lein deps and you ready to play with clj-web-crawler.

Checkout for more information

Written by Siva Jagadeesan

July 7, 2010 at 7:22 am

Posted in Clojure

Tagged with , ,


Get every new post delivered to your Inbox.

Join 147 other followers

%d bloggers like this: