In my previous post about Accessing the Datastore I set up basic security using a security-constraint element in the deployment descriptor (web.xml). This was simple, as the app didn’t have to be aware of security concerns at all. The downside is that the app doesn’t know if the user is logged in and can’t react to that. For example, the “Create new post” link is shown to all users, only after clicking it (and logging in) they get an ugly error message about missing privileges. This is bad usability, so let’s use the App Engine Users API and move the authentication and authorization into the app.

Setting up the routes

To make things easier, I changed my route definitions, separating the public routes from those that need admin privileges. All admin route URLs now start with /admin:

The admin-routes are only allowed to be accessed by logged-in users with admin privileges.appengine-clj already comes with two middleware functions that help with this: wrap-with-user-info adds references to the UserService and (if a user is logged in) User objects from the App Engine API to each request. wrap-requiring-login checks that the user is logged in before passing the request on to the wrapped handler – if not, the user is redirected to the login page.

There’s no wrap-requiring-admin (yet), so I quickly wrote it myself:

wrap-requiring-admin depends on wrap-requiring-login, which in turn depends on wrap-with-user-info, so I have to decorate my admin-routes handler with all three:

Finally, the routes are combined into my main handler:

Why can’t I just put admin-routes in there, just like public-routes? The problem is, that the middleware I wrapped around admin-routes jumps in before the route-matching. So even if admin-routes can’t match the request URL and passes on control to the next handler, it first makes sure that the user is logged in as an admin. In this case, the not-found handler (which always has to be last) could only be reached by admins, all other users would have to login and then get a 403 error when they enter a non-existing URL. Therefor, I have to make sure that the admin-routes handler is only called for URLs starting with /admin.

Checking the users login status

So far, the new code does the same thing the old configuration did, I haven’t won anything. So let’s make the site a little more dynamic and change the output depending on the users login status. I changed the sidebar to display information about the current user and login/logout links. Also, the “Create new post” link is only shown for logged-in admin users:

(Note that side-bar now is a function, since the content is dynamic.)

I’ve achieved my goals: I can login and logout and I only see the links I’m allowed to click. I can run this code using the local dev_server and I can deploy it to the Google servers (see my previous post on how to do this).

But in the interactive development environment I set up in my last post, nothing works! I’m always logged out and the login link is broken. Let’s fix that.

Making logins work in interactive development

The local implementation of the App Engine Users API calls an instance of ApiProxy$Environment, which I have to provide, to figure out if a user is logged in. In my last post, I set up a very minimal proxy, that always answers this question with “no”. Here’s the relevant snippet:

This needs to be smarter. I decided to store information about the current user globally in an atom. Of course, this implies that the server can only be used by one user at a time – for a production system this would be an incredibly stupid implementation, for local development I think it’s ok. Other options would be to store the login information in session variables or directly in a cookie. Storing it globally has the advantage, though, that I can easily view and modify the current login state from the REPL, which eases debugging (plus it’s simple to implement!).

Here’s the definition of the atom holding the login information, prefilled with some reasonable default values:

The updated Environment proxy just reads from the atom:

I added two helper functions to easily modify the atom:

Now I can login and logout by calling the functions from the REPL and the pages served by my Jetty server immediately reflect this. But the login and logout links are still broken. I need to define handlers for these:

The login-form function just builds an exact copy of the login page provided by the Google dev_server.

Last but not least, I have to update the start-server function to combine these handlers with my app (the change is in line 9):

That’s all – a functioning local implementation of the Users API complete with working login page. I hope you enjoy it!

As always, the complete source code can be found on Github, the version as of this writing is here. You can see the deployed app here (of course I’m the only admin user, so you might want to try it locally to see the full functionality…). Questions and suggestions are very welcome in the comments below!

In my last post, I promised to fix my local development setup to enable the interactive development style typical for Clojure and make the App Engine services (such as the datastore) available from the REPL.

Others have already tackled the same problem. The best resource I found is the hackers with attitude blog, another one is here. My code is largely based on these contributions, I rolled my own version mainly to get a better understanding of the setup.

Initializing the App Engine services

To use the App Engine APIs outside of the Google servers, a local ApiProxy and Environment needs to be provided. While the ApiProxy needs to be initialized only once per JVM, the Environment needs to be set for every thread on which API calls are made. This is trivial for the REPL, which runs in one thread. It’s a bit more work when you want to run a local Jetty server, since it spawns new threads for handling requests. Fortunately, the design of Ring makes it easy to add so-called middleware to an existing web app that can handle the environment setup.

Enough said, here’s the code. I’ll go through it def by def below.

The code is in a separate namespace, so it doesn’t get AOT-compiled and deployed with the rest of the app. I’m using an atom to store the Jetty server instance. Using defonce was helpful while developing this, because I could recompile the file (C-c C-k in Emacs) without losing the reference to the running Jetty server.

The two functions set-app-engine-environment and set-app-engine-delegate do the necessary setup work on the per-thread and per-jvm basis, respectively.

init-app-engine just calls these two functions. It’s intended to be called from the REPL, after which you’re able to use API calls like create-entity in the REPL.

wrap-local-app-engine is a Ring middleware that sets the environment for the current thread before passing the request on to the wrapped Ring (or Compojure) app.

The start-server function takes a Ring app, does the per-JVM setup, wraps the app with the middleware for the per-thread setup and starts a Jetty server running the app. If there already is a Jetty server stored in the atom, it is stopped first, so you can use the function to restart the Jetty as well. The :join? false argument is important, otherwise the call to run-jetty will not return.

I also added the wrap-file middleware to serve static files (and wrap-file-info to add Content-Type and Content-Length headers). This mimics the behaviour of the Google servers, which by default serve all files included in the war directory. (Note that, unlike the Google servers, this setup also serves the files in the WEB-INF directory. In a production system that would be a security concern, for local development I don’t mind.)

Last (and also least interesting), the stop-server function stops the Jetty server (and sets the atom back to nil).

Setting up the classpath

To use the local API implementations we need some additional jars on the classpath. Here’s the updated project.clj:

The new dependencies go into dev-dependencies, since they mustn’t be deployed with the app. However, in the current development version of Leiningen 1.2 (which separates dependencies and dev-dependencies into different directories), the dev-dependencies are apparently only intended for Leiningen plugins such as swank-clojure – they are not put on the classpath for the REPL. I had to patch Leiningen to make this work. Here’s the diff:

I’ll try to get this (or something similar) into Leiningen. Update: I wasn’t the only one with this problem. It was recently patched in Leiningen. Stable Leiningen (1.1) will probably work out of the box – I haven’t tried.

Running lein deps installs the new dependencies. As I said in my last post, you’ll likely have to manually install the jars from the App Engine SDK into your local Maven repository. Here’s an example command for one of the jars:

Putting it to use

Run lein swank, enter M-x slime-connect in Emacs to connect to the REPL and code away as usual. To call functions that make use of the App Engine API, enter this in the REPL:

To start a Jetty server, just enter:

example is the name of the Compojure app defined by defroutes in core.clj. Update: Using var here allows you to change the definition of example itself without having to restart the server.

What’s next?

The next thing I’m working on is using the Users API for authentication and authorization (instead of the simple security-constraint method described in my last post). I’ll need to make some changes to this local-dev code in order to properly test that locally. So stay tuned…

As always, questions and suggestions are very welcome in the comments section below!

In my last post, I managed to deploy a Compojure app to Google App Engine. Serving static content isn’t very exciting, though. Pretty much every app will need some way to store and retrieve data. So let’s try to access the App Engine Datastore.

New Dependencies

To use the datastore API I need to include a jar that comes with the GAE SDK in my app. I could call the Java API directly, but there are already a few Clojure libraries that provide friendly wrappers around it. One of the first (that I know of) was appengine-clj by John Hume (who was also one of the first to write about using Clojure on GAE).

I decided to go with this fork of appengine-clj by Roman Scherer, which seems to be more complete and actively maintained. Another interesting option would be clj-gae-datastore by the people at freiheit.com.

I updated my project.clj file with the new dependencies:

I’m using Hiccup (formerly part of Compojure) for HTML generation. Running lein deps runs into an error, because it can’t find the Google SDK jar in the public repositories. Luckily, Leiningen (or Maven) already tells me how to fix this by installing the jar from the SDK download into my local Maven repository – I just have to copy and paste the command from the error message and enter the path to the local jar. After that, lein deps executes cleanly and copies the new dependencies into my lib dir.

Along with updating the dependencies in project.clj I have to import the needed symbols into my namespace:

The choice of using :use or :require is pretty arbitrary in this case – I just used both to demonstrate the different options. With :require I need to call the functions using the full namespace (using a short alias), with :use they are imported into my namespace. For the latter case, I explicitly named all the definitions I need using :only. This is pretty verbose and not strictly necessary, but for a little howto like this I want you to immediately see where every function comes from, so you don’t have to rummage through all the libraries (although the doc function makes this easy…). I guess it generally is a good idea to not clutter your namespace with definitions you don’t need.

Storing Data

Now comes the more interesting part: Actually accessing the datastore. I want to be able to create simple blog posts, consisting of a title and a body. I need two new routes, one for displaying a form, and one that is used as the action URL for the form:

Here’s the code that handles the form submission:

Amazingly simple. The create-entity function just takes a Clojure map, which needs to have a “:kind” entry, and stores it in the datastore. After that I issue an HTTP redirect to the main page.

Retrieving Data

Retrieving data is just as simple. On the main page, I just display all posts:

The h function takes care of escaping special characters in the user input, so I don’t run into any cross-site scripting trouble. render-page is a little helper function that takes care of constructing the common HTML around the payload for all pages.

As usual, the whole code can be found at Github. The version as of this writing is here.

Basic Security

I don’t want the whole world to be able to post to my blog, so I need some authentication and authorization. I could use the App Engine Users API, but I’ll leave that for a later post. Instead I’ll go the simple route and enable security for some URLs in the deployment descriptor. That way the application itself is blissfully unaware of it. I just need to add this to the web.xml file:

Now only logged-in admin users can post new entries. You can see the deployed version of the app here: http://v0-2.latest.compojureongae.appspot.com/

What about the REPL!?

Okay, everything works fine. I can compile the project, start the dev_appserver to test it locally and deploy it to the Google cloud (see my last post for the steps). But what about interactive development? When I try to call e.g. the create-entity function from a REPL, I only get an Exception. So I can develop and deploy working software, but I’m back to the dreaded edit-compile-run cycle – that’s not the Clojure way.

I need to fix this, but it’ll have to wait until the next post. Sorry…