Category Archives: Javascript

Guided Description Forms

Today’s blog post moves a bit away from my current trend in Node.js server side code and moves on to a UX/UI problem that I solved recently.

The problem

You have a description field – just a text/html wysiwyg field that the user is supposed to fill out. The problem is that for whatever reason you want the user to add specific data to the field or in a specific format and our users are not very good at following instructions as we all know! Now you could just pre-populate the field with some titles and blanks for them to fill out, but this isn’t very “nice” to use. I came up with the following solution (cause running code is worth more than the proverbial 10,000 words):

Explained

This code runs around 2 main directives called: ‘dynamicInput’ and ‘formDescription’. Pretty simple, the former takes in some options that change how it is displayed, the latter watches all the fields passed into it to update one ng-model with the compiled code.

Hope this helps out someone else, if you have any comments/improvements or use this in some cool app – Please let me know in the comments!!

Node Routes with Functional Programming

Today’s post is a little short and a little late as I’m pretty busy at work!

I’m going to do a quick functional programming example that I do in my routing of my application, now there probably is an easier way, but as allways programming is a process.

Here’s the code:

_boolHandle = (_f) -> (req, res) ->
  _testExists req.params.ticketID, res, -></pre>
    _f req, res, (err, results) -> if err then res.status(500).json err else res.status(200).json results or true
app.post '/api/ticket/:ticketID/metadata', _boolHandle (req, res, cb) -> $tickets.metadata req.params.ticketID, req.body, cb

This is essentially a post handler in express 4 that calls a function that returns a function to call a function to call a standard response!

If we break it down, when we register the handler, normally this would be written as follows:


app.post '/api/ticket/:ticketID/metadata', (req, res) -> res.status(200).json true

My first step was figuring out that every call I made to the $ticket object calls it’s callback with (err, result) and returned the results, so my code then looks like this:


app.post '/api/ticket/:ticketID/metadata', (req, res) -> $ticket.metadata req.params.ticketID, (err, result) -> if err then res.status(500).json err else res.status(200).json result

Because I’m lazy I decided I didn’t want to write out the callback every time, it’s always the same so why bother? So my first functional programming block was this:

_boolHandle = (_f) -> (req, res) ->
  _f req, res, (err, results) -> if err then res.status(500).json err else res.status(200).json results or true
app.post '/api/ticket/:ticketID/metadata', _boolHandle (req, res, cb) -> $tickets.metadata req.params.ticketID, cb

As you can see I abstracted out the response by creating a wrapper around it. The other part I have in the code is the function that tests whether the ticketID actually exists (_testExists), if it exists it calls the callback else it returns a “ticket not found” error.

I hope this explanation helps someone out, if you have a better way of doing this – I’m all ears!!

Node.js As a Service on Windows

This week I am going to outline the difficulties I have in getting my app running as a service on windows.

Normally I would start up my server either by calling “coffee app.coffee” or “npm start”. If you look around at the blogs and posts about this you’ll find that all of them refer to running the service via “node app.js”. Which just plain won’t work for me.

Beforehand I’ve used nssm, so I decided to give it a go again, I settled for getting it to run “npm start”.

Steps!

  1. Figure out the location of your node.js installation (node.exe), for me it was “C:\Program Files\nodejs\node.exe”.
  2. Fire up nssm in the add service mode.
  3. Enter in your node installation for Application Path.
  4. Enter in the directory of your project in Startup directory.
  5. In Arguments put in the quoted location of the npm-cli.js file and then “start” (with quotes). This should be relative to your node installation, for me it’s “C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js”.
    In total the Arguments field should look like:

    "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "start"
  6. Hit save and your service should run as if you called “npm start”!

2014-08-06_1445

CRM Migration (Part 2)

Part One

Continuing on my series on gradual migration from Coldfusion to Node-js today I’ll be writing up about the authentication system that allows me to have one login that tells both node and coldfusion that the user is authenticated. (Sorry about all the &gt; showing up everywhere – Can’t figure out how to get rid of them)

Authentication

Having worked my way through many iterations of this code, I’m going to start with a quick rundown of the process it took me to get to the end result.

  1. “I can just proxy through all requests to the CF Server, that sends the cookies as well.” Good points of this is that there’s a node module for that (isn’t there always!), bad point is that if you want to have node communicate with coldfusion as well, for example sockets that call logic on CF, you can’t use the cookie from the CF Server if it’s set up correctly to prevent cookie jacking. So this was a no go as essentially I ended up with 3 sessions, Browser to CF, Browser to Nodejs and Nodejs to CF, not good!!
  2. “OK, so if I write my own proxy code I can control the cookie right?”. Well, not quite though we’re getting there, again with the same origin restriction on cookies this didn’t work so well.
  3. “Right, so I have to have all communication with CF sent via node and not store the CF cookie on the browser.” This is about right!

How It’s Done

So if you remember last week there was a function and variable I said I’d explain this week, testCookie and req.session.proxyCookie? Well both of those are to do with my authentication solution.

We start off with the basic cookie and session setup for node:


@use cookieParser security.cookieSecret
 @use session
  cookie:
    httpOnly: true
    secure: false
    maxAge: 3600000 * 8 # hour in ms * 8 = 8 hours
  secret: security.cookieSecret
  store: sessionstore
  saveUninitialized: true
  resave: true

Here we’re using a couchbase memcached session store to store our cookies with some extra settings.

Next we have three functions for working with the Cookies these are:

parseCookie = (cookie) -&gt;
 _cookie = request.jar()
 if cookie? then _cookie.setCookie require('tough-cookie').Cookie.parse(_c), security.proxyAddress for _c in cookie
 _cookie
testProxyCookie = (session) -&gt;
 try
   _foundCFID = false
   for key, cookie of parseCookie(session.proxyCookie)._jar.store.idx[security.proxyCookieDomain]['/']
     if key.toLowerCase() is 'cfid' and cookie.value then _foundCFID = true
     cookie.lastAccessed = new Date().toISOString()
   return _foundCFID
 catch
   return false
 
@testCookie = (req, resp, cb, isLogin=false) =&gt;
  if req.session? and (not testProxyCookie(req.session) or isLogin) and resp.headers and resp.headers['set-cookie']
    req.session.proxyCookie = request.jar()
    req.session.proxyCookie resp.headers['set-cookie']
    @getSessionCookie req, (sidCookie) -&gt;
    sessionstore.set sidCookie, req.session, cb
  else cb()

@getSessionCookie = (req, cb) -&gt;
  cookieParser(security.cookieSecret) req, {}, (parseErr) -&gt;
  if parseErr then return next new Error 'Error parsing cookies.'
  
  # Get the SID cookie
  EXPRESS_SID_KEY = 'connect.sid'
  cb (req.secureCookies and req.secureCookies[EXPRESS_SID_KEY]) or
     (req.signedCookies and req.signedCookies[EXPRESS_SID_KEY]) or
     (req.cookies and req.cookies[EXPRESS_SID_KEY]), sessionstore

@sessionAuth = (req, res, next) =&gt;
  @getSessionCookie req, (sidCookie) =&gt;
    req.sessionroomid = sidCookie
    if testProxyCookie req.session then next()
    else request
      uri: "#{security.proxyAddress}index.cfm/users/nodetestlogin.json"
      method: 'GET'
    , (err, resp, data) =&gt; @testCookie req, resp, next, true
@all '*', @sessionAuth

Let’s Break it Down!!

So, to start off when a user hits a page, the `@all ‘*’, @sessionAuth` gets hit. this in turn calls the `getSessionCookie` which if you haven’t guessed loads the cookies sent to node and finds the ‘connect.sid’ cookie value. We then store this cookie value in the request as the req.sessionroomid which we’ll use later for sockets and updating just one client across all his tabs (I think this may be a security hole but I’m yet to go back and review it yet). Then we call `testProxyCookie`, this function does two important things, first, it checks that the cookie we have stored in the session for communicating with CF is valid, and second it updates the lastAccessed value. Updating the lastAccessed value is important as when you send the cookie through CF uses this to detect if your session should have timed out yet or not! If you don’t update this and have say a 30 minute session timeout on CF, then even if you keep contacting CF constantly, 30 minutes after your first contact your session will expire!! Back in `sessionAuth` if the proxy cookie is valid (`testProxyCookie()` returns true) then we continue via `next()`, if not we send a request to the CF server using request and then use `@testCookie` to set the proxyCookie from the result.

In short `@testCookie` reads the response from the CF Server, parses the cookies and then saves the value in the session value, we later use that cookie with request to authenticate to the CF Server. Beyond the above you can setup your node.js server as you normally would and this will take care of all the integration.

Till Next Time (Which I am yet to decide what I’ll write about!).

UPDATE

I found a nasty bug that runs around the storage of tough-cookie objects – basically it doesn’t work! My solution was to store the ‘set-cookie’ headers instead and then add a ‘parseCookie’ method to convert it into the actual cookie jar object. This means that when you make the call to the CF server you do the following:


request
  uri: "#{security.proxyAddress}/index.cfm"
  method: 'GET'
  jar: parseCookie req.session.proxyCookie
, (err, resp, data) =&gt; #handle response here!

 

CRM Migration (Part 1)

Recently I’ve been able to start moving our internal Coldfusion based CRM onto Node.js, I’m going to start chronicling the difficulties and challenges this has produced and my solutions to these problems in this multi-part series CRM Migration. My aim is to get one of these up every wednesday.

Access

The first problem I came across is the nastiest one for a partial migration; How do I keep one login for both sites?

To solve this we need to solve a simpler question – How do I access legacy coldfusion pages that we aren’t migrating yet?

I tried using node-proxy, but that didn’t go so well after a while due to cookies making sessions do weird things, my eventual solution was as follows:

@get '/index.cfm*', (req,res) ->
	request
		uri: "#{security.proxyAddress}#{req.url.substring 1}"
		method: 'GET'
		jar: req.session.proxyCookie
	, (err, resp, data) => testCookie req, resp, -> res.send resp.statusCode, resp.body
@post '/index.cfm*', (req,res) ->
	request
		uri: "#{security.proxyAddress}#{req.url.substring 1}"
		method: 'POST'
		jar: req.session.proxyCookie
	, (err, resp, data) => testCookie req, resp, -> res.send resp.statusCode, resp.body

NOTE: I started off using zappajs, but then migrated to Express 4 so I replicated the helpers. The ‘@’ is essentially a reference to the express server variable.

I’ll start by explaining the reasoning behind the URL structure; ‘/index.cfm*’. The CRM in coldfusion runs on CFWheels, a MVC library, and I never got the url rewrites to work on IIS so an example format for Coldfusion URL’s is ‘/index.cfm/{controller}/{action}/{key}’. The two above handlers basically state that any request that starts with ‘/index.cfm’ is targeted to the coldfusion server, in case you haven’t guessed my approach is to field ALL http requests to the node.js server which will act as a proxy for the Coldfusion server if we haven’t migrated that across using node.js’s built in request library.

That’s basically it for access to the old Coldfusion pages, you may be curious about the ‘testCookie’ function and where on earth this ‘req.session.proxyCookie’ variable comes from, but that will be in Part 2.

Hint: It’s about authenticating the user for node.js and coldfusion!

See you all next week!

If you want to know about my basic setup for Node.js projects with express see my Nodejs Website Base.

Securing Express Routes with PassportJS

It’s been a while since I posted anything, but I finally found something worth blogging about! Besides my trip to India at the beginning of the year and not much work recently…

Recently I’ve been building a web app for my gaming clan to help them run their practices and also to help me learn more of Node.js and NOSQL (CouchDB in particular). My first big problem in building this app was that of user login, or authentication. After googling around a bit for some modules to help with this (I really like the modular nature of node.js!) I came across a unified login system called Auth0, which I have to say is really awesome. I decided to go with this as who hasn’t thought that letting people login using facebook or similar would be cool but it’s too hard to bother! After running through their node.js tutorial which makes use of PassportJS and their authentication strategy passport-auth0 I came up with the following basic login/authentication code. (This is stripped down a bit, for my full code see here)


http = require 'http'
express = require "express" # Note this is Express v3.5.x

app = module.exports = express()

httpServer = http.createServer app

passport = require 'passport'
Auth0Strategy = require 'passport-auth0'
security = require('./security')

strategy = new Auth0Strategy 
	domain:       security.auth0.domain
	clientID:     security.auth0.clientID
	clientSecret: security.auth0.clientSecret
	callbackURL:  '/callback'
, (accessToken, refreshToken, profile, done) ->
	console.log "Hit Auth0 strategy"
	done null, profile

passport.use strategy

# This is not a best practice, but we want to keep things simple for now
passport.serializeUser (user, done) -> done null, user

passport.deserializeUser (user, done) -> done null, user

helmet = require 'helmet'
	
app.configure ->
	app.use express.logger 'dev'
	app.set 'views', "#{__dirname}/views"
	app.engine 'jade', require('consolidate').jade
	app.set 'view engine', 'jade'
	
	app.use helmet.xframe(), helmet.iexss(), helmet.contentTypeOptions(), helmet.cacheControl()
	app.use express.json(), express.urlencoded()
	app.use express.methodOverride()
	app.use express.cookieParser()
	app.use express.session
		secret: security.cookieSecret
		#cookie:
			#httpOnly: true
			# secure: true # for HTTPS only
	app.use passport.initialize()
	app.use passport.session()
	app.use require('express-validator')()
	
app.all '/api/*', auth

auth = (req, res, next) -> if req.isAuthenticated() then next() else res.redirect '/login'

# Auth0 callback handler
app.get '/callback', passport.authenticate('auth0',
	failureRedirect: '/login'
), (req, res) ->
	if not req.user then throw new Error 'user null'
	if req.user._json.appid?
		req.session.dbconn = require('./../startup_security/nano').nano req.user._json.user_id, req.user._json.dbpass
		res.redirect "/#{req.user._json.appid}/manage"
	else res.redirect "/first-login"

# warning this would log you out of facebook if you logged in via the facebook button - don't use this in live!
app.get '/logout', (req, res) ->
	req.logout()
	delete req.session
	delete req.user
	res.redirect "https://simeonc.auth0.com/logout?returnTo=http://tawmanager.localhost:3000/"

app.get '/login', (req, res) -> res.render 'login.jade'
app.get '/first-login', auth, (req, res) -> res.render 'first-login.jade'

httpServer.listen 3000

console.log "Server Running on port 3000 via ExpressJS"
console.log "Authentication Via Passport.js and Auth0"

Now this is my final code after some help from the owner of passport-node0 here. Now I’ll detail some of the quirks I had to get through that are really, really poorly documented.

First; Authenticating individual routes. Initially from reading through what other people have written and the other information out there I tried this:

<pre>app.get '/first-login', passport.authenticate('auth0'), (req, res) -> res.render 'first-login.jade'</pre>

That is WRONG, and meant I spent many many hours looking at 302 redirects from that page back to the login page for no discernable reason…
Correct is:

auth = (req, res, next) -> if req.isAuthenticated() then next() else res.redirect '/login'</pre>
<pre>app.get '/first-login', auth, (req, res) -> res.render 'first-login.jade'</pre>
<pre>

Secondly; Authenticating all routes matching a particular prefix. From here and there around the web I thought you did this via:

app.use '/api', auth

Turns out that’s also wrong (this ones in the docs so I felt really stupid when I found it). Here’s the correct one:

app.all '/api/*', auth

textAngular

I’d like to introduce you all to a project I have been contributing to recently. It is called textAngular. It is a WYSIWYG editor designed from the ground up to work in AngularJS with no external libraries and, thanks to document.execCommand, no IFrames either!

We’ve recently released version 1.1.2 and I highly recommend that you check it out if you work with AngularJS in any way.
Check out the GitHub page for the Wiki and Source code: https://github.com/fraywing/textAngular

AngularJS drag ‘n’ drop re-order in ngRepeat

EDIT I’ve just got a github up for this here: https://github.com/SimeonC/ngRepeatReorder. I’ve fixed some of the bugs in the code and updated it to run on the 1.2.x angularjs codebase.

I’ve been busy on some small projects that I’ve been doing recently and a little problem I’ve had to overcome is the difficulty of doing drag ‘n’ drop reorders in an ngRepeat.

After looking around the web I came up with the following solution:

See the Pen AngularJS ngRepeat Drag n Drop by SimeonC (@SimeonC) on CodePen

How It Works

In short this works by starting with the ngRepeat code in angular.js. Which in itself is complex as you have to also copy over a bunch of functions that it uses that are hidden elsewhere in the codebase. Then you add all the lovely complexity of moving elements around – I decided to simplify this by limiting the movement within the list bounds and only do vertical re-ordering. I animate this by fiddling with the margins – creating a gap where the ‘dragged’ element is going to drop into and then using +/- margins to position the dragging element correctly. Also we add in classes for display purposes.

The last hurdle I had to overcome is to modify the ngRepeat element before our modified ngRepeat compiles the element, I did this with a second directive that executes before the ngRepeat and adds the event tags onto the element.

Breaking up really long words Cross browser

Best solution for cross browser css only way of stopping really long words from overflowing.

Why I’ve come to love functional programming

Recently I have been starting to learn Node.js and inadvertently changed my opinion of functional programming VS Object Oriented programming. During my work on supporting legacy Coldfusion sites I stumbled across some code that forms the basis of this comparison of functional programming VS Object Oriented programming. (PS if you want a good introduction to the differences look on over to: http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html).

This particular example will be Coldfusion and Pseudo-javascript (or coffeescript) and focuses around a multi-keyword search on 3 columns on 2 tables. Our tables are as follows:

Table Name: Jedi
With Columns: Name, Planet, Lightsaber_Colour.

Table Name: Sith
With Columns: Name, Catch_Phrase, Teacher.

First of all we will define a master function, and a convenience function for searching each of the tables, we do this for an attempt at reusability. We’ll call this main search function “galacticSearch” which will return a SQL string and take the arguments: Keyword (required, string), table (required, string) and andFlag (optional, boolean, defaults to true). The andFlag determines whether our search will return with all the keywords or any of the keywords (AND and OR searches respectively). The SQL returned will be less than perfect as really we should use something more forgiving and inclusive than the ‘=’ operator, for example in MSSQL name LIKE “%#ListGetAt(arguments.keywords, i, ‘ ‘)#%”. Also the params should be using cfqueryparam etc etc, it’s not perfect code but it doesn’t have to be production code for illustration purposes!

Here’s my implementation in Coldfusion, standing up for the Object Oriented languages:

<cffunction name="jediSearch" returnType="string">
	<cfargument name="keywords" required="true" required="false" type="string">
	<cfargument name="andFlag" required="false" type="boolean" default="true">
	<cfreturn galacticSearch(arguments.keywords,'jedi',arguments.andFlag)>
</cffunction>

<cffunction name="sithSearch" returnType="string">
	<cfargument name="keywords" required="true" required="false" type="string">
	<cfargument name="andFlag" required="false" type="boolean" default="true">
	<cfreturn galacticSearch(arguments.keywords,'sith',arguments.andFlag)>
</cffunction>

<cffunction name="galacticSearch" returnType="string">
	<cfargument name="keywords" required="true" required="false" type="string">
	<cfargument name="table" required="true" type="string">
	<cfargument name="andFlag" required="false" type="boolean" default="true">
	<cfscript>
		var sql = 'SELECT * FROM #arguments.table# WHERE ';
		//loop through the keywords
		for(var i = 1; i LTE ListLen(arguments.keywords, ' '); i++){
			//change the columns for each table
			if(arguments.table is 'jedi'){
				sql &= "(name = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR planet = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR lightsaber_colour = '#ListGetAt(arguments.keywords, i, ' ')#')";
			}else{
				sql &= "(name = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR catch_phrase = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR teacher = '#ListGetAt(arguments.keywords, i, ' ')#')";
			}
			if(i LT ListLen(arguments.keywords, ' ')){
				sql &= (arguments.andFlag)?' AND ':' OR ';
			}
		}
		return sql;
	</cfscript>
</cffunction>

Now if your like me, you look at this code and think “How on earth do I add another table easily and safely?”. Well you have to add a new function like “jediSearch” and then add an extra if/else into the loop in galacticSearch. Personally I’d rather not do that as it’s kind of messy and galacticSearch isn’t really the generic function I’d like it to be. Also when you edit like this you risk breaking something already working and increasing development time.

So next we move onto my solution in Pseudo-Javascript. Things of note are that we have changed the galacticSearch function to also take a columnsFunction that takes a keyword and returns the WHERE clause SQL for searching a tables columns for one keyword.

searchJedi = (keywords, andFlag = true) ->
	return galacticSearch keywords, 'jedi', andFlag, (keyword) ->
		return "(name = '#{ keyword }' OR planet = '#{ keyword }' OR lightsaber_colour = '#{ keyword }')"

searchSith = (keywords, andFlag = true) ->
	return galacticSearch keywords, 'sith', andFlag, (keyword) ->
		return "(name = '#{ keyword }' OR catch_phrase = '#{ keyword }' OR teacher = '#{ keyword }')"

galacticSearch = (keywords, andFlag = true, table, columnsFunction) ->
	sql = 'SELECT * FROM #{ table } WHERE '
	keys = keywords.split(' ')
	for i, key in keys
		sql += (if i gt 0 then (andFlag)?' AND ':' OR ' ELSE '') + columnsFunction(key)
	return sql

I think that the code is organised much tidier in the pseudo-javascript version. All the code referring to the interpretation of the keywords is in one place and all the code about searching the tables is held in another function. This in turn means that adding another table or more complexity to your keywords is not hard and you don’t risk messing up your previous code by doing it! That I think is very cool, especially if you have to work with other people/s code.

Admittedly you *could* do something similar in Object Oriented world, if you created a “Jedi” object and a “Sith” object and then coded the galacticSearch to take an object as an argument instead of a table name, but this seems like overly complicating things. Of course there is a case somewhere where passing the object is a better approach, but for most of my coding I’d rather the anonymous function approach I used in the functional programming version.

Chime in the comments if you have an opinion – I’m fairly new to functional programming so hearing from other people who know more than me is always nice!