Blog Archives

CRM Migration (Part 2)

Part One

Continuing on my series on gradual migration from Coldfusion to Node-js today I’ll be writing up about the authentication system that allows me to have one login that tells both node and coldfusion that the user is authenticated. (Sorry about all the > showing up everywhere – Can’t figure out how to get rid of them)

Authentication

Having worked my way through many iterations of this code, I’m going to start with a quick rundown of the process it took me to get to the end result.

  1. “I can just proxy through all requests to the CF Server, that sends the cookies as well.” Good points of this is that there’s a node module for that (isn’t there always!), bad point is that if you want to have node communicate with coldfusion as well, for example sockets that call logic on CF, you can’t use the cookie from the CF Server if it’s set up correctly to prevent cookie jacking. So this was a no go as essentially I ended up with 3 sessions, Browser to CF, Browser to Nodejs and Nodejs to CF, not good!!
  2. “OK, so if I write my own proxy code I can control the cookie right?”. Well, not quite though we’re getting there, again with the same origin restriction on cookies this didn’t work so well.
  3. “Right, so I have to have all communication with CF sent via node and not store the CF cookie on the browser.” This is about right!

How It’s Done

So if you remember last week there was a function and variable I said I’d explain this week, testCookie and req.session.proxyCookie? Well both of those are to do with my authentication solution.

We start off with the basic cookie and session setup for node:


@use cookieParser security.cookieSecret
 @use session
  cookie:
    httpOnly: true
    secure: false
    maxAge: 3600000 * 8 # hour in ms * 8 = 8 hours
  secret: security.cookieSecret
  store: sessionstore
  saveUninitialized: true
  resave: true

Here we’re using a couchbase memcached session store to store our cookies with some extra settings.

Next we have three functions for working with the Cookies these are:

parseCookie = (cookie) ->
 _cookie = request.jar()
 if cookie? then _cookie.setCookie require('tough-cookie').Cookie.parse(_c), security.proxyAddress for _c in cookie
 _cookie
testProxyCookie = (session) ->
 try
   _foundCFID = false
   for key, cookie of parseCookie(session.proxyCookie)._jar.store.idx[security.proxyCookieDomain]['/']
     if key.toLowerCase() is 'cfid' and cookie.value then _foundCFID = true
     cookie.lastAccessed = new Date().toISOString()
   return _foundCFID
 catch
   return false
 
@testCookie = (req, resp, cb, isLogin=false) =>
  if req.session? and (not testProxyCookie(req.session) or isLogin) and resp.headers and resp.headers['set-cookie']
    req.session.proxyCookie = request.jar()
    req.session.proxyCookie resp.headers['set-cookie']
    @getSessionCookie req, (sidCookie) ->
    sessionstore.set sidCookie, req.session, cb
  else cb()

@getSessionCookie = (req, cb) ->
  cookieParser(security.cookieSecret) req, {}, (parseErr) ->
  if parseErr then return next new Error 'Error parsing cookies.'
  
  # Get the SID cookie
  EXPRESS_SID_KEY = 'connect.sid'
  cb (req.secureCookies and req.secureCookies[EXPRESS_SID_KEY]) or
     (req.signedCookies and req.signedCookies[EXPRESS_SID_KEY]) or
     (req.cookies and req.cookies[EXPRESS_SID_KEY]), sessionstore

@sessionAuth = (req, res, next) =>
  @getSessionCookie req, (sidCookie) =>
    req.sessionroomid = sidCookie
    if testProxyCookie req.session then next()
    else request
      uri: "#{security.proxyAddress}index.cfm/users/nodetestlogin.json"
      method: 'GET'
    , (err, resp, data) => @testCookie req, resp, next, true
@all '*', @sessionAuth

Let’s Break it Down!!

So, to start off when a user hits a page, the `@all ‘*’, @sessionAuth` gets hit. this in turn calls the `getSessionCookie` which if you haven’t guessed loads the cookies sent to node and finds the ‘connect.sid’ cookie value. We then store this cookie value in the request as the req.sessionroomid which we’ll use later for sockets and updating just one client across all his tabs (I think this may be a security hole but I’m yet to go back and review it yet). Then we call `testProxyCookie`, this function does two important things, first, it checks that the cookie we have stored in the session for communicating with CF is valid, and second it updates the lastAccessed value. Updating the lastAccessed value is important as when you send the cookie through CF uses this to detect if your session should have timed out yet or not! If you don’t update this and have say a 30 minute session timeout on CF, then even if you keep contacting CF constantly, 30 minutes after your first contact your session will expire!! Back in `sessionAuth` if the proxy cookie is valid (`testProxyCookie()` returns true) then we continue via `next()`, if not we send a request to the CF server using request and then use `@testCookie` to set the proxyCookie from the result.

In short `@testCookie` reads the response from the CF Server, parses the cookies and then saves the value in the session value, we later use that cookie with request to authenticate to the CF Server. Beyond the above you can setup your node.js server as you normally would and this will take care of all the integration.

Till Next Time (Which I am yet to decide what I’ll write about!).

UPDATE

I found a nasty bug that runs around the storage of tough-cookie objects – basically it doesn’t work! My solution was to store the ‘set-cookie’ headers instead and then add a ‘parseCookie’ method to convert it into the actual cookie jar object. This means that when you make the call to the CF server you do the following:


request
  uri: "#{security.proxyAddress}/index.cfm"
  method: 'GET'
  jar: parseCookie req.session.proxyCookie
, (err, resp, data) => #handle response here!

 

CRM Migration (Part 1)

Recently I’ve been able to start moving our internal Coldfusion based CRM onto Node.js, I’m going to start chronicling the difficulties and challenges this has produced and my solutions to these problems in this multi-part series CRM Migration. My aim is to get one of these up every wednesday.

Access

The first problem I came across is the nastiest one for a partial migration; How do I keep one login for both sites?

To solve this we need to solve a simpler question – How do I access legacy coldfusion pages that we aren’t migrating yet?

I tried using node-proxy, but that didn’t go so well after a while due to cookies making sessions do weird things, my eventual solution was as follows:

@get '/index.cfm*', (req,res) ->
	request
		uri: "#{security.proxyAddress}#{req.url.substring 1}"
		method: 'GET'
		jar: req.session.proxyCookie
	, (err, resp, data) => testCookie req, resp, -> res.send resp.statusCode, resp.body
@post '/index.cfm*', (req,res) ->
	request
		uri: "#{security.proxyAddress}#{req.url.substring 1}"
		method: 'POST'
		jar: req.session.proxyCookie
	, (err, resp, data) => testCookie req, resp, -> res.send resp.statusCode, resp.body

NOTE: I started off using zappajs, but then migrated to Express 4 so I replicated the helpers. The ‘@’ is essentially a reference to the express server variable.

I’ll start by explaining the reasoning behind the URL structure; ‘/index.cfm*’. The CRM in coldfusion runs on CFWheels, a MVC library, and I never got the url rewrites to work on IIS so an example format for Coldfusion URL’s is ‘/index.cfm/{controller}/{action}/{key}’. The two above handlers basically state that any request that starts with ‘/index.cfm’ is targeted to the coldfusion server, in case you haven’t guessed my approach is to field ALL http requests to the node.js server which will act as a proxy for the Coldfusion server if we haven’t migrated that across using node.js’s built in request library.

That’s basically it for access to the old Coldfusion pages, you may be curious about the ‘testCookie’ function and where on earth this ‘req.session.proxyCookie’ variable comes from, but that will be in Part 2.

Hint: It’s about authenticating the user for node.js and coldfusion!

See you all next week!

If you want to know about my basic setup for Node.js projects with express see my Nodejs Website Base.

Why I’ve come to love functional programming

Recently I have been starting to learn Node.js and inadvertently changed my opinion of functional programming VS Object Oriented programming. During my work on supporting legacy Coldfusion sites I stumbled across some code that forms the basis of this comparison of functional programming VS Object Oriented programming. (PS if you want a good introduction to the differences look on over to: http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html).

This particular example will be Coldfusion and Pseudo-javascript (or coffeescript) and focuses around a multi-keyword search on 3 columns on 2 tables. Our tables are as follows:

Table Name: Jedi
With Columns: Name, Planet, Lightsaber_Colour.

Table Name: Sith
With Columns: Name, Catch_Phrase, Teacher.

First of all we will define a master function, and a convenience function for searching each of the tables, we do this for an attempt at reusability. We’ll call this main search function “galacticSearch” which will return a SQL string and take the arguments: Keyword (required, string), table (required, string) and andFlag (optional, boolean, defaults to true). The andFlag determines whether our search will return with all the keywords or any of the keywords (AND and OR searches respectively). The SQL returned will be less than perfect as really we should use something more forgiving and inclusive than the ‘=’ operator, for example in MSSQL name LIKE “%#ListGetAt(arguments.keywords, i, ‘ ‘)#%”. Also the params should be using cfqueryparam etc etc, it’s not perfect code but it doesn’t have to be production code for illustration purposes!

Here’s my implementation in Coldfusion, standing up for the Object Oriented languages:

<cffunction name="jediSearch" returnType="string">
	<cfargument name="keywords" required="true" required="false" type="string">
	<cfargument name="andFlag" required="false" type="boolean" default="true">
	<cfreturn galacticSearch(arguments.keywords,'jedi',arguments.andFlag)>
</cffunction>

<cffunction name="sithSearch" returnType="string">
	<cfargument name="keywords" required="true" required="false" type="string">
	<cfargument name="andFlag" required="false" type="boolean" default="true">
	<cfreturn galacticSearch(arguments.keywords,'sith',arguments.andFlag)>
</cffunction>

<cffunction name="galacticSearch" returnType="string">
	<cfargument name="keywords" required="true" required="false" type="string">
	<cfargument name="table" required="true" type="string">
	<cfargument name="andFlag" required="false" type="boolean" default="true">
	<cfscript>
		var sql = 'SELECT * FROM #arguments.table# WHERE ';
		//loop through the keywords
		for(var i = 1; i LTE ListLen(arguments.keywords, ' '); i++){
			//change the columns for each table
			if(arguments.table is 'jedi'){
				sql &= "(name = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR planet = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR lightsaber_colour = '#ListGetAt(arguments.keywords, i, ' ')#')";
			}else{
				sql &= "(name = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR catch_phrase = '#ListGetAt(arguments.keywords, i, ' ')#'";
				sql &= "OR teacher = '#ListGetAt(arguments.keywords, i, ' ')#')";
			}
			if(i LT ListLen(arguments.keywords, ' ')){
				sql &= (arguments.andFlag)?' AND ':' OR ';
			}
		}
		return sql;
	</cfscript>
</cffunction>

Now if your like me, you look at this code and think “How on earth do I add another table easily and safely?”. Well you have to add a new function like “jediSearch” and then add an extra if/else into the loop in galacticSearch. Personally I’d rather not do that as it’s kind of messy and galacticSearch isn’t really the generic function I’d like it to be. Also when you edit like this you risk breaking something already working and increasing development time.

So next we move onto my solution in Pseudo-Javascript. Things of note are that we have changed the galacticSearch function to also take a columnsFunction that takes a keyword and returns the WHERE clause SQL for searching a tables columns for one keyword.

searchJedi = (keywords, andFlag = true) ->
	return galacticSearch keywords, 'jedi', andFlag, (keyword) ->
		return "(name = '#{ keyword }' OR planet = '#{ keyword }' OR lightsaber_colour = '#{ keyword }')"

searchSith = (keywords, andFlag = true) ->
	return galacticSearch keywords, 'sith', andFlag, (keyword) ->
		return "(name = '#{ keyword }' OR catch_phrase = '#{ keyword }' OR teacher = '#{ keyword }')"

galacticSearch = (keywords, andFlag = true, table, columnsFunction) ->
	sql = 'SELECT * FROM #{ table } WHERE '
	keys = keywords.split(' ')
	for i, key in keys
		sql += (if i gt 0 then (andFlag)?' AND ':' OR ' ELSE '') + columnsFunction(key)
	return sql

I think that the code is organised much tidier in the pseudo-javascript version. All the code referring to the interpretation of the keywords is in one place and all the code about searching the tables is held in another function. This in turn means that adding another table or more complexity to your keywords is not hard and you don’t risk messing up your previous code by doing it! That I think is very cool, especially if you have to work with other people/s code.

Admittedly you *could* do something similar in Object Oriented world, if you created a “Jedi” object and a “Sith” object and then coded the galacticSearch to take an object as an argument instead of a table name, but this seems like overly complicating things. Of course there is a case somewhere where passing the object is a better approach, but for most of my coding I’d rather the anonymous function approach I used in the functional programming version.

Chime in the comments if you have an opinion – I’m fairly new to functional programming so hearing from other people who know more than me is always nice!

SpokeDM Documentation

SpokeDM Documentation is now LIVE!!

I have put the basic documentation for SpokeDM up at spokedemo.com. Sometime in the future I will complete the Extending section, probably when I complete the next overdue release of SpokeDM that I haven’t had time to do yet. I was expecting to have more time now that I have been contracting, but so far I have a few more weeks of solid work before I can get around to working on SpokeDM again.

SpokeDM demonstration

A short demonstration setting up a site to record the details of a few ducks using SpokeDM.

SpokeDM

I am proud to announce the fruits of some hard work and my first complete product (always a happy moment). Today I have just finished coding a CFWheels and AngularJS based Framework called SpokeDM.

This framework aims to take the hassle out of coding maintenance screens for whatever project you are building. It uses the excellent work from CFWheels models and extends it to generate the front end view for the end user.

Jump over to the website: spokedm.com for video demonstrations and more details.

Encrypting URL parameters in Coldfusion.

So on looking around the internet for a suitable solution to passing encrypted strings sensibly through URL variables. I found all the usual suspects from encrypting with base64 conversion to straight encrypting. These all had the issue of mangling the string when passed through a URL, either CFWheels would lowercase everything (hence wrecking Base64 encoding), or in the case of simply encoding and even using URLEncode several characters would get trashed completely.

I found that the best solution was to provide the Encrypt() function with it’s additional params: algorithm and encoding. See http://livedocs.adobe.com/coldfusion/8/htmldocs/help.html?content=functions_c-d_40.html#1103962 for details. I picked the Hex encoding for a sensible character set (on retrospect UUEncode would work as well if you take in account the extra characters that will need to be URL encoded). So my encode call looked as follows:

encrypt(toEncodeString, algorithmkey, "CFMX_COMPAT", "HEX")

And the Decode would go as follows:

decrypt(urlStringToDecode, algorithmkey, "CFMX_COMPAT", "HEX")

CFDocument Formatting Quick Tip

So recently I have been messing around a bit more with the CFDocument tag and keeping sections together. As noted elsewhere on the internet there is no simple way of fixing this until I quite happily stumbled on a solution (in railo at least). Basically if a table is less than the total height of a page it will be kept together! Now all those people who tell you tables are evil and should not be used may jump in horror at what I suggest you do next, which is that you code your CFDocuments as follows:

<table><tr><td> ...Some HTML/CF output code that you don't want a page break in... </tr></td></table>

Now It is neither elegant or nice to code, but for some reason it seems to work. Hope this helps someone else!

Coldfusion upload and rename a file.

Shock; my second post in a week (most likely NOT a regular occurrence). Today I solved a problem that, to me, seems difficult to find a solution online. This is the issue (this may only be specific to railo now) that if you perform the following code, (assuming a file submitted from a form) the file is uploaded but errors with “cannot move file” on the rename. I use it slightly modified to keep versions of files.

<cffile action="upload" fileField="file" destination="#ExpandPath("./")#" nameConflict="makeUnique" result="upload">
<cffile action="rename" source="#ExpandPath("./")#" destination="#ExpandPath("/document_#otherversions + 1#.#upload.serverFileExt#")#">

My solution is simple, I use the java File object to rename it, apparently this fixes the issue. As follows:

<cffile action="upload" fileField="file" destination="#ExpandPath("./")#" nameConflict="makeUnique" result="upload">
<cfdirectory directory="#ExpandPath(filedirectory)#" action="list" filter="documents_*" name="historic" sort="desc">
<cfset javaFile = CreateObject("java", "java.io.File").init(ExpandPath("./" & upload.serverfile))>
<cfset renamePass = javaFile.renameTo(CreateObject("java", "java.io.File").init(ExpandPath("/document_#otherversions + 1#.#upload.serverFileExt#")))>

CFWheels passing looping data to a specific partial.

So today I was digging around with the includePartial function of CFWheels and my favourite usage of it is:

#includePartial(somedata)# //somedata is the results of a findAll or similar

My problem was that I wanted to do this, but not repeat myself over multiple partials around the site. So I did some digging and found out that include partial has 3 undocumented arguments that get overwritten in the previous example (depending on what kind of variable somedata is), but not if you do:

#includePartial("partialName")#

These 3 arguments are; query, object and objects. These are set to the query/object/array that you pass through in the ‘somedata’ argument. So for example given that

#includePartial(somedata)#

is called, somedata is an array of objects and the result is that the partial _objectName.cfm is rendered for every object in the array. We can get exactly the same effect by calling:

#includePartial(partial="objectName", objects=somedata)#

This means that you can now store global partials that can be called from anywhere in your site (if you also want global dataFunctions put the dataFunction in Controller.cfc).