Blog Archives

CRM Migration (Part 2)

Part One

Continuing on my series on gradual migration from Coldfusion to Node-js today I’ll be writing up about the authentication system that allows me to have one login that tells both node and coldfusion that the user is authenticated. (Sorry about all the > showing up everywhere – Can’t figure out how to get rid of them)

Authentication

Having worked my way through many iterations of this code, I’m going to start with a quick rundown of the process it took me to get to the end result.

  1. “I can just proxy through all requests to the CF Server, that sends the cookies as well.” Good points of this is that there’s a node module for that (isn’t there always!), bad point is that if you want to have node communicate with coldfusion as well, for example sockets that call logic on CF, you can’t use the cookie from the CF Server if it’s set up correctly to prevent cookie jacking. So this was a no go as essentially I ended up with 3 sessions, Browser to CF, Browser to Nodejs and Nodejs to CF, not good!!
  2. “OK, so if I write my own proxy code I can control the cookie right?”. Well, not quite though we’re getting there, again with the same origin restriction on cookies this didn’t work so well.
  3. “Right, so I have to have all communication with CF sent via node and not store the CF cookie on the browser.” This is about right!

How It’s Done

So if you remember last week there was a function and variable I said I’d explain this week, testCookie and req.session.proxyCookie? Well both of those are to do with my authentication solution.

We start off with the basic cookie and session setup for node:


@use cookieParser security.cookieSecret
 @use session
  cookie:
    httpOnly: true
    secure: false
    maxAge: 3600000 * 8 # hour in ms * 8 = 8 hours
  secret: security.cookieSecret
  store: sessionstore
  saveUninitialized: true
  resave: true

Here we’re using a couchbase memcached session store to store our cookies with some extra settings.

Next we have three functions for working with the Cookies these are:

parseCookie = (cookie) ->
 _cookie = request.jar()
 if cookie? then _cookie.setCookie require('tough-cookie').Cookie.parse(_c), security.proxyAddress for _c in cookie
 _cookie
testProxyCookie = (session) ->
 try
   _foundCFID = false
   for key, cookie of parseCookie(session.proxyCookie)._jar.store.idx[security.proxyCookieDomain]['/']
     if key.toLowerCase() is 'cfid' and cookie.value then _foundCFID = true
     cookie.lastAccessed = new Date().toISOString()
   return _foundCFID
 catch
   return false
 
@testCookie = (req, resp, cb, isLogin=false) =>
  if req.session? and (not testProxyCookie(req.session) or isLogin) and resp.headers and resp.headers['set-cookie']
    req.session.proxyCookie = request.jar()
    req.session.proxyCookie resp.headers['set-cookie']
    @getSessionCookie req, (sidCookie) ->
    sessionstore.set sidCookie, req.session, cb
  else cb()

@getSessionCookie = (req, cb) ->
  cookieParser(security.cookieSecret) req, {}, (parseErr) ->
  if parseErr then return next new Error 'Error parsing cookies.'
  
  # Get the SID cookie
  EXPRESS_SID_KEY = 'connect.sid'
  cb (req.secureCookies and req.secureCookies[EXPRESS_SID_KEY]) or
     (req.signedCookies and req.signedCookies[EXPRESS_SID_KEY]) or
     (req.cookies and req.cookies[EXPRESS_SID_KEY]), sessionstore

@sessionAuth = (req, res, next) =>
  @getSessionCookie req, (sidCookie) =>
    req.sessionroomid = sidCookie
    if testProxyCookie req.session then next()
    else request
      uri: "#{security.proxyAddress}index.cfm/users/nodetestlogin.json"
      method: 'GET'
    , (err, resp, data) => @testCookie req, resp, next, true
@all '*', @sessionAuth

Let’s Break it Down!!

So, to start off when a user hits a page, the `@all ‘*’, @sessionAuth` gets hit. this in turn calls the `getSessionCookie` which if you haven’t guessed loads the cookies sent to node and finds the ‘connect.sid’ cookie value. We then store this cookie value in the request as the req.sessionroomid which we’ll use later for sockets and updating just one client across all his tabs (I think this may be a security hole but I’m yet to go back and review it yet). Then we call `testProxyCookie`, this function does two important things, first, it checks that the cookie we have stored in the session for communicating with CF is valid, and second it updates the lastAccessed value. Updating the lastAccessed value is important as when you send the cookie through CF uses this to detect if your session should have timed out yet or not! If you don’t update this and have say a 30 minute session timeout on CF, then even if you keep contacting CF constantly, 30 minutes after your first contact your session will expire!! Back in `sessionAuth` if the proxy cookie is valid (`testProxyCookie()` returns true) then we continue via `next()`, if not we send a request to the CF server using request and then use `@testCookie` to set the proxyCookie from the result.

In short `@testCookie` reads the response from the CF Server, parses the cookies and then saves the value in the session value, we later use that cookie with request to authenticate to the CF Server. Beyond the above you can setup your node.js server as you normally would and this will take care of all the integration.

Till Next Time (Which I am yet to decide what I’ll write about!).

UPDATE

I found a nasty bug that runs around the storage of tough-cookie objects – basically it doesn’t work! My solution was to store the ‘set-cookie’ headers instead and then add a ‘parseCookie’ method to convert it into the actual cookie jar object. This means that when you make the call to the CF server you do the following:


request
  uri: "#{security.proxyAddress}/index.cfm"
  method: 'GET'
  jar: parseCookie req.session.proxyCookie
, (err, resp, data) => #handle response here!

 

CRM Migration (Part 1)

Recently I’ve been able to start moving our internal Coldfusion based CRM onto Node.js, I’m going to start chronicling the difficulties and challenges this has produced and my solutions to these problems in this multi-part series CRM Migration. My aim is to get one of these up every wednesday.

Access

The first problem I came across is the nastiest one for a partial migration; How do I keep one login for both sites?

To solve this we need to solve a simpler question – How do I access legacy coldfusion pages that we aren’t migrating yet?

I tried using node-proxy, but that didn’t go so well after a while due to cookies making sessions do weird things, my eventual solution was as follows:

@get '/index.cfm*', (req,res) ->
	request
		uri: "#{security.proxyAddress}#{req.url.substring 1}"
		method: 'GET'
		jar: req.session.proxyCookie
	, (err, resp, data) => testCookie req, resp, -> res.send resp.statusCode, resp.body
@post '/index.cfm*', (req,res) ->
	request
		uri: "#{security.proxyAddress}#{req.url.substring 1}"
		method: 'POST'
		jar: req.session.proxyCookie
	, (err, resp, data) => testCookie req, resp, -> res.send resp.statusCode, resp.body

NOTE: I started off using zappajs, but then migrated to Express 4 so I replicated the helpers. The ‘@’ is essentially a reference to the express server variable.

I’ll start by explaining the reasoning behind the URL structure; ‘/index.cfm*’. The CRM in coldfusion runs on CFWheels, a MVC library, and I never got the url rewrites to work on IIS so an example format for Coldfusion URL’s is ‘/index.cfm/{controller}/{action}/{key}’. The two above handlers basically state that any request that starts with ‘/index.cfm’ is targeted to the coldfusion server, in case you haven’t guessed my approach is to field ALL http requests to the node.js server which will act as a proxy for the Coldfusion server if we haven’t migrated that across using node.js’s built in request library.

That’s basically it for access to the old Coldfusion pages, you may be curious about the ‘testCookie’ function and where on earth this ‘req.session.proxyCookie’ variable comes from, but that will be in Part 2.

Hint: It’s about authenticating the user for node.js and coldfusion!

See you all next week!

If you want to know about my basic setup for Node.js projects with express see my Nodejs Website Base.

SpokeDM Documentation

SpokeDM Documentation is now LIVE!!

I have put the basic documentation for SpokeDM up at spokedemo.com. Sometime in the future I will complete the Extending section, probably when I complete the next overdue release of SpokeDM that I haven’t had time to do yet. I was expecting to have more time now that I have been contracting, but so far I have a few more weeks of solid work before I can get around to working on SpokeDM again.

SpokeDM demonstration

A short demonstration setting up a site to record the details of a few ducks using SpokeDM.

CFWheels, OSX Lion Apache, Tomcat and Railo

Recently I got a new Macbook pro, hence this post as it took me a while to get a reliable way of getting the connectors and CFWheels URL rewriting in partial,off and on modes (Different sites to differing requirements).

Now there are a few other blog’s I’ll start off giving credit to for their solutions as follows: railodeveloper.com, getrailo installation wiki and a few others I cannot remember at the moment.

For my install I followed the get railo instructions down till “Connect Tomcat To Apache” which I could not for the life of me get to work. Then I mucked around with the many other resources you will find by doing a quick google then using the information from the railodeveloper link I came up with the following solution.

Connecting Tomcat To Railo.

An exact lift from the railodeveloper.com link under the title of “Use mod_proxy if you have Apache 2.2 or later” (I discovered OSX lion has Apache 2.2.4). Note that you do this EXACTLY, including especially the localhost:8009 – this is the port that the ajp connector runs on, it’s where Tomcat and apache talk AFAIK.

Now you should have normal coldfusion functions working handily! (Multiple hosts as set up by method 2 here: http://www.railodeveloper.com/post.cfm/investigation-on-auto-configuring-apache-vhosts-in-tomcat-for-use-with-railo#option2).

Activating CFWheels URL Rewriting For individual sites.

Here’s the code for my “contestate” site that has full URL rewriting on it, but also allows for partial URL rewriting:

<VirtualHost *:80>
 ServerName contestate
 ServerAlias contestate.x-wing
 
 Options +FollowSymLinks
 ProxyPreserveHost On
 ProxyPassReverse / ajp://localhost:8009/
 
 RewriteEngine On
 RewriteCond %{REQUEST_URI} !^.*/(flex2gateway|jrunscripts|cfide|cfformgateway|cffileservlet|railo-context|files|images|javascripts|miscellaneous|stylesheets|robots.txt|favicon.ico|sitemap.xml|rewrite.cfm|index.cfm)($|/.*$) [NC]
 RewriteRule ^(.*)$ ajp://localhost:8009/rewrite.cfm/$1 [NE,P]
 
 RewriteRule ^(.*)$ ajp://localhost:8009/$1 [P]
</VirtualHost>

Lastly you have to add a mapping in the tomcat web.xml file for the index.cfm/ redirects as follows:

<servlet-mapping>
 <servlet-name>GlobalCFMLServlet</servlet-name>
 <url-pattern>/index.cfm/*</url-pattern>
</servlet-mapping>

That’s all, any questions post in the comments, or if you found a better way I’d be happy to know!

CFWheels passing looping data to a specific partial.

So today I was digging around with the includePartial function of CFWheels and my favourite usage of it is:

#includePartial(somedata)# //somedata is the results of a findAll or similar

My problem was that I wanted to do this, but not repeat myself over multiple partials around the site. So I did some digging and found out that include partial has 3 undocumented arguments that get overwritten in the previous example (depending on what kind of variable somedata is), but not if you do:

#includePartial("partialName")#

These 3 arguments are; query, object and objects. These are set to the query/object/array that you pass through in the ‘somedata’ argument. So for example given that

#includePartial(somedata)#

is called, somedata is an array of objects and the result is that the partial _objectName.cfm is rendered for every object in the array. We can get exactly the same effect by calling:

#includePartial(partial="objectName", objects=somedata)#

This means that you can now store global partials that can be called from anywhere in your site (if you also want global dataFunctions put the dataFunction in Controller.cfc).

CFWheels Search

So today I thought I would share with you something that became very cool when I implemented the CFWheels framework in my application. Well, I was pleased with it anyway; as follows I am going to describe how I implemented a search function into my CRM across 6 tables!

Controller

In the controller is a fairly basic setup, it goes as follows (wordpress colour coding just does not want to work on cfscripts so this is the best I can be bothered to do):

<cfscript>
	if(NOT isDefined("params.search")){//check there is a search string, if not go back with error
		flashInsert(error="No Search String");
		redirectTo(back=true);
	}
	stringOrig = params.search.string;//Store the original search string, I use it to repopulate the input field

	params.search.string = "'%" & LCASE(params.search.string) & "%'";//this is the MSSQL query string you need, used in with the LIKE operator the '%' act as wild cards - so we look for anything that contains our search string

	if(isDefined("params.search.includes.notes") AND params.search.includes.notes EQ "on")//repeat this section for each table you want to search, this example only searches the notes table, the includes struct is a set of checkboxes for "advanced" search filtering, they all default to "on" i.e. search all
		searchNotes = model("note").findAll(
			where="note LIKE #params.search.string#",
			returnAs="query");
	else searchNotes = QueryNew("");//if we are not to search this table populate the variable with a blank query to avoid errors in the view
</cfscript>

Basically it’s just query stuff, as it should be.

View

The view I had more fun building (I am omitting some javascript and jquery fancy stuff). Here’s the main view file for the above query.

<cfoutput>
	#includePartial(
		partial="searchSection",
		searchResult=searchNotes,
		name="Notes",
		colList=[{title:"Label",display="label"},{title:"Note",display:"fullLeftStrip(note, 130)"},{title:"Title",display="date"}]
	)#
</cfoutput>

Now the magic is in the partial (as you can guess I used the same partial for ALL of my search results queries – one for each query dynamically picking which columns to show). As Follows:

<!---
Params
searchResult - search query to display
name - title of the section
colList = [{title:"",display=""}] - defines the headers and what is displayed
urlformat = {controller, action, keycolumn} - details for the link url (optional)
--->
<div class="contentcontainer">
	<div class="headings">
		<cfoutput><!--- The following is just styling, basically each table outputs in a table with a title and a 'result' count --->
			<h2 class="left">#arguments.name#</h2>
			<p class="notifycount<cfif arguments.searchResult.recordCount> green</cfif>"><a href="" title="" class="notifypop">#arguments.searchResult.recordcount#</a></p>
		</div>
		<div class="panes #arguments.name#">
		</cfoutput>
		<div class="contentbox">
			<table width="100%">
				<thead>
					<tr><!--- output all the columns we want to display as passed to the partial --->
						<cfloop array="#arguments.colList#" index="column">
							<cfoutput><th>#column.title#</th></cfoutput>
						</cfloop>
					</tr>
				</thead>
				<tbody>
					<cfoutput query="arguments.searchResult"><!--- Here we output the actual data --->
						<tr class="editRow" <cfif isDefined("arguments.urlformat") AND isStruct(arguments.urlformat)>href="#URLFor(controller=arguments.urlformat.controller, action=arguments.urlformat.action, key="#Evaluate(arguments.urlformat.keycolumn)#")#"</cfif>><!--- based on what the urlformat we are passed, make the row a 'link' - this requires a javascript onclick to work thou; $("tr[href]").click(function(){window.location = $(this).attr("href")}) --->
							<cfloop array="#arguments.colList#" index="column"><!--- again only show the columns we want to display --->
								<td>#Evaluate(column.display)#</td>
							</cfloop>
						</tr>
					</cfoutput>
				</tbody>
			</table>
		</div>
	</div>
</div>

Note the use of Evaluate() in displaying the column meaning that when you define the display in the includePartial() you can put in any coldFusion you want to. Apart from some styling, jquery effects and the layout that is about it. Before CFWheels this was 2 very large cfm files with a LOT of repeated code, being able to do it this simply makes me a very happy man!

Edit: Got around to adding comments to the code.

CFWheels and irregular plural table names

So much as I LOVE cfwheels as a framework I often am unsure as to whether this table name and it’s irregular plural form, e.g. person and people, will be accepted by cfwheels. So in my searching I found this excellent function called pluralize and it’s reverse singularize. These allow you to test out what cfwheels will do with the plural or singular version of your word. For quick reference I dug into the code and got the list of irregular and uncountable words as at 1.1.7.

Irregular words (single, plural): “child,children, foot,feet, man,men, move,moves, person,people, sex,sexes, tooth,teeth, woman,women”

Uncountable Words (have no plural form): “advice, air, blood, deer, equipment, fish, food, furniture, garbage, graffiti, grass, homework, housework, information, knowledge, luggage, mathematics, meat, milk, money, music, pollution, research, rice, sand, series, sheep, soap, software, species, sugar, traffic, transportation, travel, trash, water, feedback”