Debounce Your Scroll Event Handlers

I was writing some JavaScript today that would add a bunch of divs to the DOM, and then load some content dynamically from the server only when they came into view. The divs that were below the fold would wait until the user scrolled down before it would make a request to the server. The issue I was running into was that in some browsers (Chrome), the scroll event was fired way too often and slowed everything down. That’s when I found Ben Alman’s post on debouncing. Only minor thing is, I don’t use jQuery and instead I use Prototype and I wanted something a little more Prototype-y. So that’s when I whipped this up (documentation was ripped from Ben’s debounce function though).

/**
 * Debounce execution of a function. Debouncing, unlike throttling,
 * guarantees that a function is only executed a single time, either at the
 * very beginning of a series of calls, or at the very end.
 * @param {Number} delay A zero-or-greater delay in seconds. For event
 *    callbacks, values around 0.1 or 0.25 (or even higher) are most useful.
 * @param {Boolean} atBegin Optional, defaults to false. If atBegin is false or
 *    unspecified, the function will only be executed 'delay' seconds after
 *    the last debounced-function call. If atBegin is true, the function will be
 *    executed only at the first debounced-function call. (After the
 *    debounced-function has not been called for 'delay' seconds, the
 *    internal counter is reset)
 * @returns {Function} A new, debounced, function.
 */
Function.prototype.debounce = function (delay, atBegin) {
	var func = this;
	var timeout;

	return function() {
		var args = arguments;

		var delayed = function() {
			if (!atBegin) { func.apply(this, args); }
			timeout = null;
		}.bind(this);

		if (timeout) {
			clearTimeout(timeout);
		} else if (atBegin) {
			func.apply(this, args);
		}

		timeout = delayed.delay(delay);
	};
};

And you use it like so:

var myScrollEventHandler = function(e){};
Event.observe(window, 'scroll', myScrollEventHandler.debounce(0.25));

The scroll event handler will only be called once the window has started to scroll and then stopped scrolling for at least 250ms. Notice how it takes a delay in seconds, similar to how Prototype’s Function.prototype.delay function already works (in fact if you look at the code, it uses the delay function). Also, this isn’t just for scroll events, it is also useful for keypress events when you want to do autocomplete, or input validation, or many other various things.

Posted in Web Development | Tagged , | Leave a comment

Adding JavaScript Unit Tests to Your Continuous Integration Scripts

Introduction

I’ve been doing a lot more code client side lately in JavaScript and less on the server. But until recently, I haven’t had any way to writing unit tests for my code that could easily be run from the command line and incorporated into the continuous integration build, and it has made me feel dirty. So, I spent some time and came up with something that I am happy with.

Requirements

My requirements were as follows:

  • Didn’t want to have to use any new environment (like Node.js a la Yeti, or Java a la JSTestDriver)
  • Didn’t want to have to setup any server/clients before being able to run the tests (like Node.js a la Yeti, or Java a la JSTestDriver)
  • Wanted the tests to run in a full browser environment (or as close as possible), and not just a javascript interpreter (like Rhino)
  • Tests must be able to be run on the command line and output NUnit style XML (yes NUnit, not JUnit, but they are pretty close so you can easily take what I provide and make it work with JUnit)

Solution

My solution requires the following:

  • QUnit
  • Custom QUnitLogger and QUnitResultPrinter scripts
  • PhantomJS
  • Custom PhantomJS script

Step 1 – Writing Your Tests

The first step is to get QUnit and follow the instructions to write some basic tests. I’m not going to get into how to do that here as there are perfectly good examples on the QUnit site.

Step 2 – Custom QUnit Scripts

QUnit is a great unit testing framework, however it outputs the test results to the DOM. That isn’t very useful to us. Thankfully, QUnit has callback functions that allow us to gather the information we need. I packaged up this functionality in a QUnitLogger library. Second, we will need some way to take the test results and output them in an NUnit XML format. I have packaged up this functionality in a QUnitNUnitResultPrinter library.

var QUnitLogger = {
	/**
	 * Set this to be the name of your test suite
	 */
	name: "QUnit Tests",

	/**
	 * Returns whether the test is done or not
	 */
	done: false,

	/**
	 * Maintains a reference to the listeners
	 * @private
	 */
	__listeners: {},

	/**
	 * Contains the results of the test suite
	 */
	results: {
		modules: {},
		failed: 0,
		passed: 0,
		total: 0,
		runtime: 0
	},

	/**
	 * The current running module name
	 */
	__m: null,

	/**
	 * The current module's start time
	 */
	__ms: null,

	/**
	 * The current running test name
	 */
	__t: null,

	/**
	 * The current test's start time
	 */
	__ts: null,

	/**
	 * Handles the QUnit.log callbacks
	 * @private
	 * { result, actual, expected, message }
	 */
	__log: function(info) {
		this.results.modules[this.__m].tests[this.__t].asserts.push(info);
	},

	/**
	 * Handles the QUnit.testStart callbacks
	 * @private
	 * { name }
	 */
	__testStart: function(info) {
		this.__t = info.name;
		this.__ts = new Date();
		if (!this.results.modules[this.__m].tests[this.__t]) {
			this.results.modules[this.__m].tests[this.__t] = {
				asserts: [],
				failed: 0,
				passed: 0,
				total: 0,
				runtime: 0
			};
		}
	},

	/**
	 * Handles the QUnit.testDone callbacks
	 * @private
	 * { name, failed, passed, total }
	 */
	__testDone: function(info) {
		// Save the results of the module
		var now  = new Date();
		this.results.modules[this.__m].tests[this.__t].failed += info.failed;
		this.results.modules[this.__m].tests[this.__t].passed += info.passed;
		this.results.modules[this.__m].tests[this.__t].total += info.total;
		this.results.modules[this.__m].tests[this.__t].runtime += (now.getTime() - this.__ts.getTime());
		this.__t = null;
		this.__ts = null;
	},

	/**
	 * Handles the QUnit.moduleStart callbacks
	 * @private
	 * { name }
	 */
	__moduleStart: function(info) {
		this.__m = info.name;
		this.__ms = new Date();
		if (!this.results.modules[this.__m]) {
			this.results.modules[this.__m] = {
				tests: {},
				failed: 0,
				passed: 0,
				total: 0,
				runtime: 0
			};
		}
	},

	/**
	 * Handles the QUnit.moduleDone callbacks
	 * @private
	 * { name, failed, passed, total }
	 */
	__moduleDone: function(info) {
		// Save the results of the module
		var now  = new Date();
		this.results.modules[this.__m].failed += info.failed;
		this.results.modules[this.__m].passed += info.passed;
		this.results.modules[this.__m].total += info.total;
		this.results.modules[this.__m].runtime += (now.getTime() - this.__ms.getTime());
		this.__m = null;
		this.__ms = null;
	},

	/**
	 * Handles the QUnit.begin callbacks
	 * @private
	 */
	__begin: function() {
	},

	/**
	 * Handles the QUnit.done callbacks
	 * @private
	 * { failed, passed, total, runtime }
	 */
	__done: function(info) {
		this.results.failed += info.failed;
		this.results.passed += info.passed;
		this.results.total += info.total;
		this.results.runtime += info.runtime;

		this.done = true;
		this.__dispatchEvent('done');
	},

	/**
	 * Add a listener for an available event.
	 * @param {String} eventId The name of the event you want to listen for. Available events are:
	 * 'done': Raised when the test suite is complete. No arguments are passed to the callback.
	 * @param {Function} closure The callback to call
	 */
	addEventListener: function(eventId, closure) {
		if (!this.__listeners[eventId]) {
			this.__listeners[eventId] = [];
		}
		this.__listeners[eventId].push(closure);
	},

	/**
	 * Dispatches the given event, and passes any extra arguments along to the callback closure
	 * @param {String} eventId The name of the event to dispatch
	 * @private
	 */
	__dispatchEvent: function(eventId) {
		var i;
		var params = [];
		for (i = 1; i < arguments.length; ++i) {
			params.push(arguments[i]);
		}

		var closures = this.__listeners[eventId];
		if (closures) {
			for (i = 0; i < closures.length; ++i) {
				closures[i].apply(window, params);
			}
		}
	}
};

//
// Override the available overrideable functions in order to capture the test results
//
QUnit.log = function() { QUnitLogger.__log.apply(QUnitLogger, arguments); };
QUnit.testStart = function() { QUnitLogger.__testStart.apply(QUnitLogger, arguments); };
QUnit.testDone = function() { QUnitLogger.__testDone.apply(QUnitLogger, arguments); };
QUnit.moduleStart = function() { QUnitLogger.__moduleStart.apply(QUnitLogger, arguments); };
QUnit.moduleDone = function() { QUnitLogger.__moduleDone.apply(QUnitLogger, arguments); };
QUnit.begin = function() { QUnitLogger.__begin.apply(QUnitLogger, arguments); };
QUnit.done = function() { QUnitLogger.__done.apply(QUnitLogger, arguments); };
var QUnitNUnitResultPrinter = {
	/**
	 * Returns the results printed in the proper format
	 * @return {String} The results formatted properly
	 */
	print: function() {
		var now = new Date();
		var QUL = QUnitLogger;
		var rv = '\n';
		rv += '\n';
		rv += '\n\t';
		rv += '\n\t';
		rv += '\n\t\t';

		//
		// Go through all the modules
		//
		for (var mId in QUL.results.modules) {
			var m = QUL.results.modules[mId];
			rv += '\n\t\t\t';
			rv += '\n\t\t\t\t';

			//
			// Go through all the tests
			//
			for (var tId in m.tests) {
				var t = m.tests[tId];
				var result = (0 === t.failed? 'Success' : 'Failed');
				var success = (0 === t.failed? 'True' : 'False');
				rv += '\n\t\t\t\t\t';
				if ('Failed' === result) {
					rv += '\n\t\t\t\t\t\t';
					//
					// Output only the first failed assertion
					//
					for (var i = 0; i < t.asserts.length; ++i) {
						var assert = t.asserts[i];
						if (!assert.result) {
							var msg = i.toString() + ". " + assert.message;
							var src = "";
							if (assert.expected) {
								msg += "\nExpected: '" + assert.expected + "'";
							}
							if (assert.actual) {
								msg += "\nActual: '" + assert.actual + "'";
							}
							if (assert.source) {
								src = assert.source;
							}

							rv += '\n\t\t\t\t\t\t\t<![CDATA['+msg+']]></message>';
							rv += '\n\t\t\t\t\t\t\t<stack-trace><![CDATA['+src+']]>';

							break;
						}
					}
					rv += '\n\t\t\t\t\t\t';
				}

				rv += '\n\t\t\t\t\t';
			}
			rv += '\n\t\t\t\t';
			rv += '\n\t\t\t';
		}

		rv += '\n\t\t';
		rv += '\n\t';
		rv += '\n';
		rv += "\n";
		return rv;
	}
};

Step 3 – PhantomJS

PhantomJS is a headless WebKit browser that lets you load pages and execute JavaScript and then output the results to the console. It has some limitations (like I don’t think it supports cookies), but for the most part it is a full fledged browser as far as most scripts are concerned. You can also pass additional parameters on the command line. I wrote a PhantomJS script which will load an html file (the one that loads your tests), waits for the tests to complete, and then outputs the tests results to the console in an NUnit XML format.

(function() {
	var PAGE_FAILED = 1;
	var TEST_FAILED = 2;
	var EXCEPTION = 3;

	try {
		if (0 === phantom.state.length) {
			if (1 !== phantom.args.length) {
				console.log('Usage: phantomjs-driver.js URL');
				phantom.exit();
			} else {
				phantom.state = 'phantomjs-driver-running';
				phantom.viewportSize = { width: 800, height: 600 };
				phantom.open(phantom.args[0]);
			}
		} else {
			var testDone = function() {
				phantom.state = 'phantomjs-driver-finish';
				var r = QUnitLogger.results;

				var msg = QUnitNUnitResultPrinter.print();
				console.log(msg);

				msg = 'Tests completed in '+r.runtime.toString()+' milliseconds.';
				console.log(msg);

				msg = r.passed.toString()+' tests of '+r.total.toString()+' passed, '+r.failed.toString()+' failed.';
				console.log(msg);

				phantom.exit(0 < r.failed ? TEST_FAILED : 0);
			};

			if ("success" === phantom.loadStatus) {
				if (QUnitLogger.done) {
					testDone();
				} else {
					QUnitLogger.addEventListener('done', testDone);
				}
			} else {
				console.log('Page failed to load: ' + phantom.args[0]);
				phantom.exit(PAGE_FAILED);
			}
		}
	} catch(e) {
		console.log('Unhandled exception: ' + e);
		phantom.exit(EXCEPTION);
	}
})();
Posted in Web Development | Tagged , , , | 4 Comments

Making CakePHP’s Auth Component Work with Multiple Authentication Schemes

I have been trying to create a CakePHP app that will allow me to authenticate in multiple ways. For instance, I want to authenticate using a native account (username/password), Facebook Connect, Google Friend Connect, Twitter Oauth, etc. To start simple, I am just going to separate the user’s account from the user authentication. By default, CakePHP’s AuthComponent wants the user account details in the same table as the user’s authentication details. Something like:

CREATE TABLE users (
	id INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
	username VARCHAR(255) NOT NULL UNIQUE,
	password CHAR(40) NOT NULL,
	created DATETIME,
	modified DATETIME
);

But what I want is something more like:

CREATE TABLE users (
	id INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
	created DATETIME,
	modified DATETIME
);

CREATE TABLE native_accounts (
	id INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
	username VARCHAR(255) NOT NULL UNIQUE,
	password CHAR(40) NOT NULL,
	user_id INT(11) NOT NULL,
	created DATETIME,
	modified DATETIME
);

Then, I want to be able to authenticate against the native_accounts table, but set the $this->Auth->user(); to be the user instance that is associated with the native account. There are ways to change which model that the AuthComponent uses, but if I set the model to be my NativeAccount model, then the $this->Auth->user(); will return the NativeAccount that was authenticated, and not the User that is associated with it.

There are other authentication components out there (like the fantastic Authsome component), but I wanted to try and make this work with the standard AuthComponent. So, here is how I did it.

First I created an app_controller:

class AppController extends Controller {
	var $components = array('Auth', 'Session');
	var $helpers = array('Html', 'Form', 'Session');

	function beforeFilter() {
		parent::beforeFilter();
		$this->Auth->authorize = 'controller';
		$this->Auth->allow('user_login');
		$this->Auth->fields = array('username' => 'id', 'password' => 'id');
		$this->Auth->loginAction = array('controller' => 'my_app', 'action' => 'login');
		$this->Auth->logoutRedirect = array('controller' => 'my_app', 'action' => 'logout');
		$this->Auth->loginRedirect = array('controller' => 'my_app', 'action' => 'index');
	}

	function isAuthorized() {
		return true;
	}
}

This will enable the Auth component for all controllers, set the authorize method to be the controller (which currently allows all logged in users to access all content), allows the “user_login” action (which I will get to in a second), and specifies which User fields the AuthComponent should use for the username and password (which I will also get to in a second…bare with me), and sets up the default login/logout actions.

Next I created the MyAppController. This is the controller that will handle the login/logout, and other common features that aren’t really related to any model.

class MyAppController extends AppController {
	var $name = 'MyApp';
	var $uses = array();

	function index() {
		// Will redirect to the login page if the user hasn't logged in yet
	}

	function login() {
		// Just show the login form
	}

	function logout() {
		$this->redirect($this->Auth->logout());
	}
}

This controller is pretty easy. It will show the login page when needed, and log the user out when requested. The login view for this looks like:

echo $this->Session->flash('auth');
echo $this->Form->create('NativeAccount', array('action' => 'user_login'));
echo $this->Form->inputs(array(
	'legend' => __('Login', true),
	'username',
	'password'
));
echo $this->Form->end('Login');

For now, it only allows the user to log in using a native account, but when I get around to implementing the other forms of authentication, it will present the user with the various way they can log in.

Finally, the NativeAccountsController looks like:

class NativeAccountsController extends AppController {
	var $name = 'NativeAccounts';

	function user_login() {
		$this->Auth->logout();
		if (!empty($this->data)) {
			$result = $this->NativeAccount->findByUsername($this->data['NativeAccount']['username']);
			if(!empty($result)) {
				$hashedPassword = $this->Auth->password($this->data['NativeAccount']['password']);
				if ($hashedPassword == $result['NativeAccount']['password'] &amp;&amp; !empty($result['User'])) {
					$this->Auth->login($result['User']);
				} else {
					$this->Session->setFlash(__('The password entered did not match the native account\'s', true));
				}
			} else {
				$this->Session->setFlash(__('The native account could not be found', true));
			}
		}
		$this->redirect($this->Auth->redirect());
	}
}

The user_login function will try to find the NativeAccount for the given username/password. Keep in mind we need to hash the password ourself because the AuthComponent will only hash the password field for the model that is set to use, which is the User model and not the NativeAccount model. If we are able to find a native account, then we will tell the AuthComponent to login using the user account associated with the native account. When you do this, the AuthComponent will try to find an entry in the User table to validate that it is a real account, and since we the username and password fields of the AuthComponent to be the id, it will try to find the account that has the same id as the one we try to login with, which should exist. If we didn’t set the username and password fields to be the id, then it would try to find a user account using username and password columns which don’t exist on the users table.

So, that’s it. You can then add new tables/controllers to handle the different forms of authentication and then update the login view to offer the different login choices. Users can even setup multiple authentication methods, and log into the same user account in different ways.

There is one final thing to keep in mind. Savvy users could create a form which will post User data to the login action of the MyAppController, and then the AuthComponent will try to authenticate using that data. But this should not be a problem because the AuthComponent will hash the password (which we set to be the id). So, the only way someone could fake a login would be to create a request that has an id that will hash to a valid user id. If you are using the default hash (SHA1), anything id that is sent should result in a 40 character string with letters and numbers after being hashed…and this should never result in matching a user id (which is an int).

Posted in Web Development | Tagged , , , , | 3 Comments

Make CakePHP .htaccess Files Work in Personal Website on Mac OSX Snow Leopard (10.6)

CakePHP comes with default .htaccess files that work if you are working in the computers website, but if you are working in your personal websites then they need some tweaking. Since your app’s base path is not at the root, you need to specify the RewriteBase. Also, I found that the .htaccess files wouldn’t work properly until I also added the FollowSymLinks Option. Below are the three final .htaccess files I have, and they seem to work fine on my machine, with my CakePHP app under /Users/jeff/Sites/lv/:

<IfModule mod_rewrite.c>
   Options +FollowSymLinks
   RewriteEngine on
   RewriteBase /~jeff/lv/
   RewriteRule    ^$ app/webroot/    [L]
   RewriteRule    (.*) app/webroot/$1 [L]
</IfModule>
<IfModule mod_rewrite.c>
    RewriteEngine on
    RewriteBase /~jeff/lv/
    RewriteRule    ^$    webroot/    [L]
    RewriteRule    (.*) webroot/$1    [L]
</IfModule>
<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteBase /~jeff/lv/
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^(.*)$ index.php?url=$1 [QSA,L]
</IfModule>
Posted in Web Development | Tagged , , , | 1 Comment

Enable .htaccess on Mac OSX Snow Leopard (10.6)

When you enable Apache, by default you are not allowed to use .htaccess files.  Depending on if you are working on the computer’s website http://localhost or your own personal website http://localhost/~user/ you need to perform slightly different actions.

For your computer’s website, which is hosted at /Library/WebServer/Documents type the following commands in a Terminal window:

$ cd /private/etc/apache2/
$ sudo vi httpd.conf

Look for <Directory "/Library/WebServer/Documents">, and then within this element, look for AllowOverride and set it to All.

For your personal website, which is hosted at /Users/user/Sites type the following commands in a Terminal window:

$ cd /private/etc/apache2/users/
$ sudo vi user.conf

Look for AllowOverride and set it to All.

Once you have updated the appropriate conf file, restart Apache

$ sudo apachectl restart

That’s it. Your .htaccess files should now work.

Posted in Web Development | Tagged , | Leave a comment

Installing ZendDebugger on Mac OSX Snow Leopard (10.6)

Zend Studio is a great IDE for PHP development. There is also a debugger extension that you can install on your web server that will allow you to debug code while it is running on your web server. You can download the debugger at the same location as Zend Studio. The problem I had with setting it up was making sure that the plugin was in a location that was accessible. Eventually I got wise, and used <?php phpinfo(); ?> to figure out where PHP thinks my PHP extension directory is, and then put it there. So, once I put the ZendDebugger.so in the correct location, all I had to do was edit my php.ini file, and add the following:

zend_extension=/usr/lib/php/extensions/no-debug-non-zts-20090626/ZendDebugger.so
zend_debugger.allow_hosts=127.0.0.1
zend_debugger.expose_remotely='allowed_hosts'

Where /usr/lib/php/extensions/no-debug-non-zts-20090626/ is where PHP looks for extensions. Not sure if there is a better location, but for now it works, and it is only my development machine.

As a side note, I did not copy dummy.php (which comes with the Zend Debugger) to my document root location, and I don’t know why that is needed.

Posted in Web Development | Tagged , , | Leave a comment

Installing MySQL on Mac OSX Snow Leopard (10.6)

Installing MySQL on OSX is pretty straight forward, all you do is install the DMG image. Once installed, you will have a System Pref Pane that you can use to start/stop the server. Another useful thing to install is the Workbench GUI, which is an admin interface to manage your installation.

Some other things you probably want to do is make it a bit more secure, and set it up to work with PHP. In order to secure your installation, open up a terminal window, and type the following commands:

$ cd /usr/local/mysql/bin
$ ./mysql_secure_installation

Also, I found that I needed to update my php.ini file, and change the location of the mysql.sock file in order to get PHP to talk to MySQL properly. By default, PHP looks in /var/mysql/mysql.sock but by default MySQL uses /tmp/mysql.sock. I’m not sure which one is better, but as long as each is using the same location then you should be good.

UPDATE: After doing some reading, I think the /var/mysql/mysql.sock is the more correct location to use, so you will need to update your MySQL configuration to use that location.

Most of this was taken from this blog, with some of the comments added in.

Posted in Web Development | Tagged , | Leave a comment

Enable Apache and PHP on Mac OSX Snow Leopard (10.6)

Summary

There are many articles on the web related to enabling Apache and php on OSX, but I found them to be overly complicated, or scattered, so I am summarizing what I found here for my own personal records. Also, this setup is intended for developer machines only!

Enable Apache

  1. Open System Preferences Pane
  2. Go to Sharing
  3. Enable Web Sharing

The Web Sharing panel in the System Preferences
That’s it! You should now have a working instance of Apache.

Enable PHP

  1. Create the default php.ini
  2. Enable PHP in the Apache httpd.conf
  3. Restart Apache

That is the quick summary of what we need to do, and below I will explain the details.

Create the default php.ini

Open the Terminal app, and type the following commands:

$ cd /private/etc
$ sudo cp php.ini.default php.ini

Enable PHP in the Apache httpd.conf

Using the previous Terminal window, type in the following commands (if you do not know how to use vi, then replace it with your favourite text editor of choice):

$ cd /private/etc/apache2
$ sudo vi httpd.conf

Now search for the following line, uncomment it, and then save and exit.

#LoadModule php5_module libexec/apache2/libphp5.so

Restart Apache

Using the previous Terminal window, type the following command:

$ sudo apachectl restart

Test It!

Using the previous Terminal window, type the following commands:

$ cd ~/Sites
$ vi phpinfo.php

Enter the following content:

<?php phpinfo(); ?>

Save, open Safari, and then go to:
http://localhost/~user/phpinfo.php
If all goes well, you should see a page that lists your php information!

Posted in Web Development | Tagged , , | 1 Comment