Posts Tagged ‘XML’

JSON, An Alternative to XML in AJAX

Tuesday, July 18th, 2006

I admit it. I like Ajax. Sure it’s a dumb buzzword. But, as you’ll recall, in How To Make an Ajax Chat Room, I said something like, It just turns out XML is slightly more usable for complex stuff and that synchronous requests defeat the point. Though Ajat have many uses. The point I was trying to make is that sometimes sending text back instead of XML is nice. For example, I can send back a 1 if the action was successful. But sometimes that isn’t enough. Sometimes, text and XML are both the wrong tools for the job. That’s where JSON comes in.

The Problem With XML

The problem with XML is that it is a bitch to work with. XML files can get big if the file uses meaningful metadata. To boot, they are a pain in the ass to create when all you really want to do is send some simple data that is too complex for regular text. Then, when it comes time to use the XML in JavaScript, one has to do lots of DOM stuff just to get to the data.

Since I don’t do much PHP work, I don’t have to deal with the creation of the XML files. I do, however, have to access the data from the DOM. I got so irritated by it, that I wrote a function in my Ajax class that will convert simple XML files in to Javascript associative arrays. It turns out I was taking an unneeded step.

The Problem With Text

As I mentioned, sending text back is often useful. Since XMLHttpRequest is smart enough to recognize the content type of the returned code, I often send back errors as text and do something similar to this to display the error: if(!ajax_obj.responseXML){alert(ajax_obj.responseText);}. Sometimes, however, I return 1 to alert success and a string error message to give an error. While I’ve made this standard practice, it can get confusing when different Ajax requests get different styles of messages returned.

Additionally, making things more complex, sometimes sending a single raw value back doesn’t require an XML wrapper. For the sake of speeding up the request, the value is sent as text. This mucks with the paradigm I mentioned before.

JSON

XML is nice for documents. It’s a pain to work with as a data exchange format. Text is simpler, but can be inconsistent, often not providing enough useful data. That is where JSON picks up.

JSON, short for JavaScript Object Notation, is exactly what it says it is. It is the syntax used to create objects in JavaScript (and apparently Python, too). If you are curious, it would look like this in Javascript:

1
2
my_object = {"item1": "value", "item2": "value"};
document.write(my_object["item1"]); // Outputs "value"

You can nest objects inside objects, too.

Using JSON Instead of XML

Outputting XML from PHP is straight forward. JSON is similarly straight forward. The PHP script simply dumps out text similar to the object literal notation shown above (without the variable assignment). The request will be stored in the .responseText

When an XMLHttpRequest is sent to the server and XML is received, the XML is automagically parsed into a variable as a DOM object. JSON requires an extra step to assign the JSON object to a variable that you can use in your script.

1
my_object =  eval("("+request_object.responseText+")");

It’s pretty easy. If integrated into a class, the eval could be done on-the-fly.

Freddy — I mean, XML vs JSON

I’m still a JSON newb. I’ve done a pretty good deal of Ajax work, though. I’ve seen some very irritating stuff. For example, when fetching a few hundred records in XML format, the file can get large and cause the browser to freeze while the XML is parsed. Since JSON would be no different from executing Javascript code, it should avoid the crawl.

Unless XML tags are really light weight (which defeats the whole metadata concept of XML), the files can get really big. JSON doesn’t need as much tagging, so file sizes can be smaller.

XML is pretty simple to write. JSON is pretty simple to write. XML must be well-formed. JSON must be syntactically correct. JSON just happens to require less typing than XML.

I’m throwing this bonus in, even though it isn’t specific to Javascript: Not all languages have nice support for XML. PHP 4, for example, will do XML; it is just really hard. There are quite a few JSON interpreters for various languages and JavaScript and Python have it built-in. With XML, one has to learn how to use the XML library. With JSON, all one needs to know is how to work with normal objects in that language. Specifically with Javascript, you don’t need to learn DOM scripting to work with JSON. You just need to know how to use associative arrays.

Real World Testing

I did a simple test. I wrote a PHP file that creates an XML file with 300 values and a JSON file with 300 values. I later modified the loop to 1000 values to see if an article I read was true about Ajax breaking down exponentially. It’s basically true, by the way.

To be fair, I had already written the code, then modified it to run on JSON. For that to work, I had to make the JSON output match what would show after my XML file was turned into an object. The XML file was already optimized to be very small. So, while that might give some size overhead to my JSON (since it had to mimmic nodes), it also gives some parsing overhead to my Ajax (since it was converted to an object before use). What I mean is that the code I used could be more efficient (and I might re-write it eventually).

The Javascript reads the XML or JSON file. It uses the information to populate a select. I uploaded it to a server across the country from where I am and I ran the files. The server gets spikes of heavy use, so the results may be skewed.

At 300 records, I couldn’t really tell a difference between the JSON and the AJAX. Early tests at 1000 records, it was debatable. JSON was at least as fast as XML (usually around 7 seconds from refresh until was complete). Some requests were far faster on JSON (2 seconds instead of 7). Later tests revealed a much larger variation, with XML taking 40 seconds and JSON taking only 15.

The Golden Fleece

I like JSON for what it is. It’s very capable and much lighter than XML, both in terms of parsing and in terms of file size. It is a capable alternative to XML and has quite a few undeniable upsides. I’ll probably do further testing before rewriting all my legacy code, but I will be using JSON in the future.

How To Make an Ajax Chat Room

Friday, June 23rd, 2006

I’ve been helping this guy from India with an Ajax / XML chat app. I wrote one ages ago, but I think it broke. So, since I’m already giving a how-to, I might as well write it down.

Ajax For Newbies

If you are familiar with XML, the buzzword Ajax, the XMLHttpRequest() object, and DOM scripting, skip to the juicy part. If not, keep on reading.

An XML Primer for Ajax

XML looks kind of like HTML, but it is quite different. For the sake of brevity, XML is a structured data file that draws from SGML. The difference is that it must be well formed and that the tags can be arbitrarily named (unless you specify a DTD).

Going into more detail would not be pragmatic since the JavaScript XML parsers only care about whether or not the XML is well-formed. So, here is an example XML file:

<messages>
 <message>
  <from>
   Fred
  </from>
  <body>
   Fred here... Just making a post!
  </body>
 </message>
 <message>
  <from>
   Joey
  </from>
  <body>
   Hey, Fred.  Thanks for the post.
  </body>
 </message>
</messages>

The good thing about good XML is that it has metadata built in. You can read it and know what is what. The code above is obviously a group of messages. Inside the messages group is a message. In this case, there are two of them. Inside each message is a from and a body. The from, of course, contains the name of the person who sent the message. It follows that the body contains the message body. This XML file will be used later.

XMLHttpRequest Primer for Ajax

XMLHttpRequest, at least on browsers that aren’t Internet Explorer 6 or less, is a native object in JavaScript that allows the client to send a request to the server in the background. That is, the browser doesn’t have to physically go to the page. This makes it easy to update parts of pages without iframe, frame and other hacks.

I’ll give you the condensed version of how to use it for the sake of brevity. I’m sort of copy / pasting code from Apple’s Developer documents because their example does XMLHttpRequest() the right way.

// This needs to be in the global scope so we can use it in other functions.
var xml_obj;
/*
 
 We will be sending loadXMLDoc the url of the page we want to call,
 the query string to post to the url, and a function to call when
 the script updates.
 
*/
function loadXMLDoc(url,query_string,callback) {
	xml_obj = false;
    // Try native XMLHttpRequest first.
    if(window.XMLHttpRequest) {
    	try {
			xml_obj = new XMLHttpRequest();
        } catch(e) {
			xml_obj = false;
        }
    // If native fails, fall back to IE's ActiveX objects.
    } else if(window.ActiveXObject) {
       	try {
        	xml_obj = new ActiveXObject("Msxml2.XMLHTTP");
      	} catch(e) {
        	try {
          		xml_obj = new ActiveXObject("Microsoft.XMLHTTP");
        	} catch(e) {
          		xml_obj = false;
        	}
		}
    }
	if(xml_obj) {
		// Any time the state of the request changes, callback will be called.
		xml_obj.onreadystatechange = callback;
		// Set the parameters of the request.
		// POST can also be GET.  We use the URL from above.
		// The <q>true</q> tells whether the request should be asynchronous.
		xml_obj.open("POST", url, true);
		xml_obj.setRequestHeader("Content-Type","application/x-www-form-urlencoded");
		// Since we are POSTing, we send the query string as a header
		// instead of as a string at the end of the URL.
		xml_obj.send(query_string);
	}
	else{
		alert("Failed to create XMLHttpRequest!");
	}
}
/*
 
This is an example callback function.
 
*/
function messagesRetrieved(){
	if (xml_obj.readyState == 4){
		// When readyState is 4, the request is complete.
		if (xml_obj.status == 200){
			// When status is 200, we know there are no server errors or anything.
			// and we can process the data.
		}
		else{
			alert("There was a problem retrieving the XML data:\n" + xml_obj.statusText);
		}
	}
}
 
// How we would call the loadXMLDoc
loadXMLDoc("http://domain.com/myurl.php","action=get_messages",messagesRetrieved);

Hopefully, from the comments in the script, you can see what is going on. When the request starts, the xml_obj.readyState property will be update. Once the request is complete, there will be several properties available.

  • xml_obj.responseXML is the most important for Ajax, since that is where the data is stored as an object. Yes, JavaScript automagically parses the XML and puts it in an object for you.
  • xml_obj.responseText is nice because it will print out the text returned from the server. It’s great for debugging.
  • xml_obj.status is the server status code of the request. This should be 200 or something is wrong with your scripts.

What is Ajax About, Then?

As I said, the term Ajax is a buzzword. It’s a short way of saying, JavaScript using XMLHttpRequest to get XML from a server in the background.

There are other methods that are probably get called Ajax, but aren’t. For example, XMLHttpRequest allows the server to send back text instead of XML (Ajat?). Also, if the asynchronous flag is set to false, the request doesn’t run in the background. Instead the script that is running waits for a response from the server before it continues.

It just turns out XML is slightly more usable for complex stuff and that synchronous requests defeat the point. Though Ajat have many uses.

DOM Scripting

DOM scripting is the key to Ajax. Ajax is pretty useless if you can’t extract the data. Basically put, the DOM is HTML / XML represented in an object. DOM Scripting, then, is using JavaScript (or some other language, I suppose) to manipulate this object. If you manipulate the DOM of your HTML page, it will reflect visibly in the browser.

DOM scripting replaces old-school document.write coding. Instead, you insert nodes (aka tags) or remove nodes from the DOM, and thus the page. The great thing is that XML files can be represented as a DOM object. That means that you can manipulate the XML the same way you manipulate the HTML. You’ll make a lot of use of the following (where xml = xml_obj.responseXML):

  • xml.childNodes is a collection of the children of the current node. It’s handy to loop on: for(i = 0; i < xml.childNodes.length; i++){}
  • xml.nodeName is the string value of the current tag.
  • xml.nodeType is a numeric representation of what kind of node the current element is, text (3) or element (1). This is handy for conditionals: if(xml.nodeType == 3){alert(xml.nodeValue);}
  • xml.nodeValue is the string value of the current element if it is a text node.

There are more useful properties and methods, but, for now, we're going to loop through output. It isn't the most efficient method, but it is more agnostic than, say, xml.getElementsByTagName.

A Basic Ajax Chat

As we see above, we already have a good deal of code written. In fact, most of the tedious stuff is done. We just have to solve a few problems.

Server Side Scripting

Ajax isn't worth much without server side scripting. PHP is my weapon of choice, but ASP, Perl, Ruby, or any other language will work just fine. However, since I chose PHP, it will affect my data storage solution.

Data Storage

Most people use Ajax to deliver data stored on the server. All the usual methods are available including mySQL, text files, and XML. XML seems like the likely choice, since we are spitting out XML. However, at least with PHP, working with XML is hard. Further, doing queries on XML seems like it'd be severely difficult in PHP. Other languages might like XML better than PHP, though.

So, for the sake of this article, that leaves text files and mySQL. Normally, I'd go with mySQL but this is a simple article. So, we're just going to use text files. Opening files is easier than writing all the code to get the database running.

The Text Storage File

The text file needs to only hold two variables: the name of the person sending the message, and their message. We'll use a tab-separated file since it is unlikely that the user will enter a tab (because that typically changes fields) and we'll remove all carriage returns. So, the XML file mentioned at the beginning before will be stored like this:

Fred	Fred here... Just making a post!
Joey	Hey, Fred.  Thanks for the post.

Convert The Text File To XML

We'll need to write a function in PHP to convert the text file to XML for display in our chat application. It should be something like this:

function tsv_to_xml($file){
	if(file_exists($file))
		$output = trim(file_get_contents($file));
	else
		die("Couldn't open the file.");
	if($output != "")
		$output = explode("\n",$output);
	else
		$output = array();
	$return = "<messages>";
	for($i=0; $i<count($output); $i  ){
		$output[$i] = explode("\t",$output[$i],2);
		$return .= "<message>";
		$return .= "<from>";
		$return .= "".htmlentities($output[$i][0])."";
		$return .= "</from>";
		$return .= "<body>";
		$return .= "".htmlentities($output[$i][1])."";
		$return .= "</body>";
		$return .= "</message>";
	}
	$return .= "</messages>";
	return $return;
}

That should return the XML file specified at the beginning from the tab-separated file.

Adding a New Message

We also need to add to the text file. So, we'll write a PHP function for that, as well:

function tsv_add($file, $from, $body){
	if(file_exists($file))
		$output = trim(file_get_contents($file));
	else
		die("Couldn't open the file.");
	$from = preg_replace("/(\s|\r\n|\r|\n)/"," ",$from);
	$body = preg_replace("/(\s|\r\n|\r|\n)/"," ",$body);
	$tsv = $from."\t".$body;
	$my_file = fopen("$file","w");
	fwrite($my_file, trim($tsv)."\n".$output);
	fclose($my_file);
	return tsv_to_xml($file);
}

Basically, we create a tab-separated entry for our file (stripping out protected characters), dump it into a file and return the updated XML file.

Putting the PHP Together

It'd be a bummer to use several different PHP files when one would do. So, we'll use a script to manage it. We want to accept an action parameter to help our script decide what to do. It will look like this:

$file = "./data/chat.txt";
switch($_POST["action"]){
	case "get_xml":
		header("Content-type: text/xml");
		echo tsv_to_xml($file);
		die();
		break;
	case "add_post":
		header("Content-type: text/xml");
		echo tsv_add($file, $_POST["from"], $_POST["post"]);
		die();
		break;
	default:
		die("There is no action associated with that call.");
		break;
}

Once we have all that PHP in one file, it will be our request file. So, we will call it request.php and put it on our development server putting the tsv_add and tsv_to_xml functions at the top. Make sure the data folder is writable! Make sure you put an empty chat.txt file in the data folder!

The HTML

The only thing we don't have is the HTML file (and a little bit of JavaScript) we'll use to display the XML from the chat. The code for the HTML should look something like this (except you'll probably want to put the CSS in a file and use the link tag):

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
 <head>
  <title>Ajax Chat</title>
  <style type="text/css">
   label{display:block; width:100%;}
   input{display:block; width:100%;}
   input#submit{width:auto;}
  </style>
  <script type="text/javascript" src="chat.js"></script>
 </head>
 <body onload="get_xml();">
  <form method="post" action="#" name="send_form" id="send_form" onsubmit="send_xml(); return false;">
   <p>
    <label for="uname" accesskey="n">Name</label>
    <input type="text" id="uname" name="uname">
   </p>
   <p>
    <label for="mbody" accesskey="m">Message</label>
    <input type="text" id="mbody" name="mbody">
   </p>
   <p>
    <input type="submit" id="submit" value="Send Message" accesskey="s">
   </p>
  </form>
  <div id="chat_area">Loading chat...</div>
 </body>
</html>

The Rest of the JavaScript

We need just a little bit more JavaScript. The get_xml() function, the send_xml() function, and the messagesRetrieved() function (see above).

function messagesRetrieved(){
	if (xml_obj.readyState == 4){
		if (xml_obj.status == 200 &amp;&amp; xml_obj.responseXML){
			xml = xml_obj.responseXML;
			output = "";
			for(var i=0; i < xml.childNodes.length; i++){
				for(var j=0; j < xml.childNodes[i].childNodes.length; j++){
					if(xml.childNodes[i].childNodes[j].nodeType == 1){
						for(var k=0; k < xml.childNodes[i].childNodes[j].childNodes.length; k++){
							if(xml.childNodes[i].childNodes[j].childNodes[k].nodeName == "from"){
								output  = "<b>" + xml.childNodes[i].childNodes[j].childNodes[k].firstChild.nodeValue + "</b>: ";
							}
							else if(xml.childNodes[i].childNodes[j].childNodes[k].nodeName == "body"){
								output  = xml.childNodes[i].childNodes[j].childNodes[k].firstChild.nodeValue + "<br>";
							}
						}
					}
				}
			}
			document.getElementById("chat_area").innerHTML = output;
		}
		else{
			alert("There was a problem retrieving the XML data:\n" + xml_obj.statusText + "\n" + xml_obj.responseText);
		}
	}
}
 
function get_xml(){
	loadXMLDoc("request.php","action=get_xml",messagesRetrieved);
	setTimeout("get_xml()",5000);
}
 
function send_xml(){
	u = document.send_form.uname.value;
	b = document.send_form.mbody.value;
	qs = "action=add_post&amp;from=" + escape(u) + "&amp;post=" + escape(b);
	loadXMLDoc("request.php",qs,messagesRetrieved);
}

Put all the JavaScript (except the initial skeleton of messagesRetrieved()) into a file called chat.js and save it to the same folder as the HTML file and request.php.

Finishing Up

Assuming everything is set up the way I set it up on my server, you should have a rudimentary Ajax chat. Of course, there are some tweaks I highly suggest. The first being adding clip and overflow to the chat_area div. If done right, you can have a scrollable window much like an iframe.

The second thing to do is modify the script so it keeps track of the time the posts were made. Then the script could figure out which posts the user hasn't received and send only them. This will cut down on bandwidth, which is important. I let my first Ajax Chat run all day and I ended up doing three gigs of bandwidth. That's a lot considering it was all text files.

The third thing I would do is learn more about DOM scripting and get rid of those for loops in messagesRetrieved(). While they are agnostic and easy to add message items to, they aren't pretty.

Disclaimer

I wrote this code and tested it on Safari and Firefox on Mac. It worked for me with those browsers. It might not work for you and you may have to tweak it heavily. I don't have time to support it. The code is free and it is yours to use. So, feel free to modify it and ask other people for help with it. If you want to be nice, link back to this page if you get it working.

Oh, and since this dynamically updates an area of the page, it probably isn't very accessible to screen readers. Don't say I didn't warn you.

Source Code

You can download all the files mentioned above in a zip file. They are uncommented, but the code is clean enough that you should be able to figure out what is going on.