Troubleshooting JSON Errors in the Email module
03 Sep, 2008Posted By: andreassandberg
Last week I was trying to diagnose an issue in Sugar 5.0 where the email folders within the Email module would not display. The folders are actually rendered after the Email module has been loaded through an asynchronous JSON call. So, my first inclination to diagnose the issue was to fire up firebug and inspect the response from the server. The response looked correct, no PHP warnings or errors were included and the JSON response looked correctly formatted.
After further inspecting the JavaScript code, I found the error was occurring because the eval() call on the response text was failing. So I was a bit concerned and modified the server code to send back a regular string. Still the eval() call failed and I was left scratching my head. I then started thinking that perhaps there was some garbage output or empty spaces after the PHP closing tag (?>) that might have been corrupting the response from the server. I wrote a quick script to examine all of my files but still came up empty.
Since I knew this error had just recently started happening, I checked out an older revision of my code base from my local SVN repository and sure enough, the old code base worked fine. The diffs between the two code bases revealed no big changes that should have been causing the issue. So I systematically started copying files from the broken version of the code base to the working code base until I could re-create the issue.
I finally found the file that was causing the problem. It was the main language string file that Sugar uses (include/language/en_us.lang.php). However comparing the different versions using WinMerge revealed no differences in the file. This was clearly very peculiar. I then opened up a hex editor and took a peak and finally found the source of my problems.
I found that the broken file contained what is referred to as a byte order mark (bom) which is usually invisible. The mark can sometimes be added by external editors (Notepad) and can cause headers to be sent to a client even though no echo() or print() statements have been used. After removing the bom, everything worked as expected.
To ensure that none of my other files were corrupted, I put together a quick script to examine my code base. I’m including the script below, but please be aware that the normal “use at your own risk” restrictions apply. Although I’ve still to track down who or what program was responsible for inserting the bom, hopefully this post will save you some time if you see similar symptoms as it was quite tricky to trace down the exact cause.
For more information regarding the byte-order mark, Wikipedia has a quick introduction.
PHP Script to Check for Unwanted Byte Order Marks
<?php
$directory = “include/language”;$bad_files = array();
$recursive_check = TRUE;
$fix_file = TRUE;scanDirectory($directory,$bad_files);
if(count($bad_files) > 0 )
{
echo “\n\n\nFound the following files with a bad BOM:\n\n”;
printResults($bad_files);
}
else
echo “No Errors Found”;function scanDirectory($directory, &$bad_files)
{
global $recursive_check;
$ignore_files = array(”.”, “..”, “.svn”);if ($handle = opendir($directory))
{
echo “\nChecking Directory: $directory\n”;while (false !== ($file = readdir($handle)))
{
$fullFile = $directory . DIRECTORY_SEPARATOR . $file;//Recursive Call for sub-directories.
if( is_dir($fullFile) && $recursive_check )
{
if( in_array($file, $ignore_files) )
continue;
scanDirectory($fullFile, $bad_files);
}
if( checkFile($fullFile))
$bad_files[] = $fullFile;
}closedir($handle);
}
}function checkFile($file)
{
global $fix_file;
if(!is_file($file))
return;echo “\t\t$file”;
$isBad = FALSE;
$file_contents = file_get_contents($file, “+r”);$hex = bin2hex($file_contents);
$first_token = substr($hex,0,6);if($first_token == ‘efbbbf’)
{
$isBad = TRUE;
echo “……FOUND ERROR\n”;if($fix_file)
fixFile($hex,$file);
}
else
echo “……OK\n”;return $isBad;
}function printResults($results)
{
foreach ($results as $fileName)
echo “$fileName\n”;
}function fixFile($hex, $file)
{
$fp = fopen($file, ‘w’);
$good = substr($hex,6);
$good_string = pack(’H*’, $good);
fwrite($fp, $good_string);
fclose($fp);
}?>
