MySQL query returning truncated results could be because of group_concat

I recently encountered an issue where my MySQL query was returning truncated results… and by truncated I mean it was returning only a fraction of the rows it should have been returning and the last of those rows was actually missing data.

The query in question uses a few LEFT OUTER JOIN(s) and the GROUP_CONCAT function to eliminate duplicate results due to a many-to-one relationship as shown below:

Turns out that the results were getting truncated due to a GROUP_CONCAT memory limitation configured by: group_concat_max_len

I therefore had a few options:

  • Remove GROUP_CONCAT from the query and deal with the duplicate results programmatically
  • Split up the query in perhaps 2 or 3 individual queries
  • Overwrite the group_concat_max_len variable in PHP for the session

I chose the latter by adding the following snippet in my DB configuration file.

if (!mysqli_query($link, 'SET SESSION group_concat_max_len = 102400;')) {
	$error = 'Error setting group_concat_max_len: ' . mysqli_error($link);
	include $_SERVER['DOCUMENT_ROOT'].'/error.html.php';
	exit();
}

I ended up setting the value to 102400 bytes which was about the max size I could foresee the group_concat having to process.

You can check your MySQL value was by running the following in the terminal:
$ mysqld --verbose --help

Hope this helps someone!

Posted in Programming | Tagged , , , | Leave a comment

jQuery Checkbox select all

Ever wanted to allow a user to click the Select all checkbox and have all the checkboxes magically have checks in them?

You might want to do that if you have lots of options, especially if your checkboxes are being generated dynamically from information pulled from a database. The latter is what prompted me to add a select all button to one of my web apps. So like any developer who’s first, second or even third language isn’t Javascript, I fired up Google and searched for a solution.

The initial solution (not so good)

After copy pasting a bit of code here and there, I came up with a basic solution that uses document.getElementsByName

The Javascript looked like this:

<script type="text/javascript">
  function toggle(source,tgt) {
    checkboxes = document.getElementsByName(tgt);
    for(var i in checkboxes) checkboxes[i].checked = source.checked;
  }
</script>

The html:

<input type="checkbox" onClick="toggle(this, 'product[location_id][]')" />Select All<br/>

Now this worked fine until I tested it in IE. Dreaded IE! Turns out it doesn’t work because of an IE bug that doesn’t like document.getElementsByName

The best solution

So back to the drawing board or should I say Google. That’s when I found this gem: http://briancray.com/2009/08/06/check-all-jquery-javascript/

It uses jQuery which is a Javascript library. The brilliance is that it requires very little code upfront and it uses the fieldset html tag to wrap the checkboxes that you wish to check/uncheck with your select all checkbox. Therefore you can easily have many different checkbox groups with their own select all checkbox on the same page without any mixups.

Posted in Programming, Web | Leave a comment

Connect to MySQL using localhost instead of 127.0.0.1 on a MAC

For a long while now I’ve been connecting to MySQL on my development platform with 127.0.0.1 because for some reason localhost didn’t work.
Turns out it’s because 127.0.0.1 uses TCP/IP and localhost uses sockets.
The php.ini file points to the wrong place for the mysql.sock so all you have to do is change it, restart apache and voila!

  • Open php.ini: /private/etc/php.ini
  • Find the following line: mysql.default_socket = /var/mysql/mysql.sock
  • Replace with: mysql.default_socket = /tmp/mysql.sock
  • Restart apache: apachectl restart
  • Happy days!

Note: If you don’t have a php.ini file, you need to copy the provided default called php.ini.default

sudo cp /private/etc/php.ini.default /private/etc/php.ini

Thanks to Blog.Andrei.MD for the info.

Posted in General | 1 Comment

Back up! Time Machine saved my life!

Just a quick note about how important it is to backup your data! The hard drive in my MacBook Pro decided to stop working this morning and in my case, my laptop is my life. All my work, photos, music, contacts are stored on this one laptop.

Luckily, a few months ago, after staring at that inactive Time Machine icon for quite a while, I decided to activate it.

Am I ever glad I did! As I type this on my iPhone, my MBP is 22 minutes away from being fully restored thanks to Time Machine. I also took the opportunity to install a better HDD. I installed the Seagate Momentus XT, a hybrid HD that combines a solid state drive(SSD) and your standard 7200rpm mechanical drive.

I digress, the point I want to make is backing up your data is paramount! Your data is kind of like air(and sex), it’s no big deal until it’s not available.

To recap… I use:
- Time Machine for all apps and data on my MBP
- Git and Git Hub for application development
- Dropbox for miscellaneous files

Follow up note
So I’m back on my MBP, and all is good except…
I use my MBP primarily to develop web based applications. Having said that, I have a few custom configurations in files such as /etc/hosts & /etc/php.ini as well as log files in /var/log/apache2. These are not covered by Time Machine. On the plus side, /etc/apache2/users/xxx.conf is backed up by TM which is where the biggest chunk of my configurations reside.

Posted in General | Leave a comment

CSS and Javascript hover with a memory

I recently found myself needing to design a vertical menu that makes uses of the CSS :hover selector.
In this particular case I also wanted the last item that was in a hover state to stay in that state. In other words: if I was half way down the menu with my mouse then moved it away, I wanted that item to stay changed until the mouse was returned to perhaps another item in the menu.

What I ended up with was a mix of CSS and Javascript (but mostly Javascript). Please note that I’m no Javascript expert, so there may very well be a better way to do this. Having said that, I’ve successfully tested this in IE8, Firefox 4 beta, Safari 5.

Edit: I’m aware that this could be done quite easily with jQuery but I wanted to have a go at doing it without using the library.

The Javascript placed between the head tags

<script type="text/javascript">
	function trig(elid) {
		//Change the bg image for element with the current mouseover
		document.getElementById(elid).style.background = 'url(images/click.png) center left no-repeat';

		//List of all the element ids (Perhaps this can be generated dynamically?)
		var names = ["bb", "trc", "mb", "ch"];

		//Need this because indexOf is incompatible with IE8
		if (!Array.prototype.indexOf) {
			Array.prototype.indexOf = function(obj, start) {
			     for (var i = (start || 0), j = this.length; i < j; i++) {
			         if (this[i] == obj) { return i; }
			     }
			     return -1;
			}
		}

		//Find the index of the element currently triggered by the mouseover
		var idx = names.indexOf(elid);

		//Remove the element from the array
		if(idx!=-1) names.splice(idx, 1); 

		//For each element still left in the array make sure it's got the default background
		for (i=0;i<names.length;i++)
		{
			document.getElementById(names[i]).style.background = 'url(images/dot.png) center left no-repeat';
		}
	}
</script>

The HTML

<div id="locations_menu">
	<ul>
		<li class="first" id="bb"><a href="#" onmouseover="trig('bb')">MENU ITEM 1</a></li>
		<li id="trc"><a href="#" onmouseover="trig('trc')">MENU ITEM 2</a></li>
		<li id="mb"><a href="#" onmouseover="trig('mb')">MENU ITEM 3</a></li>
		<li id="ch"><a href="#" onmouseover="trig('ch')">MENU ITEM 4</a></li>
	</ul>
</div>

The end result with a bit of styling

CSS and Javascript hover with a memory

CSS and Javascript hover with a memory

The live version
CSS and Javascript hover with a memory

Posted in Programming | Leave a comment

PHP: Using mysqli_fetch_assoc more then once

In some of my code I need to use mysqli_fetch_assoc on the same result set more then once.

However the second time I fetch the associative array it returns 0 rows. Turns out this is because each time you use mysqli_fetch_assoc it moves the result pointer. You therefore have to reset the pointer.

Here’s an example:
This code will loop through the entire result set and the result pointer will be left pointing at the end

while($row = mysqli_fetch_assoc($result)) $this->grand_total += $row['total'];

To reset the pointer:

mysqli_data_seek($result,0); //Reset the result pointer
Posted in Programming | Leave a comment

Generating HTML select with PHP foreach

I’m currently programming the update part of a simple CRUD user interface. This form loads up data relating to a product we’ve previously entered and lets you update its attributes. One of the fields required a dropdown box with a selection of available options with the current selected options being the first default selection.

While I know this isn’t rocket surgery, I took sometime to code it as efficiently as I could using a shorthand if/else statement embedded in the echo.

The final code

//$product_locations is 2d array pulled from the database, each row has two associations: id and title
//$product is another array pulled from the database that contains current attribute settings for the product

<select name="product[location_id]">
	<?php
	foreach($product_locations as $product_location) :
		echo "<option value=\"".$product_location['id']."\" ".($product_location['id'] == $product['product_location'] ? "selected=\"yes\"" : "").">".$product_location['title']."</option>";
	endforeach;
	?>
</select>

Here is the embedded shorthand conditional statement pulled out of the final code

<?php
.($product_location['id'] == $product['product_location'] ? "selected=\"yes\"" : "").
?>

This could possibly be done more efficiently so if you have idea or comments, please share!

Reference
PHP Shorthand If / Else Example

Posted in Programming | Leave a comment

Photostore Purge Plugin

I recently had to purge about 100’000 photos from a Photostore site and doing it using the manager control panel is very time consuming. I developed a little plugin that will delete all photos not associated to a category and clean up DB entries. Worked great for me but use at your own risk.

<?php

/********************************************************/
/*   Photostore mass photo purge                        */
/*   Author: Trevor Wistaff                             */
/*   Site: http:/www.madproject.com						*/
/*	 Email: easycoatgmaildotcom							*/
/*   Date: 17.11.2010                                   */
/*													    */
/* This little chunk of code was developed out of       */
/* necessity. I needed to purge large amounts of        */
/* photos and I had no quick way of doing it.           */
/*													    */
/* How to use: Delete the categories containing the     */
/* photos you wish to purge. Manager->Categories.       */
/* The photos will then be listed in the Batch Edit		*/
/* section of the manager control panel. Upload this	*/
/* file in the manager folder of your photostore then 	*/
/* run it directly from you browser:					*/
/* http://www.yoursite.com/photostore/manager/purge.php */
/*                                                      */
/* Legal mumbo jumbo: Use this tool at your own risk.   */
/* I've only tested it on my own site and can't say how */
/* it might work on a different server and version.     */
/* 														*/
/* This plugin was developed for:						*/
/* Ktools.net PhotoStore Version 3.8.3					*/
/*														*/
/* Thanks and stay classy 								*/
/********************************************************/

// SETTINGS
// DATABASE
define('DBHOST','localhost'); //HOST
define('DBUSER','user'); //USER
define('DBPASS','password'); //PASSWORD
define('DBNAME','database_name'); //NAME

// IMAGE PATH
define('IMAGE_PATH','../stock_photos/');  //SHOULDN'T NEED TO BE CHANGED

// CONNECTION
$link = mysqli_connect(DBHOST, DBUSER, DBPASS);

if (!$link) {
	echo 'Unable to connect to the database server.';
	exit();
}
if (!mysqli_set_charset($link, 'utf8')) {
	echo 'Unable to set database connection encoding.';
	exit();
}

if (!mysqli_select_db($link, DBNAME)) {
	echo 'Unable to locate the ' . DBNAME . ' database.';
	exit();
}

// START PURGE

// SELECT ALL CATEGORIES NOT ASSIGNED TO CAT
$result_1 = mysqli_query($link, "SELECT id FROM photo_package WHERE gallery_id = ''");
if (!$result_1) {
	echo mysqli_error($link);
	exit();
}

while($photo_package = mysqli_fetch_assoc($result_1)) {

	// SELECT PHOTO MATCHING PHOTO_PACKAGE.ID == UPLOADED_IMAGES.REFERENCE_ID
	$result_2 = mysqli_query($link, "SELECT id, filename FROM uploaded_images WHERE reference_id = '{$photo_package['id']}'");
	if (!$result_2) {
		echo mysqli_error($link);
		exit();
	}

	while($uploaded_images = mysqli_fetch_assoc($result_2)) {

		// ECHO IMAGE DETAILS
		echo IMAGE_PATH . $uploaded_images['filename'] . "<br />";
		echo IMAGE_PATH . "PhotoID: " . $uploaded_images['id'] . " - PackageID: " . $photo_package['id'] . "<br />";

		//DELETE PHYSICAL PHOTOS
		if(file_exists(IMAGE_PATH . $uploaded_images['filename'])){
			if(unlink(IMAGE_PATH . $uploaded_images['filename'])) {
				echo "<div style=\"color: #00cc33\">image deleted successfully</div>";
			} else {
				echo "<div style=\"color: #ff0000\">image deleted unsuccessfully</div>";
			}
		}
		if(file_exists(IMAGE_PATH . "i_" . $uploaded_images['filename'])){
			if(unlink(IMAGE_PATH . "i_" . $uploaded_images['filename'])) {
				echo "<div style=\"color: #00cc33\">i_ image deleted successfully</div>";
			} else {
				echo "<div style=\"color: #ff0000\">i_ image deleted unsuccessfully</div>";
			}
		}
		if(file_exists(IMAGE_PATH . "s_" . $uploaded_images['filename'])){
			if(unlink(IMAGE_PATH . "s_" . $uploaded_images['filename'])) {
				echo "<div style=\"color: #00cc33\">s_ image deleted successfully</div>";
			} else {
				echo "<div style=\"color: #ff0000\">s_ image deleted unsuccessfully</div>";
			}
		}
		if(file_exists(IMAGE_PATH . "m_" . $uploaded_images['filename'])){
			if(unlink(IMAGE_PATH . "m_" . $uploaded_images['filename'])) {
				echo "<div style=\"color: #00cc33\">m_ image deleted successfully</div>";
			} else {
				echo "<div style=\"color: #ff0000\">m_ image deleted unsuccessfully</div>";
			}
		}

		//DELETE DATABASE ENTRIES
		$result_3 = mysqli_query($link, "DELETE FROM uploaded_images WHERE id = '{$uploaded_images['id']}'");
		if(!$result_3) {
			echo "<div style=\"color: #ff0000\">" . mysqli_error($link) . "</div>";
			exit();
		} else {
			echo "<div style=\"color: #00cc33\">image delete successfully from uploaded_images table</div>";
		}
		$result_4 = mysqli_query($link, "DELETE FROM photo_package WHERE id = '{$photo_package['id']}'");
		if(!$result_4) {
			echo "<div style=\"color: #ff0000\">" . mysqli_error($link) . "</div><br />";
			exit();
		} else {
			echo "<div style=\"color: #00cc33\">image delete successfully from photo_package table</div><br />";
		}
	}
}

?>
Posted in Programming | Leave a comment

James McMillan: A big miracle

Surf and Art by James McMillan, great stuff!

Posted in Art, Surfing | Leave a comment

Facebook: Pages vs Groups

Facebook pencil

Nowadays it’s pretty much standard practice for companies to utilise Facebook in order to extend their online presence. However, faced with the option of creating and maintaining a Facebook page or a Facebook group, which do you choose? Page, group or both? Both is definitely out of the question as you’d end up with duplicate content, a split client base and as a result: inefficient marketing.

So I did some research and found a great article detailing the pros and cons of the Facebook group and page. You can read the article here: Facebook Groups Vs Pages: The Definitive Guide

Essentially, Facebook pages is the right choice for any business that values analytics, is expecting to grow, values SEO and utilises targeted marketing.

Some noteworthy points about Facebook pages

  • You cannot send mass messages out like you can with Facebook groups(which by the way is disabled once you reach 5000) but you can send page updates out to all your members which is similar. Click on: Edit page->Marketing->Send an update
  • You have page analytics which not only tell you how many people have joined or left your page but also offer geographical, gender and age group break-down amongst otherthings.
  • You can have a human readable URL(http://www.facebook.com/madproject) which is not possible with groups

Facebook Groups vs Pages

Posted in Web | Leave a comment